[TEST] v1.8.0 release testing - Three node upgrade w/ 2 MGMT/Default Nodes & 1 Witness Node

[TachunLin]

What's the test to develop? Please describe

Perform the Three node upgrade w/ 2 MGMT/Default Nodes & 1 Witness Node for v1.8.0 release testing

Prerequisite and dependency of test

1. Setup three nodes cluster w/ witness node
2. Create a new storage class w/ replica 2 and set as default
3. VLAN 1 network on mgmt and 1 network on other NICs
4. 2 Virtual machines with data and md5sum computed- 1 running, 1 stopped
5. 2 VM backup, snapshots - 1 backup when VM is running and 1 backup when VM is stooped
6. Create a new storage class apart from default one. Use the new storage class for some basic operations.

Rancher Upgrade Path:

• H1.7.1 + Rv2.13.3 (k1.34) -> H1.7.1 + R2.14.0 (K1.34) -> H1.7.1 + R2.14.0 (K1.35)

Harvester Upgrade Path:

• v1.7.1 -> v1.8.0

OS:

• Ubuntu 22.04
• Ubuntu 24.04
• Windows 11

Pre Upgrade Test steps

1. Import to Rancher
2. Create couple of RKE2 guest clusters say gc1, gc2 provisioned on Harvester VM.
3. Create Load balancer and PVC in the guest cluster. Attach them to workloads.
4. Follow the upgrade path as mentioned in the Test plan.

Upgrade check

1. Can correctly upgrade Harvester from v1.7.1 to v1.8.0-rc5 or later
2. Can correctly upgrade Rancher from v2.13.3 to v2.14.0

Post Upgrade Check

1. Dependencies Check
   • dependency_charts
   • Versions of all the crds
2. Virtual machines are in same state as before and accessible.
3. Restore the backups, check the data
4. Image and volume status
5. Monitoring chart status
6. VM operations are highlighted and working fine.
7. Verify DHCP load balancer service and create a new Harvester PVC
8. Shutting off VM and then restarting VM
9. Create another one node guest cluster say gc3.
10. Scale down up the guest cluster gc1 (Only worker node)

[TachunLin]

Three node upgrade w/ 2 MGMT/Default Nodes & 1 Witness Node

Test environment

• Harvester: v1.7.1 upgrade to v1.8.0-rc5

• Rancher: v2.13.3

• OS:

  • Ubuntu 22.04
  • Ubuntu 24.04
  • Windows 11

• cloud provider

  • 0.2.1000

• csi-driver

  • 0.1.2500

• RKE2: v1.34.5 to v1.35

Prerequisite and dependency setup

${\textsf{✅}}$ Three nodes cluster with two management role and one witness role $~~$

n1-hp6:~ # kubectl get nodes
NAME      STATUS   ROLES                AGE   VERSION
n1-hp6    Ready    control-plane,etcd   16h   v1.34.3+rke2r3
n2-hp10   Ready    etcd                 14h   v1.34.3+rke2r3
n3-hp50   Ready    control-plane,etcd   14h   v1.34.3+rke2r3

${\textsf{✅}}$ Create a new storage class w/ replica 2 and set as default $~~$

${\textsf{✅}}$ VLAN 2012 network on mgmt and vlan 2017 network on other NICs $~~$

${\textsf{✅}}$ 2 Virtual machines with data and md5sum computed- 1 running, 1 stopped $~~$

ubuntu@ubuntu2204-running:~$ sha512sum running.txt 
d17bdc2e16e53a4b8c9a5bf9a2da02d0d3e1e1efb5729b2b30e559265ad5b3fff9bc540cfbf655e4b1cc98fb1d351da93748f283661876ba6f09d32d6e61549e  running.txt

ubuntu@ubuntu2404-stopped:~$ sha512sum stopped.txt 
9e72e3e1199a0aea3751abf3d6df3df5663a7e7677ed91b2514de74d2f169cfbba72ae83b56890444a9bc2bd6af4f04f99c0ff49cafebd725efadb67fcf30c86  stopped.txt

${\textsf{✅}}$ 2 VM backup, snapshots - 1 backup when VM is running and 1 backup when VM is stooped $~~$

${\textsf{✅}}$ Create windows 11 vm $~~$

Test step on Rancher v2.13.3 with Harvester v1.7.1

${\textsf{✅}}$ On Rancher v2.13.3 install UI extension v1.7.1 $~~$

${\textsf{✅}}$ Import Harvester to Rancher $~~$

${\textsf{✅}}$ Enabled all Harvester addons $~~$

${\textsf{✅}}$ Create v1.34.5 RKE2 cluster with ubuntu24.04 $~~$

${\textsf{✅}}$ Check cloud-provider: 0.2.1000 and csi-driver: 0.1.2500 $~~$

${\textsf{✅}}$ Create harvester pvc and attach to nginx deployment $~~$

${\textsf{✅}}$ Create load balancer service and bind to nginx deployment $~~$

[TachunLin]

Test environment

• Harvester: v1.7.1

• Rancher: v2.13.3

• OS:

  • Ubuntu 22.04
  • Ubuntu 24.04

• RKE2: v1.34.5 to v1.35

• cloud provider: 0.2.1100

• csi-driver: 0.1.2500

Upgrade to Rancher v2.14.0 with Harvester v1.7.1

${\textsf{✅}}$ Upgrade Rancher from v2.13.3 to v2.14.0 $~~$

${\textsf{✅}}$ Check cloud-provider and csi-driver status on rke2-v1345 RKE2 cluster $~~$

${\textsf{✅}}$ Upgrade `rke2-v1329` cluster from v1.34.5 to v1.35.2 $~~$

[TachunLin]

Upgrading Harvester from v1.7.1 to v1.8.0-rc5

${\textsf{✅}}$ VM, Volume and RKE2 cluster state before upgrade $~~$

${\textsf{✅}}$ Trigger upgrade Harvester to v1.8.0-rc5 $~~$

${\textsf{❌}}$ Failed to upgrade to v1.8.0-rc5, stuck in third node Pre-draining $~~$

1. The upgrade stuck on the third node Pre-draining

2. Check the pre-drain pod log, it loops in Waiting for VM live-migration or shutdown

hvst-upgrade-crtst-pre-drain-n3-hp50-kdszs Waiting for VM live-migration or shutdown...(count: 1)                                                          │
│ hvst-upgrade-crtst-pre-drain-n3-hp50-kdszs + echo 'Waiting for VM live-migration or shutdown...(count: 1)' 

vokoscreenNG-2026-04-06_22-41-41.mp4

3. Check the VMIM events, it have failure

Failed to set MigrationStat in VMI status. :the server rejected our request due to an error`

Events:                                                                                                                                                    │
│   Type     Reason              Age   From                       Message                                                                                    │
│   ----     ------              ----  ----                       -------                                                                                    │
│   Normal   SuccessfulCreate    30m   virtualmachine-controller  Created migration target pod virt-launcher-rke2-v1345-pool1-jclpl-65mld-2p968              │
│   Normal   SuccessfulCreate    30m   virtualmachine-controller  Created attachment pod hp-volume-j4czl                                                     │
│   Normal   SuccessfulHandOver  29m   virtualmachine-controller  Migration target pod is ready for preparation by virt-handler.                             │
│   Warning  FailedHandOver      29m   virtualmachine-controller  Failed to set MigrationStat in VMI status. :the server rejected our request due to an erro │
│ r in our request

• Support bundle

  supportbundle_75cf1642-606e-42a1-ba61-8ce7b1c1f666_2026-04-06T14-47-40Z.zip

• Upgrade log

  hvst-upgrade-crtst-upgradelog-archive-2026-04-06T14-46-24Z.zip

${\textsf{🛠️}}$ After apply the workaround can proceed to complete the upgrade to v1.8.0-rc5 $~~$

1. After adding a dummy annotation to the VM's virt-launcher
2. The upgrade can resume to complete

[TachunLin]

Post upgrade check

${\textsf{✅}}$ Dependency check $~~$

Chart: https://github.com/harvester/harvester/tree/v1.8.0-rc5/deploy/charts/harvester/dependency_charts
Values: https://github.com/harvester/harvester/blob/v1.8.0-rc5/deploy/charts/harvester/values.yaml

• csi-snapshotter

  $\color{green}{\textsf{PASS}}$ $~~$

  • tag: 8.5.0
    https://github.com/harvester/harvester/blob/v1.8.0-rc5/deploy/charts/harvester/dependency_charts/csi-snapshotter/values.yaml
  • Harvester: csi-snapshotter

  

• cdi

  $\color{green}{\textsf{PASS}}$ $~~$

  • tag: 11.62.0-150700.9.3.1
    https://github.com/harvester/harvester/blob/v1.8.0-rc5/deploy/charts/harvester/values.yaml
  • Harvester: cdi

  

• kubevirt-operator

  $\color{green}{\textsf{PASS}}$ $~~$

  • tag: 1.7.0-150700.3.16.2
    https://github.com/harvester/harvester/blob/v1.8.0-rc5/deploy/charts/harvester/values.yaml
  • Harvester: virt-operator (deployment)

  

• kubevirt

  $\color{green}{\textsf{PASS}}$ $~~$

  • tag: 1.7.0-150700.3.16.2
    https://github.com/harvester/harvester/blob/v1.8.0-rc5/deploy/charts/harvester/values.yaml
  • Harvester: kubevirt (crd)

  

• whereabouts

  $\color{green}{\textsf{PASS}}$ $~~$

  • tag: v0.9.3
    https://github.com/harvester/harvester/blob/v1.8.0-rc5/deploy/charts/harvester/values.yaml
  • Harvester: harvester-whereabouts- (pod)

  

${\textsf{✅}}$ Versions of all the crds $~~$

https://github.com/harvester/harvester/tree/v1.8.0-rc5/deploy/charts/harvester-crd/templates

• addons.harvesterhci.io

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_addons.yaml

  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_addons.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  

• keypairs.harvesterhci.io

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_keypairs.yaml

  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_keypairs.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  

• harvesterhci.io_preferences

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_preferences.yaml

  ```
  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_preferences.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  ```
  

• harvesterhci.io_settings

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_settings.yaml

  ```
  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_settings.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  ```
  

• harvesterhci.io_supportbundles

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_supportbundles.yaml

  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_supportbundles.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  

• harvesterhci.io_upgradelogs

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_upgradelogs.yaml

  ```
  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_upgradelogs.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  ```
  

• harvesterhci.io_upgrades

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_upgrades.yaml

  ```
  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_upgrades.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  ```
  

• harvesterhci.io_versions

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_versions.yaml

  ```
  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_versions.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  ```
  

• harvesterhci.io_virtualmachinebackups

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachinebackups.yaml

  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachinebackups.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  

• harvesterhci.io_virtualmachineimages

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachineimages.yaml

  ```
  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachineimages.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  ```
  

• harvesterhci.io_virtualmachinerestores

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachinerestores.yaml

  ```
  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachinerestores.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  ```
  

• harvesterhci.io_virtualmachinetemplates

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachinetemplates.yaml

  ```
  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachinetemplates.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  ```
  

• harvesterhci.io_virtualmachinetemplateversions

  $\color{green}{\textsf{PASS}}$ $~~$

  https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachinetemplateversions.yaml

  ```
  n1-hp6:~ # ./check-harvester-crd.sh https://raw.githubusercontent.com/harvester/harvester/v1.8.0-rc5/deploy/charts/harvester-crd/templates/harvesterhci.io_virtualmachinetemplateversions.yaml
  Observation: v1beta1 
  Expectation: v1beta1 
  
  True
  ```
  

${\textsf{✅}}$ Virtual machines are shutdown after upgrade complete $~~$

${\textsf{✅}}$ Can start all the Off vm into their original state before upgrade $~~$

${\textsf{✅}}$ Monitoring chart status $~~$

${\textsf{✅}}$ Check Addons state $~~$

${\textsf{✅}}$ VM operations are highlighted and working fine. $~~$

• Create vm
• Stop vm
• Start vm
• Restart vm
• Clone vm
• Migrate vm
• Create vm backup
• Create vm snapshot
• View logs

${\textsf{✅}}$ Verify nginx deployment and load balancer service on RKE2 cluster $~~$

${\textsf{✅}}$ Scale up the guest cluster $~~$

${\textsf{✅}}$ Upgrade csi-drive rto 0.1.28 $~~$

```
root@rke2-v1345-pool1-jclpl-65mld:~# export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
root@rke2-v1345-pool1-jclpl-65mld:~# helm upgrade harvester-csi-driver -n kube-system ./harvester-csi-driver-0.1.28.tgz
Release "harvester-csi-driver" has been upgraded. Happy Helming!
NAME: harvester-csi-driver
LAST DEPLOYED: Tue Apr  7 16:01:38 2026
NAMESPACE: kube-system
STATUS: deployed
REVISION: 3
TEST SUITE: None
NOTES:
Successfully deployed Harvester CSI driver to the kube-system namespace.
root@rke2-v1345-pool1-jclpl-65mld:~# helm history harvester-csi-driver -n kube-system
REVISION	UPDATED                 	STATUS    	CHART                        	APP VERSION	DESCRIPTION     
1       	Mon Apr  6 08:26:52 2026	superseded	harvester-csi-driver-0.1.2500	v0.2.4     	Install complete
2       	Mon Apr  6 09:20:00 2026	superseded	harvester-csi-driver-0.1.2500	v0.2.4     	Upgrade complete
3       	Tue Apr  7 16:01:38 2026	deployed  	harvester-csi-driver-0.1.28  	v0.2.8     	Upgrade complete

```

${\textsf{✅}}$ Create pvc use csi-driver 0.1.28 $~~$

${\textsf{✅}}$ Create another one node RKE2 guest cluster. $~~$

[TachunLin]

Retest Three node upgrade w/ 2 MGMT/Default Nodes & 1 Witness Node

Objective

1. Ensure can successfully upgrade to v1.8.0-rc6 without stuck in any stage
2. Ensure all vm migration works during the upgrade
3. Ensure can migrate vm after upgrade complete
4. Verify issue [BUG] Failed to upgrade from v1.7.1 to v1.8.0-rc5, stuck in third node Pre-draining state to wait vm migration start harvester#10349 (comment)

Test environment

• Harvester: v1.7.1 upgrade to v1.8.0-rc6

• Rancher: v2.13.3

• OS:

  • Ubuntu 22.04
  • Ubuntu 24.04

• cloud provider

  • 0.2.1000

• csi-driver

  • 0.1.2500

• RKE2: v1.34.5 to v1.35

Prerequisite and dependency setup

${\textsf{✅}}$ Three nodes cluster with two management role and one witness role $~~$

n1-hp6:~ # kubectl get nodes
NAME      STATUS   ROLES                AGE   VERSION
n1-hp6    Ready    control-plane,etcd   28h   v1.34.3+rke2r3
n2-hp10   Ready    etcd                 22h   v1.34.3+rke2r3
n3-hp80   Ready    control-plane,etcd   23h   v1.34.3+rke2r3

${\textsf{✅}}$ Create a new storage class w/ replica 2 and set as default $~~$

${\textsf{✅}}$ VLAN 2012 network on mgmt and vlan 2017 network on other NICs $~~$

${\textsf{✅}}$ 2 Virtual machines with data and md5sum computed - 2 running $~~$

ubuntu@ubuntu2204-running:~$ sha512sum 2204-running.txt
b4f5d2fdb3262d209c01369297aef74931c4f12d034f139d096f1adeae99e14517c06373c80a7f50db8f256f6010f291794c58d1241afbc3b4c9cf3d45fe2085  2204-running.txt

ubuntu@ubuntu2404-running:~$ sha512sum 2404-running.txt
2c1019832e5e76d2c78dc89d1f09b267bf086385edbaffa59f50f912e038e92793d494f05c3e3f90a52555b1dd24b8cc1dd0a790f6833174fd377c0e53cfb84e  2404-running.txt

${\textsf{✅}}$ 2 VM backups and snapshots $~~$

Test step on Rancher v2.13.3 with Harvester v1.7.1

${\textsf{✅}}$ On Rancher v2.13.3 install UI extension v1.7.1 $~~$

${\textsf{✅}}$ Import Harvester to Rancher $~~$

${\textsf{✅}}$ Enabled all Harvester addons $~~$

${\textsf{✅}}$ Create v1.34.5 RKE2 cluster with ubuntu24.04 $~~$

${\textsf{✅}}$ Check cloud-provider: 0.2.1000 and csi-driver: 0.1.2500 $~~$

${\textsf{✅}}$ Create harvester pvc and attach to nginx deployment $~~$

${\textsf{✅}}$ Create load balancer service and bind to nginx deployment $~~$

Upgrade to Rancher v2.14.0 with Harvester v1.7.1

${\textsf{✅}}$ Upgrade Rancher from v2.13.3 to v2.14.0 $~~$

${\textsf{✅}}$ Check cloud-provider and csi-driver status on rke2-v1345 RKE2 cluster $~~$

${\textsf{✅}}$ Upgrade `rke2-v1329` cluster from v1.34.5 to v1.35.2 $~~$

[TachunLin]

Prerequisite setup test for issue #10349

${\textsf{✅}}$ Migrate vm "ubuntu2404-running" from "n3-hp80" to "n1-hp6" $~~$

1. Before the upgrade vm ubuntu2404-running is on n3-hp80

n1-hp6:~ # kubectl get vmi -A -o wide
NAMESPACE   NAME                                 AGE   PHASE     IP              NODENAME   READY   LIVE-MIGRATABLE   PAUSED
default     rke2-v1346-u2404-pool1-mfg4w-95zvk   56m   Running   10.115.51.235   n3-hp80    True    True              
default     ubuntu2204-running                   86m   Running   10.115.49.38    n3-hp80    True    True              
default     ubuntu2404-running                   84m   Running   10.115.12.178   n3-hp80    True    True

2. Migrate vm from n3-hp80 to n1-hp6

NAMESPACE   NAME                                 AGE   PHASE     IP              NODENAME   READY   LIVE-MIGRATABLE   PAUSED
default     rke2-v1346-u2404-pool1-mfg4w-95zvk   58m   Running   10.115.51.235   n3-hp80    True    True              
default     ubuntu2204-running                   88m   Running   10.115.49.38    n3-hp80    True    True              
default     ubuntu2404-running                   86m   Running   10.115.12.178   n1-hp6     True    True

${\textsf{✅}}$ Set eviction on "n1-hp6" to trigger vm migration back to "n3-hp80" $~~$

1. Before the upgrade vm ubuntu2404-running is on n1-hp6

NAMESPACE   NAME                                 AGE   PHASE     IP              NODENAME   READY   LIVE-MIGRATABLE   PAUSED
default     rke2-v1346-u2404-pool1-mfg4w-95zvk   58m   Running   10.115.51.235   n3-hp80    True    True              
default     ubuntu2204-running                   88m   Running   10.115.49.38    n3-hp80    True    True              
default     ubuntu2404-running                   86m   Running   10.115.12.178   n1-hp6     True    True

2. Set eviction on n1-hp6

n1-hp6:~ # kubectl get nodes
NAME      STATUS   ROLES                AGE   VERSION
n1-hp6    Ready    control-plane,etcd   30h   v1.34.3+rke2r3
n2-hp10   Ready    etcd                 23h   v1.34.3+rke2r3
n3-hp80   Ready    control-plane,etcd   24h   v1.34.3+rke2r3

kubectl taint node n1-hp6 --overwrite kubevirt.io/drain=draining:NoSchedule

n1-hp6:~ # kubectl taint node n1-hp6 --overwrite kubevirt.io/drain=draining:NoSchedule
node/n1-hp6 modified

3. Ensure the vm ubuntu2404-running is migrating back to n3-hp80

n1-hp6:~ # kubectl get vmi -A -o wide
NAMESPACE   NAME                                 AGE   PHASE     IP              NODENAME   READY   LIVE-MIGRATABLE   PAUSED
default     rke2-v1346-u2404-pool1-mfg4w-95zvk   64m   Running   10.115.51.235   n3-hp80    True    True              
default     ubuntu2204-running                   94m   Running   10.115.49.38    n3-hp80    True    True              
default     ubuntu2404-running                   92m   Running   10.115.12.178   n3-hp80    True    True 

NAMESPACE   NAME                        PHASE       VMI
default     kubevirt-evacuation-7nrts   Succeeded   ubuntu2404-running
default     ubuntu2404-running-fn8zv    Succeeded   ubuntu2404-running

${\textsf{✅}}$ Migrate vm "ubuntu2204-running" from "n3-hp80" to "n1-hp6" $~~$

1. Before the upgrade vm ubuntu2204-running is on n3-hp80

n1-hp6:~ # kubectl get vmi -A -o wide
NAMESPACE   NAME                                 AGE    PHASE     IP              NODENAME   READY   LIVE-MIGRATABLE   PAUSED
default     rke2-v1346-u2404-pool1-mfg4w-95zvk   102m   Running   10.115.51.235   n3-hp80    True    True              
default     ubuntu2204-running                   132m   Running   10.115.49.38    n3-hp80    True    True              
default     ubuntu2404-running                   130m   Running   10.115.12.178   n3-hp80    True    True

2. Migrate vm from n3-hp80 to n1-hp6

n1-hp6:~ # kubectl get vmi -A -o wide
NAMESPACE   NAME                                 AGE    PHASE     IP              NODENAME   READY   LIVE-MIGRATABLE   PAUSED
default     rke2-v1346-u2404-pool1-mfg4w-95zvk   103m   Running   10.115.51.235   n3-hp80    True    True              
default     ubuntu2204-running                   133m   Running   10.115.49.38    n1-hp6     True    True              
default     ubuntu2404-running                   131m   Running   10.115.12.178   n3-hp80    True    True

${\textsf{✅}}$ Check node taint, vmim and vm state before upgrade $~~$

n1-hp6:~ # kubectl get nodes -o custom-columns=NAME:.metadata.name,UNSCHEDULABLE:.spec.unschedulable,TAINTS:.spec.taints
NAME      UNSCHEDULABLE   TAINTS
n1-hp6    <none>          <none>
n2-hp10   <none>          [map[effect:NoExecute key:node-role.kubernetes.io/etcd value:true]]
n3-hp80   <none>          <none>

n1-hp6:~ # kubectl get vmim -A
NAMESPACE   NAME                        PHASE       VMI
default     kubevirt-evacuation-7nrts   Succeeded   ubuntu2404-running
default     ubuntu2204-running-gbbv6    Succeeded   ubuntu2204-running
default     ubuntu2404-running-fn8zv    Succeeded   ubuntu2404-running

n1-hp6:~ # kubectl get vmi -A -o wide
NAMESPACE   NAME                                 AGE    PHASE     IP              NODENAME   READY   LIVE-MIGRATABLE   PAUSED
default     rke2-v1346-u2404-pool1-mfg4w-95zvk   104m   Running   10.115.51.235   n3-hp80    True    True              
default     ubuntu2204-running                   134m   Running   10.115.49.38    n1-hp6     True    True              
default     ubuntu2404-running                   132m   Running   10.115.12.178   n3-hp80    True    True

n1-hp6:~ # kubectl get nodes
NAME      STATUS   ROLES                AGE   VERSION
n1-hp6    Ready    control-plane,etcd   30h   v1.34.3+rke2r3
n2-hp10   Ready    etcd                 24h   v1.34.3+rke2r3
n3-hp80   Ready    control-plane,etcd   25h   v1.34.3+rke2r3

Before upgrade observation

• ubuntu2404-running was manually placed on n1-hp6.

• Applying

  kubectl taint node n1-hp6 --overwrite kubevirt.io/drain=draining:NoSchedule
  

  successfully triggered KubeVirt evacuation / live migration.

• ubuntu2404-running was migrated from n1-hp6 to n3-hp80.

• Related VirtualMachineInstanceMigration objects reached Succeeded.

• After migration completed, n1-hp6 showed:

  • no remaining taints
  • Unschedulable not set
  • node status remained Ready

• Therefore, the eviction test did not leave residual node state that would affect the Harvester upgrade.

[TachunLin]

Upgrading Harvester from v1.7.1 to v1.8.0-rc6

${\textsf{✅}}$ VM, Volume and RKE2 cluster state before upgrade $~~$

${\textsf{✅}}$ Trigger upgrade Harvester to v1.8.0-rc6 $~~$

${\textsf{✅}}$ Trigger AutoConverge migration strategy $~~$

n1-hp6:~ # kubectl edit kubevirt -n harvester-system
kubevirt.kubevirt.io/kubevirt edited

      configuration:
        migrations:
          allowAutoConverge: true

${\textsf{✅}}$ Ensure no vm migration trigger during upgrading system service stage $~~$

vokoscreenNG-2026-04-19_01-37-40.mp4

${\textsf{✅}}$ Ensure vm can correctly migrated while in n1-hp6 node upgrading $~~$

vokoscreenNG-2026-04-19_01-39-57.mp4

${\textsf{✅}}$ Ensure vm can correctly migrated while in n3-hp80 (last node) node upgrading $~~$

vokoscreenNG-2026-04-19_02-03-07.mp4

${\textsf{✅}}$ Can successfully upgrade to v1.8.0-rc6 $~~$

${\textsf{✅}}$ All VMs are migrated in Running state $~~$

${\textsf{❌}}$ Migrate vm "ubuntu2204-running" to node "n3-hp80" $~~$

1. Failed to migrate vm ubuntu2204-running to other node
2. It shows error message KubeVirt is not ready

vokoscreenNG-2026-04-19_02-27-14.mp4

• Support bundle

  supportbundle_01d4e04c-8a34-4e49-bbcb-d68c86ced91a_2026-04-18T18-30-17Z.zip

• Upgrade log

  hvst-upgrade-mg2mv-upgradelog-archive-2026-04-18T18-33-57Z.zip