**SOLVED** Live-migration problem after update to 5.2.1

(Max Rybas) #1

hello! We recently updated OpenNebula from 4.12 to 5.2.1 and now we have problem with live migration (migration works fine) on KVM hosts.
For shared system datastore we use ceph-fs.
When I create VM I can acces following directories and files:

ls -la /var/lib/one//datastores/0/133562/
total 366
drwxrwxr-x 1 oneadmin oneadmin       374121 Feb  6 07:29 .
drwxr-x--- 1 oneadmin oneadmin 472929437079 Feb  6 07:45 ..
-rw-rw-r-- 1 oneadmin oneadmin         1385 Feb  6 07:28 deployment.20
-rw-r--r-- 1 oneadmin oneadmin       372736 Feb  6 07:29 disk.1    

When I’m trying live-migration I have an error:

Mon Feb 6 07:09:44 2017 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/migrate ‘one-133562’ ‘host05’ ‘host04’ 133562 host04
Mon Feb 6 07:09:44 2017 [Z0][VMM][E]: migrate: Command “virsh --connect qemu:///system migrate --live one-133562 qemu+ssh://host05/system” failed: error: Cannot access storage file ‘/var/lib/one//datastores/0/133562/disk.1’ (as uid:1002, gid:1002): No such file or directory
Mon Feb 6 07:09:44 2017 [Z0][VMM][E]: Could not migrate one-133562 to host

And after this error following files are missing:

ls -la /var/lib/one//datastores/0/133562/
ls: cannot access /var/lib/one//datastores/0/133562/: No such file or directory

(Max Rybas) #2

i solved my problem.
I commented line 63 in /var/lib/one/remotes/tm/ceph/premigrate

63 #ssh_exec_and_log "$DST_HOST" "rm -rf '$DST_PATH'" \

$DST_PATH and $SRC_PATH is the same folder that contains context image on shared DS
So, when premigrate script remove$DST_PATH it remove $SRC_PATH

There is a solution

(Jaime Melis) #3

Thanks for reporting this issue (and for the patch!) we’ll review it and apply it shortly

(David Welch) #4

We are also having this issue. Is there any guess on when the fixes will be merged?

(Anton Todorov) #5


According to https://dev.opennebula.org/issues/4969 it is queued for 5.4. But if there is another maintenance release of 5.2 I believe it will be merged there too.

Kind Regards,
Anton Todorov

(David Welch) #6

Thanks, we patched the 3 files as described above and live-migrate started working again after opennebula restart