Flatten snapshot loses data on Ceph


(Jan "Yenya" Kasprzak) #1

Hello, ONe developers,

one of my users lost his data using ONe on top of Ceph, after he used “Flatten snapshot”. It seems that this action reverts to the last snapshot instead of keeping the current data, as we expected. Steps to reproduce:

  • create a Linux VM
  • create an empty datablock image on top of Ceph datastore, make it persistent
  • attach the image to the VM, let’s say as /dev/sdc
  • inside the VM, run # echo before snapshot > /dev/sdc ; sync
  • in Sunstone, open the VM and its Storage tab, click on a camera icon in the /dev/sdc row to create snapshot, name it “snap1”
  • inside the VM, run # echo after snapshot 1 > /dev/sdc ; sync
  • in Sunstone, open the VM and its Storage tab, click on a camera icon in the /dev/sdc row to create snapshot, name it “snap2”
  • inside the VM, run # echo after snapshot 2 > /dev/sdc ; sync
  • in Sunstone, detach the /dev/sdc from the VM
  • in Sunstone, open the Storage/Images list in the left panel (User view), select the image, open its “Snapshots” tab, select the latest possible snapshot (snap2), and click on “flatten”
  • after the image is flattened, attach it to the VM again
  • inside the VM, run # head -n 1 /dev/sdc

Expected behaviour: after snapshot 2
Observed behaviour: after snapshot 1

(in case it is correct, then what is the difference to reverting to the snapshot?)

Anyway, what the user wanted to do was to enlarge the image in question, and Sunstone complained that image with snapshots cannot be resized. At the Ceph level, there is no reason for it, it is possible to resize even the image with snapshots:

# rbd --pool one snap list one-668
SNAPID NAME  SIZE TIMESTAMP                
  1012 0    10GiB Tue Dec 11 11:26:39 2018 
# rbd --pool one resize  one-668 --size 11G
Resizing image: 100% complete...done.
# rbd --pool one info  one-668
rbd image 'one-668':
	size 11GiB in 2816 objects
[...]

Can the limitation of resizing the image with snapshots be removed? And, if possible, the “flatten image” to be modified to keep the latest data?

Thanks,

-Yenya


(Jan "Yenya" Kasprzak) #2

Hello,

did anyone managed to reproduce this? Could it be related to the recently released errata?

http://docs.opennebula.org/5.6/intro_release_notes/release_notes/compatibility.html#snapshots-do-not-reuse-snapshot-ids
https://github.com/OpenNebula/one/issues/2687

Thanks,

-Yenya


(Jan "Yenya" Kasprzak) #3

OK, this is independent on the above mentioned patch. Even with the database patch applied, Flatten image flattens to the last snapshot, not to the last state.