Contribution: OpenNebula backup script for QCOW2 datastores


(Kristian Feldsam) #22

Hello @mptek, I explained nature of non-persistent and persistent images/instances in my prev posts. I already discuss this problem with few my customers. It all depends on usage of opennebula.

If you use it in “cloud” way, them it is like amazon ec2 instances (vm instances), ami images (non-persistent images) and ebs volumes (persistent images). So customer deploys new instances based on ami image (non-persistent image) and configure everything “infrastructure as code” using tools like ansible. That programatic configuration also includes attaching and mounting ebs volume (persistent image) which can be backuped. So in case of crash, they can easily deploys and configure everything by running “program”. Important data will be on ebs volumes, so preserved.

On the other hand, many people use openenbula as “vps” like hosting and deploys non-persistent instances/images. In this case there are two possible solutions (in my subjective opinion).

First one is to make every customer VPS persistent, so clone base image and make it persistent, clone base VM template, map it to that image, configure persistent IP, etc… and deploy vm.

Second one is deploying non persistent instances and do backups of that “non-persistent” images created on background.

Personally I use first approach.


(Ulrich P.) #23

Hi Kristian,

I am currently using ssh transfer datastores on some of my hosts. Images are deployed persistent and with driver=qcow2.
Unfortunately your backup script does not cover this scenario (TM_MAD=ssh, Image persistent, Image “Driver = SSH”).
I believe technically it should work with the same commands from your script. Am I correct? If yes, do you see a chance to implement this?

Thanks
Uli


(Kristian Feldsam) #24

Hello, I did support in develop branch, but not tested. you can try it and let me know what happens


(Ulrich P.) #25

Hi,

the version in the develop branch does not backup the image as I expected. To be more precise about my goal:
I deployed a VM to a KVM node with local storage and a system datastore (TMD_MAD=ssh). I checked the box “persistent” while deploying the VM. Now I would like to backup the image located in the system datastore from the running VM directly from this KVM node.
When I run your script (pointing to the system datastore using a label) it just does nothing.

Best Regards
Uli


(Ulrich P.) #26

Ok,

I did some more testing and can report following results. In my first tests I only included the system ds (ID: 109) on the kvm node in the backup run. In my oppinion this should be have been sufficient because the whole persistent image is located there. The backup.sh just did nothing in this tests.

In my next tests I included only the image in the backup run (based on label, not the complete ds) and I think I found a bug in your script:

  1. Snapshot is created succesfully
  2. rsync tries to copy the image file from datastore id: 110, which is the originating image datastore only existing in the frontend node. It should copy from id: 109, which is the ssh datastore on the kvm_node.

Here is the ouptut (look at the bold part):
[oneadmin@backup addon-image-backup]$ ./backup.sh
Create live snapshot of image 5 named testpersistent-disk-0 attached to VM 16 as disk sda running on kvm01.inf.corp.loc
Run cmd: ssh oneadmin@kvm01.inf.corp.loc ‘touch /var/lib/one/snapshot_image_backup/one-16-weekly-backup’
Run cmd: ssh oneadmin@kvm01.inf.corp.loc ‘virsh -c qemu:///system snapshot-create-as --domain one-16 weekly-backup --diskspec sda,file=/var/lib/one/snapshot_image_backup/one-16-weekly-backup --disk-only --atomic --no-metadata --quiesce’ || ssh oneadmin@kvm01.inf.corp.loc ‘virsh -c qemu:///system snapshot-create-as --domain one-16 weekly-backup --diskspec sda,file=/var/lib/one/snapshot_image_backup/one-16-weekly-backup --disk-only --atomic --no-metadata’
Domain snapshot weekly-backup created
Run cmd: mkdir -p /mnt/image_backup/110/6a32b4fe232caa0b36287e2cbb91cc8e.snap
Run cmd: rsync -aHAXxWv --inplace --numeric-ids --progress -e “ssh -T -o Compression=no -x” oneadmin@kvm01.inf.corp.loc:/var/lib/one//datastores/110/6a32b4fe232caa0b36287e2cbb91cc8e /mnt/image_backup/110/6a32b4fe232caa0b36287e2cbb91cc8e.tmp
receiving incremental file list
rsync: change_dir “/var/lib/one//datastores/110” failed: No such file or directory (2)

sent 8 bytes received 100 bytes 43.20 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)


(Kristian Feldsam) #27

Hello, would you print here your datastores? If I understand correcttly, them persistent image is created in system? datastore on specific compute node? or it is image datastore?