Attach/Detach CDROM not supported


(@LINFORGE @LINBIT) #1

Hi,
I am experimenting with attaching and detaching of CDROM images in running VMs.

fails like this:

Mon Feb 6 11:39:21 2017 [Z0][VMM][D]: Message received: LOG I 69 Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/detach_disk ‘one-69’ ‘/var/lib/one//datastore /0/69/disk.0’ ‘hda’ ‘0’ 69 s001310a
Mon Feb 6 11:39:21 2017 [Z0][VMM][D]: Message received: LOG E 69 detach_disk: Command “virsh --connect qemu:///system detach-disk --domain one-69 --target hda” failed: error: Failed to detach disk
Mon Feb 6 11:39:21 2017 [Z0][VMM][D]: Message received: LOG I 69 error: Operation not supported: disk device type ‘cdrom’ cannot be detached
Mon Feb 6 11:39:21 2017 [Z0][VMM][D]: Message received: LOG E 69 Could not detach hda from one-69
Mon Feb 6 11:39:21 2017 [Z0][VMM][D]: Message received: LOG I 69 ExitCode: 1
Mon Feb 6 11:39:21 2017 [Z0][VMM][D]: Message received: LOG I 69 Failed to execute virtualization driver operation: detach_disk.
Mon Feb 6 11:39:21 2017 [Z0][VMM][D]: Message received: DETACHDISK FAILURE 69 Could not detach hda from one-69

also _a_ttaching a cdrom to running vm fails like this:

Mon Feb  6 12:02:55 2017 [Z0][VMM][D]: Message received: LOG E 69 attach_disk: Command "virsh --connect qemu:///system attach-device one-69 /var/lib/one//datastores/0/69/disk.3.attach" failed: error: Failed to attach device from /var/lib/one//datastores/0/69/disk.3.attach
Mon Feb  6 12:02:55 2017 [Z0][VMM][D]: Message received: LOG I 69 error: internal error: No device with bus 'ide' and target 'hda'. cdrom and floppy device hotplug isn't supported by libvirt
Mon Feb  6 12:02:55 2017 [Z0][VMM][D]: Message received: LOG E 69 Could not attach /var/lib/one//datastores/0/69/disk.3 (hda) to one-69

looks like it is libvirt’s fault.

ok, then for both, attaching and detaching of cdrom type images, is it just not supported by libvirt? or is there a way to achieve this with OpenNebula?

what i think is also odd is that for libvirt it doesn’t matter if you set the cdrom img to persistent, it still got the readonly tag:

<disk type='file' device='cdrom'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/var/lib/one//datastores/0/69/disk.3'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
  </disk>

so it is actually useless that opennebula supports having persistent and non persistent cdrom types.

and actually i dont really see a use case where you would need a cdrom of type persistent, you cant write into a cdroms filesystem anyway. maybe i am missing something here.

I know this issue has been discussed on this forum quite often already. I just want to set this straight: does libvirt really NOT support hotplug for cdroms?

thanks for your thoughts.

Jojo


(C. R. Oldham) #2

I think it’s more of an issue with the bus type. IDE has never supported hotplugging devices, and the CDRoms default to IDE (hda/hdb/etc). I have had some success with cdrom images on the virtio and SCSI buses, though not 100%. I have yet to understand the intricacies of why it works sometimes and not others.


(Javi Fontán) #3

I think you will understand better if you read “Attach Drive” instead of “Attach Disk”. As @cro says the problem here is that you are attaching a cdrom drive in IDE bus and is not supported.You should read

Currently OpenNebula does not support changing media of CDRoms, that I think it’s what you are looking for:

# virsh change-media --help
  NAME
    change-media - Change media of CD or floppy drive

  SYNOPSIS
    change-media <domain> <path> [--source <string>] [--eject] [--insert] [--update] [--current] [--live] [--config] [--force] [--print-xml] [--block]

  DESCRIPTION
    Change media of CD or floppy drive.

  OPTIONS
    [--domain] <string>  domain name, id or uuid
    [--path] <string>  Fully-qualified path or target of disk device
    --source <string>  source of the media
    --eject          Eject the media
    --insert         Insert the media
    --update         Update the media
    --current        can be either or both of --live and --config, depends on implementation hypervisor driver
    --live           alter live configuration of running domain
    --config         alter persistent configuration, effect observed on next boot
    --force          force media changing
    --print-xml      print XML document rather than change media
    --block          source media is a block device

There is a prefeconfigure action in KVM that uses change-media but it is only used to change the context after a new NIC is attached:

You are right that persistent CDROM makes no sense.


(@LINFORGE @LINBIT) #4

Hi C.R. and Javi,
thanks for your replies, appreciated :slight_smile:
Ok, now that you mention it, when I think about how many times I properly hotplugged a IDE CDROM in hardware back in the day…no that was not very often :wink:

I tried to change my oned.conf to have scsi drives (sd) as a default for cdroms. From this message it seems to me that it doesn’t actually mean the scsi bus, but the device type CDROM that is not supported for detaching:

Tue Feb 7 17:04:14 2017 [Z0][VMM][D]: Message received: LOG I 156 Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/detach_disk ‘one-156’ ‘/var/lib/one//datastores/0/156/disk.0’ ‘sda’ ‘0’ 156 s001310a
Tue Feb 7 17:04:14 2017 [Z0][VMM][D]: Message received: LOG E 156 detach_disk: Command “virsh --connect qemu:///system detach-disk --domain one-156 --target sda” failed: error: Failed to detach disk
Tue Feb 7 17:04:14 2017 [Z0][VMM][D]: Message received: LOG I 156 error: Operation not supported: disk device type ‘cdrom’ cannot be detached
Tue Feb 7 17:04:14 2017 [Z0][VMM][D]: Message received: LOG E 156 Could not detach sda from one-156
Tue Feb 7 17:04:14 2017 [Z0][VMM][D]: Message received: LOG I 156 ExitCode: 1
Tue Feb 7 17:04:14 2017 [Z0][VMM][D]: Message received: LOG I 156 Failed to execute virtualization driver operation: detach_disk.

I am thinking about extending the script /var/lib/one/remotes/vmm/kvm/detach_disk, so it does the change-media command when it is definitely a CDROM device that is to be detached.

but does this script actually “know” that it is a cdrom?

looks like it’s getting this variables:
DOMAIN="$1"
SOURCE="$2"
TARGET="$3"
TARGET_INDEX="$4"

TARGET tells me only that it is a hd*, vd* or hd*, which does not say anything about the image-type. what is TARGET_INDEX Javi?

thanks a lot!!!
best Jojo


(Anton Todorov) #5

Hi all

The script is run on the HV so you can query the VM XML from libvirt - virsh dumpxml $DOMAIN where you have:

<disk type='file' device='cdrom'>
  <driver name='qemu' type='raw'/>
  <source file='/var/lib/one//datastores/101/60/disk.4'/>
  <backingStore/>
  <target dev='hda' bus='ide'/>
  <readonly/>
  <alias name='ide0-0-0'/>
  <address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>

You just need to parse the VM XML :slight_smile:

Kind Regards,
Anton Todorov


(Javi Fontán) #6

I’ve been digging a bit and TARGET_INDEX is the position of the device in the bus:

target_index = target.downcase[-1..-1].unpack('c').first - 97

That is, the last letter from the target is converted this way:

sda -> 0
sdb -> 1
sdc -> 2
...

I think it was used by the old ESX driver to attach the disks to the correct position but I think is not used anymore by any driver.


(kvaps) #7

Hi @atodorov_storpool, this is my dirty solution for that:

--- a/detach_disk
+++ b/detach_disk
@@ -26,14 +26,18 @@ TARGET_INDEX="$4"
 
 DETACH_PARAMS="--domain $DOMAIN --target $TARGET"
 
-exec_and_log "virsh --connect $LIBVIRT_URI detach-disk $DETACH_PARAMS" \
-    "Could not detach $TARGET from $DOMAIN"
-
-virsh --connect $LIBVIRT_URI dumpxml $DOMAIN | grep $TARGET > /dev/null 2>&1
-
-if [ $? -eq 0 ] ; then
-    error_message "Could not detach $TARGET from $DOMAIN"
-    exit -1
-fi
-
-exit 0
+DETACH_CMD=$(cat <<EOF
+
+CDROM_TARGET="\$(virsh --connect $LIBVIRT_URI dumpxml --domain $DOMAIN | \
+    xmllint --xpath "string(/domain/devices/disk[target/@dev='${TARGET}'][@device='cdrom']/source/@file)" -)"
+
+if [ -z "\$CDROM_TARGET" ]
+then virsh --connect $LIBVIRT_URI detach-disk $DETACH_PARAMS
+else virsh --connect $LIBVIRT_URI change-media --domain $DOMAIN --eject "\$CDROM_TARGET" --force
+fi
+
+EOF
+)
+
+multiline_exec_and_log "$DETACH_CMD" \
+    "Could not detach $TARGET from $DOMAIN"

(Ruben S. Montero) #8

Just cross-linking to the development issue for this feature

https://github.com/OpenNebula/one/issues/805


(kvaps) #9

New patch:

removed
--- a/detach_disk
+++ b/detach_disk
@@ -26,8 +26,25 @@
 
 DETACH_PARAMS="--domain $DOMAIN --target $TARGET"
 
-exec_and_log "virsh --connect $LIBVIRT_URI detach-disk $DETACH_PARAMS" \
-    "Could not detach $TARGET from $DOMAIN"
+DETACH_CMD=$(cat <<EOF
+
+DISK_TYPE="\$(virsh --connect $LIBVIRT_URI dumpxml --domain $DOMAIN | \
+    xmllint --xpath "string(/domain/devices/disk[target/@dev='${TARGET}']/@device)" -)"
+
+if [ "\$DISK_TYPE" = "cdrom" ]
+then
+    DISK_SOURCE="\$(virsh --connect $LIBVIRT_URI dumpxml --domain $DOMAIN | \
+        xmllint --xpath "string(/domain/devices/disk[target/@dev='${TARGET}']/source/@file)" -)"
+    [ -z "\$DISK_SOURCE" ] || virsh --connect $LIBVIRT_URI change-media --domain $DOMAIN --eject "\$DISK_SOURCE" --force
+else
+    virsh --connect $LIBVIRT_URI detach-disk $DETACH_PARAMS
+fi
+
+EOF
+)
+
+multiline_exec_and_log "$DETACH_CMD" \
+    "Could not detach $TARGET from $DOMAIN"
 
 virsh --connect $LIBVIRT_URI dumpxml $DOMAIN | grep $TARGET > /dev/null 2>&1

I’ve got back the check for is disk was detached or not.


(Anton Todorov) #10

You should escape $DISK_SOURCE otherwise it will be always emty string. :wink:

But in case of cdrom it will remove the source entry and the check for $TARGET later in the code will detect the device and a failure will be reported to OpenNebula.

BR,
Anton


(kvaps) #11

Hi Andton, yes you true, I’m not sure that this check is really needed.
So for now I think better to continue use the previus patch.
Thanks!


(kvaps) #12

New patch which is not requires xmllint installed.

--- a/detach_disk
+++ b/detach_disk
@@ -26,14 +26,20 @@ TARGET_INDEX="$4"
 
+STAGING_DIR=/var/tmp
 DETACH_PARAMS="--domain $DOMAIN --target $TARGET"
+DETACH_PARAMS_CDROM="--domain $DOMAIN $TARGET --eject --force"
 
-exec_and_log "virsh --connect $LIBVIRT_URI detach-disk $DETACH_PARAMS" \
-    "Could not detach $TARGET from $DOMAIN"
-
-virsh --connect $LIBVIRT_URI dumpxml $DOMAIN | grep $TARGET > /dev/null 2>&1
-
-if [ $? -eq 0 ] ; then
-    error_message "Could not detach $TARGET from $DOMAIN"
-    exit -1
-fi
-
-exit 0
+DETACH_CMD=$(cat <<EOF
+
+TARGET_TYPE="\$(virsh --connect $LIBVIRT_URI domblklist --domain $DOMAIN --details | awk '\$3 == "$TARGET" {print \$2}' )"
+
+if [ "\$TARGET_TYPE" == cdrom ]; then
+    virsh --connect $LIBVIRT_URI change-media $DETACH_PARAMS_CDROM
+else
+    virsh --connect $LIBVIRT_URI detach-disk $DETACH_PARAMS
+fi
+
+EOF
+)
+
+multiline_exec_and_log "$DETACH_CMD" \
+    "Could not detach $TARGET from $DOMAIN"

Install it like:

cd /var/lib/one/remotes/vmm/kvm
patch < /tmp/1.patch

Then sync hosts:

sudo -u oneadmin onehost sync -f