Bug with App Market Disk size


(kiran Ranjane) #1

I am using ONE - 5.4.6 Version new install. I created few images in image like windows ubuntu and blank datablock disk the size in image tab shows correct until I export that image to marketapps.

After I export the Image to http marketapp it shows wrong disk size, I have tried this many time and it seems to be reproducible.

All the OS TMPL disk are of 50GB and in marketapps it shows only the size of data it has written to disk for example windows disk is a 50GB disk but shows only 13GB in appmarket which is still fine because it shows 50GB inside the VM, but the main issue comes with blank datablock disk

– Created a disk with below command
qemu-img create -f qcow2 blank-50gb 50G

– uploaded the disk to image using opennebula path
– Then exported to htp marketapp using the marketapp button

Here is the output of onemarketapp:

onemarketapp list
ID NAME VERSION SIZE STAT TYPE REGTIME MARKET ZONE
66 TMPL-Debian-9-Server Server Ver 1.8G rdy img 03/07/18 Synccore-App-Market 0
67 TMPL-Ubuntu-1604-Server Server Ver 3G rdy img 03/07/18 Synccore-App-Market 0
68 TMPL-Xubuntu-1604-Desktop Desktop Ve 7.2G rdy img 03/07/18 Synccore-App-Market 0
69 TMPL-Ubuntu-16.04-Desktop Desktop Ve 7.6G rdy img 03/07/18 Synccore-App-Market 0
70 TMPL-CentOS-6-9-Server Server Ver 1.6G rdy img 03/08/18 Synccore-App-Market 0
71 TMPL-CentOS-7-Server Server Ver 1.6G rdy img 03/08/18 Synccore-App-Market 0
72 TMPL-OPNSense-Firewall Firewall V 1.3G rdy img 03/08/18 Synccore-App-Market 0
73 TMPL-PFSense-Firewall Firewall V 863M rdy img 03/08/18 Synccore-App-Market 0
74 TMPL-Windows-2008R2-Deskt Desktop Ve 7.6G rdy img 03/08/18 Synccore-App-Market 0
75 TMPL-Windows-2012R2-Deskt Desktop Ve 8.9G rdy img 03/08/18 Synccore-App-Market 0
76 TMPL-Windows-2012R2-Serve Server Ver 8.9G rdy img 03/08/18 Synccore-App-Market 0
77 TMPL-Windows-2016R2-Serve Server Ver 12.9G rdy img 03/08/18 Synccore-App-Market 0
78 TMPL-Windows-2008R2-Serve Server Ver 7.9G rdy img 03/10/18 Synccore-App-Market 0
79 TMPL-Windows-2016R2-Deskt Desktop Ve 12.9G rdy img 03/10/18 Synccore-App-Market 0
80 ISO-Gparted-LiveCD ISO Versio 291M rdy img 03/12/18 Synccore-App-Market 0
81 ISO-Synccore-Recovery-Med ISO Versio 266M rdy img 03/12/18 Synccore-App-Market 0
84 ODB-Blank-50GB 0.0 1M rdy img 03/12/18 Synccore-App-Market 0

The last disk id 84 is a blank disk and shows 1 MB and when attached to OS it shows the same size while in qemu command it shows correct size.

qemu-img info 98d1af30b1a0b6a429557a667d90abd0
image: 98d1af30b1a0b6a429557a667d90abd0
file format: qcow2
virtual size: 50G (53687091200 bytes)
disk size: 193K
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false

This is making our users nuts as they do not have rights to create new image but only way is to download a disk from appmarket and use it.

Any help will be appreciated.


(Nicolas Beguier) #2

Hi !

Sometimes this error occured when opennebula cannot move the temporary file in /var/tmp to detect the size.
Are you using a proxy ?
What do you have into /var/log/one/oned.log ?

Nicolas.


(kiran Ranjane) #3

I am using nginx server, and ONE server is on different server and Sunstone is on a separate server. Sunstone and nginx are on same server

While uploading the image to image tab - I get correct size on the image tab :screenshot attached

Wed Mar 14 11:54:02 2018 [Z0][ReM][D]: Req:3472 UID:0 one.image.allocate invoked , “NAME=“blank-2TB”…”, 108
Wed Mar 14 11:54:02 2018 [Z0][ImM][I]: Copying /mnt/storage/blank-2tb to repository for image 89
Wed Mar 14 11:54:02 2018 [Z0][ReM][D]: Req:3472 UID:0 one.image.allocate result SUCCESS, 89
Wed Mar 14 11:54:02 2018 [Z0][ReM][D]: Req:2128 UID:0 one.image.info invoked , 89
Wed Mar 14 11:54:02 2018 [Z0][ReM][D]: Req:2128 UID:0 one.image.info result SUCCESS, “89<U…”
Wed Mar 14 11:54:02 2018 [Z0][ImM][I]: Image (89) copied and ready to use.
Wed Mar 14 11:54:02 2018 [Z0][InM][D]: Monitoring datastore NY-DS-IMAGE-LRS (108)
Wed Mar 14 11:54:02 2018 [Z0][ImM][D]: Datastore NY-DS-IMAGE-LRS (108) successfully monitored.


While exporting the image from image tad to marketapp http - After uploading I see 1MB size on Marketplace : screenshot attached

Wed Mar 14 11:56:59 2018 [Z0][AuM][D]: Message received: AUTHENTICATE SUCCESS 458 -

Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:1888 UID:0 one.marketapp.allocate invoked , “NAME=“blank-2TB”…”, 100
Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:1888 UID:0 one.marketapp.allocate result SUCCESS, 86
Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:6000 UID:0 one.marketapp.info invoked , 86
Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:6000 UID:0 one.marketapp.info result SUCCESS, “…”
Wed Mar 14 11:56:59 2018 [Z0][MKP][I]: Marketplace app successfully imported
Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:9184 UID:0 one.imagepool.info invoked , -2, 0, -200, -1
Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:9184 UID:0 one.imagepool.info result SUCCESS, “<IMAGE_POOL><…”
Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:5072 UID:0 one.marketapppool.info invoked , -2, -1, -1
Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:5072 UID:0 one.marketapppool.info result SUCCESS, “<MARKETPLACEAPP_POOL…”
Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:7920 UID:0 one.user.info invoked , 0
Wed Mar 14 11:56:59 2018 [Z0][ReM][D]: Req:7920 UID:0 one.user.info result SUCCESS, “0<GID…”
Wed Mar 14 11:57:03 2018 [Z0][MKP][D]: Monitoring marketplace Synccore-App-Market (100)
Wed Mar 14 11:57:03 2018 [Z0][MKP][D]: Marketplace Synccore-App-Market (100) successfully monitored.
Wed Mar 14 11:57:06 2018 [Z0][InM][D]: Monitoring datastore files (2)

app-tab

Thanks


(kiran Ranjane) #4

Well it seems that the problem is in the export script - /var/lib/one/remotes/datastore/fs/export

In the script it uses du -Lm which gives the incorrect disk size as the image is thin disk qcow2 and according to that image size the DD command is then executed which creates only a 1MB disk in AppMarket.

for example the size of disk in qemu-info

qemu-img info 3a5b06d543f0e0fda62347b448bd71d7
image: 3a5b06d543f0e0fda62347b448bd71d7
file format: qcow2
virtual size: 150G (161061273600 bytes)
disk size: 195K
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false


And the size in du -Lm

du -Lm 3a5b06d543f0e0fda62347b448bd71d7
1 3a5b06d543f0e0fda62347b448bd71d7

Is there a way to use thin disk in opennebula or workaround for this issue?

Thanks