OpenNebula 5.0 Beta - marketplaces

Edouard,

Can you check that there is something like:

MARKET_MAD_CONF=[
APP_ACTIONS=“monitor”,
NAME=“one”,
PUBLIC=“yes”,
SUNSTONE_NAME=“OpenNebula.org Marketplace”
]

in oned.conf

Hi Hans

We’ve taken a look to the code, and another source code change is needed to
support proxies. Sorry for that. We’ll try to priorize this issue, so an
upgrade is ready…

Cheers

Hi Ruben,

Thanks for your help. yes I have this MARKET_MAD_CONF, for “one”, “http” and “s3”.

`MARKET_MAD_CONF = [
NAME = “one”,
SUNSTONE_NAME = “OpenNebula.org Marketplace”,
REQUIRED_ATTRS = “”,
APP_ACTIONS = “monitor”,
PUBLIC = “yes”
]

MARKET_MAD_CONF = [
NAME = “http”,
SUNSTONE_NAME = “HTTP server”,
REQUIRED_ATTRS = “BASE_URL,PUBLIC_DIR”,
APP_ACTIONS = “create, delete, monitor”
]

MARKET_MAD_CONF = [
NAME = “s3”,
SUNSTONE_NAME = “Amazon S3”,
REQUIRED_ATTRS = “ACCESS_KEY_ID,SECRET_ACCESS_KEY,REGION,BUCKET”,
APP_ACTIONS = “create, delete, monitor”
]`

Well, it says that monitor is not disabled as in oned.log file. Try to
restart one, check the very first line of oned.log to check that you are
reading the proper configuration…

I don’t see any indication in oned.log about the configuration file the daemon is actually reading. But I do see the MARKET_MAD_CONF (as seen above), and no more “Monitoring disabled for market: OpenNebula Public”.

But onemarketapp list is still empty. What’s the command to refresh the market catalog ?

Are you seeing Marketplace successfully monitored? Probably the easiest
thing is to delete the marketplace and create a new one through sunstone.
Pick the “OpenNebula Public…” driver and leave the URL empty.

In the log I can only see that the MarketPlace Manager is started and that the MarketPlace Manager Driver is loaded. But nothing about any successful monitoring about it.

Will try to delete/recreate through sunstone as soon as I have access to sunstone (I only have SSH access for the moment).

Recreating the Marketplace did the trick. So the only difference I can spot except the ID, is the ZONE_ID which is now 0 instead of -1.

<MARKETPLACE> <ID>100</ID> <UID>0</UID> <GID>0</GID> <UNAME>oneadmin</UNAME> <GNAME>oneadmin</GNAME> <NAME>OpenNebula MarketPlace</NAME> <MARKET_MAD><![CDATA[one]]></MARKET_MAD> <ZONE_ID><![CDATA[0]]></ZONE_ID> <TOTAL_MB>0</TOTAL_MB> <FREE_MB>0</FREE_MB> <USED_MB>0</USED_MB> <MARKETPLACEAPPS/> <PERMISSIONS> <OWNER_U>1</OWNER_U> <OWNER_M>1</OWNER_M> <OWNER_A>0</OWNER_A> <GROUP_U>0</GROUP_U> <GROUP_M>0</GROUP_M> <GROUP_A>0</GROUP_A> <OTHER_U>0</OTHER_U> <OTHER_M>0</OTHER_M> <OTHER_A>0</OTHER_A> </PERMISSIONS> <TEMPLATE> <MARKET_MAD><![CDATA[one]]></MARKET_MAD> </TEMPLATE> </MARKETPLACE>

Should I report this bug ?

Thanks for the feedback! We’ve already fill one issue:

http://dev.opennebula.org/issues/4611

Now that the market is working, I’d like to set it behind a proxy. So in /etc/one/sunstone-server.conf I add a line:
:proxy: http://proxy.local:8080

Then I restart service opennebula and opennebula-sunstone. It does’nt work I can see the apps in the marketplace. Here what I see in the logs:

Mon Jul 11 18:18:46 2016 [Z0][MKP][D]: Monitoring marketplace OpenNebula Public Market (101) Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: Command execution fail: /var/lib/one/remotes/market/one/monitor P[..]QT4= 101 Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: /usr/share/ruby/net/http.rb:878:ininitialize’: getaddrinfo: Name or service not known (SocketError)
Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /usr/share/ruby/net/http.rb:878:in open' Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /usr/share/ruby/net/http.rb:878:inblock in connect’
Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /usr/share/ruby/timeout.rb:52:in timeout' Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /usr/share/ruby/net/http.rb:877:inconnect’
Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /usr/share/ruby/net/http.rb:862:in do_start' Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /usr/share/ruby/net/http.rb:851:instart’
Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /usr/share/ruby/net/http.rb:582:in start' Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /var/lib/one/remotes/market/one/monitor:41:inget’
Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /var/lib/one/remotes/market/one/monitor:53:in get_appliances' Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: from /var/lib/one/remotes/market/one/monitor:157:in'
Mon Jul 11 18:18:47 2016 [Z0][MKP][I]: ExitCode: 1
Mon Jul 11 18:18:47 2016 [Z0][MKP][E]: Error monitoring datastore 101: -
`

Not sure if 101 is a datastore. But it looks like http.rb is trying to resolve an hostname without asking the proxy to do so. Is there anything else to do to configure a proxy ?

Hi Edouard,

HTTP proxies are not yet supported in the new Marketplace, in OpenNebula
5.0. The issue for this feature:

http://dev.opennebula.org/issues/4388

Sorry,

Ruben

Thanks Ruben. I’m following this issue.

I just add my proxy and port

response = Net::HTTP.start(uri.hostname, uri.port, 'proxy.local', 8080) {|http|
    http.request(req)
}

in /var/lib/one/remotes/market/one/monitor and know I don’t have anymore errors about the monitoring of the marketplace. And I can browse the marketplace. Just have to find where the download part is handled since it’s the only thing that still doesn’t work.

Download to the web client (e.g. laptop…)? or to a Datastore?

to the datastore. Is it the downloader.sh ?

yes, eventually it calls downloader.sh

Thank you. At least we have a workaround and can keep OpenNebula 5.

Thanks! Works perfect!

Hello,

This issue is marked as fixed in 5.2, but I cannot figure out how to set up the proxy location in 5.2. Adding --proxy http://my.proxy:3128/ to GLOBAL_CURL_ARGS in downloader.sh and also as additional parameters to Net::HTTP.start call in market/one/monitor works for me, but is there a cleaner way?

Thanks,

-Yenya

1 Like

It should honor http_proxy environment variable, you can add the export in
the one_market executable…

Let me know if this work for you to update the documentation

@ruben, I have tried to add “touch /tmp/been-there” near the beginning of /usr/lib/one/mads/one_market, but the file in /tmp did not get created even though I tried to download images from ONe marketplace, reload the marketplace, etc. When does this command gets executed?

Anyway, the one_market script apparently reads madcommon.sh, and in this file there is a notice that /etc/one/defaultrc is sourced from there and variables exported. Wouldn’t it be cleaner to add HTTP_PROXY=... to /etc/one/defaultrc? I would rather not add my local configuration to the non-config file in /usr/lib/one.

The cleanest approach in my humble opinion would be to have a HTTP_PROXY attribute settable via Sunstone for a given marketplace, and extract this value from there.