Network contextualisation in lxd containers

Some lxd containers don’t initialize networking or host name.

Versions of the related components and OS (frontend, hypervisors, VMs):
One 5.8.3
LXD 3.0.4/stable
Ubuntu 18.04

Steps to reproduce:
Install fedora30 lxd from market place. Instantiate in a priviliged mode or default (unprivileged)
Install kali linux from the same market place
Install debian buster from the same market place
Install ubuntu bionic from the same market place

Current results:
Fedora 30:
Console doesn’t work, hostname doesn’t change to a container name, container start interface eth0 is up but no ip address assigned

Kali:
Console works, hostname doesn’t change to a container name, container start interface eth0 is up but no ip address assigned

Debian Buster:
Console works but not for root, hostname doesn’t change to a container name, container start interface eth0 and IP address assigned

Ubuntu Bionic:
All things work but only in privileged security mode.

Expected results:

  1. Console works for all users
  2. Hostname is properly set
  3. Network is working

All of this, at least 2 and 3 should work consistently in privileged and unprivileged mode.

Here a syslog on the node that started lxd. Perhaps it helps.

Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307750.747108] EXT4-fs (nbd0): mounted filesystem with ordered data mode. Opts: (null)
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307750.918929] ISO 9660 Extensions: Microsoft Joliet Level 3
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307750.966879] ISO 9660 Extensions: Microsoft Joliet Level 3
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307750.966944] ISO 9660 Extensions: RRIP_1991A
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.170200] audit: type=1400 audit(1561678538.493:275): apparmor=“STATUS” operation=“profile_load” profile=“unconfined” name=“lxd-one-28_</var/snap/lxd/common/lxd>” pid=1035176 comm=“apparmor_parser”
Jun 27 16:35:38 virt3n3-la.xcastlabs.net networkd-dispatcher[1036]: WARNING:Unknown index 62 seen, reloading interface list
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.177812] openbr.private: port 3(one-28-0) entered blocking state
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.177814] openbr.private: port 3(one-28-0) entered disabled state
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.177881] device one-28-0 entered promiscuous mode
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.177986] IPv6: ADDRCONF(NETDEV_UP): one-28-0: link is not ready
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.177988] openbr.private: port 3(one-28-0) entered blocking state
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.177990] openbr.private: port 3(one-28-0) entered forwarding state
Jun 27 16:35:38 virt3n3-la.xcastlabs.net systemd-udevd[1035177]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 27 16:35:38 virt3n3-la.xcastlabs.net systemd-udevd[1035177]: Could not generate persistent MAC address for vethY7Z0FR: No such file or directory
Jun 27 16:35:38 virt3n3-la.xcastlabs.net systemd-udevd[1035178]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.179010] openbr.private: port 3(one-28-0) entered disabled state
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.315642] eth0: renamed from vethY7Z0FR
Jun 27 16:35:38 virt3n3-la.xcastlabs.net systemd-networkd[743]: one-28-0: Gained carrier
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.347009] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.347219] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.347255] IPv6: ADDRCONF(NETDEV_CHANGE): one-28-0: link becomes ready
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.347318] openbr.private: port 3(one-28-0) entered blocking state
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.347319] openbr.private: port 3(one-28-0) entered forwarding state
Jun 27 16:35:38 virt3n3-la.xcastlabs.net kernel: [1307751.666297] audit: type=1400 audit(1561678538.989:276): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-one-28_</var/snap/lxd/common/lxd>” name="/run/systemd/unit-root/" pid=1
035329 comm="(networkd)" srcname="/" flags=“rw, rbind”
Jun 27 16:35:39 virt3n3-la.xcastlabs.net kernel: [1307751.723370] audit: type=1400 audit(1561678539.049:277): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-one-28_</var/snap/lxd/common/lxd>” name="/run/systemd/unit-root/" pid=1
035366 comm="(networkd)" srcname="/" flags=“rw, rbind”
Jun 27 16:35:39 virt3n3-la.xcastlabs.net kernel: [1307751.767779] audit: type=1400 audit(1561678539.093:278): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-one-28_</var/snap/lxd/common/lxd>” name="/run/systemd/unit-root/" pid=1
035395 comm="(networkd)" srcname="/" flags=“rw, rbind”
Jun 27 16:35:39 virt3n3-la.xcastlabs.net kernel: [1307751.822746] audit: type=1400 audit(1561678539.145:279): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-one-28_</var/snap/lxd/common/lxd>” name="/run/systemd/unit-root/" pid=1
035426 comm="(networkd)" srcname="/" flags=“rw, rbind”
Jun 27 16:35:39 virt3n3-la.xcastlabs.net kernel: [1307751.866849] audit: type=1400 audit(1561678539.193:280): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-one-28_</var/snap/lxd/common/lxd>” name="/run/systemd/unit-root/" pid=1
035452 comm="(networkd)" srcname="/" flags=“rw, rbind”
Jun 27 16:35:39 virt3n3-la.xcastlabs.net kernel: [1307752.005005] audit: type=1400 audit(1561678539.329:281): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-one-28_</var/snap/lxd/common/lxd>” name="/run/systemd/unit-root/" pid=1
035542 comm="(resolved)" srcname="/" flags=“rw, rbind”
Jun 27 16:35:39 virt3n3-la.xcastlabs.net kernel: [1307752.049383] audit: type=1400 audit(1561678539.373:282): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-one-28_</var/snap/lxd/common/lxd>” name="/run/systemd/unit-root/" pid=1
035557 comm="(r-launch)" srcname="/" flags=“rw, rbind”
Jun 27 16:35:39 virt3n3-la.xcastlabs.net kernel: [1307752.090312] audit: type=1400 audit(1561678539.413:283): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-one-28_</var/snap/lxd/common/lxd>” name="/run/systemd/unit-root/" pid=1
035571 comm="(r-launch)" srcname="/" flags=“rw, rbind”
Jun 27 16:35:39 virt3n3-la.xcastlabs.net kernel: [1307752.122809] audit: type=1400 audit(1561678539.445:284): apparmor=“DENIED” operation=“mount” info=“failed flags match” error=-13 profile=“lxd-one-28_</var/snap/lxd/common/lxd>” name="/run/systemd/unit-root/" pid=1
035586 comm="(r-launch)" srcname="/" flags=“rw, rbind”
Jun 27 16:35:39 virt3n3-la.xcastlabs.net ntpd[8175]: bind(28) AF_INET6 fe80::fce3:f8ff:fe8c:2ca5%62#123 flags 0x11 failed: Cannot assign requested address
Jun 27 16:35:39 virt3n3-la.xcastlabs.net ntpd[8175]: unable to create socket on one-28-0 (48) for fe80::fce3:f8ff:fe8c:2ca5%62#123
Jun 27 16:35:39 virt3n3-la.xcastlabs.net ntpd[8175]: failed to init interface for address fe80::fce3:f8ff:fe8c:2ca5%62
Jun 27 16:35:40 virt3n3-la.xcastlabs.net systemd-networkd[743]: one-28-0: Gained IPv6LL
Jun 27 16:35:41 virt3n3-la.xcastlabs.net ntpd[8175]: Listen normally on 49 one-28-0 [fe80::fce3:f8ff:fe8c:2ca5%62]:123

Hi

The container images imported from the marketplace are meant to be run only under privileged mode. The reasoning for this is basically this limitation.

Also

  1. Console works for all users

Are you referring to VNC ?

There is an open issue for supporting marketplace images matching https://github.com/OpenNebula/addon-context-linux#tested-platforms.

It is possible that some of those images lack a package required for context to work properly, take a look at https://github.com/OpenNebula/addon-context-linux/issues/145

That being said, images that are auto-contextualized when importing are: alpine (andy version), centos 6 and 7, ubuntu and debian (any version), fedora will be added, however kali probably not.

I’ll take a look at Debian Buster.

I refer to console via VNC

Also, fedora 30 has broken vnc and networking. So I guess some images have more problems than others. I will try centos 7 next.

See http://docs.opennebula.org/5.8/deployment/open_cloud_host_setup/lxd_driver.html#guest-issues, that is probably applicable to fedora as well, the quick test is running lxc exec fedora_container login. The network configuration doesn’t work because fedora isn’t contextualized when importing from marketplace.

You can change the login command to get direct acces to the shell and skip the login limitation.

I see that Fedora 30 contextualization has been added to tested list today. How often the images are being rebuilt? Also I tried to remove and reimport debian buster image but I am getting image 6 exist error.

Sorry, answered image in use question myself. I had to remove a container using older image and then remove the image itself. It sounds like a problem though. Let say I use image X 1.0 for lxd and then found out that bug fixed in X 1.1. Even if I use X 1.0 for current containers, shouldn’t I be able to use 1.1 for new ones? From my past I remember that the images can be versioned. When you downloading lxd image for debian buste wouldn’t it be a good idea to based id at least on the current image version? So you can have both X 1.0 and X 1.1 perhaps with the different names that incorporate the version?

I’m not sure if I understood correctly. Container images are published on a daily basis on images.linuxcontainers.org:

  • an image is imported to a datastore
  • a day passes (there is an updated version of the image)
  • you want to update your local image with the new version, kinda overwriting the linux fs ?

I download fedora on 11/14. and use it to run 3 containers. Then on. 11/30 a new fedora image becomes available with feature X. I am trying importing a new lxd image but opennebula tells me that I already have one which it can’t overwrite because it is in use by running containers. Versioning of images would allow to have and use more than one image of the same linux distribution.

To make myself a bit more clear. I am talking about the same os version i.e. lxd images for fedora 30 for example.

Couldn’t you just import it with another name?

Well, the only rename possible is the template name. Maybe adding a way to rename the image will suffice as the initial workaround. The app could, have a name suffix in both the vm template and the app template to specificy, for example, the release date, and that way it would be possible to update without issues.

Could you open a feature request on github for the team to have a nicer time evaluating this ?