Improved Security Groups in 5.6

Development
1

In the OpenNebula newsletter it mentioned that we can leave feature requests for the 5.6 roadmap, so here it goes.

OpenNebula security groups (SGs) are nothing more than firewall rules, while SGs on EC2 are a combination of firewall rules and networks. This combination is what is powerful and what allows you to have a flat network structure with flexibility of dynamically isolating instance groups one from each other.

Here is how SGs work in EC2:

  • All VMs in the same SG can communicate between each other (except if a special rule is created to disable that).
    • This means that a SG acts sort of like a special network which allows all machines in a SG to communicate with each other.
    • It also makes managing rules in a SG much easier, because VMs in a SG can talk to each other, so no additional rules for that.
  • You can put a different SG as a SRC/DEST in a SG rules
    • This is the real magic because it allows you for example to create two SGs: sg-app-worker and sg-app-mysql; then in sg-app-mysql you can just add one rule which would allow SRC: sg-app-worker to connect on TCP port 3306, which would allow all the VMs in the sg-app-worker to connect to the MySQL database.
    • What is great about this is that you don’t need to care in what networks VMs are, nor what IPs they have, nor how many app-worker VMs will you have in the future, just add them to the correct SG and it will always work, which is extremely flexible.

To be able to allow app workers to connect to MySQL in OpenNebula currently you need to:

  1. Create a virtual network app-worker-network (which can be a reservation of IPs).
    • This means that you need to plan how big this network should be for future growth (this isn’t a problem for local IPs, because you have a lot of them and can make the network very big, but it is problematic with limited public IPs).
    • Let’s not forget what complications you can have if the number of VMs in a network outgrows the network size you created.
    • The thing is why should I care how many hosts will be there in the future and what IPs do my VMs have, if I don’t have to.
  2. Create a SG sg-app-mysql which allows all hosts in network app-worker-network to connect to 3306
  3. Add rules if the VMs in the same security group need to talk to each other (the same for app-worker).

This is a very complicated process which is almost exactly the same as creating VLANs, networks and ACLs on L3 switches; and it’s not dynamic enough for cloud workloads. In EC2 just two simple security groups with 1 rule to allow one SG to connect to the other on a port does the same thing and it scales to as many VMs as you want.

There are two issues already created for this:

What do you think? How do you deal with this problem?

2 Likes
2

Hi Strahinja

THANKS for your feedback :wink:

About the SG as network

I do not really get this one. OpenNebula has richer semantics to express connectivity (networks) and security (SG). We also made an effort to make easier to use both of them. You can express that in a SG, so all the VMs in a network can communicate. Note that in OpenNebula networks includes multiple addresses spaces so they can involve different IP networks.

About using SG for targets

This is the goal of issue https://dev.opennebula.org/issues/5317. As I said before the real challenge here is to deal with the mismatch between networks and SGs.

If you replace SG for VNet in your statements it would be the same functionality as that you refer of EC2. So there is no real difference between EC2 and OpenNebula. Note that in OpenNebula you expose lower-level network details and operations.

About some of the limitations you mention, note that you can easily add AR’s to a VNET to accommodate more/less VMs in the VNET. As you mention the real problem is public IP’s, and that is a resource you need to account for.

In general, I think this is just a matter of separation of concerns, merging the VNET and SG concept in general is not a good idea as it would limit the use cases of OpenNebula. And they are different beasts. I reckon that from a final user point of view we can do it better and simplify the process (while preserving the flexibility and control for the admins).

We are planning to better expose the VNET reservations so a user can better auto-provision VNETs. Probably we could include those SG “shortcuts” as part of the process

1 Like
3

You are probably right that replacing SG with VNet would mean the same thing (except for public IPs), but if you compare how easy it is to work with SGs in Amazon compared to VNets+SGs in OpenNebula, is not even a competition. Also I don’t see any limitations with the Amazon way of doing networking, especially since you can define networks additional networks as well, but the thing is in most cases you don’t have to worry about that at all.

Here are some issue I have with VNets in One compared to Amazon:

  • As stated before, you have to worry about the size of a VNet (especially important for public addresses)
  • It’s very hard to create new VNets (especially IP reservations)
    • I’m not even sure I know how to create reservations in Sunstone (I use onevnet)
  • Because it’s hard to create a VNet, it’s almost impossible for me as an OpenNebula admin to give our users permissions to create VNets (which I want to do, so that they can easy manage everything they need for their VMs)
    • To create a SG in Amazon is very easy, you just need to specify a few firewall rules
    • It’s very easy to specify these firewall rules because you don’t have to know what is the underlying networks, because you can also specify SGs as SRC/DEST.
  • If you want to replace iptables in VMs on OpenNebula, with VNets+SGs, the number of SGs will grow with the number of VNets, instead of only one of them growing.

If you could somehow make VNets (reservations) as easy to use as SGs in Amazon with e.g. dynamic IP reservations to a VNets (from a selected IP pool) where an IP would be assigned to a VNet on VM deploy, that would probably do the thing. Or any other way so that admins don’t have to worry about IP addresses and IP networks. Networking is the most boring part of running dedicated servers, and that is one of the reason why we even switched to OpenNebula. We were expecting that One will abstract complexity of networks, but for now that is not the case. Hopefully 5.6 will fix that.

Also it would be great if you guys would share ideas when you start working on this feature (if you decide to do it for 5.6), so that the community could give you feedback and help.

1 Like
4

This week I was working with VNets a lot and I noticed two big drawbacks of VNets and their inflexibility:

  • VNet can only spread across one physical network. You cannot define multiple VLANs nor physical networks in a VNet, which means you cannot grow it later on if it become too small.
  • VNet reservation can only have IP reservations from a single VNet.

So when you said:

that’s not true. OpenNebula VNets are limited to a single physical network, while EC2 Security Groups are not, which makes them extremely flexibly and easy to use.

1 Like
5

Hi @ruben :slight_smile:
I think it would be easier if you could go try out the EC2 security groups for a few hours or so.
Right now we have a lot less possibilities, and its harder to use. Not being able to refer to another (even the same!!!) SG is one of the things that should be there since it allows to implicitly configure things.

There are some advantages I can imagine as-is, but the things that miss make too unattractive to enjoy the advantages.

1 Like
6

If anyone is interested you can follow this issue https://github.com/OpenNebula/one/issues/1301 which should tackle all (or most) of the limitation mentioned in this thread.

1 Like
7

I found this old topic while considering to prototype changes to security groups.

I dont know if the situation has evolved much since them, perhaps @ruben could point to changes/plans.

I am throwing a wild guess here, but I think the OpenNebula is providing a frontend to iptables.

and iptables in itself is fairly limited… it is very easy to end up with 100s of rules that are difficult to maintain/decipher.

I think @strahinjak is very right in pointing at other implementations as source of inspiration for functional requirements, and I would like to do also.

Pfsense has a great firewall engine, where you can basically define, aliases by name: networks, port ranges, hosts ect. Very importantly you can create aliases that include other aliases.

Then you use those to create your firewall rules.

Your rules are then readable and very easy to maintain: Allow SSH into all my Public Managed Hosts form my Management Workstations.

Public Managed Host = Product Hosts + Staging Servers
Product Hosts = Product A hosts + Product B hosts …

Any of this alias can be modified at any time, and the engine will deal with it.

They also include the possibility to use DNS names for interfaces to external entities, that the firewall engine will periodically resolve and adjust rules with those.

The closest we got to replicating this functionality in Linux is by using a combination of ferm (a frontend for iptables) and ipsets.

I was thinking in putting together a prototype of another kind of security group or something…

2 Likes