Setting up SolusVM with IPv6 Subnets

You may be like me, and read the setup documentation for SolusVM on how to setup IPv6 subnets but it doesn’t tell you what are the other things that are needed to make it work.  It doesn’t tell you what ethernet interface settings need to be enabled.  Using a single IPv6 /64 and assigning single IPs to VMs worked very easy, but when I first started I just couldn’t get things to work correctly.

Here’s what I’m using:
IPv6 /52
SolusVM 1.20.01
Xen 4.6.6

When SolusVM IPv6 Subnet not working

I assigned a range fc00:fc30:100:1000::/52, and added ::1 to my switch for the SolusVM vlan.  

I added the IPv6 range to SolusVM, and assigned it to a node.

I then created a XenPV server, where it auto assigned an IPv4 and IPv6 /64 subnet.

I login and see a subnet was assigned

$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 inet 127.0.0.1/8 scope host lo
 inet6 ::1/128 scope host 
 valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
 link/ether 00:16:3e:3d:72:70 brd ff:ff:ff:ff:ff:ff
 inet 10.211.76.221/26 brd 10.211.76.255 scope global eth0
 inet6 fc00:fc30:100:1021::9564/64 scope global 
 valid_lft forever preferred_lft forever
 inet6 fe80::216:3eff:fe3d:7270/64 scope link 
 valid_lft forever preferred_lft forever

Then if i check IPv6 routes

$ ip -6 route
unreachable ::/96 dev lo metric 1024 error -101 mtu 65536 advmss 65476 hoplimit 4294967295
unreachable ::ffff:0.0.0.0/96 dev lo metric 1024 error -101 mtu 65536 advmss 65476 hoplimit 4294967295
unreachable 2002:a00::/24 dev lo metric 1024 error -101 mtu 65536 advmss 65476 hoplimit 4294967295
unreachable 2002:7f00::/24 dev lo metric 1024 error -101 mtu 65536 advmss 65476 hoplimit 4294967295
unreachable 2002:a9fe::/32 dev lo metric 1024 error -101 mtu 65536 advmss 65476 hoplimit 4294967295
unreachable 2002:ac10::/28 dev lo metric 1024 error -101 mtu 65536 advmss 65476 hoplimit 4294967295
unreachable 2002:c0a8::/32 dev lo metric 1024 error -101 mtu 65536 advmss 65476 hoplimit 4294967295
unreachable 2002:e000::/19 dev lo metric 1024 error -101 mtu 65536 advmss 65476 hoplimit 4294967295
fc00:fc30:100:1000::1 dev eth0 metric 1024 mtu 1500 advmss 1440 hoplimit 4294967295
fc00:fc30:100:1021::/64 dev eth0 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
unreachable 3ffe:ffff::/32 dev lo metric 1024 error -101 mtu 65536 advmss 65476 hoplimit 4294967295
fe80::/64 dev eth0 proto kernel metric 256 mtu 1500 advmss 1440 hoplimit 4294967295
default via fc00:fc30:100:1000::1 dev eth0 metric 1 mtu 1500 advmss 1440 hoplimit 4294967295
default via fc00:fc30:100:1000::1 dev eth0 metric 1024 mtu 1500 advmss 1440 hoplimit 4294967295

We can see the default route is the main IP address which was assigned to our VLAN.  And if we try to ping that gateway address, I get a timeout

$ ping6 fc00:fc30:100:1000::1

This is our problem, IPv6 /64 subnet is assigned to the VM but not getting out.  So you’ll need to make some changes, SolusVM is properly assigning the settings, however IPv6 on your Node isn’t fully functioning properly.

Setting up SolusVM and CentOS Node IPv6 Forwarding

These are all the steps which you should go through to make sure you’re node is setup correctly for IPv6 and forwarding IPv6 packets to the switch.

Again I’m assuming you have your IPv6 environment working, just not on the node that is assigning IPv6 subnets.

Also I’m assuming you have your Node fully functional, if you have problems seeing setting up Xen on CentOS 7 post if you need assistance.

So we have allocated fc00:fc30:100:1000::/52

I assigned the first IP, fc00:fc30:100:1000::1/52 to the switch port or VLAN interface.  

First thing is going to be add your IPv6 block to Solus admin

Block Name 2607:fc30:100:1000::
Default Gateway 2607:fc30:0100:1000:0000:0000:0000:0001
Gateway Assignment: Default gateway
Reserve ::0000 – Checked
Reserve ::0001 – Checked
Reserve ::ffff – Checked
Netmask: /52
Nameserver: Google

The rest of the items are default. I set the gateway IP address which I had assigned to my VLAN interface, and set Default Gateway for all VMs to gateway through that address.

Next on the server we need to enable IPv6 on the interface and enable IPv6 Forwarding in the kernel. I created the file /etc/sysctl.d/solusipv6.conf and put the following contents, then reloaded.

$ vi /etc/sysctl.d/solusipv6.conf 
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.all.proxy_ndp = 1
net.ipv4.ip_forward = 1
net.ipv6.conf.xenbr0.forwarding = 1

After you’ve edit that file, now load those settings

The output will look something like this:

$ sysctl --system
* Applying /usr/lib/sysctl.d/00-system.conf ...
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
kernel.yama.ptrace_scope = 0
* Applying /usr/lib/sysctl.d/50-default.conf ...
kernel.sysrq = 16
kernel.core_uses_pid = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.conf.all.promote_secondaries = 1
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.d/solusipv6.conf ...
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.all.proxy_ndp = 1
net.ipv4.ip_forward = 1
net.ipv6.conf.xenbr0.forwarding = 1
* Applying /etc/sysctl.conf ...

After we’ve updated our kernel settings we can now update the network settings.

$ vi /etc/sysconfig/network
NETWORKING_IPV6=yes
IPV6FORWARDING=yes
IPV6_DEFAULTDEV=em1
IPV6_DEFAULTGW=fc00:fc30:100:1000::1
IPV6_AUTOCONF=no

With our SolusVM we have em1 is the interface, but all the settings are moved into xenbr0.  You’ll see in a moment.  So we enable IPv6 and enable Forwarding, and specify the Gateway and Device.  I have AUTOCONF off because I don’t have IPv6 router advertisements enabled on this VLAN, since there are many customers and different VM server i don’t want anything being auto-assigned, only what SolusVM assigns to the customer.

$ vi /etc/sysconfig/network-scripts/ifcfg-em1 
DEVICE=em1
BOOTPROTO=none
NM_CONTROLLED=yes
ONBOOT=yes
TYPE=Ethernet
UUID="14d3acfc-47e0-4365-aafd-86c4b2b39af6"
USERCTL=no
BRIDGE=xenbr0
IPV6INIT=yes
IPV6ADDR=fc00:fc30:100:1000::2

So now i edit our em1 interface configuration.  I added in there IPV6INIT and IPV6ADDR.  I enabled IPv6 and assigned the second IP to this server.

$ vi /etc/sysconfig/network-scripts/ifcfg-xenbr0 
DEVICE=xenbr0
BOOTPROTO=static
ONBOOT=yes
TYPE=Bridge
IPADDR=10.211.76.34
NETMASK=255.255.255.252
GATEWAY=10.211.76.33
IPV6INIT=yes
IPV6_AUTOCONF=no
NETWORKING_IPV6=yes
IPV6FORWARDING=yes

Now on our xenbr0 interface, we enabled IPv6, turned off AUTOCONF since we don’t want an IP auto assigned, we’re already giving it one.  Then enable Forwarding since we need to forward the /64 addresses that are assigned to individual VPS servers out to the Switch VLAN and visa versa.

You can restart the server now, or I had customers already running so I chose to individually restart interfaces which I just modified.

$ ifdown em1
$ ifup em1
$ ifdown xenbr0
$ ifup xenbr0

or

$ systemctl restart network

Afterwards try pinging out from the VPS and you should be good now.