IPv6, Docker(-compose), and Shorewall6/ip6tables
The default docker configuration is limited to IPv4 networking and the docker daemon transparently takes care of IPv4-based SNAT. However, when IPv6 is enabled, this automagic is neither included nor has been built into docker. This write-up will walkthough the 3 approaches to address this shortcoming.
IETF strongly discourages IPv6-SNAT because SNAT was conceptually just a workaround for the lack of IPv4 addresses in the addressing space.
I fully disagree!
There are benefits to running networks isolated from globally addressable IPv6-space, and using SNAT to bridge the private subnet with the public globally addressable IPv6-space. One of the biggest benefits is that your private subnet is not globally addressable, i.e., a natural “firewall” is thus created, similar to what you get in IPv4 world with RFC1918 private-space addresses (familiar to most people in the form of 192.168.0.x). Imagine if all your containers were fully reachable from the global internet just by getting hold of their IPv6 addresses, including their “unexposed” container ports!
Primer on IPv6 addressing types
So in IPv6, broadly speaking for the ease of this write-up, there are 3 major groups of addresses.
- “Global”;
- “Link-local” (now deprecated in RFC3879); and
- “Unique Local Address” (ULA).
“Link-local” addresses are automatically generated by the OS and are only valid within a link-segment, and cannot be used across routers. This is similar to the IPv4 stateless address auto-configuration (169.254.0.0/16), and cannot be used across routers. Practically useless.
ULAs can be seen as an RFC1918 IPv4 private address space equivalent (i.e., 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8), which are separate from the “Link-local” addresses. ULAs can be used across routers, depending on the routing table, but they are still not globally routable.
To enable IPv6 on docker and also within the docker-compose default network, ULA IPv6 prefixes will be used. The ULA prefix is defined as fd00::/8
. Docker requires a minimum addressable space of /80
. Generally, I will assign fd00:ffff::/80
to the default docker0
bridge, in violation of RFC4193, but I don’t see any negative impact to doing so for this docker0
bridge configuration.
Enabling IPv6 in Docker and the docker0 bridge
I’ve adapted these instructions from the docker docs.
To enable IPv6 in Docker and assign a ULA prefix to the docker0
bridge, put the following in /etc/docker/daemon.json
:
{
"ipv6": true,
"fixed-cidr-v6": "fd00:ffff::/80"
}
Then, restart the docker daemon:
$ sudo service docker restart.
You should see that the default docker0
bridge has IPv6 enabled:
$ docker network inspect bridge | grep -i "ipv6\|subnet\|gateway"
"EnableIPv6": true,
"Subnet": "172.17.0.0/16",
"Gateway": "172.17.0.1"
"Subnet": "fd00:ffff::/80",
"Gateway": "fd00:ffff::1"
Test it with this command:
$ docker run --rm -it debian ping6 google.com -c3
Success?
Probably not. The container might still not be able to obtain connectivity because there is no packet forwarding between the global addressable network and the ULA subnet. This is the automagic from docker IPv4 support that is missing in docker’s current IPv6 implementation.
3 approaches
I will discuss 3 approaches to address these shortcomings: Docker’s experimental ip6tables
support, docker-ipv6nat
, and Shorewall[6] (if you are already using Shorewall to define your firewall).
ip6tables approach
Before you proceed, you will need a very recent docker installation, version 20.10.2
or later, recently merged with this pull-request. Otherwise, manually set-up SNAT using ip6tables, or try the other 2 approaches.
To enable ip6tables
handling, the experimental
flag must be enabled. Put the following in /etc/docker/daemon.json
or adjust to fit:
{
"ipv6": true,
"fixed-cidr-v6": "fd00:ffff::/80",
"ip6tables": true,
"experimental": true
}
Then, restart the docker daemon:
$ sudo service docker restart.
Review the ip6tables
FORWARD
chain for new rules:
$ sudo ip6tables -L
Test it with this command:
$ docker run --rm -it debian ping6 google.com -c3
If it’s successful, you should see the following:
PING google.com(sa-in-x66.1e100.net (2404:6800:4003:c00::66)) 56 data bytes
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=1 ttl=110 time=3.10 ms
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=2 ttl=110 time=3.62 ms
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=3 ttl=110 time=3.14 ms
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 3.099/3.284/3.619/0.241 ms
Done!
docker-ipv6nat approach
I don’t recommend this approach as it does not work with IPv6 networking in docker-compose
. Eventually, once the ip6tables
support in docker is mature, docker-ipv6nat
will be retired. Otherwise, this is an easy approach for pure docker containers.
Run docker-ipv6nat
as a docker container:
$ docker run -d --rm --name ipv6nat --cap-add NET_ADMIN --cap-add NET_RAW --cap-add SYS_MODULE --cap-drop ALL --network host --restart unless-stopped -v /var/run/docker.sock:/var/run/docker.sock:ro -v /lib/modules:/lib/modules:ro robbertkl/ipv6nat
Or with docker-compose.yml
and docker-compose up -d
:
version: '2.3'
services:
ipv6nat:
image: "robbertkl/ipv6nat"
container_name: ipv6nat
restart: unless-stopped
network_mode: host
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /lib/modules:/lib/modules:ro
cap_add:
- NET_ADMIN
- NET_RAW
- SYS_MODULE
cap_drop:
- ALL
Review the ip6tables
FORWARD
chain for new rules:
$ sudo ip6tables -L
Test it with this command:
$ docker run --rm -it debian ping6 google.com -c3
If it’s successful, you should see the following:
PING google.com(sa-in-x66.1e100.net (2404:6800:4003:c00::66)) 56 data bytes
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=1 ttl=110 time=3.10 ms
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=2 ttl=110 time=3.62 ms
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=3 ttl=110 time=3.14 ms
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 3.099/3.284/3.619/0.241 ms
Done!
Shorewall6 approach
I’ve adapted these instructions from this gist.
Enable docker support in /etc/shorewall/shorewall.conf
:
DOCKER=Yes
Take note that this directive is not available in shorewall6.conf
, which is fine.
Define your /etc/shorewall[6]/interfaces
files to capture all the docker bridges, i.e., docker0
and br-xxx
, where docker
is the zone defined for docker traffic, using the physical
option:
?FORMAT 2
#ZONE INTERFACE OPTIONS
- lo ignore
net eth0 dhcp,routeback,accept_ra=2
docker docker0 routeback=1,physical=docker+
docker br routeback=1,physical=br-+
Be sure to set accept_ra=2
in your public-facing interface, eth0
in my case.
Enable SNAT by adding the following line to /etc/shorewall6/snat
:
#ACTION SOURCE DEST PROTO PORT IPSEC MARK USER SWITCH ORIGDEST PROBABILITY
MASQUERADE - eth0
Older versions of Shorewall use the masq
file instead. Adapt the above accordingly.
You don’t need to enable SNAT for IPv4 as docker does it for you automagically.
Check your shorewall[6] configurations:
$ sudo shorewall check
$ sudo shorewall6 check
Once error-free, activate the new rule sets:
$ sudo shorewall safe-restart
$ sudo shorewall6 safe-restart
Review the ip6tables
FORWARD
chain for new rules:
$ sudo ip6tables -L
Test it with this command:
$ docker run --rm -it debian ping6 google.com -c3
If it’s successful, you should see the following:
PING google.com(sa-in-x66.1e100.net (2404:6800:4003:c00::66)) 56 data bytes
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=1 ttl=110 time=3.10 ms
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=2 ttl=110 time=3.62 ms
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=3 ttl=110 time=3.14 ms
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 3.099/3.284/3.619/0.241 ms
Done!
docker-compose
Now that plain docker containers have IPv6 connectivity, it’s time to tackle docker-compose IPv6 bridge networking.
In a typical docker-compose.yml
, whenever there is more than one service defined, docker-compose will automatically create a default bridge network, named after the project. This project-specific default bridge network connects the containers together and configures their hostnames according to their service names. This bridge is IPv4-only and additional directives are required in the compose file to override this.
As of now, docker swarm does not support IPv6 and hence, the enable_ipv6
directive is only available for file format versions below 3, e.g., 2.4.
Similar to the fixed-cidr-v6
configuration in docker daemon.json
above, a separate ULA needs to be manually assigned to each docker-compose.yml
project, annoyingly.
To prevent conflicts (as expounded by RFC4193) with other projects and to ease tracking, I recommend generating this prefix in a simple way using the project name like this:
$ echo "projectname" | shasum | cut -c1-8
109b2e51
Using the above 8 digit hash to fill up the following 2 groups of 16-bits of the ULA prefix (fd00::/8): fd00:109b:2e51::/80
.
Append or update the following networks
key to the project docker-compose.yml
:
networks:
default:
enable_ipv6: true
ipam:
driver: default
config:
- subnet: fd00:109b:2e51::/80
gateway: fd00:109b:2e51::1
You do not need to manually assign IPv4 or IPv6 addresses to each of your containers using the networks
key under each service entry.
Test your config and bring up the services and the IPv6 enabled bridge:
$ docker-compose config
$ docker-compose down ; docker-compose up -d
Inspect the network bridge:
$ docker network inspect projectname_default | grep -i ipv6
"EnableIPv6": true,
"IPv6Address": "fd00:109b:2e51::2/80"
Test it with one of the following commands:
$ docker-compose run --rm projectname ping6 google.com -c3
$ docker-compose exec projectname ping6 google.com -c3
If it’s successful, you should see the following:
PING google.com(sa-in-x66.1e100.net (2404:6800:4003:c00::66)) 56 data bytes
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=1 ttl=110 time=3.10 ms
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=2 ttl=110 time=3.62 ms
64 bytes from sa-in-x66.1e100.net (2404:6800:4003:c00::66): icmp_seq=3 ttl=110 time=3.14 ms
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 5ms
rtt min/avg/max/mdev = 3.099/3.284/3.619/0.241 ms
Done!
Conclusion
IPv6 is now available in Docker and docker-compose, as long as you know how to set it up and populate your docker-compose.yml
file. Hopefully, the docker automagic will get implemented in IPv6 configurations by default, and the workarounds discussed in this write-up will no longer be necessary.
Special thanks to Felix and James for helping me proofread this write-up.