From 0fb55c5258a2966eb2f818471def853b43356ebb Mon Sep 17 00:00:00 2001 From: Cornelius Diekmann Date: Wed, 3 May 2017 15:46:04 +0200 Subject: [PATCH] updated docker story --- configs_corny_docker/docker.txt | 113 ++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) diff --git a/configs_corny_docker/docker.txt b/configs_corny_docker/docker.txt index f3eae46..0ddb17b 100644 --- a/configs_corny_docker/docker.txt +++ b/configs_corny_docker/docker.txt @@ -2322,7 +2322,120 @@ Only difference: (''0.0.0.0'', ''10.0.0.4'') which means: INET -> WebApp may be +=== One final connectivity check. === +The rules docker is loading by default: +# Generated by iptables-save v1.6.0 on Wed May 3 14:40:49 2017 +*nat +:PREROUTING ACCEPT [0:0] +:INPUT ACCEPT [0:0] +:OUTPUT ACCEPT [146:9341] +:POSTROUTING ACCEPT [43:2699] +:DOCKER - [0:0] +-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER +-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER +-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE +-A POSTROUTING -s 10.0.0.0/8 ! -o br-0a5ad9e85c71 -j MASQUERADE +-A DOCKER -i docker0 -j RETURN +-A DOCKER -i br-0a5ad9e85c71 -j RETURN +COMMIT +# Completed on Wed May 3 14:40:49 2017 +# Generated by iptables-save v1.6.0 on Wed May 3 14:40:49 2017 +*filter +:INPUT ACCEPT [878:3516798] +:FORWARD ACCEPT [0:0] +:OUTPUT ACCEPT [859:77933] +:DOCKER - [0:0] +:DOCKER-ISOLATION - [0:0] +-A FORWARD -j DOCKER-ISOLATION +-A FORWARD -o docker0 -j DOCKER +-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT +-A FORWARD -i docker0 ! -o docker0 -j ACCEPT +-A FORWARD -i docker0 -o docker0 -j ACCEPT +-A FORWARD -o br-0a5ad9e85c71 -j DOCKER +-A FORWARD -o br-0a5ad9e85c71 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 ! -o br-0a5ad9e85c71 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -o br-0a5ad9e85c71 -j DROP +-A DOCKER-ISOLATION -i br-0a5ad9e85c71 -o docker0 -j DROP +-A DOCKER-ISOLATION -i docker0 -o br-0a5ad9e85c71 -j DROP +-A DOCKER-ISOLATION -j RETURN +COMMIT +# Completed on Wed May 3 14:40:49 2017 + +unfortunately, my bridge interface is now br-0a5ad9e85c71 + +The rules we are installing (the same as previously, just the bridge is renamed and the docker nat table is shown): +# Generated by iptables-save v1.6.0 on Wed May 3 14:39:08 2017 +*nat +:PREROUTING ACCEPT [0:0] +:INPUT ACCEPT [0:0] +:OUTPUT ACCEPT [146:9341] +:POSTROUTING ACCEPT [43:2699] +:DOCKER - [0:0] +-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER +-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER +-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE +-A POSTROUTING -s 10.0.0.0/8 ! -o br-0a5ad9e85c71 -j MASQUERADE +-A DOCKER -i docker0 -j RETURN +-A DOCKER -i br-0a5ad9e85c71 -j RETURN +COMMIT +# ... +*filter +:INPUT ACCEPT [0:0] +:FORWARD DROP [0:0] +:OUTPUT ACCEPT [0:0] +:CUSTOM - [0:0] +-A FORWARD -j CUSTOM +-A CUSTOM +-A CUSTOM -d 193.99.144.80/32 -m recent --set --name rateheise --mask 255.255.255.255 --rsource +-A CUSTOM -d 193.99.144.80/32 -m recent --update --seconds 60 --hitcount 3 --name rateheise --mask 255.255.255.255 --rsource -j DROP +-A CUSTOM -m state --state ESTABLISHED -p tcp -m multiport --ports 22 -j ACCEPT +-A CUSTOM -p tcp -m tcp --dport 22 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.1,10.0.0.42 -o br-0a5ad9e85c71 -d 10.0.0.1,10.0.0.42 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.1,10.0.0.42 -o br-0a5ad9e85c71 -d 10.0.0.2 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.1,10.0.0.42 -o br-0a5ad9e85c71 -d 10.0.0.4 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.1,10.0.0.42 ! -o br-0a5ad9e85c71 -m iprange ! --dst-range 10.0.0.0-10.255.255.255 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.3 -o br-0a5ad9e85c71 -d 10.0.0.3 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.3 -o br-0a5ad9e85c71 -d 10.0.0.2 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.3 -o br-0a5ad9e85c71 -d 10.0.0.4 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.2 -o br-0a5ad9e85c71 -d 10.0.0.2 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.4 -o br-0a5ad9e85c71 -d 10.0.0.1,10.0.0.42 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.4 -o br-0a5ad9e85c71 -d 10.0.0.3 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.4 -o br-0a5ad9e85c71 -d 10.0.0.2 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.4 -o br-0a5ad9e85c71 -d 10.0.0.4 -j ACCEPT +-A FORWARD -i br-0a5ad9e85c71 -s 10.0.0.4 ! -o br-0a5ad9e85c71 ! -d 10.0.0.0/8 -j ACCEPT +-A FORWARD ! -i br-0a5ad9e85c71 -m iprange ! --src-range 10.0.0.0-10.255.255.255 -o br-0a5ad9e85c71 -d 10.0.0.1,10.0.0.42 -j ACCEPT +-A FORWARD ! -i br-0a5ad9e85c71 ! -s 10.0.0.0/8 ! -o br-0a5ad9e85c71 ! -d 10.0.0.0/8 -j ACCEPT +-I FORWARD -m state --state ESTABLISHED ! -i br-0a5ad9e85c71 ! -s 10.0.0.0/8 -o br-0a5ad9e85c71 -d 10.0.0.4 -j ACCEPT +COMMIT + +Checked connectivity: +Inter-container connectivity: Exactly as expected. +Tested in the busybox images with ping. One-way communication with log also works. Tested with ping, tcpdumping the docker bridge, we see echo replies generated by log, but they never arrive anywhere. + +Note: The docker host can access all containers. This is because setting up connections to the containers does not go via the FORWARD chain but uses the INPUT/OUTPUT chains. There are no restrictions currently (but the docker host can manage the containers anyway). + +An arbitrary container, spawned with ip 172.17.0.2 and not in mynet to simulate the Internet has only the desired access rights (i.e. only to webfrnt) + +Test external reachability (by exposing docker ports publicly): +Set up VirtualBox (where my docker runs) to host-only networking, so we can connect from external. +#add dummy chain t filter table, never called, just to make docker happy +iptables -N DOCKER + +#spawn containers with ports 6666 and 6667 exposed. Expected: 6666 reachable, 6667 not. +sudo docker run --rm -ti -p 0.0.0.0:6666:6666 --name webfrnt42 --hostname webfrnt42 --net mynet --ip 10.0.0.42 busybox +sudo docker run --rm -ti -p 0.0.0.0:6667:6667 --name webapp --hostname webapp --net mynet --ip 10.0.0.4 busybox + +docker automatically adds to nat table (which I let docker manage): +-A DOCKER ! -i br-0a5ad9e85c71 -p tcp -m tcp --dport 6666 -j DNAT --to-destination 10.0.0.42:6666 +-A DOCKER ! -i br-0a5ad9e85c71 -p tcp -m tcp --dport 6667 -j DNAT --to-destination 10.0.0.4:6667 +docker automatically adds to filter table (which is dead because we never call the DOCKER chain there): +-A DOCKER -d 10.0.0.42/32 ! -i br-0a5ad9e85c71 -o br-0a5ad9e85c71 -p tcp -m tcp --dport 6666 -j ACCEPT +-A DOCKER -d 10.0.0.4/32 ! -i br-0a5ad9e85c71 -o br-0a5ad9e85c71 -p tcp -m tcp --dport 6667 -j ACCEPT +from external where 192.168.56.101 is the external ip of the docker host: +nc -v 192.168.56.101 6666 #works flawlessly, as expected +nc -v 192.168.56.101 6667 #does not work, as expected, gets dropped by FORWARD default policy +Yep, connectivity works as expected.