Docker and Puppet: Firewall Contention

April 16, 2019

I learned of an interaction involving Docker and iptables recently. I was building up a Puppet profile to manage a Docker container running on an otherwise puppet-managed host. A separate but included module was responsible for managing iptables.

What appeared to be network problems would start popping up after a time. I’d lose the ability to access the container from outside a reverse proxy. The only fix was to restart the Docker daemon or to re-run puppet.

After a spell, I realized the problems sometimes lined up with hourly puppet runs. A quick check into the logs, and yep, sometimes the iptables rules from docker were being removed. This was sporadic and not always happening. Perhaps it was always happening on the first run after a refresh and subsequent runs were correct and therefore not changed.

Andrew Kroh’s post “Managing a Firewall with Puppet when using Docker” about how to handle Docker’s control of the firewall with Puppet came in very handy for this situation. In the post, Andrew spells out two ways to handle this contention. Ignoring Docker’s rules worked well in my case.