I’d been experiencing IP clashes with my router on Ubuntu, which manifested whenever I used a network with a router in the 192.168.0.0/20 block. I’d struggled to narrow down the cause, but after some discussion on the Ubuntu forums, I narrowed it down to particular bridge interface that had been set up on the same block:
> ip addr | grep inet | grep "192.168.0.1"
inet 192.168.0.1/20 brd 192.169.15.255 scope global noprefixroute br-5bf3d35782b1
Looking at this bridge with nmcli showed that it was using the same block that was causing issues:
> nmcli connection show br-5bf3d35782b1
connection.id: br-5bf3d35782b1
...
ipv4.method: manual
ipv4.dns: --
ipv4.dns-search: --
ipv4.dns-options: ""
ipv4.dns-priority: 100
ipv4.addresses: 192.168.0.1/20
ipv4.gateway: --
ipv4.routes: --
ipv4.route-metric: -1
ipv4.route-table: 0 (unspec)
ipv4.ignore-auto-routes: no
ipv4.ignore-auto-dns: no
ipv4.dhcp-client-id: --
ipv4.dhcp-timeout: 0 (default)
ipv4.dhcp-send-hostname: yes
ipv4.dhcp-hostname: --
ipv4.dhcp-fqdn: --
ipv4.never-default: no
ipv4.may-fail: yes
ipv4.dad-timeout: -1 (default)
...
To see if this was the cause of the problem, I changed the ipv4 block that it was using and restarted network-manager:
nmcli connection modify br-5bf3d35782b1 ipv4.addresses 192.169.0.1/20
sudo service network-manager restart
This fixed the issue, and checking nmcli showed the updated IP block being used. However, this didn’t resolve what had created the bridge in the first place. ip showed a large number of these bridges, which pointed to something creating them automatically:
> ip route list | grep "br-"
169.254.0.0/16 dev br-02785f378908 scope link metric 1000 linkdown
172.18.0.0/16 dev br-d70cef40006f proto kernel scope link src 172.18.0.1 linkdown
172.19.0.0/16 dev br-716b9ad47006 proto kernel scope link src 172.19.0.1 linkdown
172.20.0.0/16 dev br-f1737ad7487d proto kernel scope link src 172.20.0.1 linkdown
172.21.0.0/16 dev br-6b2783d18811 proto kernel scope link src 172.21.0.1 linkdown
172.22.0.0/16 dev br-af6549292f3c proto kernel scope link src 172.22.0.1 linkdown
172.23.0.0/16 dev br-7eb7b8ce5626 proto kernel scope link src 172.23.0.1 linkdown
172.24.0.0/16 dev br-d676325419fd proto kernel scope link src 172.24.0.1 linkdown
172.25.0.0/16 dev br-477bad8a9110 proto kernel scope link src 172.25.0.1 linkdown
172.26.0.0/16 dev br-77c1ddfe6b3b proto kernel scope link src 172.26.0.1 linkdown
172.27.0.0/16 dev br-02785f378908 proto kernel scope link src 172.27.0.1 linkdown
172.28.0.0/16 dev br-7a099a029376 proto kernel scope link src 172.28.0.1 linkdown
172.29.0.0/16 dev br-adbf9f58bedc proto kernel scope link src 172.29.0.1 linkdown
172.30.0.0/16 dev br-d68403733ee3 proto kernel scope link src 172.30.0.1 linkdown
172.31.0.0/16 dev br-662ea33fb429 proto kernel scope link src 172.31.0.1 linkdown
192.169.0.0/20 dev br-5bf3d35782b1 proto kernel scope link src 192.169.0.1 metric 425 linkdown
I suspected this was related to Docker, and after checking the documentation and seeing that the default was to use bridge networking I checked the networks created by Docker:
> docker network inspect $(docker network ls -q)
...
{
"Name": "my-project_default",
"Id": "5bf3d35782b1c260bc8382fb168cc77692cba80ad7b6b15026e3e93da35f6ded",
"Created": "2019-10-08T11:46:25.112098915+01:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "192.168.0.0/20",
"Gateway": "192.168.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {},
"Options": {},
"Labels": {
"com.docker.compose.network": "default",
"com.docker.compose.project": "my-project"
}
}
...
There was one Docker network that perfectly matched, and it turned out that the blocks 192.168.[0-240].0/20 are part of the default pool that Docker can use 1 2. Presumably at some point I’d spun up a Docker container where the 192.168.0.0/20 block wasn’t in use, and this then caused conflicts when switching networks.
To resolve this for future cases, I updated /etc/docker/daemon.json as specified in the GitHub issue to exclude this block from the pool, and removed the old bridge:
nmcli connection delete "br-5bf3d35782b1"
docker network rm "5bf3d35782b1"
As you can see here, Docker uses the short id of the Docker network as the identifier of the network interface.
My new/etc/docker/daemon.json simply removes 192.168.[0-240].0/20 from the available address pools:
{
"default-address-pools":
[
{
"base": "172.17.0.0/16",
"size": 16
},
{
"base": "172.18.0.0/16",
"size": 16
},
{
"base": "172.19.0.0/16",
"size": 16
},
{
"base": "172.20.0.0/16",
"size": 16
},
{
"base": "172.24.0.0/16",
"size": 16
},
{
"base": "172.28.0.0/16",
"size": 16
}
]
}
To make Docker recreate the network, I then had to remove the existing containers and recreate them:
docker-compose rm
docker-compose up -d
After looking at the behaviour ofdocker-compose down, I believe you could combine the removal of the network and recreation of the Docker containers by instead runningdocker-compose down, since this remove both the containers and any associated networks:
> docker-compose help down
Stops containers and removes containers, networks, volumes, and images
created by `up`.
By default, the only things removed are:
- Containers for services defined in the Compose file
- Networks defined in the `networks` section of the Compose file
- The default network, if one is used
Networks and volumes defined as `external` are never removed.
Finally, after these steps, I was back with a working network connection and Docker container.