Docker v25 Networking Issues (Self-hosted only)

Ask the Community Password Manager
1

Please be aware that a recent update, Docker version 25, has introduced networking issues in Bitwarden as well as other Docker-based software such as HomeAssistant.

Customers newly-installing self-hosted on updated systems, or running system updates that consume Docker v25, may see admin and sso containers unhealthy depending on their distribution and update settings, and all Bitwarden clients will be inaccessible; with “syncing failed” errors on logged-in clients, and “an unhandled server error has occurred” or “504 Gateway Time-Out” when attempting to log in. These errors may not occur until a restart of the Docker daemon and/or a full system reboot after applying the package update.

These error may be related to a recent update in Docker, technology which underlies your Bitwarden self-hosted server. Docker recently released version 25, which has introduced networking issues in Bitwarden as well as other Docker-based software such as HomeAssistant.

You can check in your package manager if you are running Docker 25.0.0:

sudo dpkg -l | grep docker-ce

sudo rpm -qa | grep docker-ce

If you see docker-ce 5:25.0.0-1~ubuntu.22.04~jammy, docker-ce-25.0.0-1.el9.x86_64, or similar output indicating Docker v25 is installed, this is likely the cause of the issue. While we await a fix from Docker upstream, you can resolve this issue by downgrading to Docker v24 temporarily. The steps may differ slightly depending on your distribution and the versions available; in general, however, you may downgrade using the following steps.

RHEL and RHEL-alike, versions 8.x:

sudo dnf install docker-ce-3:24.0.7-1.el8.x86_64 docker-ce-cli-1:24.0.7-1.el8.x86_64 docker-compose-plugin-2.5.0-3.el8.x86_64 docker-scan-plugin-0.8.0-3.el8.x86_64 docker-ce-rootless-extras-24.0.7-1.el8.x86_64 && sudo systemctl restart docker

RHEL and RHEL-alike, versions 9.x:

sudo dnf install docker-ce-3:24.0.7-1.el9.x86_64 docker-ce-cli-1:24.0.7-1.el9.x86_64 docker-compose-plugin-2.5.0-3.el9.x86_64 docker-scan-plugin-0.23.0-1.el9.x86_64 docker-ce-rootless-extras-24.0.7-1.el9.x86_64 && sudo systemctl restart docker

Debian 11 (Bullseye):

sudo apt install docker-compose-plugin=2.21.0-1~debian.11~bullseye docker-ce-cli=5:24.0.7-1~debian.11~bullseye docker-buildx-plugin=0.11.2-1~debian.11~bullseye docker-ce=5:24.0.7-1~debian.11~bullseye && sudo systemctl restart docker

Debian 12 (Bookworm):

sudo apt install docker-compose-plugin=2.21.0-1~debian.12~bookworm docker-ce-cli=5:24.0.7-1~debian.12~bookworm docker-buildx-plugin=0.11.2-1~debian.12~bookworm docker-ce=5:24.0.7-1~debian.12~bookworm docker-ce-rootless-extras=5:24.0.7-1~debian.12~bookworm && sudo systemctl restart docker

Ubuntu 22.04 (Jammy):

sudo apt install docker-ce=5:24.0.7-1~ubuntu.22.04~jammy docker-ce-cli=5:24.0.7-1~ubuntu.22.04~jammy docker-compose-plugin=2.21.0-1~ubuntu.22.04~jammy && sudo systemctl restart docker

To ensure the issue does not reoccur as we await the upstream fix, you should hold off on updating these packages temporarily. In Debian-like distributions, you can run:

sudo apt-mark hold docker-buildx-plugin docker-ce-cli docker-ce docker-ce-rootless-extras docker-compose-plugin

And on RHEL-alike distributions, you can install the versionlock plugin: DNF versionlock Plugin — dnf-plugins-core 4.4.2-1 documentation

Bitwarden will provide updates as it becomes available when we see a fix from Docker for this issue and normal updates can resume.

3

Hello Community,

I have downgraded Docker under Ubuntu 22.04 LTS as described. However, I still get an error 502 Bad Gateway from the NGINX server. What am I doing wrong?

bitwarden.sh version 2024.1.2
Docker version 24.0.7, build afdd53b
Docker Compose version v2.21.0
4

Hi,

bitwarden conatiners don’t play with firewalls and are not working out of the box on those setups. I had to do the following: in bwdata/docker/ create a file named docker-compose.override.yml with the following content:

version: '3'

services:
  mssql:
    networks:
      - default
      - public
  web:
    networks:
      - default
      - public

With that in place your “Bad Gateway” error should disappear. Hopefully bitwarden will do some config changes in the future so that such poor workarounds are not necessary anymore as bitwarden is my only compose app that needs such changes to run properly. The main problem seems to be that mssql and web containers do not expose their listening port to docker and docker creates no firewall rules for them.

5

Hi Tim,

Thank you very much for your quick help. Everything seems to be working so far.

6

@bw-admin Anything new on this topic?

7

@bw-admin Does the latest self-host release resolve this issue?

8

Any news on this issue? Is it safe to update Docker for an existing install?

9

yesterday i updated to docker version 25 and it did not work. sso and admin could not connect to the database. i had to revert to docker version 24

1 Like
10

@bw-admin Any progress here? Haven’t seen a github issue or something else.

11

On docker 25.0.4 the problem seems gone. No container gets unhealthy.

2 Likes
12

Confirmed for me it’s working too.