Problem

When running borgmatic create prune via the systemd service (borgmatic.service), the backup process hung indefinitely during the create phase after the log line Processing files … and the warning /var/lib/docker/containers/...-json.log: file changed while we backed it up.

The hang occurred only when the database hooks were active (mariadb_databases and sqlite_databases). These hooks automatically enable --read-special and cause borgmatic to include temporary dump files, which in turn triggered extensive special-file exclusions under /var/lib/docker. A broad exclusion - /var/lib/docker/** prevented the hang but also omitted all Docker volume data (persistent application files located under /var/lib/docker/volumes/).

Manual execution of borgmatic create from an interactive root shell completed successfully, confirming the issue was specific to the non-interactive systemd execution environment combined with Docker’s overlay2 layers, container logs, and runtime files.

Root Cause

• Database hooks force --read-special and interact with rapidly changing or special files inside /var/lib/docker.

• Borg’s file processing becomes blocked when encountering Docker overlay files, container log files that change during backup, and certain pseudo-files in the overlay2 directory.

• The broad Docker exclusion resolved the hang but removed the desired volume backups.

Solution

Retain the database hooks (they provide consistent, point-in-time dumps) and replace the broad Docker exclusion with granular exclusions that target only the ephemeral/problematic paths. Update the exclude_patterns section in /etc/borgmatic/config.yaml as follows (add/replace the Docker-related lines):

exclude_patterns:
...
  - /var/lib/docker/overlay2/**
  - /var/lib/docker/containers/**
  - /var/lib/docker/image/**
  - /var/lib/docker/tmp/**
  - /var/lib/docker/volumes/*/_data/*.log

More info about this issue in the docs: https://torsion.org/borgmatic/how-to/backup-your-databases/