mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-08 03:38:23 +00:00
kevinveenbirkenbachandClaude Opus 5
2e1057e4cd
Since the swarm-NFS migration two chains copied the same bytes: svc-bkp-nfs-2-local
reads the export from the NFS server's own filesystem, while baudolo on the manager
copied the same files again THROUGH the NFS mount. That was a migration leftover -
svc-bkp-nfs-2-local arrived in adb8bcf69 (2026-07-10) and nobody taught the older role
to step back - and the role README already forbade it as a rule nobody enforced. In the
DR drill the duplicate costs three times over: the pulled tree, the device copy and the
restore root.
swarm_nfs_backed() in utils/storage/nfs.py is now the single answer to "does the swarm
NFS rewrite back this volume". plugins/filter/compose_volumes.py (which performs the
rewrite) and this role's backup_volume lookup (which feeds baudolo's
--volumes-no-backup-required) both call it, so the two cannot drift. On the current tree
that excludes 83 of 158 type: volume entries.
Four conditions keep the exclusion from ever dropping a volume nobody else captures:
* a manager-pinned role's volumes are not on the export (web-app-seaweedfs holds
Nextcloud's object store, deliberately node-local per its meta/services.yml)
* a role forcing another mode via compose_mode_force is not on the export either.
That override is per role (roles/sys-svc-compose/defaults/main.yml:3), so it is
resolved per role inside the predicate from roles/<id>/vars/main.yml, not read once
from ambient scope. web-app-bigbluebutton pins compose statically;
web-app-matrix computes it, and a value the predicate cannot resolve counts as
forcing - unknown mode keeps the volume backed up
* an entry opting out with nfs: false stays node-local
* the exclusion applies at all only when the svc-bkp-nfs-2-local inventory group is
populated. That role enters a play by group membership alone
(tasks/groups/svc-bkp-roles.yml:19-22) and svc-storage-nfs-server ships a variant
with nfs_backup: {enabled: false}, so without the gate an export whose repository
nobody deployed would lose 83 volumes from both capture paths
Every condition fails toward the old double capture: wasted space, never lost data.
Compose mode is untouched - the exclusion cannot fire without swarm and an nfs backend.
DEPLOYMENT_MODE and storage are templated through the lookup's templar, because
group_vars/all/18_swarm.yml:2 defines the mode as a Jinja expression; comparing the raw
source string would silently disable the whole exclusion.
Twelve new tests pin the contract, including the two failure modes that would be
invisible in production: an untemplated mode, and an exclusion without an export
repository. Both were verified by mutation - removing either guard makes its test fail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
99 lines
4.1 KiB
Python
99 lines
4.1 KiB
Python
from __future__ import annotations
|
|
|
|
import contextlib
|
|
from collections.abc import Mapping
|
|
from typing import Any
|
|
|
|
from ansible.plugins.loader import lookup_loader
|
|
from ansible.plugins.lookup import LookupBase
|
|
|
|
from utils.storage.nfs import swarm_nfs_backed
|
|
|
|
|
|
class LookupModule(LookupBase):
|
|
"""Docker volume names baudolo must not copy, for its
|
|
``--volumes-no-backup-required``.
|
|
|
|
Two sources feed the list: every ``meta/volumes.yml`` entry that declares
|
|
``backup: false``, and - in swarm mode with NFS storage - every volume the
|
|
swarm NFS rewrite backs with the shared export. The latter are captured by
|
|
``svc-bkp-nfs-2-local`` on the export host; copying them here again reads
|
|
the same bytes through the NFS mount, doubling the backup tree and racing
|
|
the export-side generation (see the role README's source-of-truth rule).
|
|
|
|
The cluster mode is read here; a role's own ``compose_mode_force`` override
|
|
is resolved per role inside the predicate, because this lookup answers for
|
|
every deployed role at once and one ambient value cannot speak for all.
|
|
|
|
The NFS exclusion only applies when ``svc-bkp-nfs-2-local`` is actually
|
|
deployed. It enters a play by inventory group membership alone, and
|
|
``svc-storage-nfs-server`` ships a variant that runs without it, so an
|
|
export whose repository nobody deployed must keep its volumes here rather
|
|
than lose them from both capture paths.
|
|
|
|
The emitted name is the entry's pinned ``name:``, which is what
|
|
``docker volume ls`` reports and what baudolo matches. Exclusion is
|
|
per volume, not per image: a container may legitimately hold one
|
|
derived tree next to the state that must be preserved.
|
|
|
|
Reads the live merged ``applications`` config, whose payload carries the
|
|
canonical volumes of every deployed role. Takes no terms.
|
|
"""
|
|
|
|
def run(
|
|
self,
|
|
terms: list[Any] | None,
|
|
variables: dict[str, Any] | None = None,
|
|
**kwargs: Any,
|
|
) -> list[list[str]]:
|
|
vars_ = variables or getattr(self._templar, "available_variables", {}) or {}
|
|
|
|
applications = lookup_loader.get(
|
|
"applications",
|
|
loader=self._loader,
|
|
templar=getattr(self, "_templar", None),
|
|
).run([], variables=vars_)[0]
|
|
|
|
templar = getattr(self, "_templar", None)
|
|
deployment_mode = vars_.get("DEPLOYMENT_MODE", "")
|
|
storage = vars_.get("storage")
|
|
if templar is not None:
|
|
with contextlib.suppress(Exception):
|
|
deployment_mode = templar.template(deployment_mode)
|
|
with contextlib.suppress(Exception):
|
|
storage = templar.template(storage)
|
|
deployment_mode = str(deployment_mode or "")
|
|
storage_backend = ""
|
|
if isinstance(storage, Mapping):
|
|
storage_backend = str(storage.get("backend") or "")
|
|
groups = vars_.get("groups")
|
|
export_repo = bool(
|
|
isinstance(groups, Mapping) and groups.get("svc-bkp-nfs-2-local")
|
|
)
|
|
|
|
names: set[str] = set()
|
|
if isinstance(applications, Mapping):
|
|
for app_id, app in applications.items():
|
|
volumes = app.get("volumes") if isinstance(app, Mapping) else None
|
|
if not isinstance(volumes, Mapping):
|
|
continue
|
|
for semantic_name, entry in volumes.items():
|
|
if not isinstance(entry, Mapping):
|
|
continue
|
|
excluded = entry.get("backup") is False or (
|
|
export_repo
|
|
and entry.get("type", "volume") == "volume"
|
|
and swarm_nfs_backed(
|
|
entry,
|
|
application_id=str(app_id),
|
|
deployment_mode=deployment_mode,
|
|
storage_backend=storage_backend,
|
|
)
|
|
)
|
|
if not excluded:
|
|
continue
|
|
name = entry.get("name") or semantic_name
|
|
if isinstance(name, str) and name:
|
|
names.add(name)
|
|
return [sorted(names)]
|