A swarm job lost every published port of the mail stack without anything
reporting it. dockerd aborted the whole ingress program:
Failed to add ingress: failed to program ingress ports:
failed to bind host port 0.0.0.0:995/tcp: address already in use
/proc/net/tcp named the squatter: 10.100.1.1:995 -> 10.100.1.11:2049,
the node's own NFS client. The mount carries no noresvport, so the
kernel drew a source port from the 665-1023 reserved window and landed
on pop3s. Afterwards none of 25/465/993/995/4190/8095 was bound while a
sibling service's port was, and stalwart still reported 1/1 healthy
because its healthcheck runs inside the container. The e2e CLI probe
found it two hours later as ECONNREFUSED on SMTP 25.
993 and 995 are the only published ports repository-wide inside that
window, so this is luck of the draw on any NFS-backed swarm host that
runs the mail role.
insecure on the kernel export is not separate hygiene, it is the other
half: kernel nfsd defaults to secure and answers a mount from an
unprivileged source port with EACCES, so noresvport without it would
break every kernel-flavor mount. It reaches the kernel flavor only,
since templates/exports.j2 is reached from tasks/flavor/kernel/main.yml
alone. The ganesha flavor needs nothing: ganesha.conf.j2 sets no
Privileged_Port and the default already accepts unprivileged clients.
What it costs: the next deploy of every existing NFS client sees a
pending opts change and remounts the shared state. The role already
anticipates that, releasing DIR_BACKUPS first when the check-mode probe
reports a change. Nothing bind-mounts that path into a container, so
the exposure is a host-side job holding it at that instant, which fails
loudly rather than silently.
Not yet proven in effect: a source port is chosen when the RPC
connection is established, so a plain remount can leave a running mount
on its old reserved port until the next real mount. Reading
/proc/net/tcp after the first deploy on an existing client is what
confirms it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5