1f870b1e5 left two marks the layout rules reject. friendica's utils/ held
patch_config.yml beside patch_db_credentials.yml, which is a shared
delimiter-bounded prefix, and its numbered files ran 00, 01, 02, 04, 05 with
nothing at 03 after the config patch moved out of the sequence.
The prefix becomes the folder it was standing in for, the way 8e8dbbd08c did it
for the AI files. All four roles move, not only the one the lint could see: the
path post_deploy.yml resolves a role's declaration through has to read the same
everywhere, so a folder in one role and a prefix in the other three would be no
domain at all.
utils/patch_db_credentials.yml -> utils/patch/db_credentials.yml (4 roles)
utils/patch_config.yml -> utils/patch/config.yml (friendica)
The numbers close up behind the file that left: 04_addons and 05_user_bootstrap
become 03 and 04.
Four references follow: the include in 01_manager_ops.yml, the lookup in
post_deploy.yml, the caller path in the unit test that reads each role's
declaration, and a requirement doc that linked the old addon file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3831c8f804 gave matomo, moodle and nextcloud a patch each, called from their
own 00_core.yml. That file only runs while the role's run-once flag is unset,
so the patch runs on the deploy's first pass and never on the second -- which is
the only pass where a rotated credential exists. The app keeps the password it
was installed with while the engine moves on, and every request answers 500.
Run 33317483326 shows it on friendica, which had the same shape: svc-db-mariadb
altered the user at 15:39 and again at 16:26, either side of the 15:58 rotation,
while the app's config was patched once. The engine said what followed:
[Warning] Access denied for user 'friendica'@'192.168.201.4' (using password: YES)
Five failed healthchecks in a row, swarm replaced the task, and the convergence
gate reported "tasks not running yet" for a service that started correctly every
time.
The call moves to post_deploy.yml, which runs after every deploy and sits behind
no role's flag. It includes roles/<app>/tasks/utils/patch_db_credentials.yml when
that file exists, so a role declares what to patch and nothing else: friendica
joins the other three with its volume, its config path and its expression.
It cannot live in the role. 00_core.yml has to set the run-once flag as its very
first task or load/app.yml re-includes the role and recurses, which
tests/lint/ansible/roles/test_service_core_first_task_run_once.py enforces, and
everything after that flag is closed to a second invocation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The body was a Jinja-interpolated shell string inside the task. As a file it
can be read, shellchecked and run, and the three parameters travel through the
environment the way the password already did.
That forces one thing into the open. The old expression carried
${PATCH_PASSWORD} and the shell expanded it while assembling the command; a
value arriving in an environment variable is not expanded a second time, so the
substitution has to be explicit. The consumers now write @PASSWORD@ and the
script replaces it, which is also the version a reader can follow without
knowing that rule.
The script refuses to run without its parameters rather than treating an unset
one as empty, because an empty volume name would look exactly like a node that
does not hold the volume: a silent success on every host.
Tests move with it, to the files/ segment every other test of a shipped script
uses. They run it against a stubbed `container` in three states per consumer -
no volume, volume without the config, both present - and assert the marker the
task reads as a change. Five mutations of the script were checked against them:
dropping the substitution, the marker, the existence test, the parameter guards
and the volume filter each turn the suite red.
Layout follows what the linters already require: files/shell/ is enforced by
role-files-layout, and 65e34746fc names ansible.builtin.script as the sanctioned
way to run one from a task.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
matomo, moodle and nextcloud keep the database password inside a config
file on their data volume. When the deploy rotates credentials, the
container gets the new password while its persisted config still carries
the old one, and the app fails to authenticate against its own database.
Patch the persisted file before the health gate reads it. The value goes
through the environment rather than the command line, and sed_escape
guards the delimiter, backslash and ampersand that would otherwise corrupt
the file the replacement is meant to repair.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.
Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.
Freed from the binding, the names take the emoji prefix every other task line
already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An untyped inspect searches every docker object kind, and a role's overlay network carries the same entity name as its main container, so the call can resolve the network and report a missing State key instead of naming the container. That is how the openresty assert died on every swarm job of run 30709070050.
Extend container-exec-resolver to cover inspect, stripping raw markers and quoted spans first so the flag walker survives Go template format strings. Add container-inspect-type, scoped to roles/ because the collision follows the entity naming convention and the outer act namespace carries no role networks. Sweep 40 call sites; suppress the delegate_to-resolved CIDs and the compose co-location probes with a stated reason.
Move the five container rules into tests/lint/ansible/shell/container/ and drop the container_ prefix from their filenames. Register both markers in the suppression docs, where the resolver marker was missing entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three of the six failures triaged today were explainable only after downloading
a rescue artifact, and two not even then. The job log carried "not converged:
docker_docker", a bare "rc=1", and "failed after 3 attempt(s)".
The convergence gate now prints the task rows of a service that will not
converge - but only those whose error field is filled. A stack coming up
normally stays silent, while the one that killed svc-db-postgres would have
shown `error=Head "https://ghcr.io/...": request canceled` in the job log
instead of hiding it in an artifact. awk rather than grep: grep exits 1 when
nothing matches, which under set -e would take the script with it.
The remote pull decodes its exit status. os.system returns rc << 8, so the
message said 5888 where rsync said 23, and every attempt now names its code.
The sleep after the final attempt is gone - five wasted minutes per run.
The matomo poller hands its last 40 lines to stderr on deadline, redacted at
the source, and the failing task interpolates them the way the compose flavor
already does. No no_log: MASK_CREDENTIALS_IN_LOGS is force-set anyway and would
have censored exactly the payload this exists to deliver.
The `-p warning` filter added to trigger_units.sh a few commits ago is removed.
systemd journals a unit's stdout and stderr at info, so that filter would have
swallowed the per-provider attribution the same commit introduced.
stack_ready.sh gets its first tests: replica comparison, the completed-oneshot
branch, and both halves of the new reporting - the error reaches the caller, the
error-free rows do not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The reset introduced in 30435a625 re-probed the live container for
config.ini.php and skipped itself whenever one was present. A failed first
attempt leaves exactly that file behind, so the reset vetoed itself in the very
case it exists for, and because both flavors share the task, the veto also
disabled the swarm reset that had been working since e46659ac4.
Decide reachability and prior installation separately. The install probe in
03_install_path.yml now echoes an explicit marker instead of relying on an exit
code, and no longer swallows failures, so an unreachable container fails the
play rather than reading as a fresh install. The reset goes back to being
ungated: between that snapshot and a retry, only our own bootstrap writes, so
whatever is there belongs to the attempt being discarded.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The compose flavor retried the one-shot installer up to eight times without
touching what a failed attempt left behind. Once an attempt had half-installed
Matomo, every retry met the existing-tables dialog, erased the tables and landed
on the CoreUpdater screen, which the bootstrap image cannot drive: 41 minutes of
retries against an application that had in fact been installed since attempt 2.
Move the swarm flavor's pre-retry cleanup into tasks/utils/reset_state.yml and
drive it from both flavors, so every retry starts from a clean slate and takes
the clean-install path instead of the erase-and-wedge one.
The reset re-probes the live container before it touches anything. The caller's
gate is a snapshot taken before the attempt loop and reads an unreachable
container as "not installed", which would otherwise put a DROP DATABASE within
reach of a live installation; a probe that cannot run now fails the play.
Collapse MATOMO_BOOTSTRAP_RETRIES and MATOMO_BOOTSTRAP_SWARM_ATTEMPTS into a
single MATOMO_BOOTSTRAP_ATTEMPTS, and surface the failed attempt's stderr in the
final failure message, which the attempt loop would otherwise swallow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A shared 'resolve_' prefix on two sibling task files trips the
prefix-grouped-files lint (a delimiter-bounded prefix is a folder in
disguise). Move both into utils/resolve/ and drop the prefix:
resolve_config_host.yml -> resolve/config_host.yml,
resolve_host_cid.yml -> resolve/host_cid.yml. Update the include paths in
02_manager_ops, 03_install_path, 05_patch_database, 06_patch_trusted_hosts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swarm can replace the matomo task mid-patch; a container id resolved earlier
then points at a dead container and the config patch fails with
"container ... is not running" (observed in the swarm CI matrix deploy).
Patch config.ini.php directly on the matomo_data volume (state NFS export is
no_root_squash on both kernel and ganesha flavors), so the DB-credential (05)
and trusted-hosts (06) writes no longer depend on container liveness. Resolve
the host bind path once via a shared utils/resolve_config_host.yml.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sort the new INFINITO_SWARM_* IP/subnet keys alphabetically in default.env, wire them through the static passthrough handler so make dotenv materialises them, and shorten the matomo DB-reset task name under the 80-char limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Matomo's swarm bootstrap failed rc=1: the high-churn tmp/templates_c Twig cache lived on the soft NFS state share, so a single mid-write RPC timeout surfaced as EIO and stalled the installer 300s; the DB was already seeded, so retries took the eraseAllTables path that wedged Matomo on the CoreUpdater screen the bootstrap image cannot click through. Mount a node-local matomo_tmp volume (nfs: false) for /var/www/html/tmp so caches never traverse NFS, and drop+recreate the app's own database in the pre-retry cleanup (via a container-run reset_database.php over the app credentials, gated exactly like the config-aside task on attempt>0 + no installed config) so every retry takes the clean-install path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The container healthcheck's first page views make the installer write a
stub config before the install check runs, so the check reported
'installed', the retry clean-slate step (gated on the missing-config rc)
never fired, and every bootstrap retry inherited the previous attempt's
partial state until Matomo wedged in the CoreUpdater redirect (3x420s,
whole lab red). Installed now means the config carries a [database]
section, and the retry move-aside fires on any non-zero pre-check rc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace scripts/system/diagnostics/rescue.sh with utils/diagnostics/
container.py: one workflow call on failure captures the current level
(containers with logs+inspect, swarm services, journal, host resources,
in-play role dumps) and recurses itself through every DiD boundary via
self-copy, from the runner down to the innermost runtime (RESCUE_MAX_DEPTH).
compose and swarm now share an identical failure step; the swarm per-node
collector wrapper is gone. All 17 script-only Ansible rescue: blocks are
deleted and their bare block: wrappers inlined (plus a wordpress leftover);
the three justified blocks (postgres pg_hba, xwiki extensions, keycloak
cross-node recovery) keep their evidence tasks and now end in a fail task.
Two lints enforce the state: rescue: is forbidden without a
'# nocheck: rescue-<reason>' first-task marker, and a block without
rescue/always/when or another scoping key must be inlined.
sys-service-terminator hoists become to the include apply. Also move
routine/08_cleanup.sh to utils/clean/teardown.sh and topology/ to
utils/topology/ so routine/ holds only the sequenced flow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every Ansible rescue: block used to carry ad-hoc per-role failure dumps that
drifted apart and captured different subsets. Replace them all with one
diagnostics script (scripts/system/diagnostics/rescue.sh) that snapshots
containers (full logs plus inspect per container), swarm services (ps plus
logs per service), journal, host resources, runtime info and stats into a
per-run folder and always exits 1 so the deploy still fails. A new lint
enforces the shape: every rescue ends with the script, at most one
nocheck-justified extra task before it (postgres pg_hba dump, xwiki
extension-repo probe), and a nocheck rescue-recovery exemption for the one
block that must genuinely recover: keycloak object updates, where a
cross-node reschedule needs a swarm-wide node re-resolve plus a delegate_to
change that only a rescue can express; its recovery is kept and now also
captures diagnostics before the terminal failure.
The snapshot folder path is a group_vars SPOT: DIR_RESCUE_DIAGNOSTICS and
DIR_VAR_LIB live in group_vars/all/05_paths.yml as plain strings; new env
handlers derive INFINITO_RESCUE_DIAGNOSTICS_DIR and INFINITO_DIR_VAR_LIB
from it for the generated .env, replacing the default.env entries, and the
play-level environment forwards the value to every task host so rescue.sh
needs no second default. CI uploads the folder like the inventory and
playwright artifacts: compose copies it out of the deploy container in
dedicated.sh, swarm captures per node (manager and both workers) into
node-named subfolders.
Deleting the ad-hoc dumps orphaned helper files (postgres inspect_container,
terminator docker_diagnostics, peertube diagnostics, matomo bootstrap debug,
xwiki and openresty rescue includes) and 14 role vars; all removed. Generic
pieces of those dumps (runtime version and info, stats, dmesg oom evidence,
swapon) moved into the SPOT script instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A flaked installer that created the superuser and site but never reached
the finished step leaves Matomo redirecting every request to CoreUpdater,
so the sidecar probe never passes. The old until-retry only force-restarted
the sidecar between attempts and never reset that partial state, so all
three attempts failed identically and the job died after ~29 minutes
(everything depending on matomo hung with it).
Drive the attempts through a loop over swarm_attempt.yml that, before each
retry, moves the partial config aside (config.ini.php.wedged-<ts>, never
deleted) and clears the cache so the installer runs fresh; the eraseAllTables
step handles leftover tables. The move is gated to the config-absent-at-entry
path and skips the whole attempt once a token is obtained, so a healthy
install is never touched. bootstrap_swarm_poll.sh anchors its token scrape
with --since so a token still buffered from the previous task is not returned
before the restarted installer emits its own, and surfaces a failed move
instead of masking it. The swarm abort message reports the swarm attempt
count.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The container-exec-resolver lint only saw the inline shell form, so the
`command: argv: [container, exec, <name>]` list form (two separate lines)
slipped through, which is how checkmk's bare name reached swarm. Extend the
scanner to parse argv sequences via yaml.compose_all with node line numbers,
and annotate the nine legitimate bare-name execs (mariadb recovery container,
matomo compose-only debug, probe-gated nextcloud partner provisioning, pixelfed
runtime-resolved CID, wordpress launcher-managed discourse) with
# nocheck: container-exec-resolver.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The consumer re-render loop included sys-front-inj-all under web-svc-* application_ids in the middle of the matomo pipeline; the shared sys-svc-compose handlers rebind to the most recent role instantiation, so matomo's compose-up notifies from 06/07 flushed against a consumer's instance dir (/opt/compose/logout) that does not exist when that service is outside the play's whitelist. Move the loop into 10_reinject_consumers.yml as the last step after 08_config, behind the final flush.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a 'timeout' lookup plugin that returns round(base * TIMEOUT_FACTOR)
(group_vars default 1, so value-identical to the old literals). One knob
in group_vars stretches every wait in the stack for a slow uplink:
TIMEOUT_FACTOR: 2 doubles them all.
- plugins/lookup/timeout.py + unit tests (identity, factor scaling,
string coercion, error paths).
- group_vars/all/17_resource.yml: TIMEOUT_FACTOR: 1.
- Convert all 34 literal 'timeout:' task keywords / module args across 25
task+handler files to "{{ lookup('timeout', <base>) | int }}".
- tests/lint/repository/yaml/test_timeout_uses_lookup.py enforces the
lookup in every task/handler timeout (opt-out: # nocheck: timeout-lookup).
- swarm.yml pre-deploy build handler: add the missing timeout (same
network-build hang vector as compose.yml's Build compose).
- Drop 13 pre-existing narration comments in the touched files (comment
lint enforces whole-file on any edited file).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The re-injection loop stat-checked each web-svc consumer's compose instance dir,
which proxy-only vhosts (web-svc-html) never have, so they were skipped and kept
a placeholder matomo token, failing the Playwright tracker assertion. Stat the
per-consumer body_filter Lua that sys-front-inj-all renders (the identical
lookup used to write it), so every deployed+injected consumer is re-rendered
with the real token. compose-dir-exists implies lua-exists for all six
candidates, so no consumer that was re-injected before is now skipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
chown the config dir to www-data before the swarm installer run so the PHP writability checks pass, and raise the healthcheck start_period to 180s to cover the slower swarm/NFS first boot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:
[playbook_dir, '<rel>'] | path_join -> lookup('path_absolute', '<rel>')
playbook_dir ~ '/<rel>' -> lookup('path_absolute', '<rel>')
{{ playbook_dir }}/<rel> -> lookup('path_absolute', '<rel>')
Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.
Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.
Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The matomo bootstrap re-injects early-rendered web-svc consumers (such as
web-svc-logout) after obtaining the admin token, but in swarm this never ran:
roles_with_service filters by the manager's group_names, which omits
worker-placed web-svc services, so the re-injection loop was empty and the
logout page shipped without the matomo tracker.
- roles_with_service: add a scope kwarg; scope='all' bypasses the per-host
group_names filter (the merged applications config is already
deployment-scoped, so iterating all of it equals the deployment's consumers).
- 05_bootstrap: enumerate consumers with scope='all', stat each compose project
dir, and re-inject only deployed ones. Avoids a fatal chdir into an undeployed
consumer's missing /opt/compose/<entity>/ that scope='all' would otherwise hit.
- sys-front-inj-matomo: resolve matomo_effective_token (override else
lookup('users')); the disable-gate and the SitesManager getSitesId/addSite
calls use it, so the re-injection passes the fresh bootstrap token and bypasses
the play-start-stale controller-side users cache that left matomo disabled.
scope defaults to 'host' and the override defaults to empty, so every existing
caller and the per-consumer injection path are unchanged. Validated green in
both compose and swarm round 1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
enable_framed_pages=1 disables Matomo's JS frame-buster so the dashboard can iframe-embed it (CSP frame-ancestors still gates who may frame). Shared web-svc consumers (logout, cdn, file) render body_filter.lua in the earlier web-svc service-loader bucket, before web-app-matomo writes its API token, so the tracker is gated off and never re-rendered; after the token is persisted, re-run sys-front-inj-all for each web-svc matomo consumer so the tracker lands. The loop is restricted to web-svc-* because re-running it for a web-app consumer outside its own play would dereference unloaded role vars and crash; web-svc roles declare no javascript injection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the per-role swarm host-node/container-id resolve duplicated across fider, matomo, decidim, keycloak (login + realm-import), collabora and listmonk with two shared task utils under roles/sys-svc-compose/tasks/utils/swarm/. resolve_host_cid sets <prefix>_HOST_NODE / <prefix>_LOCAL_CID (swarm: converge + node + cid delegated to the node; compose: inventory host + given cid); wait_converged is the bare convergence gate.
Teach test_variable_definitions to read templated set_fact keys like {{ prefix }}_HOST_NODE as a *_HOST_NODE name pattern (guarded so a pure {{ x }} key cannot become a catch-all), and add the playbook_dir-relative include load-failure exemptions to .ansible-lint-ignore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DEPLOYMENT_MODE is binary (swarm / compose), so a negated `!= 'swarm'` gate
always means `== 'compose'`. Replace every negated gate across roles/ and
group_vars with the positive form for readability, and add a lint
(tests/lint/ansible/variables/test_no_deployment_mode_negation.py) that forbids
`DEPLOYMENT_MODE !=` in .yml/.yaml/.j2 with a `# nocheck:
deployment-mode-negation` escape.
The compose-verb / chdir / jinja-gate lint guidance messages now suggest the
positive form too; the gate matcher (utils/annotations/task_gate.py) and its
test inputs keep recognising the legacy `!= 'swarm'` so a nocheck-marked
exception is still detected as compose-only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Swarm bootstrap fix: the single 'container service update --force' plus one-shot log-poll aborted on a flaky installer (restart_policy:no never re-runs it). Merge force-update and token-poll into one retried block (each attempt re-forces a fresh installer task, 2-3 attempts); the poll loop lives in files/bootstrap_swarm_poll.sh (script module, vars passed as env).
Restructure: split the bootstrap into mode flavors (05_bootstrap/flavors/{swarm,compose}.yml) and per-mode debug (05_bootstrap/debug/{swarm,compose}.yml), both selected by one DEPLOYMENT_MODE-driven include; renumber 07-10 to 06-09. Swarm-only behavior changes; compose paths unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The OpenResty redeploy handler ('remounts bind to live inode') fires once per app; without --detach each call blocked until the full rolling convergence, so the per-app blocks accumulated into an apparent hang. openresty actually converges (clean Shutdown->Running, old replica keeps serving during the rolling update, readiness is gated separately by the converge-wait), so add --detach to return immediately. Also add timeout: 600 (10 min) to that handler and to the three single-fire app service-update-force tasks (baserow post-migration restart, matomo install restart + bootstrap re-run) as a hard cap: a genuinely stuck redeploy now fails cleanly instead of hanging forever. 600s comfortably exceeds the normal rolling convergence (~2-5 min, matomo healthcheck start_period 60s x replicas) and matches the existing timeout: 600 already used elsewhere in baserow; the three keep their blocking so the next step still gets the service ready.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
matomo: force-restart the swarm service after patching config.ini.php DB creds (the compose-up handler is a no-op in swarm), extracted into 10_resolve_host_cid. nextcloud: gate the redis PONG wait on the engine being local. prometheus: config 0444 (swarm config objects are root) and gate the host-monitor binds plus privileged cadvisor/node-exporter to compose only. dashboard: probe the public icon URL in swarm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Matomo image entrypoint re-owns config.ini.php to root on every container start, so the post-sed chown in 07_patch_database (which runs before 04 flushes handlers and the stack redeploys Matomo) is overwritten by the recreate, leaving the config root-owned when 09 runs its API config pass (HTTP 500, config not writable). Move the chown to 04 after the post-recreate health probe and before the API pass, so it lands after the last config-rooting restart and persists through 09.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
07_patch_database execs into the Matomo container to sed its config.ini.php. Two swarm fixes: (1) target MATOMO_LOCAL_CID (resolved via container ps) instead of container_address, whose swarm form expands to the manager-only /usr/bin/resolve-container-id that is absent on the worker MATOMO_HOST_NODE this task delegates to (No such file); same nocheck pattern as 04. (2) sed -i --user root re-owns config.ini.php to root:root, so Matomo's www-data web user can no longer write it and the config API returns 500 (config not writable); chown it back to www-data after patching. The unused MATOMO_APP_CONTAINER_ADDRESS var is dropped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In swarm the in-stack bootstrap service is deployed with the stack and races Matomo: it probes http://matomo, accepts the early 404 as reachable and drives the Playwright installer against a not-yet-ready Matomo, so the installer UI never becomes interactive and no token is emitted. Wait for Matomo to be healthy, force a fresh bootstrap task so the installer runs against a 200-serving Matomo, and strip the per-line service-log prefix before matching the token. Swarm path extracted into 05_bootstrap_swarm.yml, included only when DEPLOYMENT_MODE == swarm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both the compose and the swarm bootstrap tasks registered the same matomo_bootstrap_run. In compose mode the skipped swarm task overwrote the variable with a skipped result (no stdout), so the token parser read an empty string and the assert failed. Register each task into its own variable (compose/swarm) and pick the mode-appropriate one when parsing the token and in the rescue diagnostics.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tests/lint/ansible/tasks/test_include_uniform_apply_keyword.py: when a file pulled in via a dynamic include_tasks carries the same hoistable task keyword on every top-level task, that keyword should be set once via apply: on the include site instead of being repeated per task. The rule is blacklist-based: it flags any uniform Ansible task keyword except the per-task-binding ones (name, register, args, vars, when, loop), and never flags the module itself.
Hoist every existing occurrence to apply: across the affected roles (delegate_to, run_once, become, notify, no_log, changed_when, delay, async, poll), dropping the per-task duplication. Behaviour is identical because apply: propagates each keyword to every included task; no_log masking via apply: was empirically verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Long-string continuation breaks introduced by `make autoformat` on
the friendica addon-sync script command and the matomo bootstrap
failure message. Pure cosmetic, no semantic change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Splits the matomo install path into compose-mode and swarm-mode
manager-side flows so the role works under both deployment models:
- 03_manager_ops.yml: dual `when: DEPLOYMENT_MODE == 'swarm'` /
`\!= 'swarm'` blocks discover the matomo container differently
(`container service ps` for swarm to wait for task convergence +
resolve the hosting node; `container inspect` for compose). Both
blocks set MATOMO_HOST_NODE so the downstream steps `delegate_to:`
the correct node. Container-ID resolution then takes the right
branch via an inline Jinja toggle.
- 05_bootstrap.yml: wraps the compose and swarm bootstrap-launch
tasks in a single `block:` with a paired `rescue:` that loads
06_bootstrap_debug.yml (only under MODE_DEBUG) and then hard-fails
with the captured rc/stdout/stderr from the failed run. Drops the
redundant `| default('')` on `matomo_bootstrap_run.stdout` inside
the swarm `until:` (the lint catches it; the attribute is always
set on a registered task).
- 04_install_path.yml: removes the stale `when: matomo_token_value …`
gate on the api-config include that referenced the deleted
silent-skip rescue. 09_config.yml now runs unconditionally; the
legitimate token-shape assertions live in 05_bootstrap.yml.
- files/patch_trusted_hosts.sh: adds the missing `#\!/bin/sh` shebang
so lint-shellcheck stops flagging SC2148.
Plus a lint bugfix:
- tests/lint/ansible/structure/test_no_oversized_ansible_block.py
was counting `rescue:` / `always:` siblings as direct block
children, double-counting them under the >3 limit. Walker now
stops at `rescue:`/`always:` keys aligned with the `block:`
keyword (each section has its own child list), and computes the
terminator column from the `block:` keyword itself instead of the
regex-captured indent (which is 0 for the `- block:` form because
the dash absorbs into the prefix group).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Refactors four roles that previously used `# nocheck: compose-chdir-in-task` /
`compose-verb-in-task` / `container-exec-resolver` markers to follow the
canonical swarm-aware pattern:
- web-app-bigbluebutton: adds BBB_POSTGRES_CONTAINER_ADDRESS /
BBB_GREENLIGHT_CONTAINER_ADDRESS vars; 02a_postgres_password_sync.yml +
03_administrator.yml use `container exec -i {{ ..._CONTAINER_ADDRESS | quote }}`;
02_compose.yml's compose-build-phase tasks (generate-compose +
prebuild etherpad) move into the new
02b_bootstrap_compose_project.yml (>3 entries cannot live in a block)
under a single `when: DEPLOYMENT_MODE \!= "swarm"` include guard.
- web-app-baserow: BASEROW_APP_CONTAINER_ADDRESS var + 02_manager_ops.yml
converted to container exec + container restart canonical pattern.
- web-app-matomo: MATOMO_APP_CONTAINER_ADDRESS var; 07_patch_database.yml
swaps MATOMO_LOCAL_CID for the proper resolver-backed address.
- web-app-matrix: 01_docker.yml's two consecutive `compose pull` /
`compose up` tasks extract into 03_compose_pull_and_up.yml (with the new
`# include-gated: when: DEPLOYMENT_MODE \!= "swarm"` header marker) and
the previous 03_webserver.yml renames to 04_webserver.yml to keep the
numeric file sequence contiguous.
Supporting lint infrastructure changes so file-level include-gated
sub-task lists survive the swarm-compat lints:
- utils/annotations/task_gate.py: `is_task_compose_only_gated` now walks
up nested parent blocks (a single `when: DEPLOYMENT_MODE \!= 'swarm'`
on `- block:` exempts every child task underneath) and a new
`is_file_compose_only_by_header` recognises files declaring
`# include-gated: when: DEPLOYMENT_MODE \!= "swarm"` in their first
five lines as compose-only-by-include.
- tests/lint/ansible/shell/test_no_compose_chdir_in_shell_tasks.py and
test_no_compose_verb_in_shell_tasks.py call the header helper before
the per-line scan, so include-only sub-files are exempt.
- tests/unit/utils/annotations/test_task_gate.py covers both new
behaviours (recursive parent walk plus file header) end-to-end.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add a sibling lookup `container_service` (with 12 unit tests) that
returns the addressable form of a service for container service ps,
container service logs, and container ps --filter
label=com.docker.swarm.service.name=... It mirrors `container_address`
but emits the swarm service name (<stack>_<service_key>) instead of
the runtime container ID subshell. Compose mode returns the bare
service name.
Make the matomo manager-side ops swarm-aware without pinning matomo
to the manager. 03_manager_ops splits the wait into two mode-specific
blocks: in swarm a container service ps loop on CurrentState, in
compose the existing container inspect. After convergence the swarm
block resolves the hosting node via container service ps Node and the
compose block sets it to inventory_hostname; both set MATOMO_HOST_NODE
via separate set_fact tasks so the undefined matomo_swarm_node_query
under compose never reaches Jinja. A second unconditional task
resolves the local container ID on the hosting node (compose: bare
name; swarm: container ps with the swarm service-name label filter)
into MATOMO_LOCAL_CID. Downstream 04_install_path, 07_patch_database,
08_patch_trusted_hosts and the health wait all delegate_to
MATOMO_HOST_NODE and use MATOMO_LOCAL_CID for container exec, so the
patches and inspects always hit a local container. 05_bootstrap keeps
the compose-only profile run and adds a swarm path that tails the
in-stack matomo_bootstrap service logs for a hex token, registering
the same matomo_bootstrap_run shape downstream. 09_config delegates
to STACK_HOST with run_once so the uri_retry hits the published
matomo port from the manager regardless of where matomo is scheduled.
Drop the now-unused MATOMO_CONTAINER_ADDRESS from vars/main.yml.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Renumber the tasks/ tree to a flat NN_ scheme (drop the 02b / 03a / 03b
suffixes) and update all include_tasks references; main.yml -> 01_core
stays unchanged. Lift the hardcoded matomo bootstrap timezone label
into a global HOST_TIMEZONE_LABEL constant in group_vars/all so other
apps can share it; default to "United Kingdom" because GB has a single
IANA timezone and matomo's getTimezoneName collapses to the country
name in that case. Drop unconditional double quotes from env.j2 and
bootstrap.env.j2 since docker stack deploy preserves env-file quotes
literally and was feeding matomo "matomo" instead of matomo as the
DB name. Add MATOMO_ADMIN_EMAIL via the users lookup so the bootstrap
template can reference a typed variable instead of constructing the
address inline. Switch 08_patch_trusted_hosts to ansible.builtin.shell
with the script body piped in via stdin from lookup('file', ...) and
shrink the script file to just the inner sh body. Strip section-banner
and restate-the-code comments from vars/main.yml, env.j2 and the
extracted script; keep the two genuine trip-wire comments (matrix
deploy round-2 DB creds, include_role-chain role_vars pollution) in a
shorter Exception form.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a mode-aware `container_address` lookup plugin and a
`resolve-container-id` helper script that resolve swarm task IDs at
exec time (compose mode returns bare names unchanged). Migrates ~25
roles + ~28 task files from `command:` to `shell:` so the lookup's
`$(...)` subshell evaluates correctly, and extends `database` lookup
with an `address` key for cross-role DB exec.
Three new lint tests enforce the contract: every container exec
target goes through the resolver, lives behind an IS_STACK_HOST gate
(walks the include graph one hop deep), and uses `shell:` rather than
`command:` when the target is a Jinja reference. Existing nocheck
suppressions on bare-name DB exec sites (nextcloud redis, listmonk,
moodle OIDC, mailu) are replaced with the lookup-based form.
Fixes the systemd Environment= injection in sys-ctl-cln-disc-space
(it never expanded $(...)) by passing STACK / SERVICE_KEY parts and
resolving inside the script. Extracts shopware admin and mediawiki
extension install bodies into dedicated .sh files invoked via env.
Rewrites password-bearing shell tasks to either `| quote` directly or
the `-e VAR=... | quote` + `sh -c '... ${VAR} ...'` env-bridge pattern.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds MODE_ASSERT_LOCAL to group_vars/all/01_modes.yml:
MODE_ASSERT_LOCAL: "{{ (MODE_ASSERT | bool) and (IS_STACK_HOST | bool) }}"
The mailu preload-assertion in sys-svc-mail/tasks/01_core.yml previously
fired on every host in `'web-app-mailu' in group_names` with no
MODE_ASSERT gate at all, so swarm workers ran into the assertion every
single CI run even though the preload state lives only on the stack
host. Same shape applies to a dozen other assertions: sys-svc-rdbms
(already double-gated, now collapsed), keycloak's kcadm-driven LDAP /
RBAC / reCAPTCHA / groups-path asserts, the sys-svc-compose handler
validators for compose/swarm config, svc-prx-openresty's 02_assert,
sys-svc-webserver-core's 05_assert, web-app-matomo's bootstrap token
check, web-app-joomla's 05_assert, and the sys-ctl-alm-compose notifier
self-start.
Each asserted state (run_once_<role> flags, kcadm output, openresty
container probes, compose-tree files) lives only on IS_STACK_HOST, so
the new SPOT keeps the assertion semantics intact while skipping the
spurious worker-side failures.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Across roles/web-app-*, the typical body shape is:
- include_role: sys-stk-full (or sys-svc-compose) cluster prereqs
- <host-fs writes, container exec, compose exec> stack-host only
Under swarm-mode delegation the second group only makes sense on the
host that holds the compose stack (manager), but the role plays on
every host in groups[<role>] so workers abort on missing paths.
Two workflow sweeps (gate-manager-ops + gate-manager-ops-01core, 58
agents total, discover/plan/adversarial-verify per role) plus three
manual fixes surfaced 27 roles to refactor. Each role keeps
sys-stk-full ungated so worker-side prereqs (sys-svc-container, NFS
client) still bootstrap; the trailing manager-only block either moves
into a new tasks/02_manager_ops.yml included with
when: IS_STACK_HOST | bool (SUB_FILE pattern) or gets the gate
inlined on the one or two trailing tasks (INLINE pattern). In
compose mode IS_STACK_HOST stays true so single-host deploys are
unchanged.
Manual catches the workflow agents missed:
- web-app-fediwall: utils/once/flag.yml lives AT THE END of 01_core,
not at the start; preserved verbatim outside the gate so workers
still flip the run_once_<role> fact.
- web-app-xwiki: a pre-existing block(...) wraps sys-stk-full plus
_flush_config.yml; gated only the _flush_config.yml include so
sys-stk-full keeps running on workers.
- web-app-dashboard: 02_javascript.yml (creates /opt/compose/dashboard/
JS dirs) ran BEFORE sys-stk-full and was ungated; added the gate.
Roles touched (INLINE): akaunting, jenkins, magento, mobilizon
Roles touched (SUB_FILE): baserow, dashboard, decidim, erpnext,
fediwall, friendica, gitea, jitsi, keycloak, listmonk, littlejs,
mailu, matomo, moodle, odoo, openproject, opentalk, pixelfed,
shopware, suitecrm, taiga, xwiki, zammad
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.
Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).
Breaking change for any external override that still sets the old
name — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
