utils.cache.yaml registered its own !vault constructor that returned a plain str, and it overrides the VaultScalar constructor of utils.handler.vault whenever it registers last. Every writer that loads a host_vars file through the cached loader, changes one key and dumps it back then re-emitted each credential as an untagged quoted string, and Ansible handed the $ANSIBLE_VAULT block itself to the roles.
08680e933d and c88647b3a1 moved tor_node.py and network_mode.py to the round-trip helpers. The shared loader now returns a VaultScalar, so every yaml_io writer keeps the tag regardless of import order. VaultScalar subclasses str, so the other readers of the cached loader see the same value.
test_a_round_trip_keeps_the_vault_tag fails against the previous constructor: re-registering the plain-str constructor on CSafeLoader and SafeLoader drops '!vault |' from the dumped file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
apply_network_mode_from_env loaded the whole host_vars file through
utils.cache.yaml, set one scalar and wrote everything back. That loader's
!vault constructor returns a plain string, so every credential in the file
was re-emitted untagged and Ansible handed the raw ciphertext to the roles
instead of the secret. It only runs when the row sets `network`, which is
every multi-network round.
Run 40032069040 on the fork shows what that costs: the generated host_vars
of a failing job carries 0 `!vault` tags against 56 `$ANSIBLE_VAULT`
headers, mariadb then refuses root with 1045 on a freshly initialised
database, and the openldap bind fails with result 49.
08680e933d fixed the same class in tor_node.py, the sibling writer in this
package. This one writes through the round-trip helpers too.
The existing test did not cover the case, which is why the sibling fix did
not stop it: test_preserves_existing_vault_credentials now fails against
the previous writer with
`'!vault |' not found in "... root_password: '$ANSIBLE_VAULT;1.1;AES256`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolves 68 conflicted files. The governing rule was our structure with their
network semantics: the tor boolean becomes a network mode (clearnet | tor |
multi) throughout the variant layer, while this branch's architecture and mesh
axes, its extracted modules and its i18n content stay.
Where the two sides covered different ground, both are kept rather than one
picked:
- ca_injected answers a ca_client on the deployment's TLS mode OR on an SSO
issuer that resolves self-signed. 3a3e9ed2c9 records why: an issuer is not
an object store, so keeping only one side drops the other's coverage.
- get_merged_domains keeps the loud refusal when svc-net-tor is deployed
without a provisioned node, now gated on their tor_serves_host; their
node_onion_of returns an empty string there, which would have served every
onion-exclusive app on clearnet in silence.
- sso_cookie_domains stays the single source for oauth2-proxy's cookie scope.
Their per-network branch omitted the issuer domain on the onion arm, which
breaks the callback leg.
- The clearnet CSP batch carries --clearnet-timeout again; their batch dict
hardcoded zero while the role still passes the flag.
- The dashboard keeps its i18n content dir, image, version and design
resolution; their side reduced the logo sources to the bare service
provider values.
Their side wins where it is measurably better: the onion mirror rewrite is
gated per vhost rather than per app, the Playwright onion decision reads the
pass's own domain instead of the role's canonical one, and the matomo
reinjection and dashboard config render per network sibling.
Eight files were broken by the merge without ever carrying a conflict marker,
because git marks only the lines both sides touched:
- utils/cache/domains.py lost the onionize block that sat between two hunks
- roles/sys-front-inj-all/templates/body_filter.lua.j2 called it
- web-app-bluesky, keycloak sso_proxy: the get_entity_name filter was renamed
- trigger/__main__.py, playwright_coverage.py and its test: the deleted tor
module
- cli/administration/deploy/ci/timings.py read Label.tor
- plugins/filter/csp_filters.py called _aligned_url with four of six arguments
- tests/unit/python/utils/cache/test_domains.py referenced a renamed class
- tests/.../sys-ctl-hlth-csp/files/test_script.py mocked a function main no
longer calls
Verified: the variant layer (155 tests, 65 subtests), ca_injected (18),
the CI layer (175), the domains cache including the restored guard (32),
retrigger inputs (9), timings and the CSP health script (53). make
quality-high clears autoformat; its remaining findings are tracked separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The file is written by the test itself, so the path-keyed read cache does
not fit; opt the read out of the uncached-read lint instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruamel folds every scalar and flow collection past 80 columns, so each
run of the provision, credentials or mesh CLI broke SSH keys and alias
lists in host_vars over several lines. round_trip_yaml() now sets an
unlimited width, and the credentials CLI and the services disabler use it
instead of building their own handler.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Provisioning merges into host_vars and never deletes, so the false
enabled/shared toggles that apply_services_disabled writes for a key stayed
in the round inventory after a later run dropped that key from disable=.
That covered the provider's own entry too: with seaweedfs once disabled,
web-app-seaweedfs kept shared: false, the service registry offered no
seaweedfs provider, container_networks attached no consumer, and BookWyrm
ran USE_S3=true against http://seaweedfs:8333 without a route.
reconcile_services_disabled_from_env records the keys of each disable run
in the host_vars key _INFINITO_SERVICES_DISABLED. On the next run it deletes,
in every application block, the enabled/shared toggles that are still
exactly false for a recorded key the current disable no longer lists, drops
a block left empty, and rewrites or removes the record. It runs right after
ensure_host_vars_file, before credentials and the variant bake, so a variant
that wants a toggle off writes it again. With an unchanged or empty disable
it does nothing. An inventory written before this change has no record, so
its older toggles are only reverted after one run with the old list.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`disable=lam` aborted the local deploy of web-app-openbao with "`disable`
conflicts with the current inventory state", and web-app-moodle carries the same
construction in two of its variants.
The writer and the conflict check disagreed on scope. apply_services_disabled
walked `find_roles_with_service`, the roles whose meta/services.yml declares the
key, so neither role was in that set for `lam` and nothing was written.
find_services_disabled_conflicts walks every deployed app's host_vars instead,
found the `lam` block the variant had baked, and refused the run. Dropping
`lam` from the list is not an answer either: the round then deploys web-app-lam
although ldap is off.
The writer now also flips a key it finds in host_vars under an app whose role
does not declare it, and only the flags that are already in that block. It never
creates a key or a flag, which is what `_declared_toggles` exists to prevent:
compare_application_keys rejects a host_vars key the role contract does not
carry. Overwriting a value the variant already wrote introduces nothing, so that
rule is untouched; test_skips_app_whose_role_does_not_define_service still
passes and pins it.
The alternative the TODO proposed, declaring `lam` in meta/services.yml with the
`'web-app-lam' in group_names` flags, is the worse trade: a consumer entry is
resolved through the service registry, so it would bind web-app-lam and its
whole closure into every round that carries openbao or moodle, for a key both
roles read only to gate a Playwright spec.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`config/main.yml` exists in no role: f65f4d60fb migrated the last one. A guard
against the string already exists, `TestNoLegacyPathReferences`, and it names
`config/main.yml` among its LEGACY_TOKENS. It scanned plugins, utils, cli,
roles, scripts, filter_plugins and lookup_plugins, which is every tree except
the two the stale references actually lived in.
So nine of them survived in `tests/`, and the guard reported green the whole
time. The code underneath was already correct everywhere: `test_csp_consistency`
and `test_role_wiring` iterate ROLE_FILE_META_SERVICES, and only their docstrings
and one assertion message still said `config/main.yml`. Had any of them globbed
the old path the test would have gone vacuous rather than red, which is the
failure mode worth naming here.
`tests` and `tasks` are now scanned. The targets come from
`utils/roles/mapping.py`, not from guesswork: `config/main.yml` is
`meta/services.yml`, `schema/main.yml` is `meta/secrets.yml` (confirmed at
`cli/administration/inventory/provision/reset.py:67`, which reads exactly that
file), and `users/main.yml` is `meta/users.yml`.
Scanning `tasks` exposed a second dead pointer. The migration tool became a
package on 2026-09-14, but EXEMPT_FILES still named `migrate_meta_layout.py`, a
file that no longer exists, so the exemption was inert and only invisible
because `tasks` went unscanned. Two dead pointers covering for each other. The
exemption is now a directory, so a new module in that package cannot fall out of
it. `test_no_direct_calls.py` named the same dead path in prose; that lint
allows through nocheck markers at the call site, so nothing was broken there.
`docs/administration/configuration/email.md` was the worst of them, because it
instructed operators rather than describing tests: it told them to write
`compose.services.email` into `roles/<application_id>/config/main.yml`. The
consumer, `plugins/lookup/email.py:255`, reads `entry["services"]["email"]` and
says in its own comment that there is no `compose.services` envelope. The role
snippet, the inventory snippet and the precedence list now match the reader.
`group_vars/all/TODO.md` pointed at the same dead file.
Verified by watching the guard fail, twice. Re-introducing `config/main.yml` in
a test file reports it; pointing the new directory exemption at a wrong path
reports `tasks/utils/migrate_meta_layout/credentials.py`, which proves both that
`tasks` is scanned and that the exemption is the only thing letting the
migration tool through. Restored, 11 passed.
The changelogs under `docs/changelog/` keep their mentions, and `.md` stays out
of the scanned suffixes for that reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every deploy row now takes a node network mode, clearnet, tor or multi,
instead of an onion yes or no. The workflow input `network`
(auto|clearnet|tor|multi) replaces `tor`, and INFINITO_NETWORK replaces
INFINITO_TOR.
utils/github/variant/network.py decides which modes a row can take: clearnet
only without a Tor bond or when the variant pins Tor off, no multi for a
single_mode role, no clearnet for the Tor provider, whose rows are therefore no
longer dropped when the rotation lands on clearnet. Priority rows cover every
mode and network combination, regular rows rotate the network on sweep // 2,
and clearnet rows carry disable=tor. Selection tokens pin the axis with
+clearnet, +tor or +multi, and multi rows wear the new glyph.
A row hands its mode to the deploy as the variable `network`, which
provisioning writes into host_vars as NETWORK_MODE. Without it the node keeps
deriving its mode from the deployed providers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`disable=` wrote both `enabled: false` and `shared: false` for every role that
declares the service, but a role qualifies with any one of `enabled`, `shared`
or `provides`. compare_application_keys rejects any host_vars key the role
contract does not carry, so a role declaring only `enabled` came back as a
validation error and the deploy never reached Ansible:
Missing default for web-svc-cdn: services.dashboard.shared
Missing default for web-svc-mirror: services.dashboard.shared
[ERROR] Inventory validation failed. Aborting deployment.
_declared_toggles reads the role's meta/services.yml and writes only what is
there; a role carrying neither toggle is skipped instead of gaining an empty
block. The module already held this principle: find_roles_with_service excludes
pure metadata entries so `disable` "doesn't add bogus enabled: false overrides",
and this applies the same rule per flag rather than per role.
Delta measured over the disable list the compose iteration procedure prescribes
(matomo, dashboard, prometheus, email, css): 407 of 409 role/service pairs are
untouched because they declare both toggles, exactly 2 stop receiving a
fabricated key, and 0 fall out of the disable entirely. The 2 are the pairs the
validator rejected. Across all 266 roles with a meta/services.yml, 1224 service
blocks declare `enabled`, 1190 declare `shared`, 34 declare `enabled` alone and
none declares `provides` alone, so nothing loses its disable.
Proven by the deploy sequence rather than by a test: the first local deploy
aborted at inventory validation, the second passed into the Ansible run with the
same disable list. Two regression tests cover both shapes, and the negative
control holds: hard-coding ["enabled", "shared"] fails the new test.
Without this, `disable=` is unusable in every local deploy, which the iteration
procedure requires on every run.
Residual: the generator was exercised through compose-deploy only, not through
roundtrip or swarm-zombie, which reach the same function by other call paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make quality-high pass 75 rewrote it: a concatenation whose right operand is a
multi-line literal gets parenthesised, with the operator leading its own line.
The string itself is unchanged, so
test_narrowing_the_credentials_still_rotates_every_user keeps the declaration
that separates web-app-keycloak from web-app-a and still fails when the user
half of the rotation is narrowed along with the credentials.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The matrix gate rotates credentials between the two deploy passes so PASS 2
has to carry fresh values. It rotated every application block in the host_vars
file, and on a matrix inventory that is not the round. Provision applies its
--include as a strict intersection (filters.py:57-62), so it generates
credentials for at most derive_includes, while mirror_overrides.py adds a block
per mirror artefact carrying only image and version. Rotating those costs one
credential subprocess each for values provision never wrote.
matrix.py now passes --include derive_includes(app_id, variants=round_variants),
the same call 02_provision_inventory.sh:11 feeds provision, from the same
round_variants the provision step got one call earlier. The rotation set is
therefore a superset of what provision generated: every database_password still
rotates, because utils/manager/inventory.py:236-242 writes it under an id from
that same filtered list. Measured for web-app-wordpress: 35 ids.
The scope stops at the credentials. reset_credentials takes credential_ids
beside application_ids because the two halves shared one list, and narrowing it
narrowed _drop_user_passwords too: that resolves through
required_user_policies(roles_dir, application_ids), so 60 of 66 declared users
would have stopped rotating, root, backup and every mcp-* bot among them. The
user half keeps the full document list and is byte-identical to before.
The test fixture declared the same users in both roles, which made the
regression test pass against the regression. web-app-keycloak now declares a
user web-app-a does not, so the assertion has something to bite on.
The CLI aborts when --include matches no block instead of rotating nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The shell-safe default drew from letters and digits only, and every user without declared accounts is an identity account, so kcadm set-password rejected hermes and openclaw with invalidPasswordMinSpecialCharsMessage as soon as keycloak entered the GitLab#1 round without LDAP (4164f37981) and the extras file stopped replacing the users map (03e42bae6d; run 33689701895). The generator now draws from letters, digits and -_. and guarantees upper case, lower case, a digit and one mark, never in the leading position.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
_drop_app_credentials walked applications[app].credentials, which
f713ba8350 moved to secrets.credentials. Nothing lives at the old address
any more - the run's inventory has twelve applications under
secrets.credentials and none above it - so the isinstance check never
matched and the function returned 0 every time. The rotation ran, reported
success and rotated no application secret at all.
It stayed hidden because the sibling half worked: _drop_user_passwords
reads document["users"], which did not move, so the total was never zero
and the call looked healthy.
That matters more than a dead branch usually would. Rotating between two
deploy passes is what turns the second pass into a test of the update
path, where a role has to push a new secret into the application and the
database it already created. With the drop silently skipped, every such
run has been re-deploying the values the first pass wrote.
The fixture pinned the pre-rename shape, so the test confirmed the bug
rather than catching it. Migrated to the real layout, with the address in
one helper instead of four assertions; against the old code it now fails
1 != 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
