`disable=` wrote both `enabled: false` and `shared: false` for every role that
declares the service, but a role qualifies with any one of `enabled`, `shared`
or `provides`. compare_application_keys rejects any host_vars key the role
contract does not carry, so a role declaring only `enabled` came back as a
validation error and the deploy never reached Ansible:
Missing default for web-svc-cdn: services.dashboard.shared
Missing default for web-svc-mirror: services.dashboard.shared
[ERROR] Inventory validation failed. Aborting deployment.
_declared_toggles reads the role's meta/services.yml and writes only what is
there; a role carrying neither toggle is skipped instead of gaining an empty
block. The module already held this principle: find_roles_with_service excludes
pure metadata entries so `disable` "doesn't add bogus enabled: false overrides",
and this applies the same rule per flag rather than per role.
Delta measured over the disable list the compose iteration procedure prescribes
(matomo, dashboard, prometheus, email, css): 407 of 409 role/service pairs are
untouched because they declare both toggles, exactly 2 stop receiving a
fabricated key, and 0 fall out of the disable entirely. The 2 are the pairs the
validator rejected. Across all 266 roles with a meta/services.yml, 1224 service
blocks declare `enabled`, 1190 declare `shared`, 34 declare `enabled` alone and
none declares `provides` alone, so nothing loses its disable.
Proven by the deploy sequence rather than by a test: the first local deploy
aborted at inventory validation, the second passed into the Ansible run with the
same disable list. Two regression tests cover both shapes, and the negative
control holds: hard-coding ["enabled", "shared"] fails the new test.
Without this, `disable=` is unusable in every local deploy, which the iteration
procedure requires on every run.
Residual: the generator was exercised through compose-deploy only, not through
roundtrip or swarm-zombie, which reach the same function by other call paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The matrix gate rotates credentials between the two deploy passes so PASS 2
has to carry fresh values. It rotated every application block in the host_vars
file, and on a matrix inventory that is not the round. Provision applies its
--include as a strict intersection (filters.py:57-62), so it generates
credentials for at most derive_includes, while mirror_overrides.py adds a block
per mirror artefact carrying only image and version. Rotating those costs one
credential subprocess each for values provision never wrote.
matrix.py now passes --include derive_includes(app_id, variants=round_variants),
the same call 02_provision_inventory.sh:11 feeds provision, from the same
round_variants the provision step got one call earlier. The rotation set is
therefore a superset of what provision generated: every database_password still
rotates, because utils/manager/inventory.py:236-242 writes it under an id from
that same filtered list. Measured for web-app-wordpress: 35 ids.
The scope stops at the credentials. reset_credentials takes credential_ids
beside application_ids because the two halves shared one list, and narrowing it
narrowed _drop_user_passwords too: that resolves through
required_user_policies(roles_dir, application_ids), so 60 of 66 declared users
would have stopped rotating, root, backup and every mcp-* bot among them. The
user half keeps the full document list and is byte-identical to before.
The test fixture declared the same users in both roles, which made the
regression test pass against the regression. web-app-keycloak now declares a
user web-app-a does not, so the assertion has something to bite on.
The CLI aborts when --include matches no block instead of rotating nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The shell-safe default drew from letters and digits only, and every user without declared accounts is an identity account, so kcadm set-password rejected hermes and openclaw with invalidPasswordMinSpecialCharsMessage as soon as keycloak entered the GitLab#1 round without LDAP (4164f37981) and the extras file stopped replacing the users map (03e42bae6d; run 33689701895). The generator now draws from letters, digits and -_. and guarantees upper case, lower case, a digit and one mark, never in the leading position.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
An earlier change in this branch marked six GitLab key bases rotatable: false after
one of them, db_key_base, was proven to break a deploy. Six was the shape of
"everything in secrets.yml.j2 that could plausibly be blamed", not the shape of
"what is lost". Measured in the shipped image (gitlab-rails-ce:v19.3.1), four of
them rotate fine:
A_new_key_only => FAIL Decryption (control: reproduces the rotation)
B_old_then_new_list => s3cret-written-before-rotation
C_new_only_reads_post_rotation_write => written-after-rotation
D_salt_rotated => FAIL Decryption
B and C prove active_record_encryption_primary_key is list-rotatable and that the
old key can later be dropped; D proves the salt is not. Note the new key goes LAST
-- key_provider.rb:21 is `@keys.last` -- so a prepend would have shipped a broken
rotation.
Unpinned, with the reason each:
otp_key_base gitlab:two_factor:rotate_key:apply exists, with rollback!
encrypted_settings_key_base protects zero bytes here: grep -c encrypted on
gitlab.yml.j2 is 0, ldap and incoming_email are off,
and upstream has rotated_encrypted_settings_key_base
active_record_encryption_primary_key list rotation measured above
active_record_encryption_deterministic_key one column, a push device token the device re-registers
Still pinned, and the descriptions now name why rather than leaving it to a commit
message: db_key_base (settings.rb raises MultipleDbKeyBaseError on a second key and
upstream ships no re-encrypt task) and active_record_encryption_key_derivation_salt
(scalar PBKDF2 input, no previous-salt slot in Rails). What db_key_base protects is
also worse than first recorded -- grep -rl returns 36 files: CI/CD variable values at
every scope, terraform state, Pages TLS private keys, webhook and integration
credentials. The two tokens that actually crashed were only the ones custom-instance-setup
happened to touch.
The second half matters more than the first. _drop_generated skipped a pinned path
with a bare `continue`, and reset/cli.py printed "Rotated 66 value(s)" with no mention
that six were refused. An operator rotating a disclosed key was told it worked. A tool
that silently declines to rotate a leaked secret and reports success inverts the
property it exists to provide, so every skip now names itself on stderr.
Not addressed here, and it is the real repair: gitlab:doctor:reset_encrypted_tokens in
the db-migrate path would let any instance with undecryptable tokens heal itself,
including after a restore under a different secrets file, which no pin can help. It
changes the migrations container's command and is not measurable without a live GitLab.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_drop_app_credentials walked applications[app].credentials, which
f713ba8350 moved to secrets.credentials. Nothing lives at the old address
any more - the run's inventory has twelve applications under
secrets.credentials and none above it - so the isinstance check never
matched and the function returned 0 every time. The rotation ran, reported
success and rotated no application secret at all.
It stayed hidden because the sibling half worked: _drop_user_passwords
reads document["users"], which did not move, so the total was never zero
and the call looked healthy.
That matters more than a dead branch usually would. Rotating between two
deploy passes is what turns the second pass into a test of the update
path, where a role has to push a new secret into the application and the
database it already created. With the drop silently skipped, every such
run has been re-deploying the values the first pass wrote.
The fixture pinned the pre-rename shape, so the test confirmed the bug
rather than catching it. Migrated to the real layout, with the address in
one helper instead of four assertions; against the old code it now fails
1 != 4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The applications: host_vars block is the config namespace keyed by role id, so it also carries system services (sys-ctl-hlth-csp, sys-svc-container). Those declare system_service_id and match no application role, and the resolver died on them instead of reporting their absence.
role_name now raises RoleNotFoundError and exits 3, distinct from argparse's usage error (2) and from a genuine failure (1). run_subprocess grew ok_returncodes so a caller can accept an expected non-zero status while every other failure still surfaces, and resolve_role_path returns None for that code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
provision and credentials are sibling packages, so reaching the emit helpers
with a parent-relative import tripped TID252. Same-package imports stay
relative, as everywhere else in the package. The blank line before the
role_resolver import keeps the first-party and relative groups apart, which
I001 requires.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The host_vars merger still wrote applications.<app>.credentials and read
"credentials" out of the generated snippet. f713ba835 moved the emitter to
secrets.credentials, so the read resolved to {} on every app: nothing was ever
copied, and _ensure_ruamel_map left an empty credentials: {} shell behind as
the only trace. Every credential generated since that commit was discarded on
its way into the inventory.
Both sides now go through credentials_map() and declared_credentials(), the
accessors the emitter already uses, so the two ends cannot drift again.
_ensure_ruamel_map was a copy of emit.ensure_map and is gone.
The failure surfaced in two shapes from the one cause. Templates reading
through lookup('config', ...) raise, because get() defaults to strict:
"Key 'secrets' not found in dict at 'secrets'" for
web-app-mastodon.secrets.credentials.secret_key_base, with the dumped payload
showing 'credentials': {} and no secrets node. Templates reading through
lookup('database', ...) stay quiet, because that plugin passes strict=False
with default="": friendica's MariaDB received an empty MYSQL_ROOT_PASSWORD and
refused to initialise.
The generator's own tests pinned the pre-rename shape in both the snippet and
the assertion, which is why they stayed green through the regression.
Evidence: run 31649221474, job 94303863545.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Provisioning fills a missing credential and leaves an existing one alone, so every deploy pass reads the value the first pass wrote and the update path is never exercised. Rotating the generated values makes a redeploy carry a secret that moved.
`infinito administration inventory credentials reset` drops every vault-encrypted leaf under applications.*.credentials.* (--schema) and the pinned password of every declared user (--users), then lets the existing generators refill the gaps, so no schema logic is duplicated. Plain credentials stay unencrypted in host_vars and are therefore left alone, as are users that exist only in the inventory and would get nothing back. --except exempts application ids and user keys, --backup keeps the pre-rotation file as <name>.YYYYMMDDTHHMMSSZ.backup, and --mirror realigns the other hosts of a multi-host inventory.
Adds the command only; no deploy path calls it yet. Measured against web-svc-coturn and web-app-flowise: eleven generated secrets move between two runs while ansible_become_password and the administrator password stay pinned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A user password is pinned once by inventory creation from a shell-safe
alphabet, because it travels through Ansible, a shell and container exec.
An application whose registration demands a special character then had no
way to say so, and roles worked around it with a second credential beside
the account they already owned. The account now carries algorithm and
validation itself, two roles disagreeing about one account abort instead of
one silently winning, and an unsatisfiable pair fails naming both halves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A user password reaches its application through Ansible, a shell, a container
exec -e and finally the runtime's own environment reader. The generator seeded
it from the same alphabet as the become password, punctuation included, and
that does not survive the chain intact.
Three roles had already worked around it by declaring a separate alphanumeric
credential for an identity that their meta/users.yml already declares, and
saying so in the description. Removing those without fixing the generator would
have reintroduced exactly the bug they were built against, so the generator is
fixed first and all three now read the user's own password. At length 64 the
smaller alphabet costs nothing worth measuring, and the become password keeps
the old one because it travels a different path.
Zammad additionally used the bot's password for its wizard-bypass admin, one
secret for two users. Each has its own declaration and now its own value.
A test asserts the generated password is alphanumeric, so a future change to
the alphabet fails there rather than inside a Ruby ENV.fetch in a container.
A user definition that declares no password falls back to
{{ 42 | strong_password }}. That is a template, not a value: it is re-rendered
at every use site, so the task that creates an account and the task that later
authenticates as it read two different secrets. The account exists and the
login fails. Inventory creation now decides the value once, after the roles are
resolved and next to the role credentials that were already generated there.
The scope follows the deployment: only roles resolved into this inventory
contribute users, an existing password is never overwritten, and a second run
writes nothing.
Building it duplicated what the package already carried three to seven times
over: an ensure-map helper defined three times, the same fifteen-line
round-trip load repeated seven times, and the vault encrypt-and-reparse dance
twice. They now live in one module, the round-trip counterpart to the existing
read-only yaml_io. host_vars drops from 213 to 130 lines, and its two override
entry points share the merge they used to copy. Behaviour is unchanged, and
that is measured rather than asserted: the committed versions and the new ones
were driven against the same host_vars and produced byte-identical output for
all six writers, comment placement and vault block layout included.
The adapter probes move into their own folder because a shared file-name prefix
reads as a folder in disguise, and the lint that enforces it scans the tracked
file list, so the repository was failing on it. A test that was defined at
module level instead of inside its class, and therefore never ran, is indented
back in.
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.
meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.
Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.
tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.
meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.
Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.
tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The provisioned become password doubles as the Keycloak administrator realm password (user-administrator/meta/users.yml) and the guide flow generated it 64-char alphanumeric, so kcadm set-password failed the realm passwordPolicy specialChars(1) on every fresh single-app deploy. Delegate generate_random_password to ValueGenerator.generate_strong_password, the same class-guaranteeing generator every other realm user already gets via the strong_password filter; the realm policy stays strict. Also drop the commented-out Trusted Hosts block from the realm template.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The disable env only matched service/entity keys (email, prometheus);
naming the provider role (web-app-mailu) silently did nothing. A shared
expand_service_tokens (registry SPOT) now normalizes either spelling to
the provided service key(s), and every disabler resolver plus the
conflict check normalize at entry, so compose and swarm accept both
forms.
derive_includes additionally prunes the disabled providers from the
swarm include closure. Without it the provision-time disabler removed
the provider role but extend_inventory re-added it via the same dep
walk, so the deploy tripped the 'provider role still active' conflict.
Live-verified on a swarm-dashboard run: disable='web-app-mailu,...'
expands to email->web-app-mailu and deploy round 1/1 proceeds with no
conflict.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mint the node onion during provisioning and write its address straight into applications.svc-net-tor.services.tor.node, dropping the INFINITO_TOR_NODE_ONION env indirection: ensure_node_onion (was init_env) mints/reuses the .onion-identity key files + returns the address (no env write); identity_hs_dir/IDENTITY_DIRNAME move to utils.tor_onion so cli and utils share the path SPOT; provision.apply_tor_node_onion sets services.tor.node in host_vars when svc-net-tor is deployed (run by 'inventory provision' at deploy time, and the svc-net-tor role copies the same key files into the daemon); domains.py reads services.tor.node; nginx_vhosts reads the key file instead of os.environ; INFINITO_TOR_NODE_ONION removed from default.env/passthrough/compose + the inventory env-lookup. Bonus: the address now lives in the persistent key files, surviving .env regeneration by make test/autoformat.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turn a NAT/edge node into a full Tor v3 onion node: DOMAIN_PRIMARY
becomes the node's .onion, so the whole stack (web, SSO, LDAP, CA)
resolves onion domains and is reachable over Tor without a public IP,
port forwarding, VPS or DynDNS.
- utils/tor_onion.py: offline v3 key mint (address + HiddenServiceDir
key files), verified against a known key->address test vector.
- cli .../inventory/onion: `init-env` pre-mints the node onion, writes
INFINITO_DOMAIN(+_RE) into .env and authoritative key files into
.onion-identity/hs (idempotent). The onion is a deploy-time input;
no in-deploy random minting (that would not match the pre-set domain).
- roles/svc-net-tor: Tor sidecar (Debian-built, network_mode: host),
installs the minted keys, torrc with one node HiddenService + a SOCKS
proxy restricted to loopback/DiD ranges. files/test.sh probes every
served onion domain over the Tor SOCKS.
- tls: .onion -> TLS off (onion v3 crypto is the transport auth; LE
cannot sign .onion) + app-level flavor override.
- harness/DNS: coredns resolves the onion zone via INFINITO_DOMAIN;
skip the clearnet DNS sanity check for onion; raise nginx
server_names_hash_bucket_size to 128 for long onion server_names.
- test-e2e-playwright: gated browser SOCKS proxy (PLAYWRIGHT_PROXY) +
navigation/action timeouts so specs reach .onion over Tor and fail
fast; the onion node points it at the Tor SOCKS.
- docs/requirements/031: full design.
Verified: dashboard + SSO (Keycloak) + 9 apps reachable over Tor
(HTTP 200 through the node hidden service). make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".
Fix:
- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
full_cycle, variant) from `passthrough.STATIC_KEYS` and from
`default.env`. They are per-invocation; .env is the persistent SPOT
and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
`os.environ.get("KEY", "")` so the missing-key path stops at "no
filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
(lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
name in every doc; routing-knob env vars now use the same name so
the cmdline form, the recipe env-prefix, and the consumer reads
all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
knobs whose alias differs from the env name (`type` →
INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
auto-exports cmdline variables into the recipe's shell env, so the
identical-name `$(if $(apps),apps="$(apps)")` lines were pure
noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
`extra_env` per `docker exec` call and forwards `disable` via
`-e`, so the container env never needs the value frozen at
compose-up time.
- Update the 3 actual leak call sites
(07/08/09_deploy_*.sh) to the make-alias form
(`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
the deliberate make/env parity, not a typo.
Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.
Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
InventoryManager.resolve_schema_includes_recursive previously read the
root role's base meta/services.yml when discovering shared providers,
regardless of which matrix variant the planner had pinned for the
round. For a role whose variants.yml enables additional services per
variant (e.g. svc-bkp-container-2-local variant 2 enabling
services.ldap), the base config had no `enabled+shared` flag for those
services, so the recursive resolver never visited the variant's
shared providers — their credentials were never generated, and the
deploy crashed in the provider's env.j2 with "credentials is defined
in schema at roles/<provider>/meta/schema.yml but not set in
application '<provider>'".
Wires the active variant through the existing inventory pipeline:
- utils/manager/inventory.py: InventoryManager(variant=None|int). When
set, load_role_config_by_path overlays variants.yml[N] onto the root
role's meta config (only for the root — transitive providers keep
their own base lookup). variant=None preserves the pre-refactor base
behavior for every standalone caller.
- cli/administration/inventory/credentials: new --variant N flag,
forwarded to InventoryManager.
- cli/administration/inventory/provision: new --app-variants <JSON>
flag ({app_id: variant_index}), forwarded per-app to the credentials
subprocess.
- cli/administration/deploy/development/inventory/builder.py: emits
--app-variants from spec.variant_selectors() when the round pins any
variant. Apps without variants.yml collapse to a single empty
overlay in get_variants so this is a no-op for them.
10 unit tests cover: variant overlay for root role, variant=None
fallback to base, out-of-range fallback, no overlay on non-root
paths, --variant CLI forwarding (set + default), --app-variants
generator forwarding (set + omit), builder --app-variants emission
(set + omit).
Caveat: bkp-c2l CI variant 1 still loads svc-db-openldap transitively
at runtime (outside the planner's variant closure) — a separate
sys-service-loader / role-meta bug, not addressed here.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
\`--workers\` for credentials generation kept a hardcoded default of 4
regardless of host capacity. Using \`os.cpu_count() or 4\` scales the
default to whatever the runner actually has (with the same 4-worker
floor as a safety net for the rare case where the OS does not report
a CPU count), so a 16-core developer machine no longer pessimistically
ceilings credentials generation at 4 while CI runners keep the
same effective behaviour they had before.
Stays at the same SPOT (argparse default) so callers can still
override per-invocation with \`--workers N\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause of the DinD DNS-check failure on test-deploy-server
(networks.internet.ip4 falling through to 127.0.0.1 despite
INFINITO_IP4=172.30.0.10 reaching every link of the env pipeline):
cli/administration/inventory/provision/cli.py declared --ip4 with a
hardcoded default of "127.0.0.1" and ensure_host_vars_file wrote that
value unconditionally into host_vars/<host>.yml. group_vars'
defaults_networks.internet.ip4 resolves "172.30.0.10" via
\`lookup('env', 'INFINITO_IP4')\`, but tasks/stages/01_constructor.yml
merges \`defaults_networks | combine(networks | default({}, true))\` so
host_vars wins. INFINITO_IP4 looked broken end-to-end; the breakage
actually lived in the provision-CLI host_vars overlay.
INFINITO_DNS_IP and INFINITO_DOMAIN propagated correctly because
the provision CLI had no --dns and the --primary-domain write was
None-gated and never invoked.
While here, --ssl-disabled and --primary-domain are removed too:
neither had any effect in the dev flow. provision/cli.py applies
\`--vars-file inventories/development/default.yml\` AFTER
ensure_host_vars_file via apply_vars_overrides_from_file, whose
_deep_update_commented_map unconditionally overwrites
(\`target[key] = value\`). The dev vars-file always set
\`TLS_ENABLED: true\`, so \`--ssl-disabled\` (which forced
\`TLS_ENABLED: false\` in host_vars) was systematically reverted to
true. Dev/CI ran with TLS enabled the whole time. Inventory stays at
\`TLS_ENABLED: true\` to preserve that effective behaviour.
Removed from cli/administration/inventory/provision/cli.py:
- --ip4, --ip6: defaulted to 127.0.0.1 / ::1, shadowed group_vars
- --primary-domain: never set by any caller, None-gated write
- --ssl-disabled: always re-overwritten by --vars-file later in the
same provision pass
Removed from cli/administration/inventory/provision/host_vars.py the
matching writes in ensure_host_vars_file; the function now only sets
\`ansible_connection: local\` for localhost-class hosts. Defaults for
DOMAIN_PRIMARY / TLS_ENABLED / networks.internet.{ip4,ip6} live in
group_vars (env-driven) and the inventory vars-file (deployment-wide
flags); host_vars stops duplicating them.
Callers cleaned up:
- cli/administration/deploy/development/inventory/builder.py
- cli/administration/deploy/container/command.py
- scripts/tests/deploy/local/utils/fresh-kept-all-init-and-deploy.sh
- scripts/tests/deploy/local/reset/inventory.sh
Tests + docs follow:
- tests/unit/cli/administration/inventory/provision/test_host_vars.py
now asserts host_vars no longer carries baked DOMAIN_PRIMARY /
TLS_ENABLED / networks defaults
- docs/administration/deploy.md drops --ssl-disabled from the
provision example
- docs/contributing/design/inventory.md re-scopes the host_vars
column: the DOMAIN_PRIMARY / TLS_ENABLED / networks.internet.{ip4,ip6}
values arrive via --vars / --vars-file overlays, otherwise group_vars
provides the env-driven default
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
empty-string fallback (which silently masks an unset key). The
nocheck check walks every physical line the call spans so an
autoformat-split call still honours the marker.
Cache namespace flattens to a single prefix:
INFINITO_PACKAGE_CACHE_* -> INFINITO_CACHE_PACKAGE_*
INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.
SPOT-strict rewrites:
- 19 sites: `something.get("INFINITO_X", "")` -> subscript
`something["INFINITO_X"]` (loud KeyError when env loader did
not run; all keys live in env/default.env).
- 2 handler-internal bootstrap reads suppressed with
`# nocheck: handler-bootstrap-read`.
- 30 runtime-only / driver-injected / test-fixture references
annotated with `# nocheck: <reason>` per-line.
- Unit tests that previously removed SPOT-guaranteed keys via
`{k: v ... if k \!= KEY}` or `clear=True` switch to setting
the key to "" explicitly, matching the new contract.
Side fixes:
- INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
shell-side defaults in common.sh to env/default.env and
passthrough.STATIC_KEYS.
- dns_systemd_is_operational() drops the dead
INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
variable was never set anywhere).
- INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
to `os.environ[KEY]`.
- scripts/tests/code/wrapper.sh exec args move into a bash
array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
own element line (SC2215 fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidates image declarations under a single schema: every role's
`<svc>.{image,version}` lives in `meta/services.yml` alongside compose
service entries. The `image` lookup now reads from disk per role,
`iter_role_images` collapses to a single source, and the mirror chain
(resolver + apply_mirror_overrides) drops the dead `images:` branch.
`test-e2e-playwright` resolves `TEST_E2E_PLAYWRIGHT_IMAGE` directly via
`lookup('image', 'playwright', 'ref')` in vars/main.yml; the runtime
set_fact + override-empty-string dance is gone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).
* Document the rule in `docs/contributing/documentation.md` under
"Comments". Code MUST describe the *why* in present-state language;
contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
`cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
`eslint.config.js`. Pure comment / docstring / log-message edits;
no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
enforce the rule going forward across `.js`, `.py`, `.yml`,
`.yaml`, `.j2`, and `.sh` files. The legacy
`test_no_stale_requirement_refs.py` is exempt as the only file
whose purpose is to detect these very tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TC (flake8-type-checking): typing-only imports moved into
``if TYPE_CHECKING:`` blocks across 8 source files via auto-fix
(117 hits). Reduces runtime import surface and rules out circular
imports for type-only deps. Project-wide ``from __future__ import
annotations`` (FA, already enabled) keeps annotations as strings,
so the moved imports never need to load at runtime.
N (pep8-naming): 32 hits handled.
Renames (semantic + lint):
- ``isSmallerThenMaximumBackupSize`` -> ``is_larger_than_maximum_backup_size``
(snake_case + fixes both the typo "Then -> Than" and the
semantic mismatch: the function returns ``current > maximum``)
- ``isDirectoryDeletable`` / ``deleteVersion`` /
``getAmountOfIteration`` / ``deleteIteration`` -> snake_case
- 8 test functions with ALL-CAPS env-var fragments
(``test_..._DOMAIN_PRIMARY_...`` etc.) -> snake_case
- Mock alias ``VH`` -> ``vh`` in test_host_vars.py
Exception class renames (``N818`` Error suffix):
- ``TimeLimitExceededException`` -> ``TimeLimitExceededError``
- ``AttemptException`` -> ``AttemptError``
- ``PathNotFound`` -> ``PathNotFoundError`` (propagated across
4 callers under tests/integration/lookups/config/)
- ``_ExitJson`` / ``_FailJson`` -> ``*_Error``
Refactor:
- ``METADATA = {...}`` local constant in two functions of
``plugins/filter/invokable_paths.py`` lifted to module-level
``_METADATA_KEYS``.
- ``from typing import TYPE_CHECKING`` in
``tasks/utils/migrate_meta_layout/__main__.py`` moved above
the ``sys.path.insert`` hack to satisfy E402 (collateral from
the TC auto-fix that landed it after the path mutation).
Per-file ignores in pyproject.toml for the legitimate cases the
rule cannot accept:
- ``pull-specific-host.py`` -> ``N999`` (Ansible-invoked role-level
script, hyphenated filename pinned by the surrounding stack).
- 4 matrix-deploy files -> ``N806`` for the ``include_R`` /
``raw_include_R`` round-index notation; the uppercase ``R``
here mirrors the formula in the docstring rather than naming
a constant.
- ``plugins/filter/value_generator.py`` -> ``N806`` for the
lazy-loaded class binding ``ValueGenerator =
_value_generator_cls()``, which retains the imported class
identifier per PEP 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
