Role README Credits now name galaxy_info.author (plain bold or wrapped
in a Markdown link), matched byte-for-byte against the role's own
meta/main.yml, instead of a hard-coded maintainer name. Regenerate all
245 role README Credits blocks to the new 'Implemented by ...' format.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.
- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).
- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.
- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).
- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/naming/test_branch_name.py validating the checked-out branch against the naming policy in branch.md. Rejects unknown prefixes (e.g. feat/ instead of feature/), non-kebab-case descriptions, and fix/ branches missing a ticket id; exempts main, master, and detached HEAD. Document the local enforcement in branch.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switch the label that flips a fork PR to the trusted (PR-head, with
credentials) path from 'trusted-pr' to '🛡️ Trusted', in the
pull_request_target gating and the orchestrator text, plus the docs.
The trusted_pr input names are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
actions/checkout now refuses to fetch PR head/merge code under
pull_request_target without an explicit opt-in, which broke the fork
image build and the fork mirror job for every fork PR.
Set allow-unsafe-pr-checkout on the fork source checkouts (new
allow_unsafe_checkout input on images-build-ci.yml and
images-mirror-missing.yml, passed true by the fork producer jobs) so
the producer builds again — untrusted forks from the safe base merge
ref, trusted forks (trusted-pr label) from the PR head.
Harden the untrusted path: the fork build no longer inherits
organization secrets (it authenticates to GHCR with the per-job
github.token only, and the build needs no other secret), and Docker Hub
mirror credentials are passed only for trusted PRs (untrusted forks
mirror anonymously). Switch the GHCR logins to github.token so auth no
longer depends on secrets inheritance.
Docs updated in pipeline.md and authentication.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A maintainer-applied trusted-pr label opts a fork PR out of the cautious fork-prereq dance: the orchestrator skips the wait for privileged fork images, and the pull_request_target build switches its source from the base merge ref to the raw PR head (head.repo.full_name@head.sha). The image keeps the merge-SHA tag so the pull_request orchestrator and its downstream consumers find it unchanged.
Security gates: only the label flips the build source (label-application is collaborator-only on GitHub, so fork authors cannot grant it); build-orchestration scripts and the helper checkout always come from the base repo at github.sha, only the Docker build context is the fork head; persist-credentials stays false and USE_NIX_TOKEN stays disabled for fork-sourced builds; fork pull_request runs keep their GitHub-enforced read-only token. Untrusted fork PRs and same-repo PRs are unchanged.
Wiring: ci-orchestrator gains a trusted_pr boolean input resolved once into a WAIT_REQUIRED job env; images-build-ci gains an optional source_repository input; both pull_request and pull_request_target carry a labeled trigger so applying the label starts fresh runs that observe it. Documented in pipeline.md (Trusted fork PRs) and workflows.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
entry-push-latest.yml gains a sync-main job that force-pushes the current repository's main from a source repo (default infinito-nexus/core) before CI policy and deploy discovery derive scope from origin/main. The source is configurable and can be disabled via CI_SYNC_MAIN_SOURCE_REPOSITORY; the same-repository case and disabled values skip the sync.
The sync-main checkout uses fetch-depth: 0 and the push is --force to avoid shallow-update and non-fast-forward rejections. Docs warn that main is force-overwritten and must not be used as a working branch on synced repositories.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Delete the per-deploy-type workflows (test-deploy-local/universal/
workstation) and the redundant act-all/act-app runner scripts (replaced
by make act-workflow). Only compose and swarm deploy tests remain.
Both now discover the full application matrix (every deployment type,
lifecycle-filtered to alpha/beta/rc/stable) instead of just the server/
web-* scope, via apps.sh resolving applications.type without --type.
Retire INFINITO_DEPLOY_TYPE entirely: discovery is now unconditional and
the deploy step never read it. Removed from apps.sh, ci/all.sh, the local
deploy scripts, the two workflows, default.env, the env passthrough list,
the dotenv generator test, the Makefile compose-deploy target, and docs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.
objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.
deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.
Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two scalar group-vars are referenced verbatim in
`roles/user-*/meta/users.yml` (`lastname: "{{ ORGANIZATION }}"`,
`description: "owned by {{ SOFTWARE_NAME }}"`). Ansible 2.19's
TrustedAsTemplate gate silently leaves those untagged Jinja
literals unrendered, so downstream consumers received the raw
`{{ ORGANIZATION }}` string. Keycloak rejected the resulting user
updates with `error-person-name-invalid-character` because `{` /
`}` are not valid in a person name.
Add `substitute_scalar_placeholders` next to the existing
`substitute_primary_domain_placeholder`. Both helpers run from
`get_merged_users` immediately before `_render_with_templar`, so the
cache returns fully-resolved scalars regardless of the templar's
trust gate.
Also restructure: move the cache into a `utils.cache.users` package
so the placeholder helpers no longer need a redundant `users_`
prefix:
utils/cache/
users/
__init__.py was utils/cache/users.py (public API
`get_user_defaults` / `get_merged_users` is
unchanged; consumers using
`from utils.cache.users import ...` keep
working via package __init__).
placeholders.py the substitution helpers, dropping the
redundant `users_` prefix.
tests/unit/utils/cache/
users/
__init__.py
test_module.py was test_users_module.py
test_placeholders.py new: 15 unit tests covering both
helpers (nested-structure walk, URL /
port handling for DOMAIN_PRIMARY,
Jinja-in-raw-value resolution, empty /
missing skip paths, and a guardrail
pinning the current
`_SCALAR_USER_PLACEHOLDERS` tuple).
Update README links under `utils/cache/` and
`docs/contributing/artefact/files/plugins/lookup/users.md` to point
at the new package path.
Placeholders covered today: ORGANIZATION, SOFTWARE_NAME. Extend the
tuple in `utils/cache/users/placeholders.py` when a new scalar
group-var appears in user metas.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The test timed out: the publish-button locator no longer matched the v8.2.1
manage-video wizard, and the inline 78-byte MP4 stub was structurally invalid,
so PeerTube rejected the upload as "Video file unreadable" and left the Save
button disabled.
- test-seaweedfs.js: target the v8.2.1 save control (.save-button > button),
upload PeerTube's real video_short.mp4 fixture, and assert a version-tolerant
saved marker.
- add fixture roles/web-app-peertube/files/playwright/fixtures/video_short.mp4.
- test-e2e-playwright: stage role-local files/playwright/fixtures/ next to the
tests (the *.js glob ships neither binaries nor subdirectories), so specs load
a fixture via path.join(__dirname, "fixtures", <name>).
- docs: document the fixtures/ folder and the per-scenario test-*.js split in the
playwright spec contract and the runner README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Use whale (🐳) for the per-app matrix job to mirror the swarm
workflow's bee (🐝) pattern, and magnifying glass (🔍) for the
discover step. Keep the workflow header on the existing 📦 prefix to
stay consistent with the other test-deploy-* siblings.
Rename the workflow file and align all references:
- ci-orchestrator.yml job key + uses: path
- artifact names: inventory-server-* -> inventory-compose-*,
playwright-server-* -> playwright-compose-*
- scripts/github/inventory/zip_compose_for_upload.sh output path
- 3 docs files
INFINITO_DEPLOY_TYPE: server stays - that key is wired to the
invokable-rule registry in utils/roles/validation/invokable.py and is
a separate refactor concern.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".
Fix:
- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
full_cycle, variant) from `passthrough.STATIC_KEYS` and from
`default.env`. They are per-invocation; .env is the persistent SPOT
and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
`os.environ.get("KEY", "")` so the missing-key path stops at "no
filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
(lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
name in every doc; routing-knob env vars now use the same name so
the cmdline form, the recipe env-prefix, and the consumer reads
all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
knobs whose alias differs from the env name (`type` →
INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
auto-exports cmdline variables into the recipe's shell env, so the
identical-name `$(if $(apps),apps="$(apps)")` lines were pure
noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
`extra_env` per `docker exec` call and forwards `disable` via
`-e`, so the container env never needs the value frozen at
compose-up time.
- Update the 3 actual leak call sites
(07/08/09_deploy_*.sh) to the make-alias form
(`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
the deliberate make/env parity, not a typo.
Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.
Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three docs still referenced the pre-collapse service keys:
* docs/contributing/artefact/files/role/playwright.specs.js.md — biber
and administrator auth-chain diagrams listed
`skipUnlessServiceEnabled('oidc' | 'oauth2' | 'ldap')`; the unified gate
is `'sso' | 'ldap'`.
* docs/contributing/design/role/services/base.md — the `lookup('service',
...)` example block carried the now-invalid `'oidc'` key; replaced
with `'sso'`.
* docs/requirements/019-playwright-meta-services-parity.md — the
prometheus, matomo, and mailu rows still spelled
`safeSkipUnlessEnabled("oauth2"|"oidc")` and
`isServiceEnabled("oauth2")`; all rewritten to `"sso"`.
`_build_application_defaults` returned variant 0 of every role with a
meta/variants.yml, which then deep-merged with the variant-N override
that build_dev_inventory bakes into the inventory. Any service flag
that variant N did not explicitly disable carried through from
variant 0 into the runtime applications map. The most visible casualty
was svc-bkp-container-2-local variant 1 (postgres + mariadb only),
where gitea + keycloak leaked in via variant 0, sys-service-loader
pulled in svc-db-openldap via keycloak.services.ldap, and the deploy
crashed in sys-svc-compose/04_files.yml on openldap's env.j2 with
"credentials is defined in schema at roles/svc-db-openldap/meta/schema.yml
but not set in application 'svc-db-openldap'".
Defaults now come from each role's variant-free base config
(meta/<topic>.yml plus schema-defaults, no variants overlay). Variants
stay accessible via get_variants for any caller that explicitly needs
a specific variant. Inventory-level applications.<app> overrides win
exactly as before.
Supporting changes that keep the variant matrix consistent under the
new defaults model:
- New lint guard tests/integration/roles/meta/variants/
test_non_baseline_explicit_disables.py: for every dynamic
service-key declared in a role's meta/services.yml that variant 0
pins to literal true, every non-baseline variant MUST explicitly
pin the same key to either literal true or to literal
enabled: false + shared: false. Statically-enabled keys are exempt
(same rule as test_auth_isolation.py).
- roles/web-app-{fusiondirectory,lam,nextcloud,phpldapadmin}/meta/
variants.yml: explicit `false` pins for dashboard / email / logout /
matomo / prometheus / talk in restrictive variants so deep-merge no
longer hands them the deleted variant-0 inheritance.
- roles/web-app-moodle/templates/playwright.env.j2: gate the LAM
block on `lookup('config', application_id, 'services.lam.enabled',
false)`. LAM is a transitive helper, not a moodle-owned service, so
the default-bearing lookup tolerates the missing path when the
variant does not deploy LAM.
- tests/unit/utils/cache/test_variants.py: rename the legacy
"always variant zero" coverage to TestMergedApplicationsUsesBaseConfig
and add test_get_application_defaults_omits_variant_zero_only_keys
as the regression guard for the leak this commit fixes.
- docs/contributing/artefact/files/role/variants.md +
docs/contributing/design/variants.md: variant 0 now manually
enumerates every dynamic services.yml key with
enabled: true + shared: true; `{}` is only permitted when the role
has no dynamic-enabled service-key.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
design/variants.md gains a "Credentials Generation" section that
explains both flows the credential machinery can take:
- CI/CD path: planner picks the round's variants -> build_dev_inventory
emits --app-variants -> provision CLI forwards --variant -> credentials
CLI hands the index to InventoryManager, which resolves the root
role's variants.yml overlay so resolve_schema_includes_recursive
sees the variant-merged services map and discovers every shared
provider that the variant enables.
- Standalone path: any caller that omits --app-variants / --variant
gets variant=None, InventoryManager skips the overlay, and shared-
provider discovery walks the base meta config only.
Also adds four reference-table rows for the involved files and fixes
two unrelated wording issues in the surrounding text:
- "every dep round R needs MUST be redeployed" -> grammatical fix.
- "historical payload" / "rename the file back" -> drop historical
framing per documentation.md's current-state-only rule.
artefact/files/role/variants.md gains a short "Credentials
Interaction" section that anchors back to the design doc so a role
author setting services.<key>.enabled+shared in a variant knows that
the credential generator will pick the provider up.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Per-role Playwright runs no longer call `npm install` against
registry.npmjs.org on every invocation. A new `01_setup.yml` runs once
per host/version, populates `_shared/node_modules/` inside the Playwright
container, and stamps the installed version in `.installed-version`.
Each `02_run_one.yml` mounts that tree read-only at `/e2e/node_modules`
and the container command becomes `npx --no-install playwright test`.
This eliminates the npmjs.org TLS-handshake flake that took out the
keycloak playwright in a recent variant-2 update, saves ~4 s per role,
and gives version bumps a single rebuild point.
Three retry layers cover transient registry failures: Ansible (3× with
5 s delay), npm fetch (5× with 10-60 s backoff), and image pull (3×).
Stage-base / shared-dir / stamp / node_modules paths live as SPOT
constants in `vars/main.yml` so call sites never reconstruct them.
Renames `tasks/run_one.yml` → `tasks/02_run_one.yml` (and adds
`01_setup.yml`) to make the include order visible by filename; updates
cross-references in `AGENTS.md`, the two `docs/contributing/...` pages,
`dump_disk_diagnostics.sh`, and the lint test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`lookup('image', '<role>', '<svc>', 'ref')` only consulted
`images_overrides.*` plus role defaults, which the mirror pipeline
never populates — the pipeline writes into
`applications.<role>.services.<svc>.{image,version}`. As a result,
`sys-svc-container`'s `BUSYBOX_IMAGE` / `NODE_IMAGE`,
`sys-ctl-hlth-csp`'s `HEALTH_CSP_IMAGE`, and `test-e2e-playwright`'s
`TEST_E2E_PLAYWRIGHT_IMAGE` always resolved to the raw upstream ref
and hit Docker Hub directly. In CI this surfaced as
`toomanyrequests: ...unauthenticated pull rate limit` on
`busybox:1.37`.
Switches every productive `lookup('image', ...)` to
`lookup('config', '<role>', 'services.<svc>.image|version')` — the
same path templates like `web-app-oauth2-proxy` already use, which
resolves through the merged `applications.*` map and so transparently
picks up the GHCR mirror override written by the inventory creator.
Removes the now-unused `plugins/lookup/image.py` plugin and its unit
test (zero remaining consumers across roles, scripts, and tests).
Rewrites the image-origin / image-mirror / dockerfile docs to teach
the `lookup('config', ...)` pattern, dropping references to the
deleted plugin and the dead `images_overrides.*` variable. Doc edits
follow the `documentation.md` rules (no em dashes, one sentence per
line, no historical phrasing).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compose.yml no longer hides services behind \`profiles: ["ci"]\`. The
profile existed solely as a no-op safety net against a bare
\`docker compose up\` in the repo root, but the top-level compose.yml is
already explicitly scoped to local dev / CI testing (production goes
through the Ansible roles under roles/<role>/templates/), so the safety
net is redundant with the documented scope.
Every \`--profile ci\` callsite collapses:
- compose.yml: \`profiles: ["ci"]\` removed from coredns + infinito
- Profile.args() deleted; Profile keeps is_ci() and
registry_cache_active() for the cache-override gating
- compose.py / down.py: profile.args() splat removed from the
docker-compose command builder; down.py drops its now-unused
Profile import
- 5 shell wrappers (install, lint, tests/code, meta/resolve/apps,
meta/resolve/diff/affected_roles): plain \`docker compose\` instead
of \`docker compose --profile ci\`
- tests/unit/.../test_profile.py: TestProfileArgs class removed
- tests/unit/.../test_compose.py: two cmd-shape asserts updated to
reflect the missing --profile pair
Docs:
- docs/contributing/artefact/files/compose.yml.md: the
\"every service MUST be behind a profiles: entry\" rule is gone
- docs/contributing/actions/debugging/ci.md: the
\`docker compose ... --profile ci config\` debug snippet drops the
flag; while touching the file the two em-dash clause separators in
the reproduction-profiles shell block are rewritten to colons
(documentation.md em-dash rule), and one list-item in
compose.yml.md is collapsed to one sentence per line for
consistency with its siblings
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
With env.development and env/ci.env both gone, env/default.env was the
last inhabitant of the directory. Hoisting it to the repo root removes
the empty-shell indirection: the SPOT is now [default.env](default.env),
sitting next to .env (its generated companion) and other repo-root
configuration artefacts like compose.yml, Dockerfile, and Makefile.
Path updates land in:
- generator: cli/meta/env/__main__.py (REPO_ROOT / "default.env")
- workflow: .github/workflows/test-install-make.yml (awk reads from
default.env, error message updated in lockstep)
- lint tests: test_used_keys_declared.py and test_infinito_prefix.py
switched from globbing env/*.env to targeting default.env directly;
test_default_env_keys_handled.py points at the new path
- drift test: test_inventory_vars_file_spot.py whitelist + STATIC_ENV
- smoke test: tests/integration/meta/env/test_dotenv_generator.py
STATIC_ENV + every docstring/error message
Doc-string and prose references in utils/env/*, scripts/*,
cli/administration/deploy/development/{common,coredns}.py,
Makefile, Dockerfile, and every doc page that mentioned env/default.env
are rewritten in the same pass so future readers see one path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four ci.env keys (INFINITO_DNS_IP, INFINITO_DOMAIN, INFINITO_DOMAIN_RE,
INFINITO_IP4) were only honored by docker compose --env-file, so processes
that sourced .env via load.sh saw INFINITO_IP4 as unset. As a result
networks.internet.ip4 in the Ansible playbook fell through to the
127.0.0.1 default and the DinD DNS-check handler failed on the deploy
server matrix.
Moving the defaults into env/default.env makes them flow through the same
.env / load.sh / os.environ chain as every other INFINITO_* key, so
\${INFINITO_IP4} in compose.yml, lookup('env', 'INFINITO_IP4') in
group_vars, and envsubst over the CoreDNS Corefile template all see the
same value deterministically.
Side fix: utils/env/parser.py now undoes the writer's \\\\/\\" escaping for
quoted values, closing a round-trip gap that surfaced once
INFINITO_DOMAIN_RE (the only value containing a literal backslash) started
flowing through the generator.
All --env-file env/ci.env call sites (compose.py, down.py, install/lint/
test wrappers, resolve scripts, test-dns workflow + script) are removed.
coredns.py now reads .env via parse_static_env, docs and unit/integration
tests follow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/docker/entry.sh's two compile-control reads (INFINITO_COMPILE
and INFINITO_COMPILE_SILENCE) crashed during Dockerfile RUN because
that context has no env-pipeline: env/default.env exists in the repo
but no make/BASH_ENV/load.sh chain materialises it into the running
shell. The bare reads under `set -u` triggered unbound-variable exit
even though the keys were registered in default.env + passthrough.
Move build/silence control from env to script flags so entry.sh is
self-contained at bootstrap time. Each flag now triggers its action
directly inside the parse loop -- no COMPILE/SILENCE state vars:
--compile -> run_local_build (streaming)
--compile-silent -> run_local_build >/dev/null 2>&1
-- -> end flag parsing, rest is the exec target
Drop INFINITO_COMPILE + INFINITO_COMPILE_SILENCE from the whole
registry surface: env/default.env, passthrough.STATIC_KEYS, the
infinito_compile handler (gone entirely), compose.yml's env
injection block, the dotenv-generator baseline test, the
build_env unit-test fixture, and the now-no-op `-e INFINITO_COMPILE=0`
in test-install-{make,pkgmgr} workflows.
Dockerfile RUN switches to `entry.sh --compile -- true`; runtime
ENTRYPOINT receives no flags so the default (no-compile) holds and
exec hands off to systemd via the CMD.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
empty-string fallback (which silently masks an unset key). The
nocheck check walks every physical line the call spans so an
autoformat-split call still honours the marker.
Cache namespace flattens to a single prefix:
INFINITO_PACKAGE_CACHE_* -> INFINITO_CACHE_PACKAGE_*
INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.
SPOT-strict rewrites:
- 19 sites: `something.get("INFINITO_X", "")` -> subscript
`something["INFINITO_X"]` (loud KeyError when env loader did
not run; all keys live in env/default.env).
- 2 handler-internal bootstrap reads suppressed with
`# nocheck: handler-bootstrap-read`.
- 30 runtime-only / driver-injected / test-fixture references
annotated with `# nocheck: <reason>` per-line.
- Unit tests that previously removed SPOT-guaranteed keys via
`{k: v ... if k \!= KEY}` or `clear=True` switch to setting
the key to "" explicitly, matching the new contract.
Side fixes:
- INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
shell-side defaults in common.sh to env/default.env and
passthrough.STATIC_KEYS.
- dns_systemd_is_operational() drops the dead
INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
variable was never set anywhere).
- INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
to `os.environ[KEY]`.
- scripts/tests/code/wrapper.sh exec args move into a bash
array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
own element line (SC2215 fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Group the shell helpers under scripts/github/ into five subfolders
that match how the workflow files consume them:
runner/ host-runner hygiene (disk, swap, Docker daemon, diagnostics)
cancel/ run cancellation on PR-close and branch-delete events
resolve/ derivation of workflow inputs and outputs from repo state
update/ scheduled Update: Versions pipeline
release/ release-time gates
Drop the now-redundant subfolder prefix from each filename, e.g.
dump_runner_state.sh becomes dump_state.sh and cancel_pull_request_runs.sh
becomes pull_request_runs.sh. Each subfolder carries a README that
describes its scope per documentation.md rules; the top-level README
points at the subfolders without inventorying files.
Propagate the new paths through ten workflow files, three docs
(workflow.md, pipeline.md, secrets.md), and two tests. Fix a real
runtime bug in enlarge_swap.sh, which still referenced the
pre-rename dump_runner_state.sh path. Extend the inline-block-size
lint hint to describe the new subfolder convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The internal container port slot under <entity>.ports is now a
category-keyed map (matching the local/public shape) instead of a
single int. "internal" replaces "inter" because the latter is
ambiguous in English (could read as "between").
Affected:
- 8 role meta/services.yml (matrix, ollama, mig, shopware, oauth2-proxy,
gitea, listmonk, sphinx) and every caller (vars, templates, group_vars).
- web-app-matrix: 8008 hardcoded across compose.yml.j2, homeserver.yaml.j2,
tasks/01_docker.yml, and 7 mautrix bridges now reads
MATRIX_SYNAPSE_CONTAINER_PORT from services.synapse.ports.internal.http.
- web-app-matrix: admin username/password lookups consolidated as
MATRIX_ADMINISTRATOR_{USERNAME,PASSWORD} in vars/main.yml, sourced from
lookup('users', 'administrator').
- Tooling: cli/meta/ports/suggest, utils/meta/scan, integration scan,
migrate_meta_layout/role_builder updated to the new key name.
- Docs: design/role/services/layout.md, tools/ports-suggest.md,
requirements 009/021, image/origin.md, PR template.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `image` lookup inferred `role_id` from `role_name` when only
`service_name` was passed. Inference broke lazy re-evaluation: a default
like `HEALTH_CSP_IMAGE: "{{ lookup('image', 'csp-checker', 'ref') }}"`
defined in sys-ctl-hlth-csp got re-evaluated inside sys-service's
systemctl.service.j2 template (via `system_service_tpl_exec_start`),
where `role_name='sys-service'`. The lookup then searched
`roles/sys-service/meta/services.yml` for `csp-checker` and aborted with
`no image mapping found for role 'sys-service' service 'csp-checker'`.
CI run 25976825584 surfaced this on six apps (web-app-{bridgy-fed,chess,
mig,hugo,mini-qr,postmarks}) whose deploy reached sys-svc-webserver-core
outside the service-loader preload context, which is what actually
triggers the sys-ctl-hlth-csp -> sys-service include chain.
Make `role_id` mandatory in the plugin signature, remove `_infer_role_id`,
and update the three callsites that relied on inference
(sys-ctl-hlth-csp, sys-svc-container, test-e2e-playwright) to pass the
owning role id explicitly. A migration guard catches the removed
`(service_name, want)` form and emits a clear error instead of the
confusing "missing services.yml" file-not-found message.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.
utils/
update/
base.py # semver primitives shared by both backends
docker.py # OCI registry lookups + version: line rewriter
repository.py # git ls-remote walker + ref: line rewriter
cli/contributing/update/
docker/ # python -m cli.contributing.update.docker
repository/ # python -m cli.contributing.update.repository
scripts/github/update/
image_versions.sh
repository_refs.sh
Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.
Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`INFINITO_PACKAGE_CACHE_MAX_AGE_MIN` was 129600 (90 days), 13× longer
than the 7-day `Valid-Until` window Debian and Ubuntu generate by
default in their apt `Release` files. Nexus held cached `Release`
manifests for 90 days, so once the upstream `Valid-Until` elapsed the
manifest stayed in the cache for another ~83 days and every Docker
build that ran `apt-get update` against the package-cache aborted with
E: Release file for http://deb.debian.org/debian/dists/.../InRelease
is expired (invalid since 5h 30min 54s).
Drop the default to 8640 minutes (6 days). Stays one day below the
7-day `Valid-Until` so Nexus always re-validates the cached `Release`
against upstream before apt would see it expired. The single value
still applies to every proxy repo (apt, yum, pypi, npm, gem, go, helm,
raw-*), but the `Valid-Until` enforcement only exists in apt — for
the other repos the change merely makes the cache 84 days fresher
instead of staler, no regression.
Update both docs sites that document the default:
- docs/contributing/artefact/files/compose.yml.md (env-var table)
- docs/contributing/environment/cache.md (operator narrative)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidates image declarations under a single schema: every role's
`<svc>.{image,version}` lives in `meta/services.yml` alongside compose
service entries. The `image` lookup now reads from disk per role,
`iter_role_images` collapses to a single source, and the mirror chain
(resolver + apply_mirror_overrides) drops the dead `images:` branch.
`test-e2e-playwright` resolves `TEST_E2E_PLAYWRIGHT_IMAGE` directly via
`lookup('image', 'playwright', 'ref')` in vars/main.yml; the runtime
set_fact + override-empty-string dance is gone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Nine role specs (bigbluebutton, dashboard, discourse, keycloak,
matomo, matrix, moodle, openwebui, wordpress) each carried inline
copies of the same six CSP helpers and the `EXPECTED_CSP_DIRECTIVES`
constant. The bodies were semantically identical; only message
wording, brace style, and one captured-field-set differed.
Changes
-------
- Extends `roles/test-e2e-playwright/files/personas/utils/csp.js`
(already home to `assertCspInjections`) with
`installCspViolationObserver`, `readCspViolations`,
`parseCspHeader`, `assertCspResponseHeader`,
`assertCspMetaParity`, `expectNoCspViolations`, and the
`EXPECTED_CSP_DIRECTIVES` constant. The helpers are re-exported
through `personas/utils/index.js` so specs that
`require("./personas")` get them via the existing aggregator.
- Strips the inline definitions from the nine role specs and amends
each spec's `require("./personas")` destructuring with whichever
helpers the body still references.
Drift handling
--------------
- moodle's reduced violation-record shape (only `violatedDirective`
+ `blockedURI`) is replaced by the canonical 5-field record.
Moodle's downstream consumer asserts `.toEqual([])` on the array
only, so the extra fields are diagnostic-only — no behaviour
change.
- dashboard's 2-arg `expectNoCspViolations(page, label)` call site
is updated to the canonical 3-arg form
`expectNoCspViolations(page, null, label)`. The shared helper
guards `if (diagnostics && Array.isArray(diagnostics.cspRelated))`
before checking the diagnostics arm, so passing `null` keeps the
exact pre-migration behaviour for that call site.
- bigbluebutton and matrix had subtly different failure-message text
in `assertCspResponseHeader` and `expectNoCspViolations`. The
canonical wording is more descriptive; passing tests are
unaffected, failing tests get clearer diagnostics.
Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` is
extended in two places: the invariants section names the full CSP
test surface and pins the "import from `./personas`, no inline copy"
rule, and the per-service catalogue's CSP / injectors row lists the
helpers by name.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline).
- `markdownlint-cli2`: 0 errors on the touched doc.
- 0 inline definitions of any extracted helper / constant remain in
any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both helpers were inlined verbatim in 71 of the role-local
`playwright.spec.js` files (52 with the compact form, 19 with the
verbose form, plus one parameter-name variant). All bodies were
semantically identical.
Changes
-------
- New `roles/test-e2e-playwright/files/personas/utils/dotenv.js` holds
the canonical pair (`decodeDotenvQuotedValue` and `normalizeBaseUrl`),
re-exported through `personas/utils/index.js` so specs that
`require("./personas")` get them via the existing aggregator.
- Every role spec drops its inline copies and pulls the helpers from
`./personas` instead. Specs that only reference one of the two
carry only that one in the destructuring (no unused-import noise).
Doc
---
`docs/contributing/artefact/files/role/playwright.specs.js.md` no
longer carries the "once a third spec needs them, MUST be promoted"
directive: the helpers live at the shared path and inline copies are
forbidden.
Verification
------------
- 419 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors, 283 warnings (unchanged from
baseline; no new unused-imports introduced).
- 0 inline `function decodeDotenvQuotedValue|normalizeBaseUrl`
definitions remain in any role spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The persona-flow runners now fail loudly when biber / administrator
cannot reach an authenticated surface on the role under test. The
previous runtime auto-detection silently `test.skip(...)`'d, hiding
real regressions (broken OIDC mapping, removed logout button,
misconfigured oauth2-proxy, drifted UI selectors) behind a green
deploy.
Roles where the persona legitimately has no access (admin-only roles,
auth-provider roles) MUST declare an explicit env opt-out
`PERSONA_BIBER_BLOCKED=true` / `PERSONA_ADMINISTRATOR_BLOCKED=true`
in `templates/playwright.env.j2` with a documented rationale; without
the flag the helper fails the test with a diagnostic naming the
last-seen URL.
A new lint guard (`test_persona_strict_mode.py`) hard-fails any
future `test.skip(...)` in the persona helpers that is not guarded
by an allowed env opt-out / auth-less-collapse / service-gate
condition, and any `status === 200` branch in the prometheus / matomo
deny-check helpers that does not validate role-specific body markers.
Req 019 Rule 11 (strict-skip contract) and Rule 12 (200-body validation)
plus a matching "Strict failure on un-executable persona cases"
section in `playwright.specs.js.md` document the contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a lint that rejects empty/tautological/TODO-marked persona and
contract test bodies in roles/<role>/files/playwright.spec.js, plus
the spec-contract doc updates (playwright.specs.js.md, req-017/018/019)
and the integration-test parity-guard tweak that articulate it.
Reason: previous batch updates produced "contract: <svc> gate is wired"
stubs whose bodies only proved what the gate itself already proved; the
lint catches both the title shape and the body shape so a rename is not
enough.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
