The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.
iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.
Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
One boolean answered five unrelated questions: whether the pull-through cache stack runs, whether image pulls take the GHCR mirror, whether the docker data root may be wiped, plus runtime naming and inventory resolution. The workspace is the environment where those answers diverge — it simulates a developer machine on a real runner — so getting one of them meant misreporting where the suite runs and silently paying for the other four.
is_ci() and runs_on_github() stay facts about the machine. cache_stack_enabled, image_mirror_enabled and docker_root_ephemeral become decisions that default off those facts and can be declared explicitly, so an environment states what it wants. Every default reproduces the previous condition exactly, including the distinction the old mirror and wipe gates drew between a real runner and act.
73ae40585 got the cache stack by flipping the identity flag and lost the docker-root wipe with it; the workspace now declares all three capabilities instead. Pairs with 4c068d1cf, which makes the resulting stray bind sources harmless rather than fatal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Parallel checkouts collided on the compose subnet, host ports and container
names, so a quality run in a worktree died at compose-up. Slot each checkout
via custom.env and layer two overrides on top: the shared cache network in
place of a second cache stack, and the shared .git directory without which
git aborts inside the container.
Both overlays gate on the resource they need, not on the slot number, so a
stray INFINITO_INSTANCE cannot make compose demand a value nobody set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolve the 80 open CodeQL/code-scanning alerts on this branch.
Security-sensitive fixes:
- cli/administration/deploy/development: break the unsafe import cycle by
extracting the env/arg helpers into a leaf module `env.py`, giving the
load-time DAG compose -> common -> deps -> env.
- web-app-bluesky login-broker server.js: prototype-pollution guard on
cookie parsing, CR/LF + control-char log sanitisation, HTML-escaped
exception text, and a ReDoS-bounded handle regex.
- web-app-baserow SSO: harden the open-redirect `next` handling.
- network diagnose probes: require TLS >= 1.2.
- actionlint installer: validate tar members against path traversal.
- decidim/apt-purge lint regexes: rewrite to avoid catastrophic backtracking.
- 6x Playwright URL regexes anchored; crs-k/stale-branches pinned to a commit SHA.
Mechanical cleanups: unused imports/globals/locals, empty-except comments,
import-and-import-from, implicit-string-concat, multiple-definition,
unreachable/useless-assignment.
False positives are intentionally left in place (Django settings patches,
Odoo __manifest__, side-effect imports, cross-module symbols).
Recurrence guards:
- import-linter contract (pyproject + tests/lint) locks the dev-deploy
package acyclic; a new cycle now fails CI instead of only CodeQL.
- new lint test requires every bandit (S###) `noqa` to carry a justification.
- BLE001 (blind-except) added to ruff as a ratchet (tests/roles exempt,
current production offenders grandfathered).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
compose.yml no longer hides services behind \`profiles: ["ci"]\`. The
profile existed solely as a no-op safety net against a bare
\`docker compose up\` in the repo root, but the top-level compose.yml is
already explicitly scoped to local dev / CI testing (production goes
through the Ansible roles under roles/<role>/templates/), so the safety
net is redundant with the documented scope.
Every \`--profile ci\` callsite collapses:
- compose.yml: \`profiles: ["ci"]\` removed from coredns + infinito
- Profile.args() deleted; Profile keeps is_ci() and
registry_cache_active() for the cache-override gating
- compose.py / down.py: profile.args() splat removed from the
docker-compose command builder; down.py drops its now-unused
Profile import
- 5 shell wrappers (install, lint, tests/code, meta/resolve/apps,
meta/resolve/diff/affected_roles): plain \`docker compose\` instead
of \`docker compose --profile ci\`
- tests/unit/.../test_profile.py: TestProfileArgs class removed
- tests/unit/.../test_compose.py: two cmd-shape asserts updated to
reflect the missing --profile pair
Docs:
- docs/contributing/artefact/files/compose.yml.md: the
\"every service MUST be behind a profiles: entry\" rule is gone
- docs/contributing/actions/debugging/ci.md: the
\`docker compose ... --profile ci config\` debug snippet drops the
flag; while touching the file the two em-dash clause separators in
the reproduction-profiles shell block are rewritten to colons
(documentation.md em-dash rule), and one list-item in
compose.yml.md is collapsed to one sentence per line for
consistency with its siblings
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The env.development marker file existed for two purposes that no longer
justify the wiring cost: a per-machine compose --env-file override layer
appended after auto-loaded .env, and a fallback signal for
detect_runtime() to return "dev" when the marker exists. With every
INFINITO_* default now collapsed into env/default.env -> .env,
persistent per-machine overrides are best expressed as
\`export INFINITO_FOO=…\` before invoking make (load.sh honors
caller-set values via setdefault). The marker fallback in
detect_runtime was redundant with the explicit RUNTIME=dev that every
dev-flow callsite already passes (fresh-*.sh, reset/inventory.sh).
Removed:
- Makefile mark-development target + .PHONY entry
- env.development from .gitignore / .dockerignore
- env_local conditional in compose.up()
- env_development conditional in down._compose_run()
- env.development conditional in scripts/meta/resolve/apps.sh and
scripts/meta/resolve/diff/affected_roles.sh
- env.development marker branch + the now-unused project_root parameter
from cli/meta/runtime/detect_runtime; __main__.py caller follows
- "Optional Development Marker" section in docs/contributing/environment/
setup.md and the marker reference in docs/administration/installation.md
Also drops two RUNTIME=='dev' guards in roles/web-app-bigbluebutton that
relied on the auto-detect path:
- the compose build etherpad task in tasks/02_compose.yml
- the freeswitch iptables-REJECT entrypoint override in
templates/compose.override.yml.j2
With auto-detection of "dev" gone the guards would otherwise only fire
on explicit RUNTIME=dev runs, silently regressing the local-dev path
operators rely on; both behaviors now apply unconditionally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four ci.env keys (INFINITO_DNS_IP, INFINITO_DOMAIN, INFINITO_DOMAIN_RE,
INFINITO_IP4) were only honored by docker compose --env-file, so processes
that sourced .env via load.sh saw INFINITO_IP4 as unset. As a result
networks.internet.ip4 in the Ansible playbook fell through to the
127.0.0.1 default and the DinD DNS-check handler failed on the deploy
server matrix.
Moving the defaults into env/default.env makes them flow through the same
.env / load.sh / os.environ chain as every other INFINITO_* key, so
\${INFINITO_IP4} in compose.yml, lookup('env', 'INFINITO_IP4') in
group_vars, and envsubst over the CoreDNS Corefile template all see the
same value deterministically.
Side fix: utils/env/parser.py now undoes the writer's \\\\/\\" escaping for
quoted values, closing a round-trip gap that surfaced once
INFINITO_DOMAIN_RE (the only value containing a literal backslash) started
flowing through the generator.
All --env-file env/ci.env call sites (compose.py, down.py, install/lint/
test wrappers, resolve scripts, test-dns workflow + script) are removed.
coredns.py now reads .env via parse_static_env, docs and unit/integration
tests follow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes a stack of env-pipeline regressions left over from the recent
INFINITO_* namespace refactor by:
* DOMAIN_PRIMARY now reads INFINITO_DOMAIN strictly (was: legacy DOMAIN
env var → defaulted to infinito.localhost, which never resolved
because CoreDNS serves the INFINITO_DOMAIN value, infinito.example).
* group_vars/all/08_networks.yml IPs (ip4/ip6/dns) now read
INFINITO_IP4 / INFINITO_IP6 / INFINITO_DNS_IP.
* PLAYWRIGHT_KEEP_ALL renamed to INFINITO_PLAYWRIGHT_KEEP end-to-end
(env-builder, role default, run_one.yml docker exec, playwright.config.js,
lint allowlist, docs anchor, CI workflow vars).
* PKGMGR_IMAGE renamed to INFINITO_PARENT_IMAGE; Dockerfile ARG drops
the inline default so env/default.env is the only SPOT.
* INFINITO_SRC_DIR centralized: Dockerfile ARG (no default) + ENV;
compose forwards it as build-arg, volume mount path, and container env;
every Python CLI + shell helper + CI workflow now reads it strictly
(os.environ[...] / "${INFINITO_SRC_DIR:?...}") instead of the literal
/opt/src/infinito.
* New entries in env/default.env: INFINITO_IP6=::1, INFINITO_PARENT_IMAGE,
INFINITO_PLAYWRIGHT_KEEP=false. All three propagated through
utils/env/handlers/passthrough.STATIC_KEYS and (where consumed in the
container) through compose.yml's environment block.
Validated end-to-end with `make test` (2360 tests across lint, lint-*,
test-lint, test-external, test-unit, test-integration).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promote the key from GHA-only (gha_passthrough) to always-emitted (passthrough)
so the deploy CLI can read it strictly via os.environ[...]. The previous
INFINITO_NO_BUILD layout left the key unset locally, which crashed
cli/.../up.py and cli/.../compose.py with KeyError after the env-pipeline
refactor that introduced gha_passthrough.
New semantics:
INFINITO_BUILD=1 build the local image (default; local mode)
INFINITO_BUILD=0 skip build, pull image (CI overrides via -e)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
empty-string fallback (which silently masks an unset key). The
nocheck check walks every physical line the call spans so an
autoformat-split call still honours the marker.
Cache namespace flattens to a single prefix:
INFINITO_PACKAGE_CACHE_* -> INFINITO_CACHE_PACKAGE_*
INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.
SPOT-strict rewrites:
- 19 sites: `something.get("INFINITO_X", "")` -> subscript
`something["INFINITO_X"]` (loud KeyError when env loader did
not run; all keys live in env/default.env).
- 2 handler-internal bootstrap reads suppressed with
`# nocheck: handler-bootstrap-read`.
- 30 runtime-only / driver-injected / test-fixture references
annotated with `# nocheck: <reason>` per-line.
- Unit tests that previously removed SPOT-guaranteed keys via
`{k: v ... if k \!= KEY}` or `clear=True` switch to setting
the key to "" explicitly, matching the new contract.
Side fixes:
- INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
shell-side defaults in common.sh to env/default.env and
passthrough.STATIC_KEYS.
- dns_systemd_is_operational() drops the dead
INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
variable was never set anywhere).
- INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
to `os.environ[KEY]`.
- scripts/tests/code/wrapper.sh exec args move into a bash
array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
own element line (SC2215 fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):
- cli/deploy -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror -> cli/contributing/mirror
The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".
Help system redesign in cli/core/{discovery,help,app}.py:
- Discovery now distinguishes runnable commands (directories with
__main__.py) from category folders (directories that only group
further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
as the title and its first paragraph as the one-line description in
the help output.
- "infinito" / "infinito --help" prints only the top-level entries
(one level deep). "infinito <category>" navigates into a folder and
shows its sub-folders and commands one level deep, with the README
title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
prefix recursively. N caps the depth (1-based); omit N for an
unbounded walk. The output uses Unicode tree connectors, an emoji
per entry (folder for categories, gear for commands), a globally
aligned description column, and multi-line wrapped descriptions
with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
("infinito meta callorder") rather than the bare leaf name, so the
output is copy-pasteable.
Convention doc + 17 CLI READMEs:
- New docs/contributing/artefact/files/cli/readme_md.md describes the
H1+paragraph layout the help system reads, and how it differs from
role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
and their immediate sub-folders, plus cli/meta/roles/applications
and its resolution sub-folder. Each follows the "title with trailing
emoji + single descriptive paragraph" convention.
Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.
Fixes uncovered by the move:
- import unittest.mock added to four dedicated-deploy tests so
unittest.mock.patch resolves under stdlib unittest discovery
(Python 3.13 no longer surfaces unittest.mock through "import
unittest" alone).
- parents[N] index in the new
cli/administration/inventory/validate/__init__.py corrected from
3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
substring assertions updated to match the new validator path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5.5