moby/buildkit was hardcoded in four places across two workflows and catthehacker/ubuntu in two act scripts, none of them reachable by the image mirror: utils/docker/image/discovery.py only scans roles/**/meta/services.yml. So the one image the CI build cannot proceed without was pulled straight from Docker Hub, unmirrored, which is what broke the fedora build in run 30160077828.
Declare both as mirror-only services on svc-runner, following the test-e2e-playwright precedent, so the existing mirror pipeline picks them up unchanged. New scripts/meta/resolve/image/service_ref.sh reads image+version back out of the declaration and is the single source for consumers outside Ansible; its mode argument is mandatory (upstream never touches the network, mirror prefers the GHCR tag and falls back to upstream). The fallback is why the mirror job can keep running in parallel with the build instead of gating it.
ACT_PLATFORM_IMAGE moves to an env handler so act consumers read the declared value via BASH_ENV, and workflow.sh now demands it instead of defaulting to a literal. Operator overrides still win through eb.setdefault.
Verified: service_ref.sh in all four paths (upstream, mirror hit, missing mode, invalid mode); the mirror path matches GHCRProvider.image_base() character for character and resolves against the live registry anonymously; ACT_PLATFORM_IMAGE resolves through the real BASH_ENV mechanism; make quality green on all four targets.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Mark the merged best-effort '|| true' sites (guide EXIT trap, baseline
fetch fallback) with shell-or-true nocheck reasons, parenthesize the
implicitly concatenated message strings ISC004 enforces since the ruff
0.16 pin, and drop the regex narration comments the comment lint flags
on the now-changed test files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
load.sh short-circuits on INFINITO_ENV_LOADED, so a shell that already
sourced the env kept a stale copy after 'make dotenv' regenerated .env
(make runs in a subshell and cannot reset the parent's guard), and a
freshly added key was missing until a new shell. Gate the short-circuit
on the .env mtime: re-source when it changed, still a fast no-op when it
did not. A stat failure falls back to the previous idempotent behaviour.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A push resolves the application roles that exist under roles/ locally but
not at origin/main and passes them as the orchestrator priority line, so
brand-new roles deploy and gate ahead of the regular matrix. The resolver
is stdlib-only (git ls-tree plus an application_id line scan) so it runs
on a bare runner without installing the resolver stack, and yields an
empty set when the baseline ref cannot be resolved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The discovery SPOT (cli.meta.ci.query) now runs one query shape per mode:
compose and host stay whole-role with variants packed into bundles, swarm
ranks and budget-cuts on per-variant rows and emits role#variant tokens
through apps.sh into variant_bundles, so the cut can pick a subset of a
role's variants and the --matrix view is the selection priority.
Replace the --unique flag with visible dna/clone columns (heaviest role
per dna cluster stays ahead of the cut), add the integrated column and
sort, count host bundles like compose, and give the deploy plan an id
counter, per-variant weights and display-width-aware alignment matching
the matrix renderer. Add i8ma*/i8pa* inspection aliases per mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cli.meta.ci.query is the single source of truth for the app-discovery
query (filter, coverage-first sort, lifecycle envelope, compose
--unique, INFINITO_MAX_JOBS cap with 'auto'); apps.sh shrinks to one
call through it. cli.meta.ci.plan renders one section per deploy mode
in the query's own order (priority line starred, capped regular line,
then the candidates behind the cut), one row per variant, glossary
symbols in the header, with --cli for fixed-width terminal tables.
New glossary symbols: skip, priority, distros, unavailable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cli.meta.ci.slots statically and conservatively counts every job the
CI run spawns outside the deploy matrices (reusable workflows opened
recursively, static matrices multiplied out, dynamic ones estimated,
worst entry-*.yml overhead included) and splits the remaining
256-job run budget between compose/swarm/host by the _SHARES weights,
with a per-mode floor and _SLOT_OVERRIDES pins. INFINITO_MAX_JOBS
defaults to 'auto', resolved per mode in apps.sh; the host workflow
drops its hardcoded cap. --matrix renders the budget as a table.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
test-deploy-single-node.yml bundles the compose and host deploy tests
behind one reusable workflow; the orchestrator gains a priority input
that deploys focus roles first per mode and runs the remaining roles
only after every priority job succeeded, excluded via the new
INFINITO_BLACKLIST discovery filter in apps.sh. entry-manual exposes
the priority field and treats priority-only dispatches as roles-only.
Workflow dependency diagram added under .github/workflows/README.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
meta/services.yml modes states where a role RUNS; a new meta/tests.yml
skip list deactivates TESTING a mode without touching that capability.
The table gains test_compose/test_swarm/test_host columns (base column
minus skip), --filter/--sort and the JSON/YAML payloads know the new
fields, and CI discovery (apps.sh) now filters on test_<mode> == true.
svc-storage-nfs-client is the first consumer and skips all three modes.
Also replaces the nine VS16-based header/lifecycle emojis (like the
old host/jobs/weight symbols) with East-Asian-Wide ones: terminals
render VS16 sequences 2 cells wide while the width model said 1,
shifting every header right of them off its column. A guard test pins
all table symbols to wide, VS16-free codepoints.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surfaced by make quality: alias comments on i8matiso/i8mativaso/i8s; self-path nocheck + drop two narration comments in load.sh (venv note kept in the header); shorten the host deploy job name to <=80 chars; move the legend rationale from an inline comment into the docstring. Also move utils/env/handlers/infinito/fork_{account,repository_url}.py into a fork/ subfolder (shared-prefix rule) and update the handler imports.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Host lane for invokable non-stack roles (drivers, backup, wireguard, swapfile, ...). modes.host.enabled declared on 56 such roles; new lint test_role_host_mode enforces it. The complexity matrix gains a host column (not stack and modes.host.enabled), --deploy-mode host, and host bundles (one per variant); apps.sh accepts INFINITO_DEPLOY_MODE=host and filters host == true. New test-deploy-host.yml workflow (INFINITO_MAX_JOBS=10) deploys discovered host roles via the mode-agnostic ci/all.sh path, wired into ci-orchestrator (modes default 'swarm compose host').
Also: complexity -s/--symbol compact view (emoji-only headers, true/false as check/cross, lifecycle stage symbols, grouped legend with per-entry explanations, display-width aligned); stack column moved after host. scripts/meta/env/load.sh made zsh-compatible by delegating to bash and importing INFINITO_/PYTHON/VENV/PATH back.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lifecycle envelope for test-deploy discovery is now driven by
INFINITO_LIFECYCLES (default: alpha beta rc stable) instead of a
hardcoded constant. apps.sh forwards it to the complexity CLI via a
new --lifecycles flag (comma- or whitespace-separated); the model
falls back to the built-in default when unset.
Manual CI (entry-manual) gains two override fields: 'lifecycles' to
override the envelope per run, and 'modes' (default 'swarm compose')
to pick which deploy jobs run. ci-orchestrator gates test-deploy-compose
and test-deploy-swarm on the selected modes and threads lifecycles into
both. The new key is registered in the passthrough handler so it lands
in the generated .env.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The complexity report orders roles coverage-first and caps them with --max-jobs; that order must drive selection. Once selected, re-sort the surviving roles alphabetically so the swarm and compose deploy matrices list jobs in a stable ascending order. The storage filter and variant_bundles both preserve input order, so the final matrix stays sorted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drive the test-deploy matrix discovery through the complexity report instead of the type CLI: apps.sh selects the mode's tested roles via the 'compose'/'swarm' column (which already bakes in invokable + tested-lifecycle + per-mode skip), orders them coverage-first, and hard-caps the cumulative per-mode runner budget.
- complexity gains --deploy-mode {compose,swarm} (makes 'bundles'/'jobs' the runner count for that mode: compose packs variants into size/storage bundles, swarm runs one per deployable variant) and --max-jobs N (keep only the leading rows whose cumulative jobs stay below N).
- apps.sh emits whole role names; variant_bundles/output_apps.sh expand them into the matrix unchanged. Manual INFINITO_WHITELIST becomes a 'name %% {…}' filter clause applied before the cut, so a manual run only counts the apps it actually invokes.
- INFINITO_DEPLOY_MODE is now required; INFINITO_MAX_JOBS, INFINITO_DISCOVERY_SORT and INFINITO_REQUIRED_STORAGE move to default.env (the SPOT) + the passthrough handler, so no shell-side defaults remain.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.
- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).
- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.
- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).
- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a `skip: [compose, swarm]` field on a role's meta/services.yml primary entity
so an app can opt out of the test-deploy matrix per deploy mode, replacing the
hardcoded exclude regex in apps.sh (the "TODO: Remove exclude condition"). The
filter threads through both discovery paths: the CI workflows pass
INFINITO_DEPLOY_MODE -> apps.sh -> `cli meta roles applications type
--skip-mode`, and roundtrip.sh honours the same get_role_skip SPOT per app/mode.
Apply skip: [compose, swarm] to svc-storage-nfs-server and svc-storage-nfs-client
(NFS storage is swarm-only and set up via the cluster inventory, not the matrix)
and web-opt-rdr-www (the previously regex-excluded app).
Covered by get_role_skip and skip_mode-filter unit tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Delete the per-deploy-type workflows (test-deploy-local/universal/
workstation) and the redundant act-all/act-app runner scripts (replaced
by make act-workflow). Only compose and swarm deploy tests remain.
Both now discover the full application matrix (every deployment type,
lifecycle-filtered to alpha/beta/rc/stable) instead of just the server/
web-* scope, via apps.sh resolving applications.type without --type.
Retire INFINITO_DEPLOY_TYPE entirely: discovery is now unconditional and
the deploy step never read it. Removed from apps.sh, ci/all.sh, the local
deploy scripts, the two workflows, default.env, the env passthrough list,
the dotenv generator test, the Makefile compose-deploy target, and docs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scripts/meta/wait/mirrors.sh embedded its logic as a `python - <<PY` heredoc
plus a bash skopeo/retry loop, and the heredoc imported iter_role_images from
utils.docker.image_discovery — a module that does not exist (it is
utils.docker.image.discovery). That raised ModuleNotFoundError, so the process
substitution yielded zero refs and the script logged "Nothing to wait for" and
exited 0, silently turning the fork-PR "wait for mirrored images" safeguard
into a no-op.
Move the whole thing into cli/contributing/mirror/wait/ (sibling to sync and
resolver): an argparse entrypoint that enumerates role images via
iter_role_images, computes each GHCR destination via GHCRProvider, and polls
GHCRProvider.tag_exists with --attempts/--sleep-seconds until all are present
or it times out. mirrors.sh becomes a thin env->args wrapper invoking
python -m cli.contributing.mirror.wait, matching scripts/meta/mirror/sync_missing.sh.
Behaviour is identical to the previous bash, minus the broken import.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds INFINITO_VENV_DIR=/opt/venvs/infinito to default.env (registered
in passthrough's STATIC_KEYS so make dotenv propagates it to .env) and
threads it through the build chain: compose.yml forwards it as a build
arg; scripts/image/push.sh asserts and --build-args it; the Dockerfile
declares the matching ARG/ENV and derives PYTHON/PIP/PATH from the
variable. scripts/meta/env/python.sh now prefers the value when set
before its fallback discovery so the SPOT is effective in env
resolution.
Fixes the dev image build regression where 'infinito --version' in
scripts/docker/entry.sh exited 127. Root cause: the Dockerfile RUN
SHELL uses bash -lc, which sources /etc/profile and resets PATH to
the Debian default, dropping ${INFINITO_VENV_DIR}/bin from it. The
fix invokes the script via its absolute path "${INFINITO_VENV_DIR}/bin/infinito"
so the post-install version check no longer depends on a PATH that
the login-shell init has just clobbered.
scripts/lint/wrapper.sh's existing PATH-reset note now references the
variable instead of the literal path. test-install-make.yml and
test-install-pkgmgr.yml read INFINITO_VENV_DIR from default.env up
front, pass it into their pkgmgr containers, and source the venv
activate script via the variable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
push_ci_policy.sh now hard-fails on missing GH_TOKEN/GITHUB_REPOSITORY
and, for non-main branches, calls gh pr list to detect whether the
push is already covered by an open pull request. If so, emits
should_run=false (skip_reason=pr-ci-covers-this-push) so the push
workflow exits cleanly and the pull_request workflow handles the
actual CI run.
To absorb the bot's push -> gh-pr-create latency on update/* and
alert-autofix-* branches, the PR check retries up to 10x with 2s
sleeps before falling through to a real CI run. Human branches
(feature/**, hotfix/**, fix/**, chore/**) check once with no delay.
entry-push-latest.yml's run-policy step now forwards GH_TOKEN so the
gh call works against the repo.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a detect-affected-roles job to entry-pull-request-change.yml that
runs the affected-roles resolver upfront on host python (via make
install) and emits whitelist + roles_only. ci-orchestrator skips
test-development when roles_only=true; done job tolerates the skip.
entry-manual derives roles_only from its whitelist input.
affected_roles.sh now invokes the resolver host-side only (compose-exec
fallback removed; it was a no-op in every existing caller because
compose was never up there, always emitting __ALL__ fail-safe). The
PR-flow gate consolidates what test-deploy-* used to recompute per
job; downstream effective_whitelist.sh now consumes the pre-resolved
whitelist verbatim.
Test fixture switched from a fake docker binary to a fake PYTHON
binary and clears BASH_ENV so the bash-startup env loader does not
reset PYTHON.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the seven separate deploy-* make targets with a single `make
deploy` recipe that maps short Make variables to the canonical
INFINITO_* environment variables and dispatches via the new
scripts/tests/deploy/local/deploy/main.sh router. main.sh and all
sub-scripts continue to read the long-form names, so Ansible passthrough
and container forwarding stay unchanged.
Short Make CLI now exposes apps, mode, purge, type, bundles, disabled,
and full_cycle as aliases. The router picks the verb folder
(initialize, reinstall, update) and scope (all, selection) based on the
chosen mode plus the presence of INFINITO_APPS or INFINITO_BUNDLES.
Also folds in the global INFINITO_TEST_DEPLOY_TYPE -> INFINITO_DEPLOY_TYPE
rename across default.env, passthrough handler, dotenv test, CI deploy
workflows, discovery scripts, and the server stage task.
Active contributor and agent-iteration docs, role-level Administration
and README files, the bundles README, and the deploy reset helper are
all retargeted to the new short CLI. docs/requirements/ entries stay
untouched as historical audit logs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compose.yml no longer hides services behind \`profiles: ["ci"]\`. The
profile existed solely as a no-op safety net against a bare
\`docker compose up\` in the repo root, but the top-level compose.yml is
already explicitly scoped to local dev / CI testing (production goes
through the Ansible roles under roles/<role>/templates/), so the safety
net is redundant with the documented scope.
Every \`--profile ci\` callsite collapses:
- compose.yml: \`profiles: ["ci"]\` removed from coredns + infinito
- Profile.args() deleted; Profile keeps is_ci() and
registry_cache_active() for the cache-override gating
- compose.py / down.py: profile.args() splat removed from the
docker-compose command builder; down.py drops its now-unused
Profile import
- 5 shell wrappers (install, lint, tests/code, meta/resolve/apps,
meta/resolve/diff/affected_roles): plain \`docker compose\` instead
of \`docker compose --profile ci\`
- tests/unit/.../test_profile.py: TestProfileArgs class removed
- tests/unit/.../test_compose.py: two cmd-shape asserts updated to
reflect the missing --profile pair
Docs:
- docs/contributing/artefact/files/compose.yml.md: the
\"every service MUST be behind a profiles: entry\" rule is gone
- docs/contributing/actions/debugging/ci.md: the
\`docker compose ... --profile ci config\` debug snippet drops the
flag; while touching the file the two em-dash clause separators in
the reproduction-profiles shell block are rewritten to colons
(documentation.md em-dash rule), and one list-item in
compose.yml.md is collapsed to one sentence per line for
consistency with its siblings
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The env.development marker file existed for two purposes that no longer
justify the wiring cost: a per-machine compose --env-file override layer
appended after auto-loaded .env, and a fallback signal for
detect_runtime() to return "dev" when the marker exists. With every
INFINITO_* default now collapsed into env/default.env -> .env,
persistent per-machine overrides are best expressed as
\`export INFINITO_FOO=…\` before invoking make (load.sh honors
caller-set values via setdefault). The marker fallback in
detect_runtime was redundant with the explicit RUNTIME=dev that every
dev-flow callsite already passes (fresh-*.sh, reset/inventory.sh).
Removed:
- Makefile mark-development target + .PHONY entry
- env.development from .gitignore / .dockerignore
- env_local conditional in compose.up()
- env_development conditional in down._compose_run()
- env.development conditional in scripts/meta/resolve/apps.sh and
scripts/meta/resolve/diff/affected_roles.sh
- env.development marker branch + the now-unused project_root parameter
from cli/meta/runtime/detect_runtime; __main__.py caller follows
- "Optional Development Marker" section in docs/contributing/environment/
setup.md and the marker reference in docs/administration/installation.md
Also drops two RUNTIME=='dev' guards in roles/web-app-bigbluebutton that
relied on the auto-detect path:
- the compose build etherpad task in tasks/02_compose.yml
- the freeswitch iptables-REJECT entrypoint override in
templates/compose.override.yml.j2
With auto-detection of "dev" gone the guards would otherwise only fire
on explicit RUNTIME=dev runs, silently regressing the local-dev path
operators rely on; both behaviors now apply unconditionally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four ci.env keys (INFINITO_DNS_IP, INFINITO_DOMAIN, INFINITO_DOMAIN_RE,
INFINITO_IP4) were only honored by docker compose --env-file, so processes
that sourced .env via load.sh saw INFINITO_IP4 as unset. As a result
networks.internet.ip4 in the Ansible playbook fell through to the
127.0.0.1 default and the DinD DNS-check handler failed on the deploy
server matrix.
Moving the defaults into env/default.env makes them flow through the same
.env / load.sh / os.environ chain as every other INFINITO_* key, so
\${INFINITO_IP4} in compose.yml, lookup('env', 'INFINITO_IP4') in
group_vars, and envsubst over the CoreDNS Corefile template all see the
same value deterministically.
Side fix: utils/env/parser.py now undoes the writer's \\\\/\\" escaping for
quoted values, closing a round-trip gap that surfaced once
INFINITO_DOMAIN_RE (the only value containing a literal backslash) started
flowing through the generator.
All --env-file env/ci.env call sites (compose.py, down.py, install/lint/
test wrappers, resolve scripts, test-dns workflow + script) are removed.
coredns.py now reads .env via parse_static_env, docs and unit/integration
tests follow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`make test-unit` (docker runner) reached run.sh inside the container via
`bash --login`, which never sources the project's env-loader. PYTHON stayed
unset, so the test invocation fell back to the system python3 — which has
none of the runtime deps (ruamel, ansible, bcrypt, requests, certifi,
humanfriendly), producing 132+ ModuleNotFoundError errors.
run.sh now sources scripts/meta/env/load.sh itself (idempotent via
INFINITO_ENV_LOADED), so the env layer applies regardless of how the shell
was spawned.
While load.sh's detect.sh (now scripts/meta/env/python.sh) correctly walks
/opt/venvs and ~/.venvs to find the venv, an ambient VIRTUAL_ENV from a
different project (e.g. /root/.venvs/pkgmgr in pkgmgr's container) would
hijack the resolution. python.sh now scans the project-named venv candidates
first and only falls back to VIRTUAL_ENV when no project venv exists.
Also moves scripts/meta/python/detect.sh to scripts/meta/env/python.sh so the
env-related helpers live under one directory; updates the load.sh source
pragma and the lint-test path that scans it for INFINITO_* registrations.
`make test-unit` goes from 1262 tests / 132 errors to 1999 tests / 0 errors.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Virgin pkgmgr containers ship without /opt/venvs, so the prior
'-d /opt/venvs && -w /opt/venvs' check fell through to ${HOME}/.venvs,
breaking the test-install-{make,pkgmgr} workflows that hardcode
'source /opt/venvs/infinito/bin/activate'. Probe the parent instead —
venv.sh mkdir -p's the leaf later.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls VENV/PYTHON/PIP/PYTHONPATH out of the dotenv handler pipeline
into scripts/meta/python/detect.sh. load.sh now sources detect.sh and
prepends ${VENV}/bin to PATH above the INFINITO_ENV_LOADED guard, so
every BASH_ENV re-entry re-resolves PYTHON the moment a venv
materialises mid-chain — the same fix as b79ed2869 but at the SPOT,
which retires the reload-trick from ansible.sh, python.sh and
pre-commit.sh.
Also renames INVENTORY_VARS_FILE → INFINITO_INVENTORY_VARS_FILE under
the new env-prefix lint, drops the 5 dead INFINITO_ENV_*_LOADED unsets
left over from the multi-loader era, and adds a lint guarding the
INFINITO_ prefix invariant on env/*.env keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four test-deploy workflows (server/universal/workstation/local)
each started the full compose stack just so scripts/meta/resolve/
apps.sh could call its meta-CLI inside the container. The stack
spin-up dwarfed the actual discovery query.
Introduce INFINITO_APP_DISCOVERY_RUNNER (host|docker, default
`docker` for local dev) alongside the existing INFINITO_TEST_RUNNER
/ INFINITO_LINT_RUNNER dispatch knobs. In host mode apps.sh runs
the meta-CLI through the venv python directly; docker mode keeps
the existing `docker compose --profile ci exec` path.
Workflows switch to host mode and replace `make up` with
actions/setup-python@v5 + `make install`. make install was already
a prereq of make up, so the install cost is unchanged -- only the
docker pull + container start time disappears.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new lint tests close gaps in the INFINITO_* SPOT contract:
* tests/lint/filesystem/env/test_default_env_keys_handled.py
Every default.env key must be wired through a handler.
* tests/lint/filesystem/env/test_used_keys_declared.py
Every INFINITO_* in .sh/.py must live in env/{default,ci}.env or
a handler; per-line `# nocheck:` suppresses runtime-only refs.
* tests/lint/filesystem/python/test_no_infinito_get_defaults.py
Forbid ANY default in `.get("INFINITO_*", ...)` -- including the
empty-string fallback (which silently masks an unset key). The
nocheck check walks every physical line the call spans so an
autoformat-split call still honours the marker.
Cache namespace flattens to a single prefix:
INFINITO_PACKAGE_CACHE_* -> INFINITO_CACHE_PACKAGE_*
INFINITO_REGISTRY_CACHE_* -> INFINITO_CACHE_REGISTRY_*
The five previously driver-injected paths (APT_LIST, NPMRC,
PIP_CONF, REGISTRY_PROXY_CONF, FRONTEND_CA_FILE) move into
env/default.env and passthrough.STATIC_KEYS; the
`cache_env_overrides()` helper and its two call sites are gone
along with the test-fixture override for FRONTEND_CA_DIR.
SPOT-strict rewrites:
- 19 sites: `something.get("INFINITO_X", "")` -> subscript
`something["INFINITO_X"]` (loud KeyError when env loader did
not run; all keys live in env/default.env).
- 2 handler-internal bootstrap reads suppressed with
`# nocheck: handler-bootstrap-read`.
- 30 runtime-only / driver-injected / test-fixture references
annotated with `# nocheck: <reason>` per-line.
- Unit tests that previously removed SPOT-guaranteed keys via
`{k: v ... if k \!= KEY}` or `clear=True` switch to setting
the key to "" explicitly, matching the new contract.
Side fixes:
- INFINITO_DNS_DOMAIN + INFINITO_DNS_HOSTS_FILE move from
shell-side defaults in common.sh to env/default.env and
passthrough.STATIC_KEYS.
- dns_systemd_is_operational() drops the dead
INFINITO_DNS_FORCE_HOSTS_FALLBACK override branch (the
variable was never set anywhere).
- INFINITO_WORKER_FETCH switches from `os.environ.get(KEY, "1")`
to `os.environ[KEY]`.
- scripts/tests/code/wrapper.sh exec args move into a bash
array so `# nocheck:` on INFINITO_TEST_TYPE can live on its
own element line (SC2215 fix).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace 8 bash env modules with a single Python generator
(cli.meta.env) that materialises env/static.env + runtime context
into a gitignored .env. Each variable lives in its own handler
under utils/env/handlers/; defaults carry inline one-line `# ...`
comments.
Rename to the INFINITO_* schema:
- workflow inputs: APPS, SERVICES_DISABLED, CMD, DEBUG, WHITELIST,
BUNDLES, FULL_CYCLE, VARIANT, RUN_FLAGS, LIMIT_HOST, PARALLEL
- inventory: INVENTORY_DIR / INVENTORY_FILE / HOST_VARS_FILE,
PW_FILE -> INVENTORY_PASSWORD_FILE
- venv: VENV_BASE / VENV_FALLBACK / VENV_NAME
- new SPOT defaults: COMPILE_SILENCE, CONTAINER_NO_CA,
WAIT_HEALTH_TIMEOUT_S, PACKAGE_CACHE_FRONTEND_INIT_IMAGE,
promoted IMAGE_TAG from GHA-only to general passthrough
compose.yml hardening:
- inline \${VAR:-default} removed; strict \${VAR:?...} everywhere
- new integration lint blocks regressions
Tooling:
- scripts/install/all.sh owns the stamp-file install logic
(Makefile target delegates)
- scripts/meta/env/load.sh is the single shell entry point;
8 bash modules deleted
- new shell lint: no INFINITO_* defaults outside env/static.env
Tests:
- 93 new unit tests for utils/env/{builder,parser,runtime,writer}
- restructure tests/lint into repository/ (file-content rules) vs
filesystem/{shell,python,env}/ (filesystem-shape rules)
- drift + smoke integration tests against the generator
Docs:
- variables.md is now a pointer; per-variable docs live next to
the key in env/static.env / env/ci.env
- compose.yml.md drops the Default column
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirror scope.sh's is_markdown_or_rst skip-list in affected_roles.sh so a
PR that touches both a role and a doc file outside roles/<role>/ resolves
to that role only, not __ALL__. Covered by tests/integration/meta/workflows/test_diff_affected_roles.py.
Three classes of drift cleaned up across 85 .md files:
- Em-dashes: replaced ` — ` with the contextually correct separator
(colon for `term: explanation`, period to split clauses across
sentences/source lines, parentheses for asides, semicolon for
tight-linked continuations). Two intentional exceptions remain
in `docs/contributing/documentation.md` (the rule literal) and
inside a fenced ```sh block in `docs/contributing/actions/debugging/ci.md`.
- SPOT self-labels: rewrote "This is the SPOT for X" to describe
the subject directly ("This directory holds X" / "This file
describes X"). Cross-references to other artefacts' canonical
role remain unchanged.
- Heading-emoji position: moved leading-emoji headings (`# 🧩 Title`)
to trailing position (`# Title 🧩`) per the headline rule that
emojis MUST come after the title text.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
