WP 6.3+ renders the block editor inside an `<iframe name="editor-canvas">`,
so the existing top-level `getByRole("textbox", { name: /add title/i })`
no longer reached the title field on freshly-deployed instances and the
Discourse round-trip spec timed out. Now probes the iframe first and
falls back to the top-level locator for older WP versions.
Also pins `wordpress.version` to `7.0.0` so the spec assertion targets
a known compatible editor revision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dashboard card icons were rendered with `http://172.17.0.1:8044/<slug>.svg`,
the in-cluster sync URL that only the Ansible runner can reach. The
browser saw `NS_ERROR_CONNECTION_REFUSED` for every Simple Icons card.
The `add_simpleicon_source` filter now takes two URLs: it probes the
local sync URL (plain HTTP, no TLS, no redirect chain — reachable
from the deploy container's `requests` without a CA bundle) and
writes the public `lookup('tls', 'web-svc-simpleicons', 'url.base')`
URL into `icon.source`. Cards render `<img src="https://icon.<DOMAIN>/<slug>.svg">`,
which is reachable from the browser and CSP-permitted (`img-src *`).
Drops the dead local-cache scaffolding (static-subdir vars, volume
mount, `Ensure directory` task, refresh-download step) since the
browser now fetches Simple Icons directly. Renames the
previously-static `templates/style.css.j2` to `files/style.css`
(no Jinja markers, matches the new template-uses-jinja lint warning).
Adds a Playwright spec `test-icons.js` that asserts the regression:
no card may still reference `172.17.0.1`, at least one card must
resolve under `icon.<DOMAIN>`, with a favicon fallback path when the
Simple Icons service is disabled.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a warning-only lint that walks every `roles/<role>/templates/**/*.j2`
file (via the cached `iter_project_files_with_content` helper) and flags
those that contain no `{{ }}`, `{% %}`, or `{# #}` markers. Each
offender gets a GitHub Actions `::warning ::` annotation pointing at
the file path, plus a consolidated stdout list for local runs, with
the suggestion to move the file under `roles/<role>/files/` and load
it via `copy:` / `script:`. The test does not assert — purely warning
surface so we can drain the backlog incrementally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`lookup('image', '<role>', '<svc>', 'ref')` only consulted
`images_overrides.*` plus role defaults, which the mirror pipeline
never populates — the pipeline writes into
`applications.<role>.services.<svc>.{image,version}`. As a result,
`sys-svc-container`'s `BUSYBOX_IMAGE` / `NODE_IMAGE`,
`sys-ctl-hlth-csp`'s `HEALTH_CSP_IMAGE`, and `test-e2e-playwright`'s
`TEST_E2E_PLAYWRIGHT_IMAGE` always resolved to the raw upstream ref
and hit Docker Hub directly. In CI this surfaced as
`toomanyrequests: ...unauthenticated pull rate limit` on
`busybox:1.37`.
Switches every productive `lookup('image', ...)` to
`lookup('config', '<role>', 'services.<svc>.image|version')` — the
same path templates like `web-app-oauth2-proxy` already use, which
resolves through the merged `applications.*` map and so transparently
picks up the GHCR mirror override written by the inventory creator.
Removes the now-unused `plugins/lookup/image.py` plugin and its unit
test (zero remaining consumers across roles, scripts, and tests).
Rewrites the image-origin / image-mirror / dockerfile docs to teach
the `lookup('config', ...)` pattern, dropping references to the
deleted plugin and the dead `images_overrides.*` variable. Doc edits
follow the `documentation.md` rules (no em dashes, one sentence per
line, no historical phrasing).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CI `📊 Top role runtimes` step parses `$ANSIBLE_LOG_PATH` on the
runner, but ansible-playbook actually runs inside the dev container via
`docker compose exec` — and the env var was never forwarded there, so
the log file never materialised. Even when set, the log would have
lived inside the container, invisible to the runner-side parser.
Forwards `ANSIBLE_LOG_PATH` through `compose.exec`'s `extra_env` so
ansible-playbook writes the log at the configured path inside the
container, then `docker cp`s it back to the runner from the cleanup
trap before the stack is torn down.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tags `Free disk space` / `Enlarge swap space` / `Resolve repository owner` /
`Install jq` / `Deploy app` / `Dump runner state` across the four
`test-deploy-*` flows plus `images-build-ci`, `images-cleanup-ci`, and
`test-dns` — mirroring the existing 📊 prefix on `Top role runtimes`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure cosmetic YAML reflow of the `or`-chain in the refresh handler.
Same operands, same precedence — no behavioural change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Splits mixed-content target descriptions into a single description line
plus `Note:` follow-ups, and dresses the targets that carry real
parameters (`compose-deploy`, `compose-exec`, `compose-inner-run`) with
explicit `Usage:` / `Example:` / `Param <name>:` lines. Adds a
`variant=<idx>` Make var alias to `compose-deploy` so multi-variant
pinning works without dropping to `INFINITO_VARIANT=` env prefixes.
Removes redundant inline `compose-deploy` / `compose-inner-run` param
enumerations from the docs and replaces them with
`make help target=<name>` pointers, so the help banner stays the SPOT.
Also fixes SC2016 in `scripts/make/help.sh` (literal backticks in the
help banner) with a scoped suppression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `make compose-playwright role=<role> [pw="..."] [keep=true]` as the
canonical inner-loop entry point for rerunning a role-local Playwright
spec against the live stack — replaces the raw
`scripts/tests/e2e/rerun-spec.sh` invocation across docs.
Defines a small comment-block schema for Makefile targets (description
line on top, followed by `Usage:` / `Example:` / `Note:` / `Param <name>:`
markers). Lint enforces it; `make help` shows the description per target
in the short list and `make help target=<name>` prints the full block.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move `COMPOSE_HTTP_TIMEOUT`, `DOCKER_CLIENT_TIMEOUT`, `DOCKER_BUILDKIT`
and `COMPOSE_DOCKER_CLI_BUILD` from per-task `environment:` blocks to
the play-level `environment:` directive in playbook.yml so every Ansible
task that shells out to docker/compose picks them up automatically.
Removes three duplicate blocks (sys-svc-compose handlers `compose pull`
+ `Build compose`, web-app-matrix `compose up`, web-app-bigbluebutton
`create compose.yml`). The `compose up` handler keeps its
`COMPOSE_PARALLEL_LIMIT` (not in the centralised set).
The env-gated DOCKER_BUILDKIT/COMPOSE_DOCKER_CLI_BUILD toggle did a bare
`os.environ["INFINITO_BUILDKIT_ENABLED"]` lookup which KeyError'd in any
subprocess where the variable wasn't exported (notably the cert-deploy
systemd oneshot, whose unit only sets `compose_cmd`). The KeyError exited
/usr/bin/compose with rc=1, which failed the `mtn-cert-deploy.*.service`,
which never restarted `mailu_front` to pick up the freshly issued certs,
which left dovecot dead and IMAP/STARTTLS ports closed, which made
Roundcube loop on /webmail/sso.php after OIDC callback, which surfaced as
chrome-error://chromewebdata/ (net::ERR_TOO_MANY_REDIRECTS) in the mailu
Playwright spec and aborted the whole playbook run on the post-Playwright
gate.
The block is now removed entirely: it was never reachable under systemd,
and buildkit is the docker default since 23.0 so the off-switch carries
no real value here. INFINITO_BUILDKIT_ENABLED remains defined in
default.env / compose.yml / inventory passthrough for any consumer that
wants to read it, just not from compose.py.
Verified in-container: KeyError reproduced under `env -i ... compose`,
fix applied, cert-deploy systemd unit Result=success on retrigger,
mailu_front auto-restarted with dovecot up, ports 10143/993/etc. open,
mailu Playwright spec (5 tests) now 3 passed / 2 skipped / 0 failed
against the live stack (was 1/2/2 before).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `Set systemctl service state` handler called the systemd module with
the resolved `{{ system_service_name }}` (e.g.
`mtn-cert-deploy.7.0.0.infinito.nexus.service`), but the subsequent
`systemctl show -p Result/ExecMainStatus` queries used
`{{ system_service_id }}.infinito.nexus.service` (e.g.
`sys-ctl-mtn-cert-deploy.infinito.nexus.service`) — a different,
non-existent unit. systemctl returned the default `success`/`0` for the
non-existent unit, so the diagnostic-on-failure task was skipped and
real oneshot failures (Mailu cert-deploy in the test-deploy-server CI
runs) were silently swallowed by the playbook.
Use `{{ system_service_name }}` for both queries so the actual unit's
result is inspected, and add `systemd_set_state.failed` to the
diagnostic trigger as a belt-and-suspenders against any future
systemctl-show false positives.
Path-style invocation in scripts/system/network/dns/common.sh failed
with `ImportError: attempted relative import with no known parent
package` because cli/meta/domains/__main__.py uses
`from . import PROJECT_ROOT`. The DNS host generator silently fell
back to a static 3-host list on every bootstrap run (visible in the
test-development CI logs).
Switch the caller to `PYTHONPATH=<repo> python3 -m cli.meta.domains`
so the package context resolves. Keeps the PROJECT_ROOT SPOT
(tests/lint/filesystem/python/test_project_root_import.py) intact —
no local redefinition.
Also: one-line YAML rewrap in compose/cache.override.yml (formatter).
`entry.sh true` skips `run_local_build` because `true` is treated as
the exec target rather than a flag, so the in-container reinstall step
never ran. `--compile -- true` triggers the build first and then runs
`true` as a no-op exec target.
Catches refactor drift across `.sh`, `Makefile` recursive calls, and
GitHub workflows. Includes fixes for the 5 bugs the new test surfaced
on first run: `make -C ROOT down`/`up` in the network stack refresh
helper (missed during the compose-* rename) and four
`make compose-deploymode=…` / `compose-deployapps=…` typos in
07/08/09_deploy_*.sh plus three docstring/echo references
(missing space after `compose-deploy`, introduced by the
deploy → compose-deploy sed-rename).
Move the deploy router into the compose- domain prefix to match the
rest of the local stack lifecycle (compose-up/down/exec/...). Reorder
its block alphabetically between compose-down and compose-entity-purge
to keep `make help` and the alphabetical-targets lint happy.
Update every active call site: contributor docs, agent-iteration docs,
role-level Administration/README/TODO files, the bundles README, the
test-environment scripts, the in-repo deploy helpers and main.sh, and
the server stage task. The two wildcard prose references (`make
deploy-*` in CLAUDE.md and testing/playwright.md) become single
`make compose-deploy` mentions since no other deploy-* target exists.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the three container-* targets into the compose- domain prefix to
match their actual scope (compose stack maintenance) and to make `make
help` group them with the other compose lifecycle commands:
- container-purge-entity -> compose-entity-purge
- container-purge-system -> compose-system-purge
- container-refresh-inventory -> compose-inventory-refresh
Update the prereq chain (compose-system-purge -> compose-entity-purge),
the local-deploy testing doc table, and reorder the block so the
compose- family stays alphabetically sorted between compose-down and
compose-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Set `.DEFAULT_GOAL := help` so bare `make` prints the target index
instead of falling through to the alphabetically-first recipe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reorder every target block in the Makefile so the definitions land in
strict ASC order — the order broke during the cascading domain-prefix
renames (install-agent, security-apparmor-*, etc.) and now matches what
test_makefile_targets_sorted_ascending requires.
Rewrite the two for-loops in test_phony_declarations as list-comprehension /
list.extend forms so ruff PERF401 stops complaining and the file passes
autoformat.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Place a single `.PHONY: <target>` line directly above each target's
comment block instead of clustering all phony declarations at the top
of the Makefile. The `help.sh` parser already resets its comment state
on lines beginning with `.`, so the PHONY entry must sit ABOVE the
comment (not between comment and target) to keep `make help` accurate.
Add the previously-missing PHONY entries for `dotenv` and `dotenv-force`
so every task target carries one.
Introduce `tests/lint/filesystem/makefile/test_phony_declarations.py`
to enforce two invariants going forward:
- every target has a matching `.PHONY: <target>` above its comment, and
- every `.PHONY: <target>` references an actually-defined target (no orphans).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `Debug: dump status + ansible_env + hostvars` task ran
`hostvars[inventory_hostname] | to_nice_yaml`, which can't serialise
Ansible's internal `<<Omit>>` sentinel that surfaces inside hostvars
once any role references `omit` defaults. The task was MODE_DEBUG-gated
diagnostic code, not part of any production path, so remove it entirely
rather than wrap it in an error-suppressing workaround.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the seven separate deploy-* make targets with a single `make
deploy` recipe that maps short Make variables to the canonical
INFINITO_* environment variables and dispatches via the new
scripts/tests/deploy/local/deploy/main.sh router. main.sh and all
sub-scripts continue to read the long-form names, so Ansible passthrough
and container forwarding stay unchanged.
Short Make CLI now exposes apps, mode, purge, type, bundles, disabled,
and full_cycle as aliases. The router picks the verb folder
(initialize, reinstall, update) and scope (all, selection) based on the
chosen mode plus the presence of INFINITO_APPS or INFINITO_BUNDLES.
Also folds in the global INFINITO_TEST_DEPLOY_TYPE -> INFINITO_DEPLOY_TYPE
rename across default.env, passthrough handler, dotenv test, CI deploy
workflows, discovery scripts, and the server stage task.
Active contributor and agent-iteration docs, role-level Administration
and README files, the bundles README, and the deploy reset helper are
all retargeted to the new short CLI. docs/requirements/ entries stay
untouched as historical audit logs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The MODE_DEBUG dump in tasks/stages/01_constructor.yml crashed
`to_nice_yaml` on the internal `<<Omit>>` sentinel that Ansible plants
into `vars`. Dump `hostvars[inventory_hostname]` instead and drop the
now-unused `vars` allowance from the jinja variable-definition lint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Split the local deploy scripts into a verb-based hierarchy under
scripts/tests/deploy/local/deploy/apps/{initialize,reinstall,update}/
plus scripts/tests/deploy/local/deploy/bundles/, with each verb folder
exposing all.sh (full discovery) and selection.sh (one or more apps via
INFINITO_APPS). The Makefile targets retain their existing names and now
dispatch to the new paths. The reinstall and bundles flows take over the
down + up cycle that used to be a Make prerequisite, and bundles/fresh.sh
now correctly delegates to apps/reinstall/selection.sh (the previous
delegate path was broken after the move). Each new directory carries a
README.md documenting scope, entry points, and when to use which verb.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
\`utils/env/writer.py\` rejects multi-line values because bash \`source\`
and docker-compose \`--env-file\` cannot round-trip them
(introduced by a3782bf8d). \`scripts/tests/code/run.sh\` simultaneously
appended \`accept-flake-config = true\` to NIX_CONFIG before invoking
\`make setup\`, producing exactly the multi-line shape the writer
refuses. The test-environment workflow tripped this on every run
where \`06_test.sh\` → \`make test\` → \`scripts/tests/code/run.sh\`
fired (see actions/runs/26227822536/job/77180908867).
Fix the producer side, not the writer's layering:
* utils/env/handlers/nix_config.py: pull only the \`access-tokens\`
line out of NIX_CONFIG and write that single line. Other Nix
settings (\`accept-flake-config\`, \`experimental-features\`, …)
already live in /etc/nix/nix.conf via the Dockerfile inside the
dev container; host operators set them once in
~/.config/nix/nix.conf.
* scripts/tests/code/run.sh: drop the NIX_CONFIG mutation block.
It was a no-op for in-container test runs (the container's
/etc/nix/nix.conf already has accept-flake-config) and an active
bug for the env-pipeline (multi-line value crashed make dotenv).
The writer keeps its multi-line guard unchanged — the contract is
that .env carries single-line values, and producers must respect it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Enable the ansible.posix.profile_roles callback alongside profile_tasks
so every deploy run emits a per-role runtime block. Each test-deploy
workflow now writes the Ansible log to ANSIBLE_LOG_PATH and runs a new
post-step that parses the block, aggregates duplicate role entries,
sorts by total duration, and writes the top-N (default 20, override via
INFINITO_PROFILE_TOP_N) as a Markdown table to $GITHUB_STEP_SUMMARY so
the table appears on the Actions job overview page. Includes 15 unit
tests covering aggregation, ANSI stripping, task/total filtering,
env-knob handling, and step-summary writes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four follow-ups uncovered by \`make test\` after 3c9db6142:
* tests/lint/ansible/variables/test_role_and_group_vars_used:
TLS_ENABLED is declared in group_vars/all/02_tls.yml but never
read via \`{{ TLS_ENABLED }}\`; it is consumed by
plugins/lookup/{tls,email,cert}.py, which the YAML/Jinja-scoped
lint cannot see. Tag the declaration with the project's standard
\`# nocheck: unused-var\` marker and document the consumer.
* tests/lint/filesystem/python/test_no_raw_project_walk: the new
test_no_ansible_env_lookups.py used \`Path.rglob("*")\` for its
walk, which is the exact shape the no-raw-walks lint forbids in
test code. Switch to \`iter_project_files(extensions=_SCAN_EXTS)\`
with a relative-path prefix filter; same coverage, shared cache.
* tests/unit/cli/administration/deploy/development/test_inventory_vars_file_spot:
the new lint test legitimately names the dev inventory path
(\`inventories/development/default.yml\`) in its failure message.
Add it to the SPOT-drift-guard allowlist with a justification.
* tests/integration/ansible/jinja/test_variable_definitions: the
debug task added in 0884eb4e2 prints \`ansible_env\` and \`vars\`,
both Ansible magic variables. Add them to the test's reserved-name
list so the static analyzer stops flagging them as undefined.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
\`lookup('env', 'INFINITO_*')\` calls in group_vars / roles couple
deployment-agnostic Ansible code to default.env, which only exists in
dev/CI. Production deploys never source .env, so the lookups silently
fall back to their group_vars defaults and the inventory's intent gets
lost on the way (see the IP4 host_vars-overlay incident fixed in
8980b00b1).
Migrated keys (group_vars/roles -> inventory):
- DOMAIN_PRIMARY (00_general.yml)
- BUILDKIT_ENABLED (00_general.yml)
- CA_CONTAINER_ENABLED (02_tls.yml)
- defaults_networks.internet.{ip4,ip6,dns} (08_networks.yml)
- TEST_E2E_PLAYWRIGHT_KEEP (roles/test-e2e-playwright/defaults/main.yml)
group_vars and the role default now carry literal fallback values;
the dev/CI inventory at \`inventories/development/default.yml\` pulls
the live values from .env via straight \`lookup('env', 'INFINITO_*')\`
calls without any \`| default(...)\` filter, because load.sh guarantees
the keys are set whenever .env exists.
The convention is enforced by the new lint test
\`tests/lint/filesystem/env/test_no_ansible_env_lookups.py\`, which
scans group_vars/, tasks/, roles/, and playbook.yml for any
\`lookup('env', 'INFINITO_*')\` against a default.env key and fails
with a pointer to the dev inventory as the injection site.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
\`--workers\` for credentials generation kept a hardcoded default of 4
regardless of host capacity. Using \`os.cpu_count() or 4\` scales the
default to whatever the runner actually has (with the same 4-worker
floor as a safety net for the rare case where the OS does not report
a CPU count), so a 16-core developer machine no longer pessimistically
ceilings credentials generation at 4 while CI runners keep the
same effective behaviour they had before.
Stays at the same SPOT (argparse default) so callers can still
override per-invocation with \`--workers N\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`python -m cli.console -h` and `--help` now print a local usage
summary and exit without importing `cli.console.repl`. The
description covers the wrapper's behaviour (prefix stripping,
help-alias handling, Ctrl+C semantics, exit tokens) so callers can
discover the REPL surface without entering it.
The repl import is moved into the run branch so the help path
stays cheap and free of repl-side imports.
Wrap `python -m cli` in a host-side REPL launched via `make console`. The
loop normalizes input (strips leading `infinito`/`cli`, treats bare `help`
as `--help`), ignores Ctrl+C so accidental input cancels don't kill the
session, and exits only on `exit`/`quit`/`:q` or Ctrl+D. Includes a banner
with help/web/docs/license/author hints. Drops the now-redundant
`infinito --help` footer from `make help`. Adds 22 unit tests covering
normalization, subprocess dispatch, and main-loop control flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause of the DinD DNS-check failure on test-deploy-server
(networks.internet.ip4 falling through to 127.0.0.1 despite
INFINITO_IP4=172.30.0.10 reaching every link of the env pipeline):
cli/administration/inventory/provision/cli.py declared --ip4 with a
hardcoded default of "127.0.0.1" and ensure_host_vars_file wrote that
value unconditionally into host_vars/<host>.yml. group_vars'
defaults_networks.internet.ip4 resolves "172.30.0.10" via
\`lookup('env', 'INFINITO_IP4')\`, but tasks/stages/01_constructor.yml
merges \`defaults_networks | combine(networks | default({}, true))\` so
host_vars wins. INFINITO_IP4 looked broken end-to-end; the breakage
actually lived in the provision-CLI host_vars overlay.
INFINITO_DNS_IP and INFINITO_DOMAIN propagated correctly because
the provision CLI had no --dns and the --primary-domain write was
None-gated and never invoked.
While here, --ssl-disabled and --primary-domain are removed too:
neither had any effect in the dev flow. provision/cli.py applies
\`--vars-file inventories/development/default.yml\` AFTER
ensure_host_vars_file via apply_vars_overrides_from_file, whose
_deep_update_commented_map unconditionally overwrites
(\`target[key] = value\`). The dev vars-file always set
\`TLS_ENABLED: true\`, so \`--ssl-disabled\` (which forced
\`TLS_ENABLED: false\` in host_vars) was systematically reverted to
true. Dev/CI ran with TLS enabled the whole time. Inventory stays at
\`TLS_ENABLED: true\` to preserve that effective behaviour.
Removed from cli/administration/inventory/provision/cli.py:
- --ip4, --ip6: defaulted to 127.0.0.1 / ::1, shadowed group_vars
- --primary-domain: never set by any caller, None-gated write
- --ssl-disabled: always re-overwritten by --vars-file later in the
same provision pass
Removed from cli/administration/inventory/provision/host_vars.py the
matching writes in ensure_host_vars_file; the function now only sets
\`ansible_connection: local\` for localhost-class hosts. Defaults for
DOMAIN_PRIMARY / TLS_ENABLED / networks.internet.{ip4,ip6} live in
group_vars (env-driven) and the inventory vars-file (deployment-wide
flags); host_vars stops duplicating them.
Callers cleaned up:
- cli/administration/deploy/development/inventory/builder.py
- cli/administration/deploy/container/command.py
- scripts/tests/deploy/local/utils/fresh-kept-all-init-and-deploy.sh
- scripts/tests/deploy/local/reset/inventory.sh
Tests + docs follow:
- tests/unit/cli/administration/inventory/provision/test_host_vars.py
now asserts host_vars no longer carries baked DOMAIN_PRIMARY /
TLS_ENABLED / networks defaults
- docs/administration/deploy.md drops --ssl-disabled from the
provision example
- docs/contributing/design/inventory.md re-scopes the host_vars
column: the DOMAIN_PRIMARY / TLS_ENABLED / networks.internet.{ip4,ip6}
values arrive via --vars / --vars-file overlays, otherwise group_vars
provides the env-driven default
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnostic for the INFINITO_IP4-doesn't-propagate-to-Ansible CI symptom
seen in test-deploy-server (run 26227822536, job 77184640736):
networks.internet.ip4 keeps falling through to the 127.0.0.1 default
even though INFINITO_IP4=172.30.0.10 is wired through default.env,
the passthrough handler, the .env generator, and the compose env
block. INFINITO_DNS_IP and INFINITO_DOMAIN flow through the same
mechanism and land in Ansible correctly, so the asymmetry must be
visible in either the controller process env (ansible_env) or the
play namespace (vars). Dumping both at the start of the constructor
stage, alongside the existing status lookup, gives a complete
snapshot to diff against on the next CI run.
The dump is gated on MODE_DEBUG, so it stays a no-op in non-debug
deploys.
Also tighten roles/sys-svc-container/files/test-dns.sh: the
BUSYBOX_IMAGE / NODE_IMAGE env vars now fail loud via \${VAR:?...}
instead of silently defaulting to busybox:1.37 / node:26-alpine.
With the handler in roles/sys-svc-container/handlers/main.yml
unconditionally injecting both via the image-lookup plugin, a missing
override is a real misconfiguration, not something a stale literal
should paper over.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
compose.yml no longer hides services behind \`profiles: ["ci"]\`. The
profile existed solely as a no-op safety net against a bare
\`docker compose up\` in the repo root, but the top-level compose.yml is
already explicitly scoped to local dev / CI testing (production goes
through the Ansible roles under roles/<role>/templates/), so the safety
net is redundant with the documented scope.
Every \`--profile ci\` callsite collapses:
- compose.yml: \`profiles: ["ci"]\` removed from coredns + infinito
- Profile.args() deleted; Profile keeps is_ci() and
registry_cache_active() for the cache-override gating
- compose.py / down.py: profile.args() splat removed from the
docker-compose command builder; down.py drops its now-unused
Profile import
- 5 shell wrappers (install, lint, tests/code, meta/resolve/apps,
meta/resolve/diff/affected_roles): plain \`docker compose\` instead
of \`docker compose --profile ci\`
- tests/unit/.../test_profile.py: TestProfileArgs class removed
- tests/unit/.../test_compose.py: two cmd-shape asserts updated to
reflect the missing --profile pair
Docs:
- docs/contributing/artefact/files/compose.yml.md: the
\"every service MUST be behind a profiles: entry\" rule is gone
- docs/contributing/actions/debugging/ci.md: the
\`docker compose ... --profile ci config\` debug snippet drops the
flag; while touching the file the two em-dash clause separators in
the reproduction-profiles shell block are rewritten to colons
(documentation.md em-dash rule), and one list-item in
compose.yml.md is collapsed to one sentence per line for
consistency with its siblings
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
With env.development and env/ci.env both gone, env/default.env was the
last inhabitant of the directory. Hoisting it to the repo root removes
the empty-shell indirection: the SPOT is now [default.env](default.env),
sitting next to .env (its generated companion) and other repo-root
configuration artefacts like compose.yml, Dockerfile, and Makefile.
Path updates land in:
- generator: cli/meta/env/__main__.py (REPO_ROOT / "default.env")
- workflow: .github/workflows/test-install-make.yml (awk reads from
default.env, error message updated in lockstep)
- lint tests: test_used_keys_declared.py and test_infinito_prefix.py
switched from globbing env/*.env to targeting default.env directly;
test_default_env_keys_handled.py points at the new path
- drift test: test_inventory_vars_file_spot.py whitelist + STATIC_ENV
- smoke test: tests/integration/meta/env/test_dotenv_generator.py
STATIC_ENV + every docstring/error message
Doc-string and prose references in utils/env/*, scripts/*,
cli/administration/deploy/development/{common,coredns}.py,
Makefile, Dockerfile, and every doc page that mentioned env/default.env
are rewritten in the same pass so future readers see one path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The env.development marker file existed for two purposes that no longer
justify the wiring cost: a per-machine compose --env-file override layer
appended after auto-loaded .env, and a fallback signal for
detect_runtime() to return "dev" when the marker exists. With every
INFINITO_* default now collapsed into env/default.env -> .env,
persistent per-machine overrides are best expressed as
\`export INFINITO_FOO=…\` before invoking make (load.sh honors
caller-set values via setdefault). The marker fallback in
detect_runtime was redundant with the explicit RUNTIME=dev that every
dev-flow callsite already passes (fresh-*.sh, reset/inventory.sh).
Removed:
- Makefile mark-development target + .PHONY entry
- env.development from .gitignore / .dockerignore
- env_local conditional in compose.up()
- env_development conditional in down._compose_run()
- env.development conditional in scripts/meta/resolve/apps.sh and
scripts/meta/resolve/diff/affected_roles.sh
- env.development marker branch + the now-unused project_root parameter
from cli/meta/runtime/detect_runtime; __main__.py caller follows
- "Optional Development Marker" section in docs/contributing/environment/
setup.md and the marker reference in docs/administration/installation.md
Also drops two RUNTIME=='dev' guards in roles/web-app-bigbluebutton that
relied on the auto-detect path:
- the compose build etherpad task in tasks/02_compose.yml
- the freeswitch iptables-REJECT entrypoint override in
templates/compose.override.yml.j2
With auto-detection of "dev" gone the guards would otherwise only fire
on explicit RUNTIME=dev runs, silently regressing the local-dev path
operators rely on; both behaviors now apply unconditionally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four ci.env keys (INFINITO_DNS_IP, INFINITO_DOMAIN, INFINITO_DOMAIN_RE,
INFINITO_IP4) were only honored by docker compose --env-file, so processes
that sourced .env via load.sh saw INFINITO_IP4 as unset. As a result
networks.internet.ip4 in the Ansible playbook fell through to the
127.0.0.1 default and the DinD DNS-check handler failed on the deploy
server matrix.
Moving the defaults into env/default.env makes them flow through the same
.env / load.sh / os.environ chain as every other INFINITO_* key, so
\${INFINITO_IP4} in compose.yml, lookup('env', 'INFINITO_IP4') in
group_vars, and envsubst over the CoreDNS Corefile template all see the
same value deterministically.
Side fix: utils/env/parser.py now undoes the writer's \\\\/\\" escaping for
quoted values, closing a round-trip gap that surfaced once
INFINITO_DOMAIN_RE (the only value containing a literal backslash) started
flowing through the generator.
All --env-file env/ci.env call sites (compose.py, down.py, install/lint/
test wrappers, resolve scripts, test-dns workflow + script) are removed.
coredns.py now reads .env via parse_static_env, docs and unit/integration
tests follow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* tests/integration/meta/env/test_dotenv_generator.py picks up a
module-level `STATIC_READS` tuple on every dynamic handler module
under utils/env/handlers/, in addition to PASSTHROUGH_STATIC_KEYS
and GHA_STATIC_KEYS. So a handler that pulls its fallback from
`ctx.static[KEY]` (e.g. infinito_outer_network_mtu) can declare
the key it reads without flipping the drift detector.
* utils/env/handlers/infinito_outer_network_mtu.py exports
`STATIC_READS = (KEY,)` accordingly and marks the two live system
reads (`/proc/net/route`, `/sys/class/net/<iface>/mtu`) with
`# nocheck: cache-read` since the cached-read lint forbids
uncached `Path.read_text` calls inside utils/.
* utils/env/loader.py marks the `INFINITO_ENV_LOADED` sentinel with
`# nocheck: env-loader-internal-guard` so the registry-drift lint
does not treat the idempotency flag as a missing config key.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Relocates `cli/meta/{ports,networks/suggest}` to
`cli/contributing/network/{ports,address/suggest}` so port and subnet
allocators live alongside the rest of the contributor-side tooling
(mirror, update, changelog) instead of `meta`. Mirrors the tests
into `tests/unit/cli/contributing/network/` and renames the contract
docs to `docs/contributing/tools/network/{port,address}.md`. Prog
names update from `cli meta ports suggest` / `cli meta networks
suggest` to `cli contributing network ports suggest` / `cli
contributing network address suggest`; the corresponding lint
references and `utils.meta.scan` docstring follow.
Adds a third subcommand: `cli contributing network diagnose`
(stdlib-only DNS / TCP / TLS / PMTU probe against the standard
infrastructure hosts plus interface MTU, `/etc/resolv.conf`,
`/etc/hosts`, proxy env, CA-bundle summaries). Split across `cli.py`
(orchestration), `config.py` (SPOT for hosts / per-distro install
commands), `format.py` (output + subprocess helpers), `tools.py`
(distro-aware auto-install of `iputils-ping` + `iproute2`),
`report.py` (system-info dumps + IPv6-default-route detection), and
`probes.py` (per-host probes). Exposed as `make debug-network`.
Companion `docs/contributing/actions/debugging/network.md` absorbs
the diagnose-tool walkthrough (incl. how to read MTU-induced TLS
handshake timeouts) so the merged page covers DNS / MTU / IPv6 /
connectivity end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After moving or deleting a Python module, the surrounding directory
often ends up holding nothing but a stale `__pycache__/`. `git clean`
does not remove the parent because the parent is still tracked, so
these orphans stay around indefinitely (or until ansible-lint /
test-init checks start complaining).
Adds `utils.cleanup.pycache_only_dirs` (walked post-order so emptying
a child can collapse a parent on the next pass through the sorted
list), skips `.git`, `.venv`, `venv`, `node_modules`, and tolerates
permission errors from container-root-owned bytecode. Exposed as
`make clean-pycache-only-dirs`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`pull.py` always ran `compose build --pull` so BuildKit refetched the
FROM image even when it was already local. When the registry was
unreachable mid-handshake (sandboxed bridge, MTU mismatch dropping
TLS-ServerHello fragments) the build aborted on
`failed to resolve source metadata` even though the base image was
on disk.
The buildable path now runs `compose build --pull` first and falls
back to plain `compose build` on non-zero exit. The fallback uses the
local FROM image directly; if that build also fails, run_or_fail
still raises with the labelled error.
Test suite reworked to track every `run_cmd` invocation (the build
path no longer routes through run_or_fail on the happy path) and
covers both the fallback branch and the local-images-only pull
tolerance path that was already in main.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The role-include files under `tasks/groups/*-roles.yml` are generated
by `make setup` and tracked as `.gitignore`d artefacts. CI checks out
the repo and runs `make lint-ansible` before `make setup`, so
ansible-lint statically resolves the literal include in
`tasks/stages/02_workstation.yml:7` against a missing
`tasks/groups/desk-roles.yml` and surfaces an un-ignored
`load-failure[filenotfounderror]` warning that, under the `min`
profile, exits with status 2.
Making `lint-ansible` depend on `setup` materialises the include
files before the lint walks them, so the literal path resolves
normally and no per-file ignore entry is needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`make install-lint` could hang indefinitely on the actionlint /
markdownlint binary download because the underlying `urlopen` calls
in `utils.install.github_release.resolve_latest_tag` and
`utils.install.primitives.download_file` ran without a timeout
parameter. Slow / dropped TLS handshakes against `github.com` (e.g.
behind an MTU-broken bridge) translated into a frozen install step
with no error.
Adds keyword-only `timeout=` parameters with sensible defaults:
30s for the latest-tag redirect resolve, 60s for the actual binary
download. Either can be lowered per call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`make dotenv` previously copied the static 1500 default from
env/default.env into .env for the compose bridge MTU, regardless of the
host's actual outbound MTU. On hosts where the default-route interface
sits below 1500 (VPN, tunnels, jumbo-frame negotiation failure) the
bridge ended up larger than the path MTU and TLS handshakes silently
lost their ServerHello fragments — every dotenv regeneration without
an explicit INFINITO_OUTER_NETWORK_MTU= override silently re-introduced
the regression.
Replaces the static-passthrough entry with a dedicated handler that
walks /proc/net/route to find the default-route interface, reads
/sys/class/net/<iface>/mtu, and seeds the value via setdefault so any
caller-set env still wins. Falls back to the env/default.env static
default when detection cannot complete (no default route, unreadable
sysfs entry, foreign host layout).
Adds 13 unit tests covering the /proc parse, the sysfs read, the
composition, and the handler's apply contract (detected wins,
static fallback, no-op when both empty, caller-env override).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Direct `infinito <subcmd>` calls from an interactive shell skipped the
bash env loader (`scripts/meta/env/load.sh`) and crashed on any module
that strict-reads an INFINITO_* key without a Python-side default —
the trigger was `infinito contributing update repository` raising
`KeyError: 'INFINITO_WORKER_FETCH'` from `resolve_max_fetch_workers`,
which only exists in `.env` because it is computed dynamically by
`make dotenv` (INFINITO_WORKER_CPU * 20).
Adds `utils.env.loader.load_dotenv_once()`: walks up from a start
path until it finds `.env`, parses it via the existing
`parse_static_env`, and injects each key with `os.environ.setdefault`.
Idempotent via `INFINITO_ENV_LOADED=1` to mirror the bash loader so
make-dispatched calls (which already set the flag) skip the work.
`cli/__main__.py` calls it once right after the sys.path bootstrap and
before importing `cli.core.app`, so every subcommand and transitive
import sees the defaults. Caller-set env vars still win — only
unset keys are filled.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`make test-unit` (docker runner) reached run.sh inside the container via
`bash --login`, which never sources the project's env-loader. PYTHON stayed
unset, so the test invocation fell back to the system python3 — which has
none of the runtime deps (ruamel, ansible, bcrypt, requests, certifi,
humanfriendly), producing 132+ ModuleNotFoundError errors.
run.sh now sources scripts/meta/env/load.sh itself (idempotent via
INFINITO_ENV_LOADED), so the env layer applies regardless of how the shell
was spawned.
While load.sh's detect.sh (now scripts/meta/env/python.sh) correctly walks
/opt/venvs and ~/.venvs to find the venv, an ambient VIRTUAL_ENV from a
different project (e.g. /root/.venvs/pkgmgr in pkgmgr's container) would
hijack the resolution. python.sh now scans the project-named venv candidates
first and only falls back to VIRTUAL_ENV when no project venv exists.
Also moves scripts/meta/python/detect.sh to scripts/meta/env/python.sh so the
env-related helpers live under one directory; updates the load.sh source
pragma and the lint-test path that scans it for INFINITO_* registrations.
`make test-unit` goes from 1262 tests / 132 errors to 1999 tests / 0 errors.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renames the container-wrapper CA toggle from the negated INFINITO_CONTAINER_NO_CA
to the positive INFINITO_CONTAINER_CA_ENABLED, mirroring the BUILDKIT_ENABLED
pattern: SPOT in env/default.env (default 1), passthrough handler + .env baseline
test, compose.yml env block, the CA_CONTAINER_ENABLED ansible var in
group_vars/all/02_tls.yml, a playbook-level CA_CONTAINER_ENABLED task-env
projection, and a strict `== "1"` read in container.py. Also drops the
redundant INFINITO_ prefix from the DNS/DOMAIN/IP4 vars that the DinD-DNS
test-dns.sh handler injects, since those names are local to the script's
caller-controlled env block. Finally lets pull.py fall back to a plain
`compose build` when `compose build --pull` fails — BuildKit can then use the
local FROM image instead of requiring a registry round-trip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lets operators force the legacy docker builder via INFINITO_BUILDKIT_ENABLED=0
(propagated through env/default.env, the compose env block, the BUILDKIT_ENABLED
ansible var and both the compose wrapper and the Build-compose handler), and
makes sys-svc-compose pull.py treat a failed `docker compose pull` as success
when every required image is already present locally — both needed to deploy
in sandboxed environments where docker.io is unreachable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
