Commit Graph
578 Commits
Author SHA1 Message Date
kevinveenbirkenbach 85c1ed1a6e Merge branch 'main' into feature/web-app-zammad
# Conflicts:
#	CHANGELOG.md
#	packaging/arch/PKGBUILD
#	packaging/debian/changelog
#	packaging/fedora/infinito-nexus.spec
#	pyproject.toml
#	roles/sys-svc-compose/README.md
#	scripts/docker/entry.sh
#	scripts/github/update/image_versions.sh
#	scripts/github/update/repository_refs.sh
2026-05-29 11:18:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f286ce54dd fix(update/open_pr): scope dedup to changed files
The previous tree-hash dedup compared whole-commit trees, so unrelated
drift on main (e.g. a manual edit to any roles/* file) caused the same
auto-update delta to appear "new" each daily cron and a fresh PR to open.

Now: fingerprint only the files that this run actually committed
(git diff --name-only HEAD~1 HEAD), both locally and on each candidate
open PR with the same UPDATE_BRANCH_PREFIX. If any open PR carries the
identical content for those files, skip push + PR creation entirely.

Branch name now carries the fingerprint's short hash so parallel PRs
of the same class can coexist (the close-superseded loop is dropped).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-29 10:37:21 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c680be12ea fix(image/build): propagate INFINITO_VENV_DIR to docker build
scripts/image/build.sh drives 'make build' and the test-environment
matrix's dev-image build, but missed the new INFINITO_VENV_DIR build
arg. Dockerfile's ARG resolved to empty, ENV set INFINITO_VENV_DIR=""
in the image, and entry.sh's assert tripped on first --compile.

Adds the assert and the matching --build-arg, parallel to the
INFINITO_SRC_DIR plumbing already in this file and to the equivalent
plumbing in scripts/image/push.sh.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 22:33:15 +02:00
kevinveenbirkenbachandClaude Opus 4.7 00c4246c3a refactor(env): introduce INFINITO_VENV_DIR SPOT, fix dev image build
Adds INFINITO_VENV_DIR=/opt/venvs/infinito to default.env (registered
in passthrough's STATIC_KEYS so make dotenv propagates it to .env) and
threads it through the build chain: compose.yml forwards it as a build
arg; scripts/image/push.sh asserts and --build-args it; the Dockerfile
declares the matching ARG/ENV and derives PYTHON/PIP/PATH from the
variable. scripts/meta/env/python.sh now prefers the value when set
before its fallback discovery so the SPOT is effective in env
resolution.

Fixes the dev image build regression where 'infinito --version' in
scripts/docker/entry.sh exited 127. Root cause: the Dockerfile RUN
SHELL uses bash -lc, which sources /etc/profile and resets PATH to
the Debian default, dropping ${INFINITO_VENV_DIR}/bin from it. The
fix invokes the script via its absolute path "${INFINITO_VENV_DIR}/bin/infinito"
so the post-install version check no longer depends on a PATH that
the login-shell init has just clobbered.

scripts/lint/wrapper.sh's existing PATH-reset note now references the
variable instead of the literal path. test-install-make.yml and
test-install-pkgmgr.yml read INFINITO_VENV_DIR from default.env up
front, pass it into their pkgmgr containers, and source the venv
activate script via the variable.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 22:12:19 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a561aa76b5 ci(push): skip push CI when an open PR already covers the branch
push_ci_policy.sh now hard-fails on missing GH_TOKEN/GITHUB_REPOSITORY
and, for non-main branches, calls gh pr list to detect whether the
push is already covered by an open pull request. If so, emits
should_run=false (skip_reason=pr-ci-covers-this-push) so the push
workflow exits cleanly and the pull_request workflow handles the
actual CI run.

To absorb the bot's push -> gh-pr-create latency on update/* and
alert-autofix-* branches, the PR check retries up to 10x with 2s
sleeps before falling through to a real CI run. Human branches
(feature/**, hotfix/**, fix/**, chore/**) check once with no delay.

entry-push-latest.yml's run-policy step now forwards GH_TOKEN so the
gh call works against the repo.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 21:06:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 af628847bb fix(update/open_pr): delete head branch when closing superseded PRs
The previous gh pr close call left the bot's daily update branch
behind on the remote, so stale update/<prefix>-YYYYMMDD refs
accumulated indefinitely. --delete-branch removes the head ref on
close. Scope-safe because the loop only iterates branches matching
${UPDATE_BRANCH_PREFIX}-.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 21:06:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 806a3e93db ci(pr-flow): gate env-matrix on cross-cutting diffs, dedup resolution
Adds a detect-affected-roles job to entry-pull-request-change.yml that
runs the affected-roles resolver upfront on host python (via make
install) and emits whitelist + roles_only. ci-orchestrator skips
test-development when roles_only=true; done job tolerates the skip.
entry-manual derives roles_only from its whitelist input.

affected_roles.sh now invokes the resolver host-side only (compose-exec
fallback removed; it was a no-op in every existing caller because
compose was never up there, always emitting __ALL__ fail-safe). The
PR-flow gate consolidates what test-deploy-* used to recompute per
job; downstream effective_whitelist.sh now consumes the pre-resolved
whitelist verbatim.

Test fixture switched from a fake docker binary to a fake PYTHON
binary and clears BASH_ENV so the bash-startup env loader does not
reset PYTHON.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 21:06:34 +02:00
kevinveenbirkenbach 84bd56e72c Merge branch 'main' into feature/req-21-sso-flavor-migration 2026-05-28 18:07:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8e293bd921 fix(compose-deploy): stop persisting runtime routing knobs into .env
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".

Fix:

- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
  full_cycle, variant) from `passthrough.STATIC_KEYS` and from
  `default.env`. They are per-invocation; .env is the persistent SPOT
  and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
  `os.environ.get("KEY", "")` so the missing-key path stops at "no
  filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
  (lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
  disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
  full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
  INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
  name in every doc; routing-knob env vars now use the same name so
  the cmdline form, the recipe env-prefix, and the consumer reads
  all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
  knobs whose alias differs from the env name (`type` →
  INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
  auto-exports cmdline variables into the recipe's shell env, so the
  identical-name `$(if $(apps),apps="$(apps)")` lines were pure
  noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
  `extra_env` per `docker exec` call and forwards `disable` via
  `-e`, so the container env never needs the value frozen at
  compose-up time.
- Update the 3 actual leak call sites
  (07/08/09_deploy_*.sh) to the make-alias form
  (`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
  the deliberate make/env parity, not a typo.

Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.

Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 17:47:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ad270fbbeb feat(cli): add --version / -V flag to infinito CLI
The Docker entry script's `[docker-infinito] Installed:` banner used
to call `pkgmgr version infinito`; after the pkgmgr removal that line
became `infinito --version || true`, which printed
`Error: command '--version' not found.` to stderr because the CLI's
argparse-style dispatcher treated `--version` as an unknown
subcommand.

Hook a real top-level flag into cli.core.app.parse_flags that prints
`infinito <version>` from importlib.metadata (single source of truth:
pyproject.toml's `version` field; falls back to `unknown` when the
package isn't pip-installed) and exits 0. entry.sh drops the
`|| true` since the call is now well-defined.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 16:24:06 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8ab37347b1 feat(test-e2e-playwright): add host.docker.internal:host-gateway to sidecar runs
Gives Playwright sidecars a stable name for the docker host so role specs
can reach services whose port is bound on the host (e.g. Mailu's SMTP
submission on 465/587 in the Zammad email-channel test) without
attaching to that service's docker network.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:53:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1bbdbdb877 fix(update): load .env before invoking python updaters
`update.yml` jobs (Docker image versions, repository refs) shelled
straight into `python3 -m cli.contributing.update.{docker,repository}`
without sourcing the project dotenv. After eb4bcd23a removed the
`.get(..., default)` fallback in resolve_max_fetch_workers(),
KeyError: 'INFINITO_WORKER_FETCH' surfaced on every run.

Both wrapper scripts now `make dotenv` (idempotent) and `set -a; . .env;
set +a` before exec'ing the updater, restoring the SPOT env chain on
CI runners that don't go through compose.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 15:53:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2883025c8e fix(docker): drop pkgmgr from dev container, install in-tree via make
`pkgmgr install infinito` started failing in CI after the floating
`pkgmgr-<distro>:stable` base image was rebuilt with kpmx 1.15.x, which
no longer registers the `infinito` identifier in its default config:

    Identifier 'infinito' did not match any repository in config.

The error string then flowed through `pkgmgr path infinito` in
entry.sh and was treated as the install directory, leading to
`make install` in a bogus cwd. This broke test-development,
test-install-pkgmgr, and build-ci-images for every distro.

The pkgmgr step in the dev Dockerfile is redundant — `make install`
runs `pip install -e .` which already produces the `infinito` CLI
shim via pyproject.toml's [project.scripts] entry. Drop the pkgmgr
calls, the rsync-into-INFINITO_PATH dance, and the `INFINITO_PATH`
variable; everything operates on `INFINITO_SRC_DIR` directly.

Production `pkgmgr install` paths (roles/sys-cli, roles/web-app-navigator,
.github/workflows/test-install-pkgmgr.yml) are intentionally untouched —
the test-install-pkgmgr job is what surfaces the upstream kpmx breakage.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 15:53:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 dd7e84257f fix(update/open_pr): skip non-existent credentials paths via truncate -c
actions/checkout records two includeif targets per repo: one under
the host path (/home/runner/work/_temp/...) and one under the
container-internal path (/github/runner_temp/...). Only the host path
exists on the runner; the container path is a placeholder for
docker-based actions.

`truncate` (without -c) aborts on the missing container path with
exit 123. Add `-c` (--no-create) so missing paths are skipped
silently and the existing credentials file still gets blanked.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:42:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6dc402e2b7 fix(update/open_pr): blank checkout's external credentials file
actions/checkout v6 doesn't put `http.https://github.com/.extraheader`
inside .git/config; it writes that header to an external file at
/home/runner/work/_temp/git-credentials-*.config and pulls it in via
local `includeif.gitdir:<repo>/.git.path = <credfile>` directives.
That's why every earlier `git config --local --unset-all` /
`--remove-section` round was a silent no-op (verified by the previous
debug commit's dump: the extraheader survives because the source-of-
truth lives outside .git/config).

Read every includeif `.path` value from local config, dedupe, and
`truncate -s 0` the targets so the included header chunk becomes
empty. The App installation token then enters the wire via the
URL-embedded `x-access-token:${GH_TOKEN}` credentials with no
competing Authorization header.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:37:08 +02:00
kevinveenbirkenbachandClaude Opus 4.7 86a998159b debug(update/open_pr): dump every config scope + probe token identity
The previous strip targeted only `--local` config and the resulting
CI log showed local was already empty, yet the push still authed
as github-actions[bot]. The auth has to be coming from --global,
--system, --worktree, or GH_TOKEN itself is GITHUB_TOKEN.

Dump every scope's http/credential/url entries before and after the
strip, and probe /installation/repositories with GH_TOKEN so the
HTTP status alone (no secret in logs) tells us whether it's an App
installation token or the workflow GITHUB_TOKEN.

Will replace this with the real fix once the next CI run surfaces
where the rogue Authorization actually originates.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:30:02 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3dffbb8493 fix(update/open_pr): strip every extraheader via regex + log what was hit
Previous attempts targeted a specific config key path
(`http.https://github.com/.extraheader`). If actions/checkout v6
stores it under a different subsection naming, our unset/remove-section
no-ops silently and the GITHUB_TOKEN-bearing header keeps overriding
the App-token URL credentials.

Walk every `http.*.extraheader` entry the local config holds, pipe
the matched key names through xargs to unset-all each one, and dump
the remaining `http.*` config before pushing so the next CI run
proves what was stripped. The push itself still uses an
x-access-token URL so the App's installation token is the only
Authorization that reaches the wire.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:23:27 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d950770385 fix(update/open_pr): nuke checkout's http section and push via tokenised URL
`--replace-all` on http.https://github.com/.extraheader still left the
wire with two Authorization headers ("remote: Duplicate header" → 400).
Drop the existing extraheader **and** the whole `http.https://github.com/`
subsection from local config, then push to a URL that embeds
`x-access-token:${GH_TOKEN}` directly so only one Authorization header
(the App's installation token) reaches the server.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:16:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 34f0c1d6c8 fix(update/open_pr): replace checkout extraheader instead of unsetting
Last attempt (unset + URL-embedded token) still pushed as
github-actions[bot] because actions/checkout's
http.https://github.com/.extraheader Authorization survived the
unset (likely silenced by `|| true`) and outranked URL-embedded
credentials on the wire.

Force-replace the header value with the App installation token's
HTTP Basic equivalent (`x-access-token:<token>` base64'd, same shape
checkout itself uses for GITHUB_TOKEN). Drops the URL rewrite, drops
the unset-then-rewrite dance, and guarantees a single Authorization
header that GitHub honours as the App.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:08:35 +02:00
kevinveenbirkenbach 6b63c12573 Merge branch 'main' into feature/req-21-sso-flavor-migration 2026-05-28 13:01:55 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b2a3d003af fix(update/open_pr): suppress url-reachability lint on extraheader unset
`git config --unset-all "http.https://github.com/.extraheader"` is a
git config key path, not a fetchable URL, but
tests/external/repository/test_urls_reachable.py extracts the URL-like
substring and probes it, which returns 404 and fails the suite.

Tag the line with `# nocheck: url` so the URL test skips it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:01:55 +02:00
kevinveenbirkenbachandClaude Opus 4.7 55e2a5a4d5 fix(update): push update branches with the App installation token
`actions/checkout` configures git's http.extraheader to authenticate
as GITHUB_TOKEN, which update.yml scopes to `contents: read`. The
subsequent `git push --force` in open_pr.sh therefore tripped 403
("Permission to infinito-nexus/core.git denied to github-actions[bot]")
even though the token-minting step had already produced an App
installation token with `contents: write` in GH_TOKEN.

Drop checkout's extraheader and rewrite origin's URL to embed
GH_TOKEN as x-access-token before pushing, so the App's permissions
actually reach the wire.

Also pin two actionlint `-ignore` patterns in scripts/lint/action.sh
to silence false positives for the `client-id` input on
actions/create-github-app-token@v3 (the action's action.yml carries
the input and marks `app-id` deprecated; actionlint 1.7.12's bundled
schema lags behind the upstream release).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:01:55 +02:00
kevinveenbirkenbachandClaude Opus 4.7 4085dc1ab1 fix(update): tree-hash dedup + shellcheck SC1091 suppression
open_pr.sh's PR-dedup loop relied on `gh pr diff <num>` which the
GitHub API refuses with HTTP 406 once a PR exceeds 300 changed files.
Under `set -euo pipefail` that killed the workflow whenever any
unrelated large PR was open against the same base branch.

Replace the diff-hash check with a tree-SHA comparison: fetch each
PR's head commit shallowly, compare `git log -1 --format=%T` against
HEAD's tree. Same-tree means same end-state regardless of base SHA,
whitespace, file order, or patch context. Unfetchable heads (deleted
branches, lost permissions) fall through as non-duplicates instead of
aborting the run.

image_versions.sh / repository_refs.sh also pick up an explicit
`# shellcheck disable=SC1091` above their `. .env` source since the
dotenv file is generated at runtime and never present during
shellcheck's static parse.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:01:54 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ee9800200e fix(update/open_pr): suppress url-reachability lint on extraheader unset
`git config --unset-all "http.https://github.com/.extraheader"` is a
git config key path, not a fetchable URL, but
tests/external/repository/test_urls_reachable.py extracts the URL-like
substring and probes it, which returns 404 and fails the suite.

Tag the line with `# nocheck: url` so the URL test skips it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:01:36 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ce30948ed4 fix(update): push update branches with the App installation token
`actions/checkout` configures git's http.extraheader to authenticate
as GITHUB_TOKEN, which update.yml scopes to `contents: read`. The
subsequent `git push --force` in open_pr.sh therefore tripped 403
("Permission to infinito-nexus/core.git denied to github-actions[bot]")
even though the token-minting step had already produced an App
installation token with `contents: write` in GH_TOKEN.

Drop checkout's extraheader and rewrite origin's URL to embed
GH_TOKEN as x-access-token before pushing, so the App's permissions
actually reach the wire.

Also pin two actionlint `-ignore` patterns in scripts/lint/action.sh
to silence false positives for the `client-id` input on
actions/create-github-app-token@v3 (the action's action.yml carries
the input and marks `app-id` deprecated; actionlint 1.7.12's bundled
schema lags behind the upstream release).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 12:47:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a988c12606 fix(update): tree-hash dedup + shellcheck SC1091 suppression
open_pr.sh's PR-dedup loop relied on `gh pr diff <num>` which the
GitHub API refuses with HTTP 406 once a PR exceeds 300 changed files.
Under `set -euo pipefail` that killed the workflow whenever any
unrelated large PR was open against the same base branch.

Replace the diff-hash check with a tree-SHA comparison: fetch each
PR's head commit shallowly, compare `git log -1 --format=%T` against
HEAD's tree. Same-tree means same end-state regardless of base SHA,
whitespace, file order, or patch context. Unfetchable heads (deleted
branches, lost permissions) fall through as non-duplicates instead of
aborting the run.

image_versions.sh / repository_refs.sh also pick up an explicit
`# shellcheck disable=SC1091` above their `. .env` source since the
dotenv file is generated at runtime and never present during
shellcheck's static parse.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 12:23:15 +02:00
kevinveenbirkenbach 1e0f86d0d0 Merge branch 'main' into feature/req-21-sso-flavor-migration 2026-05-28 11:50:53 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3537d4d0a6 fix(update): load .env before invoking python updaters
`update.yml` jobs (Docker image versions, repository refs) shelled
straight into `python3 -m cli.contributing.update.{docker,repository}`
without sourcing the project dotenv. After eb4bcd23a removed the
`.get(..., default)` fallback in resolve_max_fetch_workers(),
KeyError: 'INFINITO_WORKER_FETCH' surfaced on every run.

Both wrapper scripts now `make dotenv` (idempotent) and `set -a; . .env;
set +a` before exec'ing the updater, restoring the SPOT env chain on
CI runners that don't go through compose.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 11:26:23 +02:00
kevinveenbirkenbachandClaude Opus 4.7 848fd08308 fix(docker): drop pkgmgr from dev container, install in-tree via make
`pkgmgr install infinito` started failing in CI after the floating
`pkgmgr-<distro>:stable` base image was rebuilt with kpmx 1.15.x, which
no longer registers the `infinito` identifier in its default config:

    Identifier 'infinito' did not match any repository in config.

The error string then flowed through `pkgmgr path infinito` in
entry.sh and was treated as the install directory, leading to
`make install` in a bogus cwd. This broke test-development,
test-install-pkgmgr, and build-ci-images for every distro.

The pkgmgr step in the dev Dockerfile is redundant — `make install`
runs `pip install -e .` which already produces the `infinito` CLI
shim via pyproject.toml's [project.scripts] entry. Drop the pkgmgr
calls, the rsync-into-INFINITO_PATH dance, and the `INFINITO_PATH`
variable; everything operates on `INFINITO_SRC_DIR` directly.

Production `pkgmgr install` paths (roles/sys-cli, roles/web-app-navigator,
.github/workflows/test-install-pkgmgr.yml) are intentionally untouched —
the test-install-pkgmgr job is what surfaces the upstream kpmx breakage.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 10:50:35 +02:00
kevinveenbirkenbach 21d81ccdc2 Merge branch 'main' into feature/req-21-sso-flavor-migration 2026-05-27 18:24:00 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ad778dd1c2 fix(svc-bkp-container-2-local): wipe stale databases.csv rows on purge
Matrix-deploy variant transitions wipe a consumer's compose stack and
its named DB volume (`compose down -v` in
scripts/container/purge/entity/compose.sh) but `databases.csv` at
$FILE_DATABASE_SECRETS survives the purge. When the next round drops
the consumer from its inventory, the stale baudolo-seed row remains
behind. The subsequent backup run reads that row, attempts
`mariadb-dump -u<entity> -p<old_pw>` against the freshly initialised
RDBMS user that no longer exists, and the systemd unit fails with
"Access denied for user 'matomo'@'127.0.0.1' (using password: YES)"
(job 77797356000 on run 26428080957: matomo dump in
svc-bkp-container-2-local variant 1, where round 1 seeded matomo but
round 2 only deploys postgres + mariadb + svc-bkp itself).

Add `utils.cleanup.databases_csv` mirroring the existing
`utils.cleanup.tokens` pattern: parse the baudolo-seed CSV format
(';'-separated, four columns instance/database/username/password, with
header — verified against baudolo.seed.__main__:check_and_add_entry)
and strip every row whose `database` or `username` column matches an
entity derived from the supplied app ids via
utils.roles.entity_name.get_entity_name. Header and blank lines are
preserved verbatim; the file is rewritten only when at least one row
was removed.

Wire the helper into scripts/container/purge/apps.sh right after the
token-store wipe so every matrix-deploy inter-round purge invokes it
alongside the other cleanups. Operator-triggered purges
(`INFINITO_APPS=<app> bash scripts/container/purge/apps.sh`) also get
the cleanup — that is a behaviour change relative to before, where
the row would survive an operator-triggered entity teardown.

Coverage: 12 unit tests in tests/unit/utils/cleanup/test_databases_csv.py
covering the happy path, the multi-app case, the
`database='*'`/username-match case, the unknown-app no-op, header and
blank-line preservation, the all-rows-removed case, malformed short
rows, and the legacy-header-less file. `make test` green across all 5
targets.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 13:25:12 +02:00
kevinveenbirkenbachandClaude Opus 4.7 183f41845b refactor(env): SPOT pkgmgr base image via handler, drop slim variant
default.env decomposed into INFINITO_PARENT_IMAGE_{OWNER,TAG,VARIANT}
atoms. New utils/env/handlers/infinito_parent_image.py owns the URL
composition `ghcr.io/{owner}/pkgmgr-{distro}:{tag}` as the single
definition site; scripts/image/{push,build}.sh consume the handler-
emitted INFINITO_PARENT_IMAGE from .env instead of inlining their own
format string.

Side effect: `compose up --build` now picks the parent image that
matches INFINITO_DISTRO. Previously default.env's hardcoded
pkgmgr-arch:stable won regardless, so debian-named containers were
silently built FROM the arch base. scripts/image/build.sh already
overrode per-distro, so its behavior is unchanged.

Other cleanups landed in the same pass:
- Slim variant removed end-to-end (Dockerfile stage, --variant CLI
  flag, slim suffix logic). Zero in-repo callers used it.
- compose.yml build-args INFINITO_PARENT_IMAGE / INFINITO_SRC_DIR now
  carry the same `:?Run 'make dotenv'…` guard as the runtime entries,
  so a missing .env fails fast with a clear message instead of with
  buildkit's `base name should not be blank`.

Note: fork-CI (pull_request_target trusted-helpers path) will stay red
until main's scripts/image/push.sh and scripts/meta/wait/images.sh
adopt the INFINITO_* names too — that is the deliberate cost of
keeping a single SPOT instead of carrying a bridge alias.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-24 10:44:50 +02:00
kevinveenbirkenbachandClaude Opus 4.7 dd57167615 fix(install): chmod build/ to 0777 so userns-remapped container shares it
build/ is bind-mounted; under docker userns-remap the container root
maps to host uid 65534 (nobody), so files written from inside the
container are nobody-owned on the host. Whichever side created build/
first locked the other out: a stale nobody-owned build/ blocked the
host touch of build/install.stamp, and a host-owned 0755 build/
blocked the container from writing build/install-lint-<hash>.stamp.

Stamp writers now chmod the build dir to 0777 after creating it. The
first creator wins (only owner can chmod); both writers then place
their own stamps without colliding on directory ownership.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:52:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 bc3bdf3dc4 chore(scripts/meta/resolve/apps): drop web-opt-rdr-www from global exclude
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:52:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 39ef28ea9e fix(scripts/tests/env/common): drop single-quoted printf that breaks compose-exec wrap
Inner `'==> %s\n'` collided with Make's `cmd='$(cmd)'` recipe wrap, mis-tokenizing the line and exiting 2 in every test-development variant.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:52:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ff84378bfa feat(make): add clean-container-owned target
Adds a Makefile target plus helper script that removes container-owned
generated artefacts (build/install.stamp, tasks/groups/*-roles.yml)
which the host user cannot rm directly. The helper auto-starts a
stopped infinito compose container before deletion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:52:41 +02:00
kevinveenbirkenbachandClaude Opus 4.7 24d4ab8e4a refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:

* Schema: every role's meta/services.yml replaces the parallel
  `oauth2:` / `oidc:` top-level blocks with a single `sso:` block
  carrying `enabled`, `shared`, and a `flavor` discriminator
  (`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
  sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
  allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
  entirety; its 5 sidecar templates move under
  roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
  rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
  web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
  centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
  `is_oidc_native`) and the oauth2 sub-values
  (`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
  Three lookup plugins consume the resolver: `sso` (single-app
  predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
  consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
  plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
  and plugins/filter/compose_volumes.py all gate on
  `lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
  `enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
  → `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
  `OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
  `oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
  `OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
  `SSO_SERVICE_ENABLED`; Playwright gating helpers
  (isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
  safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
  is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
  `flavor: oauth2` with their existing `oauth2.acl.blacklist`
  (/user/login) and `oauth2.acl.whitelist` (federation + discovery
  endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
  (tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
  test_acl_mutual_exclusive.py) are deleted per Decision #10;
  test_oauth2_contract.py is renamed and rewritten to
  test_sso_contract.py, asserting that `flavor: oauth2` roles carry
  `oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
  (web-app integration) now requires the unified `sso:` block with an
  explicit flavor or a `# nocheck: sso` opt-out; new
  tests/lint/repository/no_legacy_sso_paths/ guards the source tree
  against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
  009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
  docs/contributing/actions/testing/suppression.md are updated to the
  unified SSO naming.

Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).

BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 12:59:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 490f0919ab refactor(make): replace INFINITO_CMD with cmd parameter
INFINITO_CMD is a per-invocation argument, not a config value — drop it
from default.env, the passthrough STATIC_KEYS, and the dotenv-generator
test, and rename to lowercase `cmd` in Makefile targets, exec scripts,
internal callers, and user-facing docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 05:31:15 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c3ff093df0 refactor(scripts/deploy/local): mirror host wrappers with container/ subtree
Renames the three in-container deploy helpers out of `utils/` (where
they lived next to unrelated bash libs) into a `container/` subtree
that mirrors the host-side `apps/` layout one-to-one:

  utils/fresh-kept-all-init-and-deploy.sh  -> container/initialize/all.sh
  utils/reuse-kept-all-deploy.sh           -> container/update/all.sh
  utils/reuse-kept-app-deploy.sh           -> container/update/selection.sh

The `fresh-/reuse-/kept-` prefixes were overloaded ("kept" was
universal, "fresh/reuse" duplicated the verb already encoded in
initialize/update). New names use the same verbs + scope as the host
wrappers, so the host/container halves of each flow share a name.

Updates the three host wrappers under apps/{initialize,update}/ to
point at the new in-container paths and modernizes their
cache-retry keys (`fresh-kept-all` -> `initialize-all`, `reuse-all` ->
`update-all`, `reuse-<app>` -> `update-<app>`).

Adds READMEs for container/, container/initialize/, container/update/
following the project documentation conventions (H1 + trailing emoji,
RFC 2119 keywords, env-var tables, cross-links to the matching host
wrappers) and extends deploy/README.md with a pointer to the new
subtree.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 04:23:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 81a4f7fdde feat(make): add clean-container-owned target
Adds a Makefile target plus helper script that removes container-owned
generated artefacts (build/install.stamp, tasks/groups/*-roles.yml)
which the host user cannot rm directly. The helper auto-starts a
stopped infinito compose container before deletion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 18:26:20 +02:00
kevinveenbirkenbachandClaude Opus 4.7 54f88275ce feat(cli/console): stateful navigation, ls, and modular split
Reworks the REPL into a stateful shell that tracks a category path: typing a bare category name cd's into it, `/` returns to root, `..` walks one level up (chained as `../..` etc.), path tokens with slashes or a leading `..` navigate, `infinito [path]` always jumps absolute, and `ls` renders the current folder with a 🗂️/⚙️ emoji column, name, and a 30-char-truncated description (folders sorted before commands). Executable leaves (sub-packages with `__main__.py`) run instead of cd-ing into them. The prompt prints `🖥️  infinito <space-separated path>> `; the screen clears (incl. scrollback via `\033[3J`) before each command run, and the run echoes the resolved invocation in the fresh frame so the user sees what is about to execute.

Splits `repl.py` (~250 lines) into focused modules: `constants.py` (sentinels), `navigation.py` (path/category resolve + prompt + normalize), `ls.py` (folder rendering + truncate), `runner.py` (subprocess dispatch + clear_screen), `banner.py` (startup banner). `repl.py` is now a thin `main()` loop with backward-compat re-exports so external consumers keep the `repl.<symbol>` surface. Mirrors the split on the test side into `test_navigation.py`, `test_runner.py`, `test_ls.py`, and a `test_repl.py` that owns only main-loop integration with a shared `_MainLoopHarness` mixin.

Adds `scripts/tests/environment/11_console.sh` as a CI smoke that feeds a navigation sequence through the REPL and asserts every prompt shape plus the `ls` emoji output, so a regression in the prompt format, the path resolver, or the folder rendering is caught by the environment-test orchestrator.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 13:06:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7ad7b5e76c feat(lint): add lint-playwright for per-role spec parse checks
Each role's Playwright spec is staged the same way `test-e2e-playwright` does at deploy time (central config + personas tree + service-gating) and `npx playwright test --list` runs against it. `--list` skips the browser launch and any test body, so this catches refactor regressions like missing helpers, broken exports, or syntax errors in seconds instead of waiting for a full deploy. Stub env values keyed off the variable-name suffix satisfy the few specs that build URLs or call `.replace()` at module-load time.

Parallelism caps at `INFINITO_WORKER_CPU` (same env knob as `scripts/make/parallel.sh`); output is replayed in `SPEC_FILES` order after all workers drain so stdout stays deterministic. `@playwright/test` ships as a devDep via the existing eslint `npm ci`. Workflow wires into `ci-orchestrator.yml` behind the `lint-gate`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 10:24:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b5c3f33b56 fix(ci): surface in-container ansible log on the runner for role-runtime summary
The CI `📊 Top role runtimes` step parses `$ANSIBLE_LOG_PATH` on the
runner, but ansible-playbook actually runs inside the dev container via
`docker compose exec` — and the env var was never forwarded there, so
the log file never materialised. Even when set, the log would have
lived inside the container, invisible to the runner-side parser.

Forwards `ANSIBLE_LOG_PATH` through `compose.exec`'s `extra_env` so
ansible-playbook writes the log at the configured path inside the
container, then `docker cp`s it back to the runner from the cleanup
trap before the stack is torn down.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 02:53:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c36929090d refactor(make): apply comment-block schema across Makefile targets
Splits mixed-content target descriptions into a single description line
plus `Note:` follow-ups, and dresses the targets that carry real
parameters (`compose-deploy`, `compose-exec`, `compose-inner-run`) with
explicit `Usage:` / `Example:` / `Param <name>:` lines. Adds a
`variant=<idx>` Make var alias to `compose-deploy` so multi-variant
pinning works without dropping to `INFINITO_VARIANT=` env prefixes.

Removes redundant inline `compose-deploy` / `compose-inner-run` param
enumerations from the docs and replaces them with
`make help target=<name>` pointers, so the help banner stays the SPOT.

Also fixes SC2016 in `scripts/make/help.sh` (literal backticks in the
help banner) with a scoped suppression.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 02:53:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 91fc422f19 feat(make): add compose-playwright target + schema-aware help
Adds `make compose-playwright role=<role> [pw="..."] [keep=true]` as the
canonical inner-loop entry point for rerunning a role-local Playwright
spec against the live stack — replaces the raw
`scripts/tests/e2e/rerun-spec.sh` invocation across docs.

Defines a small comment-block schema for Makefile targets (description
line on top, followed by `Usage:` / `Example:` / `Note:` / `Param <name>:`
markers). Lint enforces it; `make help` shows the description per target
in the short list and `make help target=<name>` prints the full block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 02:53:05 +02:00
kevinveenbirkenbach 92bebe5e11 fix(dns): invoke cli.meta.domains as a module
Path-style invocation in scripts/system/network/dns/common.sh failed
with `ImportError: attempted relative import with no known parent
package` because cli/meta/domains/__main__.py uses
`from . import PROJECT_ROOT`. The DNS host generator silently fell
back to a static 3-host list on every bootstrap run (visible in the
test-development CI logs).

Switch the caller to `PYTHONPATH=<repo> python3 -m cli.meta.domains`
so the package context resolves. Keeps the PROJECT_ROOT SPOT
(tests/lint/filesystem/python/test_project_root_import.py) intact —
no local redefinition.

Also: one-line YAML rewrap in compose/cache.override.yml (formatter).
2026-05-21 22:58:28 +02:00
kevinveenbirkenbach 5545cbe9c7 fix(test-deploy): run entry-bootstrap with --compile -- true
`entry.sh true` skips `run_local_build` because `true` is treated as
the exec target rather than a flag, so the in-container reinstall step
never ran. `--compile -- true` triggers the build first and then runs
`true` as a no-op exec target.
2026-05-21 22:58:28 +02:00
kevinveenbirkenbach 664457b739 test(lint): verify every make <target> reference resolves
Catches refactor drift across `.sh`, `Makefile` recursive calls, and
GitHub workflows. Includes fixes for the 5 bugs the new test surfaced
on first run: `make -C ROOT down`/`up` in the network stack refresh
helper (missed during the compose-* rename) and four
`make compose-deploymode=…` / `compose-deployapps=…` typos in
07/08/09_deploy_*.sh plus three docstring/echo references
(missing space after `compose-deploy`, introduced by the
deploy → compose-deploy sed-rename).
2026-05-21 22:58:27 +02:00
kevinveenbirkenbachandClaude Opus 4.7 605f29a2f8 refactor(make): rename deploy to compose-deploy
Move the deploy router into the compose- domain prefix to match the
rest of the local stack lifecycle (compose-up/down/exec/...). Reorder
its block alphabetically between compose-down and compose-entity-purge
to keep `make help` and the alphabetical-targets lint happy.

Update every active call site: contributor docs, agent-iteration docs,
role-level Administration/README/TODO files, the bundles README, the
test-environment scripts, the in-repo deploy helpers and main.sh, and
the server stage task. The two wildcard prose references (`make
deploy-*` in CLAUDE.md and testing/playwright.md) become single
`make compose-deploy` mentions since no other deploy-* target exists.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:38:08 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c02229b4be refactor(make): align targets to domain prefixes (compose-, network-, etc.)
Apply a consistent naming convention to the Make target inventory so
every target is grouped by its domain:

- compose-*  (compose-up/down/restart/stop/exec/inner-run)
- network-*  (network-dns-setup/remove, network-ipv6-disable/restore,
              network-refresh, network-trust-ca)
- security-* (security-apparmor-restore/teardown)
- diagnose-* (diagnose-network, diagnose-disk-usage)
- fix-*      (fix-chmod, fix-dockerignore)
- clean-*    (clean-cache, clean-pycache-dirs)
- install-*  (install-agent)
- meta-*     (meta-list, meta-tree, meta-mig)

Drop the stale debug-network target and reintroduce its functionality
as diagnose-network. Cascade renames through:

- Makefile (.PHONY entries, prereqs, recursive $(MAKE) calls)
- contributor docs, agent-iteration docs, role-level docs
- bundle README, environment scripts, install/lint/test wrappers
- AGENTS.md, CLAUDE.md, compose/cache.override.yml, .github/workflows

Historical entries under docs/requirements/ and changelog/ stay
untouched as audit logs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 21:00:45 +02:00