test-environment.yml and test-guide.yml become test-workspace.yml and
test-instructions.yml. Orchestrator job ids, the distros-SPOT matrix
lint, the act example, the diagnostics slug and the rescue artefact
names follow.
Replace the roles_only input with per-job workspace/instructions
tri-state inputs (auto|true|false). 'auto' derives the former
roles_only skip straight from whitelist: both jobs run on a global
scope (whitelist empty or __ALL__) and are skipped on a role-scoped
run, which keeps the PR behaviour identical because the diff resolver
never emits an empty whitelist. entry-manual exposes both as choice
dropdowns defaulting to false, so a manual run opts in explicitly.
The now redundant roles_only output is dropped from
pr_affected_roles.sh and subset_roles.py.
Branch-protection required-status-check names change along with the
job renames.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guide job builds its boot image from a pinned pkgmgr-arch parent that
can be weeks old, and `pacman -Sy systemd` against it is the partial
upgrade arch upstream documents as unsupported. It pulled systemd 261.2-1
next to the image's systemd-sysvcompat, which pins systemd exactly, so the
transaction could not be satisfied and every arch guide run died before
the README sequence started.
Refreshing archlinux-keyring first and then running a full -Syu --needed
fixes it; the cache wipe matches what the apt and dnf branches already do.
The upgrade lands in the PREP container, which runs sleep infinity, so
systemd is only ever replaced before it becomes PID 1 in the committed
image.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
guide_select picked the role, the mode and the distro in one draw, so a run proved the README instructions worked on exactly one randomly chosen distribution. The distro list was also a literal in the workflow, which meant the manual-CI distros input never reached the guide at all.
The distro draw is removed. guide_select still picks the role and derives the mode, while the shared loop from 1bbe7a0eb drives the distro dimension, and ci-orchestrator forwards its distros input.
The four mode-specific steps collapse into guide/one.sh, which the loop runs once per distro: it renders the pkgmgr base image for the current distro, then either execs host_deploy.sh or brings the compose server up and replays the README block against it.
one.sh releases the compose server in an EXIT trap. Without it the second distro would start against the stack the first one left behind. Rescue diagnostics gain a distro segment for the same reason the swarm collectors did.
The job timeout goes from 90 to 345 minutes because up to five rounds now have to fit; the loop's own budget still decides how many actually run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.
meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.
Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.
tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.
meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.
Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.
tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Swarm's nested DinD-node daemons plus the backup DR drill (LUKS loop image,
per-node registry pulls) fill the ~50 GB root fs and trip the <6 GB abort
guard in utils/tests/swarm/matrix.py, while /mnt is a separate ~74 GB
partition sitting idle behind only a swapfile.
Add a runner step that moves the host Docker data-root to /mnt/docker when
/mnt is a separate, roomier partition (detection mirrors enlarge_swap.sh),
before any Docker usage. Best-effort: any failure rolls Docker back onto /
and the step still exits 0, so it can never turn a green deploy red. Wired
into the compose, swarm, and host deploy workflows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mark the merged best-effort '|| true' sites (guide EXIT trap, baseline
fetch fallback) with shell-or-true nocheck reasons, parenthesize the
implicitly concatenated message strings ISC004 enforces since the ruff
0.16 pin, and drop the regex narration comments the comment lint flags
on the now-changed test files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
host_deploy.sh drops its bespoke dump: the EXIT trap now runs
utils/diagnostics/container.py on failure before removing the boot
container, because the workflow's failure() collect step only fires
after this script has already torn the container down. That yields the
full snapshot (logs, inspect, in-container systemctl and journal, DiD
recursion) instead of three ad-hoc files, and successful runs skip the
capture entirely.
The compose-mode README replay moves out of test-guide.yml into
scripts/github/guide/compose_deploy.sh unchanged; the workflow step
becomes a one-liner.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host-deploy EXIT trap now dumps failed units, the tail of the
journal, and the container log from the pkgmgr boot container into the
rescue-diagnostics base before removing the container, so a failed guide
run leaves a readable artifact instead of tearing down silently. The
base directory is hard-required from the workflow environment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The hlth-webserver health service HEAD-requests the deploy's *.infinito.localhost vhosts, but the guide boot container has no resolver mapping for them (no CoreDNS sidecar, no dev-flow dns setup), so the service failed every start and burned 25x60s retries. Install libnss-myhostname on the apt branch and ensure the myhostname NSS module is listed in nsswitch.conf on every distro; it resolves any *.localhost name to loopback where the openresty vhosts answer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host-mode guide booted the pkgmgr distro image with --entrypoint
/sbin/init, but that image ships no init at all (it is a package-manager CLI
tool), so runc rejected the container with exit 127. Install systemd in a
throwaway container, commit an image that carries /sbin/init, and boot that
instead. Also read the post-loop readiness state into a variable so a
degraded systemd is not masked by pipefail.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host-deploy base image ships systemd-logind masked, so loginctl
enable-linger cannot activate org.freedesktop.login1 over D-Bus and the
guide fails after the image pull with "Could not activate remote peer".
Unmask and start logind before enabling linger; it is a no-op where logind
is already active.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the inline fork-prerequisites result assertion out of the
fork-prereqs-ready job into scripts/github/gate/fork_prereqs.sh (result
passed via the RESULT env var; the job now checks out the repo).
The staged diff also drops the code-quality-gate and test-install-gate
aggregator jobs and wires their members (lint, test, security-codeql,
test-install-make, test-install-pkgmgr) directly into the done job's needs,
which still hard-fails on any upstream failure or cancellation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The README Production block went provision -> deploy with no admin-key step, so user-administrator failed early (authorized_keys empty) for both the Guide CI jobs and any real user copying the README. Add a visible 'provision --vars' step that sets users.administrator.authorized_keys (merged into host_vars) in both Production variants of the role README template, regenerate all role READMEs, and replace the invisible CI payload splice with a visible substitution of the documented <your-ssh-public-key> placeholder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The inline run: block for the host-mode deploy exceeded the 11-line
inline-script-block cap. Move the body to scripts/github/guide/host_deploy.sh
and reduce the workflow step to a single bash invocation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 🧩 Subset path in pr_affected_roles.sh exec'd a bare `python`, the setup-python interpreter without PyYAML, so subset_roles.py failed with `ModuleNotFoundError: No module named 'yaml'`. Source scripts/meta/env/load.sh so ${PYTHON} resolves to the venv (where PyYAML is installed), matching the diff resolver path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The compose and swarm inventory collectors hard-failed at the 'Zip generated
inventory' step because they looked in the wrong place. Inventory is written on
the runner host at $INFINITO_INVENTORY_DIR (resolve.sh yields
$HOME/inventories/github in CI), but:
- zip_compose_for_upload.sh ran 'docker cp' from infinito_nexus_* containers,
which dedicated.sh's cleanup trap removes before the always() step runs, so it
never found a container.
- zip_swarm_for_upload.sh globbed /tmp/inv*, which the matrix orchestrator never
populates; it defaults its base to $INFINITO_INVENTORY_DIR.
Both now source scripts/meta/env/load.sh to resolve INFINITO_INVENTORY_DIR and
zip that dir plus its per-round '-<n>' variants.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.
- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).
- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.
- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).
- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zip_swarm_for_upload.sh only checked the base /tmp/inv, but the matrix
orchestrator writes per-round dirs /tmp/inv-<n> (and runs as root), so the
artifact was always empty. Collect every /tmp/inv* dir with sudo and de-root
the staged copy before zipping.
Both the swarm and compose collectors now hard-fail (exit 1) when they find no
inventory to copy, instead of echoing and exiting 0, so a silently-missing
artifact surfaces a real provisioning/path bug.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The roles updaters (Docker image versions, repository refs) opened a
single combined PR for all role changes, and the jobs checked out the
trigger commit (actions/checkout default), so updates were computed
against a stale tree and could re-propose already-merged changes.
- Check out 'ref: main' in all update jobs so the diff is always
computed against the latest main tip at run time.
- Add open_role_prs.sh: split the working-tree changes per role and
open one isolated branch/PR per changed role, reusing open_pr.sh for
the dedup/push/PR logic (new UPDATE_DEDUPE_PREFIX hook scopes the
duplicate check per role; default preserves prior single-PR behavior
for the skills job).
- Wire the Docker and repository-refs jobs to open_role_prs.sh; skills
stays a single PR.
Branch: update/<type>-<role>-<date>-<fp>; commit/title gain a (<role>)
suffix and a Scope line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 'Set up Docker Buildx' step boots BuildKit, which pulls
moby/buildkit from registry-1.docker.io. A transient Docker Hub
timeout there had no retry (only the build step did), so the whole
build job failed at setup.
Pre-pull the BuildKit image with a 7x/20s retry before the buildx
setup, and pin the action to that exact tag via driver-opts. Once the
image is local, buildx falls back to it if the bootstrap pull fails
('pulling failed, using local image'), so a flaky Docker Hub no longer
breaks setup. Applied to both images-build-ci.yml and
release-version.yml.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The cleanup step ran silently, so a no-op run was indistinguishable
from a broken one. Add a config banner, per-deletion lines, and a
per-package plus grand-total summary with a kept-reason breakdown, and
an explicit message when nothing matches. Add DRY_RUN support.
Deletion rule: delete a version only when it carries a ci-* tag, has no
semver (v*) or latest tag, and is older than MAX_AGE_DAYS. Released
images and untagged multi-arch child layers are never touched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
entry-push-latest.yml gains a sync-main job that force-pushes the current repository's main from a source repo (default infinito-nexus/core) before CI policy and deploy discovery derive scope from origin/main. The source is configurable and can be disabled via CI_SYNC_MAIN_SOURCE_REPOSITORY; the same-repository case and disabled values skip the sync.
The sync-main checkout uses fetch-depth: 0 and the push is --force to avoid shallow-update and non-fast-forward rejections. Docs warn that main is force-overwritten and must not be used as a working branch on synced repositories.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Delete the per-deploy-type workflows (test-deploy-local/universal/
workstation) and the redundant act-all/act-app runner scripts (replaced
by make act-workflow). Only compose and swarm deploy tests remain.
Both now discover the full application matrix (every deployment type,
lifecycle-filtered to alpha/beta/rc/stable) instead of just the server/
web-* scope, via apps.sh resolving applications.type without --type.
Retire INFINITO_DEPLOY_TYPE entirely: discovery is now unconditional and
the deploy step never read it. Removed from apps.sh, ci/all.sh, the local
deploy scripts, the two workflows, default.env, the env passthrough list,
the dotenv generator test, the Makefile compose-deploy target, and docs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
