Introduces per-role `meta/variants.yml` for declaring multi-variant
deploys (e.g. WordPress Single-Site vs. Multisite) and reshapes the
development deploy CLI to materialise one inventory folder per variant
and iterate them with co-located sync+async passes.
Why:
- The previous single-inventory + Pass-1-then-Pass-2 split deployed each
variant twice and ran the async pass against a host left in a
different variant's state, masking real bugs.
- Inventory and runtime concerns were tangled: variant selection lived
as a runtime extra-var and required cross-stage coordination.
How:
- `roles/<role>/meta/variants.yml` (file format docs at
docs/contributing/artefact/files/role/variants.md): YAML list of
deep-merge overrides on top of `config/main.yml`. Empty entry `{}` is
variant 0 (= legacy payload). 6 roles get their dev overrides moved
here from inventories/dev.yml; WordPress declares 2 variants.
- `utils/runtime_data.py` gains `_build_variants` / `get_variants` and
drops the retired `_active_variants` runtime selector. Loader always
returns variant 0 as the default; non-zero variants reach
`get_merged_applications` as inventory-level `applications.<app>`
overrides baked at init time.
- New `cli/deploy/development/inventory.py`: `DevInventorySpec`,
`plan_dev_inventory_matrix` (pure planner shared between init and
deploy), `build_dev_inventory_matrix` (materialises one folder per
round at `<dir>-<R>`, or `<dir>` when single-variant). Single-folder
paths preserve full backward compat.
- `cli/deploy/development/deploy.py`: matrix loop with per-round
cleanup (only apps whose variant changed AND that are about to
re-deploy), `--variant <idx>` (or `VARIANT=` env) to pin a single
round, and `--full-cycle` (or `FULL_CYCLE=true`) which interleaves
sync + async per round so the async re-deploy targets exactly the
host state the matching sync deploy just produced.
- `cli/deploy/development/exec.py` gains repeatable `--env KEY=VAL` so
bash callers can inject per-run context into in-container helpers
without inlining heredocs.
- `scripts/meta/env/inventory.sh` exports `INVENTORY_VARS_FILE`
(Python mirror in `cli.deploy.development.common.DEV_INVENTORY_VARS_FILE`;
test_inventory_vars_file_spot.py locks them against drift).
- `make deploy-fresh-purged-apps` and `scripts/tests/deploy/ci/dedicated.sh`
switch to a single init + single deploy `--full-cycle`. The
`make deploy-reuse-*` bash scripts honour `VARIANT=` by suffixing
`INVENTORY_DIR`/`INVENTORY_FILE`.
- The 5 `scripts/tests/deploy/local/deploy/*.sh` scripts now delegate
their in-container heredocs to dedicated helpers in
`scripts/tests/deploy/local/utils/`, removing the bash-in-bash
quoting hell.
Tests:
- `tests/unit/utils/test_variants.py` (renamed from
`test_application_variants.py`): loader edge cases, variant-zero
default, retired-`_active_variants` guard.
- `tests/unit/cli/deploy/development/test_inventory.py` (new): planner,
variant baking, builder, `filter_plan_to_variant`, `DevInventorySpec`.
- `tests/unit/cli/deploy/development/test_matrix_deploy.py` (new):
per-round deploy filter, cleanup decisions, `--variant` pinning,
`--full-cycle` per-round interleave.
- `tests/unit/cli/deploy/development/test_inventory_vars_file_spot.py`
(new): drift guard for the bash <-> Python literal pair.
- `tests/unit/cli/deploy/development/test_exec.py` extended for `--env`.
Docs:
- `docs/contributing/artefact/files/role/variants.md` (new): file format.
- `docs/contributing/design/matrix-deploy.md` (new): runtime mechanism
(folder layout, --variant, --full-cycle, cleanup logic).
- `docs/contributing/actions/deploy.md` (new): contributor dev-deploy
flow (make targets, --variant pinning).
- `docs/administration/deploy.md` slimmed to admin-only (was mixed
admin+dev), cross-links to the new contributor doc.
- `docs/agents/action/iteration/role.md` keeps only agent-iteration
recipes; mechanism explanation moved out and linked.
- `docs/contributing/documentation.md` adds the "no self-SPOT
declaration" rule; `applications.md` renamed its SoT header for
consistency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The tool now pins branch.main.pushRemote to origin, so main pushes go
upstream while other branches keep landing on the personal fork. The
layout table and the new Push Routing section reflect this and list
the three git-config keys the tool writes. Also tighten the sandbox
rule to MUST per the docs RFC-2119 style.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Drop the project-local copy of scripts/git/sign-push.sh in favour of
the git-maintainer-tools CLIs (git-setup-remotes, git-sign-push),
which now ship as a standalone PyPI/GitHub package so the same helpers
can be reused across every maintainer repo without vendoring shell
scripts.
The package is declared in pyproject.toml's `dev` extras and lands on
$PATH via `make install-python-dev` (which already runs
`pip install -e .[dev]`). Consumers invoke the CLIs directly; there
are no make wrappers because doing so would just duplicate help text
the tool itself already prints.
Fallout:
* scripts/git/sign-push.sh deleted.
* Makefile: sign-push target removed (no replacement target).
* .claude/settings.json: deny rule for `make sign-push*` replaced
with direct `git-setup-remotes*` / `git-sign-push*` denies so the
sandbox blocks the real binaries, not just the make alias.
* CLAUDE.md: operator instruction points at the bare `git-sign-push`
CLI + `make install-python-dev` for setup.
* docs/contributing/artefact/git/remotes.md: new tool-focused page.
Describes the layout and the two CLIs; does not duplicate raw git
commands already covered by git-maintainer-tools.
* docs/contributing/artefact/git/README.md: trimmed per readme_md.md
(docs READMEs must not act as a file index).
* docs/contributing/tools/make.md: Git section now documents the
external CLIs and the install entry point, not a wrapper target.
Add three follow-up requirements that extend the work landed in 004:
- 005 WordPress Multisite support + LDAP per-application OU refactor:
introduces opt-in rbac.tenancy schema, migrates every rbac-carrying
role onto the ou=<application_id>,ou=roles,... hierarchy, adds a
central rbac_group_path lookup plugin as the only sanctioned way to
compute OIDC group paths, mandates one-time LDAP rewrite plus a
Keycloak logout-all to flush stale tokens, and specifies the
WordPress mu-plugin contract for per-site roles and grant_super_admin
semantics.
- 006 Service-gated Playwright tests: per-service <NAME>_SERVICE_ENABLED
env flags, shared helper at roles/test-e2e-playwright/files/
service-gating.js with hard-fail on unknown services, test.skip() with
flag-named reason, explicit per-role refactor checkboxes for every
existing spec, and a new MODE_CI variable (GitHub Actions + nektos/act
+ make-driven deploy) replacing the accidental DOCKER_IN_CONTAINER
proxy at the test-e2e-playwright include site.
- 007 WordPress -> Discourse post round-trip: Playwright scenario that
publishes a WP post and asserts the corresponding Discourse topic
exists, gated on DISCOURSE_SERVICE_ENABLED, with teardown that covers
draft and partial states and an API key scoped for topic delete.
Also update the role Playwright spec guide to carry the Option B
service-gating rules (per-service boolean flags, helper-only reads,
hard-fail on unknown services) so contributor and requirement SPOTs
stay in sync.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- playwright.spec.js: add "biber logs into nextcloud via OIDC and logs
out" scenario covering the non-administrator persona required by the
MUSTs; parameterize loginToStandaloneNextcloud on username/password,
add a logoutStandaloneNextcloud helper, and wrap biber's first login
in a retry so the LDAP first-login-provisioning delay does not flake.
- playwright.env.j2: expose BIBER_USERNAME and BIBER_PASSWORD so the
spec can read biber credentials from the rendered env.
- docs/contributing/.../playwright.specs.js.md: codify the MUST that
existing scenarios may not be deleted while optimizing a spec.
- roles/web-app-nextcloud/docs/LDAP.md: document the biber first-login
caveat against fresh Nextcloud + LDAP deployments.
Validated via scripts/tests/e2e/rerun-spec.sh web-app-nextcloud:
3 passed (talk admin settings, dashboard to nextcloud login, biber).
Shorten the module name now that it backs more than just lookup plugins
(validator CLI, tests, dashboard role). Updates all 26 import sites and
adds a unit test covering the pure helpers.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Resolve merge conflict in web-app-peertube/vars/main.yml; migrate
remaining `applications |` and `users.*` references to
`lookup('applications')` / `lookup('users', ...)` in web-app-odoo and
web-app-decidim (compose, env, playwright, setup tasks).
- web-app-peertube: split OIDC tasks into oidc/{enable,disable}.yml,
add install_mem_limit filter + PEERTUBE_MEM_LIMIT /
PEERTUBE_OIDC_PLUGIN_INSTALL_MEM_LIMIT vars to raise the cgroup ceiling
during the one-off `npm run plugin:install` exec (prevents OOM-SIGKILL
that was surfacing as "Container is restarting"); add
upsert-oidc-plugin.sql.j2 and diagnostics.sh.
- CI: test-deploy-{local,server,universal,workstation}.yml tweaks plus
scripts/github/{dump_runner_state,enlarge_swap}.sh for constrained
runners.
- cli/meta/applications/ressources: new CLI entry with unit tests.
- Misc config/docs updates across dashboard, mailu, mattermost, asset,
cdn, file roles and contributing docs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Swap size was free(/)-25, which took ~99 GB on the 145 GB root fs.
Remaining ~25 GB then ran out mid-deploy during Playwright image
pulls (ENOSPC). Replace with a fixed 16 GB cap and use mountpoint
to detect a real /mnt partition instead of [ -d /mnt ], which
matched the empty placeholder dir on single-partition runners.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Introduce scripts/github/enlarge_swap.sh that consumes all remaining
space on / minus a 25 GB buffer (Docker layers, checkout, runtime
state), falling back to /mnt when / is too small. Replaces the
default ~4 GB runner swap so transient memory spikes (e.g. peertube
plugin install, issue #162) don't trip the host OOM-killer on the
16 GB GitHub-hosted runner.
Wire the script into all four deploy test workflows (local, universal,
server, workstation) and flip free-disk-space to the aggressive config
(tool-cache, large-packages, docker-images, swap-storage all true):
the deploy actually runs inside the infinito container, so host-side
toolchains, language runtimes, and the default Docker layer cache are
dead weight. Update the workflow SPOT doc accordingly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Expose INFINITO_MEM_LIMIT/INFINITO_MEMSWAP_LIMIT/INFINITO_CPUS on the
infinito service so operators can cap local runs to GitHub-hosted runner
specs and reproduce OOM failures like #162. Document the compose.yml as
a dev/CI-only artefact under docs/contributing/artefact/files/ and link
to it from the CI debugging guide.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Split docs/agents/action/iteration.md into role.md, playwright.md,
workflow.md, and a scope-focused README. Move Certificate Authority
and Inspect rules into role.md. Rewrite playwright.md as a machine-
oriented state machine with explicit preconditions, procedure, exit
gates, and escape path.
Extract Playwright spec content MUSTs into the new contrib SPOT
docs/contributing/artefact/files/role/playwright.specs.js.md. Reduce
the agent-side spec and env.j2 pages to procedure only. Align the
framework SPOT and the test-e2e-playwright role README with the
central templates/package.json.j2 + files/playwright.config.js model
and images.playwright.version as single source of truth.
Sweep em dashes and sentence-case role headings across all touched
docs per documentation.md and readme_md.md rules.
Remove the direct push-on-main trigger from security-codeql.yml so that
the concurrency group no longer cancels the orchestrator-invoked scan.
CodeQL now runs exactly once per CI run via ci-orchestrator.yml, and
its failure continues to block code-quality-gate and all downstream
deploy, install, and environment tests.
Split the CI documentation along SPOT lines: workflows.md owns the
per-workflow catalog (trigger, inputs, short description); pipeline.md
owns the flow (stages, gates, concurrency, fork handling). Drop the
duplicated entry-points table and per-stage workflow bullet lists from
pipeline.md in favour of cross-references. Fix the stale "push to
latest" wording that pre-dated the current branch prefix list.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Treat chore/** branches like feature/fix/etc. for CI: trigger the
push pipeline on chore/** pushes and accept chore as a valid PR
branch prefix in full scope. This unblocks the chore/update-* PRs
that update.yml creates automatically, which previously failed
branch-prefix validation and never triggered the push pipeline.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add repository variable CI_RUN_ON_MAIN that gates whether pushes to
main trigger CI. When unset (default), pushes to main skip CI;
set to 'true' to enable. Pushes to other branches are unaffected.
Reorganize GitHub Actions docs into a dedicated tools/github/actions/
directory and consolidate every CI repository variable in a single
SPOT (configuration.md). Move actions.md to actions/workflows.md and
branch/configuration.md to actions/configuration.md, then update
every cross-reference. Trim README files to scope-only per
docs/contributing/artefact/files/docs/readme_md.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Moves the GitHub Actions workflow catalog under tools/github/ alongside the other GitHub-scoped docs, and updates the internal workflow/artefact links plus the two external references in workflow.md and pipeline.md.
Centralise the full GitHub Actions workflow catalog in
docs/contributing/tools/actions.md as the single source of truth, link
pipeline.md and workflow.md to it, and introduce
docs/agents/action/push-trigger-pull.md as the authoritative procedure
for the push to draft CI to ready-for-review loop. Retire the stale
flat docs/contributing/tools/agents/claude.md and drop docs README
page-inventories that violate the readme_md.md scope rule.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Centralise the full GitHub Actions workflow catalog in
docs/contributing/tools/actions.md as the single source of truth, link
pipeline.md and workflow.md to it, and introduce
docs/agents/action/push-trigger-pull.md as the authoritative procedure
for the push to draft CI to ready-for-review loop. Retire the stale
flat docs/contributing/tools/agents/claude.md and drop docs README
page-inventories that violate the readme_md.md scope rule.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Create contributor rules for the applications and users lookup plugins so
contributors understand the lookup-only access pattern and are not tempted
to reintroduce inline data in the removed group_vars files. Mark all 34
acceptance criteria in requirement 003 as satisfied.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Use vars.CI_CANCEL_IN_PROGRESS in entry-push-latest.yml so cancellation
can be disabled per-repo without changing workflow code
- Move docs/contributing/tools/github/branch.md into branch/ subdirectory
as security.md; add configuration.md for CI repository variables and
README.md as directory index
- Fix all cross-references in SECURITY.md, pipeline.md, branch.md,
pull-request.md, workflow.md, and repository.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
large-packages, docker-images and swap-storage can remove build
deps, cached layers and swap needed by memory-intensive services
(e.g. Keycloak 503 under memory pressure). Only android/dotnet/
haskell are guaranteed unused on deploy runners.
Rationale linked from each step comment and documented in
docs/contributing/artefact/files/github/workflow.md.
Keep live registry lookups out of the default make test flow.
Add a dedicated external suite, workflow, and docs so the check stays opt-in but still gates code CI when enabled.
Add tests/lint/repository/test_markdown_links.py — a hard-fail lint
test that scans all git-tracked markdown files for broken relative and
repo-root-absolute links. Skips external URLs, anchor-only links, fenced
code blocks, and placeholder targets.
Apply documentation.md rules to all staged files: add emojis after
headlines in docs files, remove emojis from role README headings,
standardize Credits sections to the canonical format, remove empty
sections and orphaned sentences referring to missing documentation files,
fix raw URLs and incorrect link text across role READMEs and docs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move emojis after headline text (CONTRIBUTING.md, actions/, design/, tools/, tests/)
- Replace em dashes with colons, semicolons, or complete sentences throughout
- Add missing emojis to all headlines in exceptions.md, principles.md, services.md, shell.md, setup.md, pull-request.md, commit.md, act.md, docker.md, pr/README.md
- Update H1 titles to reflect renamed directories (actions/, design/)
- Fix broken relative links in workflow.md after directory moves
- Fix full-path link texts in services.md (style.css.md, javascript.js.md)
- Fix broken principles.md link to lint.md after testing/ → actions/testing/ move
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- document docs README purpose and scope in a dedicated SPOT
- add scope-focused README.md files across docs directories
- remove lint rule that banned README.md files under docs
- Remove duplicated role README requirement from documentation.md (SPOT is readme_md.md)
- Add cross-reference from readme_md.md to documentation.md for general rules
- Fix em dashes in readme_md.md
- Apply RFC 2119 keywords where missing
- Add emojis to all headlines in both files
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move flow/security/ghcr.md to tools/ghcr/authentication.md
- Move flow/security/mirror-cleanup.md to tools/ghcr/cleanup.md
- Move flow/security/branch.md to tools/github/branch.md
- Move flow/security/repository.md to tools/github/repository.md
- Move code/frameworks/ansible.md to tools/ansible.md
- Merge code/frameworks/Makefile.md into tools/make.md and extract file rules to artefact/files/makefile.md
- Rename tools/makefile.md to tools/make.md
- Add .github/workflows/images-mirror-cleanup.yml for manual GHCR cleanup
- Apply documentation.md standards: emojis on headlines, no em dashes, filename link text
- Fix all cross-references broken by directory restructuring
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rename artefact/image.md to artefact/image/origin.md and merge code/role/image.md into it
- Move artefact/mirror.md to artefact/image/mirror.md
- Move docs/contributing/code/role/ to docs/contributing/artefact/files/role/
- Rename readme_base.md to readme_md.md
- Apply documentation.md standards (emojis, RFC 2119, link text) to all modified files
- Fix all cross-references broken by the directory restructuring
- Remove publish section from mirror.md (module deleted in prior commit)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
