Adds a `swarm:` subcategory under `svc:` in roles/categories.yml so the
shared-cluster role is properly invokable, and renames the existing
`svc-swarm` role + inventory group to `svc-swarm-node` (with
`get_entity_name`-aligned primary key `node:` in meta/services.yml).
`svc-swarm-manager` stays as the manager-marker role, becoming invokable
via the same new category and re-keyed to `manager:`.
Consumers of the cluster-orchestration service (svc-registry-cache,
svc-registry-docker, svc-swarm-manager) now reference the service-key
`node:` (was `swarm:`), matching the renamed primary entry. The planner
empirically resolves `service_registry['node']` -> `svc-swarm-node` and
emits it into --include for stack-host deploys, so the CI failure on
`universal-svc-registry-docker` (`svc-swarm: not present in inventory`)
no longer reaches `ValidDeployId`.
Breaking change for operators: inventory groups named `svc-swarm:` must
be renamed to `svc-swarm-node:`. `DEPLOYMENT_MODE` reads
`groups['svc-swarm-node']`, so a stale `svc-swarm:` group silently falls
back to compose mode.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Match the workflow's actual purpose (building OCI images) more
literally than the Docker mascot, freeing 🐳 for compose-specific
contexts (see test-deploy-compose.yml).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Use whale (🐳) for the per-app matrix job to mirror the swarm
workflow's bee (🐝) pattern, and magnifying glass (🔍) for the
discover step. Keep the workflow header on the existing 📦 prefix to
stay consistent with the other test-deploy-* siblings.
Rename the workflow file and align all references:
- ci-orchestrator.yml job key + uses: path
- artifact names: inventory-server-* -> inventory-compose-*,
playwright-server-* -> playwright-compose-*
- scripts/github/inventory/zip_compose_for_upload.sh output path
- 3 docs files
INFINITO_DEPLOY_TYPE: server stays - that key is wired to the
invokable-rule registry in utils/roles/validation/invokable.py and is
a separate refactor concern.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
svc-cache-registry -> svc-registry-cache
svc-docker-registry -> svc-registry-docker
svc-docker-swarm -> svc-swarm
svc-docker-swarm-manager -> svc-swarm-manager
categories.yml: drop svc.docker, add svc.registry. Deliberately no svc.swarm
category so get_entity_name('svc-swarm') stays 'swarm' instead of collapsing
to '' (which would break discover_role_services).
Entity keys updated to match the new svc.registry prefix:
services.cache-registry.* -> services.cache.*
services.registry.* -> services.docker.*
01_constructor: replace svc-docker bootstrap-loop entry with explicit
include_role of svc-swarm + bootstrap loop over [svc-registry, svc-storage].
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace per-key `compose_container_name` lookup with a generic
`compose_only(key, value)` SPOT that emits `key: "<value>"` only in
compose mode and nothing in swarm. Same gating logic now serves
container_name (157 call sites) and pull_policy (17 call sites)
without duplicate plugin code.
Why this consolidation:
- The earlier `build-key` lint rule was misdirected. Swarm tolerates
`build:` with a harmless warning when `image:` is pre-pushed (via
the existing handlers/swarm_registry_push.py flow in
roles/sys-svc-compose). The actual swarm-rejected key is
`pull_policy:` ("Additional property pull_policy is not allowed"),
and 10 templates were silently broken because they declared inline
`pull_policy: never` outside the existing `{% if DEPLOYMENT_MODE
\!= 'swarm' %}` gate in roles/sys-svc-container/templates/build.yml.j2.
- A single generic lookup is cleaner than N per-key wrappers when
the gating logic is identical (emit-key-or-nothing).
Lint changes in tests/lint/ansible/jinja/test_mode_incompatible_keys.py:
- New `pull-policy-key` rule, regex `^\s*pull_policy\s*:`, remediation
points at compose_only.
- Existing `container-name` rule keeps its raw-key regex; remediation
now also points at compose_only.
- Broken-split / broken-embed regexes rekeyed on the `compose_only`
lookup name (was `compose_container_name`).
- `build-key` rule and its 12 nocheck annotations across templates
removed.
Coordinated changes:
- 86 templates rewritten (157 container_name calls migrated;
17 inline pull_policy lines replaced).
- 1 redundant `{% if DEPLOYMENT_MODE \!= 'swarm' %}` guard dropped in
build.yml.j2 (the lookup self-gates).
- 12 unit tests pin the compose_only contract.
- Catalog (suppression.md) updated: `pull-policy-key` row added,
`container-name` row remediation updated.
- Old `plugins/lookup/compose_container_name.py` and its test module
deleted.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirror compose_container_name pattern: plugins/lookup/compose_restart.py
emits `restart: <policy>` only in compose mode. In swarm the key is
silently ignored (replaced by deploy.restart_policy) and produces
"Ignoring unsupported options" warnings at docker stack deploy time.
Sweep 19 raw restart: sites across 10 templates to the no-arg form
`{{ lookup('compose_restart') }}` (the lookup falls back to
DOCKER_RESTART_POLICY from ansible-vars).
base.yml.j2 keeps the explicit-arg form
`{{ lookup('compose_restart', docker_restart_policy | default(DOCKER_RESTART_POLICY)) }}`
so the 5 templates that override the policy via Jinja-scope
`{% set docker_restart_policy = ... %}` (erpnext, zammad, shopware,
matomo, openproject) retain their override path. Jinja-scope bindings
are visible only in the rendering template, never in the Ansible
variable context that the lookup sees, so the caller must resolve them.
Generalize the container-name lint into a data-driven multi-rule
scanner at tests/lint/ansible/jinja/test_mode_incompatible_keys.py
covering both container-name and restart-key. Adding a new rule is one
ModeRule row sharing the file-walk / suppression / line-grouping
scaffolding.
Also gate roles/svc-db-redis/templates/service.yml.j2's
resource.yml.j2 include with DEPLOYMENT_MODE \!= 'swarm'. Mirrors
base.yml.j2's existing pattern; prevents cpus / mem_limit /
mem_reservation / pids_limit from leaking into swarm output where
deploy.resources.* in deploy.yml.j2 already handles them.
11 unit tests pin the compose_restart lookup contract. Both rules
registered in docs/contributing/actions/testing/suppression.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
DEPLOYMENT_MODE is now derived per role from group membership:
swarm iff len(groups[application_id]) > 1, else compose. The cluster
infrastructure flag SWARM_CLUSTER_AVAILABLE is true whenever any host
sits in svc-docker-swarm, so DB and proxy services render attachable
overlay networks even when their own group lists only one host.
All service_is_stateful flags are removed across svc-db-mariadb,
svc-db-postgres, svc-db-openldap, svc-prx-openresty, svc-docker-registry
and svc-db-mariadb's compose template; the inventory shape now expresses
the same intent. Cluster-wide handlers (swarm build/push/validate/deploy,
external network creation, DB healthcheck) gain run_once + delegate_to
the manager so they execute exactly once regardless of how many hosts
the role's play targets. CA-injection and compose validate are gated to
compose-mode only; their compose-config strictness breaks when the
deploy block coexists with container_name under swarm.
Dead per-host swarm-resolve infrastructure is removed: swarm_resolve_
container.sh, resolve_container.yml, resolve_self.yml, the MariaDB-
specific swarm_mariadb_cid.sh, the cross-host template replication
step, and the _swarm_retries / _swarm_delay defaults. With run_once +
delegate_to: manager replacing the old "render every template on every
swarm peer" pattern, every consumer becomes a noop on workers and
collapses to the manager's one-shot path.
The web-app-nextcloud memcache override mapping flips from the legacy
"single" sentinel to "compose" (semantically equivalent under the new
model). Requirement 023 is updated to reflect the new model and drops
the now-irrelevant per-role opt-out / placement-constraint clauses.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the host-scoped set_fact resolvers in sys-svc-compose/tasks/01_core.yml
and tasks/stages/01_constructor.yml with a single group_vars definition driven
by group_names. Uppercase the variable to match the SPOT convention and drop
every `| default('compose')` fallback now that the resolution is unconditional.
Role-var opt-out on a swarm host still works via Ansible's role-vars > group_vars
precedence.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New requirement for a `web-app-erpnext` role that integrates ERPNext
(Frappe Framework) into Infinito.Nexus with OIDC-direct SSO against
Keycloak, reusing central `svc-db-mariadb` and `svc-db-redis` (three
Redis logical roles split by DB number 0/1/2 for cache/queue/socketio).
Captures 13 proposed design decisions for operator review before
implementation: upstream `frappe/erpnext` image pinning, central-service
reuse (Decision #7), wizard-bypass `bench` bootstrap (Decision #9),
Keycloak-group → Frappe-role reconciliation (Decision #5), Mailu IMAP
inbound + SMTP outbound (Decision #8), and the standard
V1/V2/V3 variant matrix (Decision #11) mirroring web-app-kix /
web-app-zammad. Out of scope for v1: multi-site / multi-tenant Frappe
(Decision #12).
Modeled on the layout precedents from web-app-odoo (ERP shape,
dual HTTP + WebSocket vhost) and web-app-zammad (central-service
reuse pattern, three-variant matrix).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the svc-docker-swarm, svc-docker-swarm-manager, svc-storage-nfs-server
and svc-storage-nfs-client roles, the compose_volumes filter NFS support,
the test-deploy-swarm-nfs CI workflow with its swarm-nfs-pilot script set,
and the deploy_mode-aware compose/swarm wiring across mediawiki, mariadb
and sys-svc-* helpers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Defines the v1 scope for multi-node Docker Swarm support: group-based
trigger (svc-docker-swarm + svc-docker-swarm-manager), three new roles
(svc-docker-swarm, svc-storage-nfs-server, svc-storage-nfs-client),
per-volume NFS opt-in (DB volumes stay local + pinned), overlay
networks with default encryption, and openresty kept as the swarm-mode
edge (pinned to manager, L7 LB via tasks.<service> upstream blocks) to
preserve the sys-front-inj-* injection pipeline unchanged. Pilot role
is web-app-mediawiki, validated by a 3-node DinD CI workflow whose
inventory is bootstrapped via the canonical inventory provision CLI.
Procedure block follows the autonomous-run pattern from 014.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Migrates the role from the pre-v9.0.0 legacy oidc/oauth2 top-level
service blocks to the unified services.sso shape with flavor: oidc.
Drops the playwright.spec.js require for the test-alias-domain module
that was deleted in 82491791e, and removes the README link to the
TODO.md file dropped in c48386eee. Archives the in-flight 022 zammad
requirement doc into the README index since the role itself is now
landed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops markdown links in 017/018 and role TODOs that pointed at the
016 and 019 requirement files archived in 027ba06ee. The text
references stay; only the now-broken link targets are removed so the
markdown-links lint passes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Moves 008/009/010/011/016/019/020 to the archive list in
docs/requirements/README.md and removes the per-requirement files;
their acceptance criteria are considered closed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".
Fix:
- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
full_cycle, variant) from `passthrough.STATIC_KEYS` and from
`default.env`. They are per-invocation; .env is the persistent SPOT
and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
`os.environ.get("KEY", "")` so the missing-key path stops at "no
filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
(lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
name in every doc; routing-knob env vars now use the same name so
the cmdline form, the recipe env-prefix, and the consumer reads
all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
knobs whose alias differs from the env name (`type` →
INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
auto-exports cmdline variables into the recipe's shell env, so the
identical-name `$(if $(apps),apps="$(apps)")` lines were pure
noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
`extra_env` per `docker exec` call and forwards `disable` via
`-e`, so the container env never needs the value frozen at
compose-up time.
- Update the 3 actual leak call sites
(07/08/09_deploy_*.sh) to the make-alias form
(`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
the deliberate make/env parity, not a typo.
Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.
Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
actions/create-github-app-token@v3 emits "Input 'app-id' has been
deprecated with message: Use 'client-id' instead." on every run.
Swap the input + rename the corresponding repository secret from
BOT_APP_ID to BOT_APP_CLIENT_ID so the value reflects what the
action actually expects: the OAuth-style Client ID (`Iv…` format
shown on the App's General page), not the numeric App ID.
Requires the operator to create a fresh `BOT_APP_CLIENT_ID`
repository secret on infinito-nexus/core before merging; the old
`BOT_APP_ID` secret becomes orphaned and SHOULD be deleted in the
same settings pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
actions/create-github-app-token@v3 emits "Input 'app-id' has been
deprecated with message: Use 'client-id' instead." on every run.
Swap the input + rename the corresponding repository secret from
BOT_APP_ID to BOT_APP_CLIENT_ID so the value reflects what the
action actually expects: the OAuth-style Client ID (`Iv…` format
shown on the App's General page), not the numeric App ID.
Requires the operator to create a fresh `BOT_APP_CLIENT_ID`
repository secret on infinito-nexus/core before merging; the old
`BOT_APP_ID` secret becomes orphaned and SHOULD be deleted in the
same settings pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three docs still referenced the pre-collapse service keys:
* docs/contributing/artefact/files/role/playwright.specs.js.md — biber
and administrator auth-chain diagrams listed
`skipUnlessServiceEnabled('oidc' | 'oauth2' | 'ldap')`; the unified gate
is `'sso' | 'ldap'`.
* docs/contributing/design/role/services/base.md — the `lookup('service',
...)` example block carried the now-invalid `'oidc'` key; replaced
with `'sso'`.
* docs/requirements/019-playwright-meta-services-parity.md — the
prometheus, matomo, and mailu rows still spelled
`safeSkipUnlessEnabled("oauth2"|"oidc")` and
`isServiceEnabled("oauth2")`; all rewritten to `"sso"`.
Three iteration-rule sharpenings, all surfaced by repeated foot-guns in
the web-app-zammad role-build loop:
- Promote "validate the fix in-container before redeploying" from SHOULD
to MUST. Adds an explicit reproducible-there carve-out (container
crashes at startup, misconfigured env var, image-side bug, in-container
script failure) so the rule is unambiguous on when it applies; the
only exception is failures that genuinely depend on multi-container or
post-deploy Ansible state.
- Playwright-specific addendum: when the failing unit is a Playwright
spec, "in-container validation" means running
`make compose-playwright role=<role>` against the live stack — not
just `curl` against the underlying API. The spec runs in a browser
context with env loading, locator strategies, and cookie state that
`curl` does not exercise.
- New "Full-matrix iteration flow" subsection under "Matrix variants"
documenting the variant-pinned `mode=update` → per-remaining-variant
`mode=reinstall full_cycle=true` → final unpinned full-matrix
`mode=reinstall full_cycle=true` recipe. Closes the gap where the
existing "Matrix variants" rules only described focused single-variant
debug and full-matrix initiation, but not the cycle that keeps full
cross-variant coverage as the iteration target.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three iteration-rule sharpenings, all surfaced by repeated foot-guns in
the web-app-zammad role-build loop:
- Promote "validate the fix in-container before redeploying" from SHOULD
to MUST. Adds an explicit reproducible-there carve-out (container
crashes at startup, misconfigured env var, image-side bug, in-container
script failure) so the rule is unambiguous on when it applies; the
only exception is failures that genuinely depend on multi-container or
post-deploy Ansible state.
- Playwright-specific addendum: when the failing unit is a Playwright
spec, "in-container validation" means running
`make compose-playwright role=<role>` against the live stack — not
just `curl` against the underlying API. The spec runs in a browser
context with env loading, locator strategies, and cookie state that
`curl` does not exercise.
- New "Full-matrix iteration flow" subsection under "Matrix variants"
documenting the variant-pinned `mode=update` → per-remaining-variant
`mode=reinstall full_cycle=true` → final unpinned full-matrix
`mode=reinstall full_cycle=true` recipe. Closes the gap where the
existing "Matrix variants" rules only described focused single-variant
debug and full-matrix initiation, but not the cycle that keeps full
cross-variant coverage as the iteration target.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds roles/web-app-zammad/ — a brand-new web-app role deploying Zammad as
the Infinito.Nexus helpdesk with direct OpenID Connect SSO against the
shared Keycloak client. Tracks requirement 022.
Stack: ghcr.io/zammad/zammad 6.5.0 (init / railsserver / websocket /
scheduler / nginx, all driven by upstream command modes), bundled
elasticsearch 8.13.4 + bundled memcached (no central svc role exists),
central svc-db-postgres reused via lookup('database', application_id).
Variant matrix mirrors web-app-kix: V1 oidc+ldap, V2 all-false, V3 ldap-only.
Bootstrap flow (tasks/01_core.yml):
- Stage auto_wizard.json into the role's volumes dir AFTER sys-stk-full
creates the directory; bind-mount it into zammad-init so the upstream
rake zammad:setup:auto_wizard bypasses the setup UI on first boot.
- Stage zz_omniauth_allow_get.rb initializer + bind-mount into rails /
websocket / scheduler / nginx, since the Zammad SPA's SSO link is a
plain GET and omniauth 2.x forbids GET by default.
- ALTER ROLE <zammad-db-user> WITH CREATEDB so Zammad's unconditional
`rake db:create` doesn't trip on insufficient privilege when the
database is pre-created by svc-db-postgres.
- Apply post-bootstrap settings via apply_oidc_settings.sh
(rails-runner inside zammad-railsserver): fqdn, http_type so Zammad
emits the correct redirect_uri to Keycloak; auth_openid_connect +
auth_openid_connect_credentials so the SSO button works.
Workarounds embedded in compose.yml.j2:
- zammad-nginx command override that copies /docker-entrypoint.sh to
/tmp and patches `grep "nameserver"` to `grep -E "^nameserver"`,
because Docker writes comment lines like `# Overrides: …` into
/etc/resolv.conf that the upstream grep matches as substrings,
yielding an invalid `resolver … Overrides: …` directive nginx rejects.
- DATABASE_URL pre-populated in env.j2 so post-bootstrap
`container exec rails r` calls (which bypass the entrypoint) can
load Rails without database.yml.
Playwright (files/playwright/playwright.spec.js, single-file lint-conform
layout): landing-reachable + administrator OIDC + biber OIDC + guest
public-landing scenarios. Personas skip-correctly via
skipUnlessServiceEnabled('oidc') under V2 / V3.
V1 end-to-end green: admin + biber Playwright pass through the full
Keycloak SSO round-trip. V2 (all-false) and V3 (ldap-only) green on
baseline reinstall with full_cycle=true.
Follow-ups indexed in roles/web-app-zammad/TODO.md: Zammad LdapSource
auto-create for V3, mail channel auto-config + end-to-end mail-to-ticket
test, Keycloak-group-to-Zammad-role mapping (Zammad doesn't ship a
built-in claim-to-role translator — currently all OIDC logins default to
Customer per Decision #5 fallback), Playwright assertion for the
`zammad.helpdesk.*` server-name alias.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New requirement for a `web-app-zammad` role that integrates Zammad as
the Infinito.Nexus helpdesk with OIDC-direct SSO against Keycloak.
Captures 11 operator-confirmed design decisions before implementation:
- Canonical `helpdesk.infinito.example` + true server-name alias
`zammad.helpdesk.infinito.example` (no 301 redirect).
- SSO flavor: OIDC-direct via Zammad's native OmniAuth strategy
(no oauth2-proxy sidecar). Uses the legacy `services.oidc.*` shape
(pre-021); does not block on 021.
- Keycloak OIDC client auto-provisioned via `web-app-keycloak`.
- Keycloak group -> Zammad role mapping (Admin/Agent/Customer) if
Zammad's OIDC strategy supports group claims natively; otherwise
documented limitation.
- Upstream `ghcr.io/zammad/zammad` image pinned to latest stable
semver. Self-built image (a la 015 Moodle) explicitly out of scope.
- Central-service reuse: `svc-db-postgres`, `svc-db-redis`,
`svc-db-memcached`. Elasticsearch bundled in-role (no central
`svc-db-elasticsearch` exists today).
- Email in scope: SMTP outbound via `sys-svc-mail-smtp` /
`web-app-mailu`, IMAP inbound (mail-to-ticket) auto-wired.
- Setup wizard bypassed on first deploy via env-vars + post-start
API call so a fresh deploy lands a ready-to-use instance.
- Playwright biber + administrator personas in scope (per 019).
- `meta/variants.yml` defines three variants mirroring web-app-kix:
V1 oidc+ldap, V2 all-false, V3 ldap-only.
Acceptance criteria are grouped per concern (routing/TLS, role
layout & image, central-service reuse, SSO/OIDC, LDAP, email,
first-admin bootstrap, variants, playwright, health & quality,
documentation) so each item is independently verifiable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ran `python -m cli.contributing.requirements.archive` after marking all
remaining acceptance criteria as `[x]` in:
- 012 - Package Cache (Nexus 3 OSS)
- 013 - Sub-Beta-to-Beta Lifecycle Promotion for `web-*` Roles
- 015 - OpenCloud and OpenTalk IAM integration
- 015 - Self-built Moodle image
Their headings are appended to the `## Archive` block of
`docs/requirements/README.md` and the per-requirement files are deleted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ran `python -m cli.contributing.requirements.archive` to move the
following fully-checked-off requirement files into the `## Archive`
block of `docs/requirements/README.md` and delete the per-requirement
files:
- 001 - Nextcloud Talk TURN relay
- 002 - Service Registry Refactoring
- 003 - Reduce `applications` and `users` to lookup-only access
- 004 - Generic RBAC role auto-provisioning via OpenLDAP
- 005 - WordPress Multisite support for OIDC-driven RBAC
- 006 - Service-gated Playwright tests
- 007 - WordPress to Discourse post round-trip
- 015 - KIX Service Management integration
- 016 - web-app-hugo
Cross-references from the still-open requirements (015-moodle,
015-opencloud, 017, 018, 019, 021) rewritten from the deleted file
paths to \`README.md#archive\`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`_build_application_defaults` returned variant 0 of every role with a
meta/variants.yml, which then deep-merged with the variant-N override
that build_dev_inventory bakes into the inventory. Any service flag
that variant N did not explicitly disable carried through from
variant 0 into the runtime applications map. The most visible casualty
was svc-bkp-container-2-local variant 1 (postgres + mariadb only),
where gitea + keycloak leaked in via variant 0, sys-service-loader
pulled in svc-db-openldap via keycloak.services.ldap, and the deploy
crashed in sys-svc-compose/04_files.yml on openldap's env.j2 with
"credentials is defined in schema at roles/svc-db-openldap/meta/schema.yml
but not set in application 'svc-db-openldap'".
Defaults now come from each role's variant-free base config
(meta/<topic>.yml plus schema-defaults, no variants overlay). Variants
stay accessible via get_variants for any caller that explicitly needs
a specific variant. Inventory-level applications.<app> overrides win
exactly as before.
Supporting changes that keep the variant matrix consistent under the
new defaults model:
- New lint guard tests/integration/roles/meta/variants/
test_non_baseline_explicit_disables.py: for every dynamic
service-key declared in a role's meta/services.yml that variant 0
pins to literal true, every non-baseline variant MUST explicitly
pin the same key to either literal true or to literal
enabled: false + shared: false. Statically-enabled keys are exempt
(same rule as test_auth_isolation.py).
- roles/web-app-{fusiondirectory,lam,nextcloud,phpldapadmin}/meta/
variants.yml: explicit `false` pins for dashboard / email / logout /
matomo / prometheus / talk in restrictive variants so deep-merge no
longer hands them the deleted variant-0 inheritance.
- roles/web-app-moodle/templates/playwright.env.j2: gate the LAM
block on `lookup('config', application_id, 'services.lam.enabled',
false)`. LAM is a transitive helper, not a moodle-owned service, so
the default-bearing lookup tolerates the missing path when the
variant does not deploy LAM.
- tests/unit/utils/cache/test_variants.py: rename the legacy
"always variant zero" coverage to TestMergedApplicationsUsesBaseConfig
and add test_get_application_defaults_omits_variant_zero_only_keys
as the regression guard for the leak this commit fixes.
- docs/contributing/artefact/files/role/variants.md +
docs/contributing/design/variants.md: variant 0 now manually
enumerates every dynamic services.yml key with
enabled: true + shared: true; `{}` is only permitted when the role
has no dynamic-enabled service-key.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
design/variants.md gains a "Credentials Generation" section that
explains both flows the credential machinery can take:
- CI/CD path: planner picks the round's variants -> build_dev_inventory
emits --app-variants -> provision CLI forwards --variant -> credentials
CLI hands the index to InventoryManager, which resolves the root
role's variants.yml overlay so resolve_schema_includes_recursive
sees the variant-merged services map and discovers every shared
provider that the variant enables.
- Standalone path: any caller that omits --app-variants / --variant
gets variant=None, InventoryManager skips the overlay, and shared-
provider discovery walks the base meta config only.
Also adds four reference-table rows for the involved files and fixes
two unrelated wording issues in the surrounding text:
- "every dep round R needs MUST be redeployed" -> grammatical fix.
- "historical payload" / "rename the file back" -> drop historical
framing per documentation.md's current-state-only rule.
artefact/files/role/variants.md gains a short "Credentials
Interaction" section that anchors back to the design doc so a role
author setting services.<key>.enabled+shared in a variant knows that
the credential generator will pick the provider up.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Renames the shared-services loader role from sys-utils-service-loader to
sys-service-loader (shorter, no orphaned "utils" prefix; entity name in
meta/services.yml realigns to service-loader per get_entity_name).
Introduces an explicit two-phase contract in the loader: after the
ordered preload pass, flush a `_service_loader_post_run` queue that
service roles register into during preload via the new helper
`list_or_shoot.yml` (list during preload, shoot directly outside it).
Replaces the `when: not _service_loader_preloading` guards in
sys-svc-compose and sys-svc-webserver-core with helper calls. Closes
the gap where non-web deploys would never load these post-load
dependencies; in normal web deploys behavior stays the same thanks to
each sub-role's `run_once_<role>` gate.
Docs, comments, test docstring, and constructor-stage invocation
follow the rename. Historical changelogs (CHANGELOG.md, packaging/*)
intentionally left at the original name.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Trailing textual references missed in the prior cmd-migration commit:
three requirement docs and two lint-test docstrings still pointed at
`make exec`, but the target has always been `make compose-exec`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the gap left by test_image_versions, which silently skips any
non-semver pin (latest, stable, branch tags). The new test_image_reachable
walks every entry from iter_role_images and probes the Docker Registry v2
manifest endpoint via HTTPS with anonymous bearer-token negotiation
(docker.io, ghcr.io, quay.io, mcr.microsoft.com). HTTP 404 hard-fails as
a broken pin; auth/rate-limit/5xx/network errors warn. Bounded by per-
request, per-probe, and global deadlines and parallelized via
INFINITO_WORKER_FETCH.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
INFINITO_CMD is a per-invocation argument, not a config value — drop it
from default.env, the passthrough STATIC_KEYS, and the dotenv-generator
test, and rename to lowercase `cmd` in Makefile targets, exec scripts,
internal callers, and user-facing docs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Auth-isolated matrix variants (exactly one of LDAP / OIDC-OAuth2
enabled, and no non-auth `enabled: true` alongside) must explicitly
pin every non-auth service-key in the role's variant union to literal
`enabled: false, shared: false`. Without this, the variant's
auth-only test scope balloons into a full integration deploy via the
services.yml-default propagation, wasting CI minutes and obscuring
the regression signal.
Adds `tests/integration/roles/meta/variants/test_auth_isolation.py`
(rule key `variants-auth-isolation`) plus the matching catalog entry.
The trigger is narrow on purpose: a variant that enables auth
alongside other services is a deliberate integration test, not an
auth-isolated one, and the rule does not fire there.
Mass-fixes the 24 legacy auth-isolated variants the rule turned up:
each gets the missing `enabled: false, shared: false` declarations
appended to its `services:` block. Where the false-override clashes
with a static-literal base in meta/services.yml, the override line
also picks up a `# nocheck: variants-static-override` marker. The
svc-bkp-container-2-local variant 2 additionally carries
`# nocheck: variants-services-match` on the new keys because they
are resolver-only matrix hooks with no consumer contract in the base
meta.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The variants → services match guard (`test_services_match.py`) used to
have no exemption mechanism, forcing every variant service-key to also
appear as a top-level entry in `meta/services.yml`. That contract is
correct for application roles whose variants pin per-app service
flags, but it forces noise on invokable infrastructure roles whose
variants exist only to pin the round's companion topology — there the
service-key is a resolver hook, not a consumer-side contract, and
adding a stub in `meta/services.yml` just duplicates the variant
declaration.
Add a `# nocheck: variants-services-match` opt-out (same line or line
above the key) so legitimate resolver-only matrix hooks can stay in
variants.yml without a synthetic services.yml stub. Mirrors the
`variants-static-override` parser pattern (text walk + per-leaf line
indices + is_suppressed_at).
Adopts the new exemption on `svc-bkp-container-2-local`:
meta/services.yml drops the gitea / keycloak / postgres / mariadb /
ldap stubs, meta/variants.yml carries the nocheck marker on each
variant service-key entry. The resolver closure per round stays
identical (16 / 3 / 2 roles).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per-role Playwright runs no longer call `npm install` against
registry.npmjs.org on every invocation. A new `01_setup.yml` runs once
per host/version, populates `_shared/node_modules/` inside the Playwright
container, and stamps the installed version in `.installed-version`.
Each `02_run_one.yml` mounts that tree read-only at `/e2e/node_modules`
and the container command becomes `npx --no-install playwright test`.
This eliminates the npmjs.org TLS-handshake flake that took out the
keycloak playwright in a recent variant-2 update, saves ~4 s per role,
and gives version bumps a single rebuild point.
Three retry layers cover transient registry failures: Ansible (3× with
5 s delay), npm fetch (5× with 10-60 s backoff), and image pull (3×).
Stage-base / shared-dir / stamp / node_modules paths live as SPOT
constants in `vars/main.yml` so call sites never reconstruct them.
Renames `tasks/run_one.yml` → `tasks/02_run_one.yml` (and adds
`01_setup.yml`) to make the include order visible by filename; updates
cross-references in `AGENTS.md`, the two `docs/contributing/...` pages,
`dump_disk_diagnostics.sh`, and the lint test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`lookup('image', '<role>', '<svc>', 'ref')` only consulted
`images_overrides.*` plus role defaults, which the mirror pipeline
never populates — the pipeline writes into
`applications.<role>.services.<svc>.{image,version}`. As a result,
`sys-svc-container`'s `BUSYBOX_IMAGE` / `NODE_IMAGE`,
`sys-ctl-hlth-csp`'s `HEALTH_CSP_IMAGE`, and `test-e2e-playwright`'s
`TEST_E2E_PLAYWRIGHT_IMAGE` always resolved to the raw upstream ref
and hit Docker Hub directly. In CI this surfaced as
`toomanyrequests: ...unauthenticated pull rate limit` on
`busybox:1.37`.
Switches every productive `lookup('image', ...)` to
`lookup('config', '<role>', 'services.<svc>.image|version')` — the
same path templates like `web-app-oauth2-proxy` already use, which
resolves through the merged `applications.*` map and so transparently
picks up the GHCR mirror override written by the inventory creator.
Removes the now-unused `plugins/lookup/image.py` plugin and its unit
test (zero remaining consumers across roles, scripts, and tests).
Rewrites the image-origin / image-mirror / dockerfile docs to teach
the `lookup('config', ...)` pattern, dropping references to the
deleted plugin and the dead `images_overrides.*` variable. Doc edits
follow the `documentation.md` rules (no em dashes, one sentence per
line, no historical phrasing).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Splits mixed-content target descriptions into a single description line
plus `Note:` follow-ups, and dresses the targets that carry real
parameters (`compose-deploy`, `compose-exec`, `compose-inner-run`) with
explicit `Usage:` / `Example:` / `Param <name>:` lines. Adds a
`variant=<idx>` Make var alias to `compose-deploy` so multi-variant
pinning works without dropping to `INFINITO_VARIANT=` env prefixes.
Removes redundant inline `compose-deploy` / `compose-inner-run` param
enumerations from the docs and replaces them with
`make help target=<name>` pointers, so the help banner stays the SPOT.
Also fixes SC2016 in `scripts/make/help.sh` (literal backticks in the
help banner) with a scoped suppression.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `make compose-playwright role=<role> [pw="..."] [keep=true]` as the
canonical inner-loop entry point for rerunning a role-local Playwright
spec against the live stack — replaces the raw
`scripts/tests/e2e/rerun-spec.sh` invocation across docs.
Defines a small comment-block schema for Makefile targets (description
line on top, followed by `Usage:` / `Example:` / `Note:` / `Param <name>:`
markers). Lint enforces it; `make help` shows the description per target
in the short list and `make help target=<name>` prints the full block.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the deploy router into the compose- domain prefix to match the
rest of the local stack lifecycle (compose-up/down/exec/...). Reorder
its block alphabetically between compose-down and compose-entity-purge
to keep `make help` and the alphabetical-targets lint happy.
Update every active call site: contributor docs, agent-iteration docs,
role-level Administration/README/TODO files, the bundles README, the
test-environment scripts, the in-repo deploy helpers and main.sh, and
the server stage task. The two wildcard prose references (`make
deploy-*` in CLAUDE.md and testing/playwright.md) become single
`make compose-deploy` mentions since no other deploy-* target exists.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the three container-* targets into the compose- domain prefix to
match their actual scope (compose stack maintenance) and to make `make
help` group them with the other compose lifecycle commands:
- container-purge-entity -> compose-entity-purge
- container-purge-system -> compose-system-purge
- container-refresh-inventory -> compose-inventory-refresh
Update the prereq chain (compose-system-purge -> compose-entity-purge),
the local-deploy testing doc table, and reorder the block so the
compose- family stays alphabetically sorted between compose-down and
compose-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
