Commit Graph
342 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.7 b34131ca1d feat(package-cache): add Sonatype Nexus 3 OSS pull-through cache (req-012)
Brings up the package-cache compose service alongside the existing
registry-cache, gated by the same `cache` profile. Bootstrap helper
(scripts/docker/cache/package.sh) auto-rotates the admin password from
the well-known default, accepts the Sonatype CE EULA (3.71+ blocks
artifact downloads with HTTP 403 until /v1/system/eula is POSTed —
disclaimer must be sent verbatim with its unicode quotes), enables
anonymous read, creates one default blobstore, and registers the MVP
proxy set: apt-debian, apt-ubuntu, pypi-proxy, npm-proxy, helm-bitnami,
raw-githubusercontent. The runner gets pip.conf / .npmrc /
sources.list snippets bind-mounted in (each defaulting to /dev/null
when the cache profile is inactive) so the deploy is byte-identical
with cache off.

Cache plumbing:
- scripts/meta/env/cache/{registry,package}.sh: per-cache env defaults,
  hostname-derived admin password seed (uses `hostname` command, not
  $HOSTNAME shell var, so the value is identical across interactive
  shells, non-interactive subshells, and Python subprocesses).
- scripts/docker/cache/{registry-ca,package}.sh: container-side
  bootstrap helpers, both moved under scripts/docker/cache/.
- scripts/system/cache/clean.sh + `make cache-clean`: wipes both
  /var/cache/infinito/core/cache/{registry,package} dirs and stops
  the cache containers.
- cli/deploy/development/compose.py: Profile gate exports the three
  new INFINITO_PACKAGE_CACHE_* client-snippet paths and runs the
  bootstrap helper after the package-cache reports healthy.

Compose service uses sonatype/nexus3:3.91.1-ubi pinned, runs as
root (UBI image refuses to chown a fresh bind-mount), and forces
NEXUS_SECURITY_RANDOMPASSWORD=false so the bootstrap can log in with
admin/admin123 before rotating; the random-password mode otherwise
auto-completes onboarding silently and leaves no readable secret on
disk for an automated bootstrap to pick up.

Tests: tests/unit/cli/deploy/development/test_compose.py grows two
new assertions (cache services listed in `up` args when local; build
tests pin CI=true to retain ci-only assertions).

Closes req-012.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 03:11:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b37f05ffff docs(requirements): add 012 package-cache (Nexus 3 OSS) spec
Adds Sonatype Nexus 3 OSS as a second service under the existing
`cache` compose profile, complementing (not replacing) the rpardini
docker registry-cache. Covers apt / pypi / npm / helm / raw via
proxy repos. Mirrors the registry-cache wiring conventions: env
script under scripts/meta/env/, strict ${VAR:?…} contract, Profile
gate, /dev/null bind-mount fallback when the profile is inactive.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-28 03:11:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b8edbe10da docs(requirements): add 011 role meta/info.yml migration spec
Specifies the move of project-internal descriptive role-level fields
(`logo`, `homepage`, `video`, `display`) out of
`meta/main.yml.galaxy_info` into a new project-owned
`roles/<role>/meta/info.yml` (file-root, no `info:` wrapper).

Same anti-pattern that req-010 fixed for `run_after` / `lifecycle` and
that the strip in 0ab817a83 already handled for
`license_url` / `repository` / `documentation`. After this requirement
lands, the Galaxy-schema lint
(`tests/lint/repository/test_role_meta_main_galaxy_schema.py`) will
go green for every remaining offender.

Includes:
  * target layout + file-root convention;
  * known consumer (`web-app-dashboard/lookup_plugins/docker_cards.py`);
  * acceptance criteria for file migration, schema rules, loader,
    consumer rewrites, lint, docs, atomicity;
  * validation app set + final iteration loop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 10:54:26 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00
kevinveenbirkenbachandClaude Opus 4.7 da8109883f docs(requirements): add 010 role meta run_after/lifecycle migration spec
Captures the agreed-upon move of run_after: and lifecycle: from
meta/main.yml.galaxy_info into meta/services.yml.<primary_entity> alongside
the rest of each role's service metadata. Primary entity is derived via
get_entity_name(role_name); multi-entity roles (bluesky/matrix) get a
dedicated metadata-holder entry next to their compose entities. Lifecycle
allow-list anchored on values actually present in roles today (planned,
pre-alpha, alpha, beta) plus stable/deprecated forward-compat. A small
helper (utils/roles/meta_lookup.py) hides the entity derivation from
callers and degrades gracefully on missing meta/services.yml. Depends on
both req-008 and req-009.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 12:21:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c5ebdd3e22 docs(requirements): add 009 per-role networks and ports migration spec
Captures the agreed-upon migration of group_vars/all/{08_networks,09_ports}.yml
into per-role meta/server.yml.networks and meta/services.yml.<entity>.ports
with the always-category-keyed shape, the inter/local/public split, and the
top-level port: -> ports.inter rename + int-cast. Defines PORT_BANDS in
group_vars/all/08_networks.yml as the single source for suggester and lint,
specifies cli meta {ports,networks} suggest helpers, and pins the lint rules
including the flat-set host-bound port collision check that excludes inter.
Depends on req-008.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 12:21:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0ce0d16740 docs(requirements): add 008 role meta layout refactoring spec
Captures the agreed-upon migration of every role's metadata to single-
responsibility files under meta/: meta/schema.yml (with new optional default:
field), meta/users.yml, meta/server.yml, meta/rbac.yml, meta/services.yml
(file-root keyed by entity_name; long-tail config inlined), and
meta/volumes.yml. Drops the compose. prefix from all consumer paths and
enumerates the lookup plugins to update.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 12:21:34 +02:00
kevinveenbirkenbach f5e81ee56b refactor(modes): retire MODE_CI in favour of a direct RUNTIME check
The Playwright E2E gate previously fired through `MODE_CI`, a boolean
that OR-ed three env-var markers (`GITHUB_ACTIONS`, `ACT`,
`INFINITO_MAKE_DEPLOY`) AND-ed with an `INFINITO_SKIP_E2E` opt-out.
That mechanism duplicated information `RUNTIME` already carries:

- `make deploy-*` already bakes `RUNTIME=dev` into the inventory at
  init time (`RUNTIME_VARS_JSON='{"RUNTIME":"dev"}'`).
- CI runners (act / GitHub Actions) detect `RUNTIME=act|github` from
  their canonical env markers via `cli.meta.runtime.detect_runtime`
  during init.
- A bare `ansible-playbook` from a production shell stays at the
  default `RUNTIME=host`.

Switching the gate to `RUNTIME in ['dev', 'act', 'github']` therefore
covers every case `MODE_CI` covered, AND folds the gate into the
single existing runtime SoT rather than maintaining a parallel one.

Changes:
- `tasks/stages/02_server.yml` E2E include now keys on `RUNTIME`.
- `group_vars/all/01_modes.yml` drops the `MODE_CI` block; a short
  note explains the retirement.
- `scripts/meta/env/ci.sh` deleted (it only existed to export
  `INFINITO_MAKE_DEPLOY=1`).
- The 5 `scripts/tests/deploy/local/deploy/*.sh` entry-points stop
  sourcing `ci.sh`.
- `cli/deploy/development/deploy.py` no longer forwards the four
  retired env markers into the container; the runtime decision is
  already serialised into the inventory the deploy stage consumes.
- `scripts/tests/deploy/local/deploy/reuse-kept-app.sh` drops the
  same four `-e` flags from its `docker exec`.
- `docs/requirements/006-playwright-service-gated-tests.md` gets a
  prepended "MODE_CI retired" update note; the historical sections
  remain for context but the dead `ci.sh` link target was reduced to
  inline code.

Behaviour: any runtime where E2E was fired before still fires it now,
and the previous opt-outs have a single replacement: pass
`RUNTIME=host` (or `--vars '{"RUNTIME":"host"}'`) to skip Playwright on
a deploy that would otherwise have run it.
2026-04-25 14:21:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d97ed7a9e8 feat(deploy): per-role matrix-deploy variants with folder-per-round model
Introduces per-role `meta/variants.yml` for declaring multi-variant
deploys (e.g. WordPress Single-Site vs. Multisite) and reshapes the
development deploy CLI to materialise one inventory folder per variant
and iterate them with co-located sync+async passes.

Why:
- The previous single-inventory + Pass-1-then-Pass-2 split deployed each
  variant twice and ran the async pass against a host left in a
  different variant's state, masking real bugs.
- Inventory and runtime concerns were tangled: variant selection lived
  as a runtime extra-var and required cross-stage coordination.

How:
- `roles/<role>/meta/variants.yml` (file format docs at
  docs/contributing/artefact/files/role/variants.md): YAML list of
  deep-merge overrides on top of `config/main.yml`. Empty entry `{}` is
  variant 0 (= legacy payload). 6 roles get their dev overrides moved
  here from inventories/dev.yml; WordPress declares 2 variants.
- `utils/runtime_data.py` gains `_build_variants` / `get_variants` and
  drops the retired `_active_variants` runtime selector. Loader always
  returns variant 0 as the default; non-zero variants reach
  `get_merged_applications` as inventory-level `applications.<app>`
  overrides baked at init time.
- New `cli/deploy/development/inventory.py`: `DevInventorySpec`,
  `plan_dev_inventory_matrix` (pure planner shared between init and
  deploy), `build_dev_inventory_matrix` (materialises one folder per
  round at `<dir>-<R>`, or `<dir>` when single-variant). Single-folder
  paths preserve full backward compat.
- `cli/deploy/development/deploy.py`: matrix loop with per-round
  cleanup (only apps whose variant changed AND that are about to
  re-deploy), `--variant <idx>` (or `VARIANT=` env) to pin a single
  round, and `--full-cycle` (or `FULL_CYCLE=true`) which interleaves
  sync + async per round so the async re-deploy targets exactly the
  host state the matching sync deploy just produced.
- `cli/deploy/development/exec.py` gains repeatable `--env KEY=VAL` so
  bash callers can inject per-run context into in-container helpers
  without inlining heredocs.
- `scripts/meta/env/inventory.sh` exports `INVENTORY_VARS_FILE`
  (Python mirror in `cli.deploy.development.common.DEV_INVENTORY_VARS_FILE`;
  test_inventory_vars_file_spot.py locks them against drift).
- `make deploy-fresh-purged-apps` and `scripts/tests/deploy/ci/dedicated.sh`
  switch to a single init + single deploy `--full-cycle`. The
  `make deploy-reuse-*` bash scripts honour `VARIANT=` by suffixing
  `INVENTORY_DIR`/`INVENTORY_FILE`.
- The 5 `scripts/tests/deploy/local/deploy/*.sh` scripts now delegate
  their in-container heredocs to dedicated helpers in
  `scripts/tests/deploy/local/utils/`, removing the bash-in-bash
  quoting hell.

Tests:
- `tests/unit/utils/test_variants.py` (renamed from
  `test_application_variants.py`): loader edge cases, variant-zero
  default, retired-`_active_variants` guard.
- `tests/unit/cli/deploy/development/test_inventory.py` (new): planner,
  variant baking, builder, `filter_plan_to_variant`, `DevInventorySpec`.
- `tests/unit/cli/deploy/development/test_matrix_deploy.py` (new):
  per-round deploy filter, cleanup decisions, `--variant` pinning,
  `--full-cycle` per-round interleave.
- `tests/unit/cli/deploy/development/test_inventory_vars_file_spot.py`
  (new): drift guard for the bash <-> Python literal pair.
- `tests/unit/cli/deploy/development/test_exec.py` extended for `--env`.

Docs:
- `docs/contributing/artefact/files/role/variants.md` (new): file format.
- `docs/contributing/design/matrix-deploy.md` (new): runtime mechanism
  (folder layout, --variant, --full-cycle, cleanup logic).
- `docs/contributing/actions/deploy.md` (new): contributor dev-deploy
  flow (make targets, --variant pinning).
- `docs/administration/deploy.md` slimmed to admin-only (was mixed
  admin+dev), cross-links to the new contributor doc.
- `docs/agents/action/iteration/role.md` keeps only agent-iteration
  recipes; mechanism explanation moved out and linked.
- `docs/contributing/documentation.md` adds the "no self-SPOT
  declaration" rule; `applications.md` renamed its SoT header for
  consistency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 13:47:48 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6b5aeabc3f refactor(env): move INFINITO_MAKE_DEPLOY SPOT from Makefile into scripts/meta/env/ci.sh
Drop the `DEPLOY_CI_EXPORTS := INFINITO_MAKE_DEPLOY=1` declaration
from the Makefile and the per-recipe `$(DEPLOY_CI_EXPORTS)` prefix
from every `deploy-*` target. The marker now lives in a dedicated
on-demand env module `scripts/meta/env/ci.sh` and is sourced by every
deploy entry-point under `scripts/tests/deploy/local/deploy/*.sh`,
which keeps the export aligned with the rest of the project's env
layer (`scripts/meta/env/all.sh` and friends).

`ci.sh` is intentionally NOT composed into `all.sh`: the marker
flips `MODE_CI` to `true` and MUST surface only when a deploy
actually runs, not for unrelated `make test*` / `make build*`
recipes that share the same `BASH_ENV`.

`docs/requirements/006-playwright-service-gated-tests.md` and the
`scripts/meta/env/README.md` document the new SPOT and the
always-on / on-demand distinction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 09:50:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 48766f0bf9 docs: align rbac/req docs with documentation.md and readme_md.md guidelines
- Remove every em-dash thought break (4 in 004, 24 in 006, 2 in
  the WordPress role README) per the "no em-dashes" rule in
  `docs/contributing/documentation.md`.
- Replace "is forbidden" with the RFC 2119 keyword "MUST NOT" in
  005 and 006.
- Disambiguate the ambiguous `[rbac.md](...)` links in the role
  README, the administration SPOT, and the design SPOT by giving
  each link a descriptive label ("Administration RBAC" /
  "RBAC design"), per the link-text rules in `documentation.md`.
- Move the emoji decorations on `roles/web-app-wordpress/tasks/
  plugins/README.md` headings from before to after the heading
  text and turn the U+2192 arrow into prose so the document
  matches the heading-emoji rule.
- Update `docs/contributing/design/iam/rbac.md` so the LDAP
  layout, Keycloak mapper, and `path_join` paragraphs reflect
  the flat-CN + per-app mapper implementation that requirement
  005 actually ships, instead of the hierarchical-OU vision the
  page was originally drafted against.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 09:34:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d009a35116 feat(rbac): hierarchical /roles/<app>/<role> + service-gated Playwright + Discourse round-trip
Implements requirements 005, 006, and 007.

005: per-application Keycloak `group-ldap-mapper`s anchor each app's
role groups under `/roles/<application_id>/<role_name>` (and, for
tenant-aware apps, `/roles/<application_id>/<tenant>/<role_name>`).
The flat-CN LDAP layout keeps each CN globally unique so Keycloak's
group-tree resolver never collides; per-app mappers use the LDAP
`description` as the visible Keycloak group name to keep the path
free of redundant prefixes. Migration purges legacy pre-005 entries
and forces a Keycloak `logout-all` so stale tokens drop pre-005
group paths.

006: per-service `<SERVICE>_SERVICE_ENABLED` env flags and the
`service-gating.js` helper let role specs `skipUnlessServiceEnabled`
cleanly when a shared service is missing or `SERVICES_DISABLED` is
set; `MODE_CI` markers (`INFINITO_MAKE_DEPLOY`, `GITHUB_ACTIONS`,
`ACT`) propagate through the deploy CLI into the test stage.

007: WordPress->Discourse round-trip Playwright scenario gated on the
`web-app-discourse` deployment.

WordPress Playwright runs 5/5 green on the single-site flow; the
multisite + Discourse scenarios are correctly skipped when their
service prerequisites are absent. The `rbac_group_path` lookup is the
single sanctioned producer for OIDC group paths and is wired into
every consumer (yourls, lam, minio, openwebui, pgadmin, phpldapadmin,
phpmyadmin, prometheus, xwiki).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 08:51:01 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f0f802faf0 docs(requirements): add 005/006/007 and tighten Playwright spec guide
Add three follow-up requirements that extend the work landed in 004:

- 005 WordPress Multisite support + LDAP per-application OU refactor:
  introduces opt-in rbac.tenancy schema, migrates every rbac-carrying
  role onto the ou=<application_id>,ou=roles,... hierarchy, adds a
  central rbac_group_path lookup plugin as the only sanctioned way to
  compute OIDC group paths, mandates one-time LDAP rewrite plus a
  Keycloak logout-all to flush stale tokens, and specifies the
  WordPress mu-plugin contract for per-site roles and grant_super_admin
  semantics.

- 006 Service-gated Playwright tests: per-service <NAME>_SERVICE_ENABLED
  env flags, shared helper at roles/test-e2e-playwright/files/
  service-gating.js with hard-fail on unknown services, test.skip() with
  flag-named reason, explicit per-role refactor checkboxes for every
  existing spec, and a new MODE_CI variable (GitHub Actions + nektos/act
  + make-driven deploy) replacing the accidental DOCKER_IN_CONTAINER
  proxy at the test-e2e-playwright include site.

- 007 WordPress -> Discourse post round-trip: Playwright scenario that
  publishes a WP post and asserts the corresponding Discourse topic
  exists, gated on DISCOURSE_SERVICE_ENABLED, with teardown that covers
  draft and partial states and an API key scoped for topic delete.

Also update the role Playwright spec guide to carry the Option B
service-gating rules (per-service boolean flags, helper-only reads,
hard-fail on unknown services) so contributor and requirement SPOTs
stay in sync.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 13:25:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 cc0a4301d9 docs(requirements): add 004 — generic RBAC role auto-provisioning via OpenLDAP
Defines the end-to-end contract for auto-provisioning LDAP groups for
every role that declares an rbac: block, gated on group_names, plus
the OIDC flow from Keycloak into WordPress and the Playwright
verification that exercises the full pipeline for three exemplary
WordPress roles (subscriber, editor, administrator).

Key decisions captured in the criteria:

- Scope covers every role under roles/ with an rbac.roles block, not
  just web-app-* — svc-*, web-svc-* etc. included when they declare
  rbac.
- Gating: a role's groups are provisioned only when application_id is
  in group_names on the current host.
- Naming is the flat pattern <application_id>-<role_name>; full DN
  stays under the existing ou=roles container (no new OU introduced),
  which preserves every existing <application_id>-administrator DN
  exactly so current bind/search filters keep working.
- Administrator is still implicitly added even when not listed in
  rbac.roles, matching today's build_ldap_role_entries behavior.
- Keycloak sync + OIDC groups claim must deliver WordPress role
  mapping without any alternative path (no direct LDAP bind from WP);
  if the current OIDC plugin config doesn't deliver that, it is fixed
  in the same work item.
- Idempotency is required at two levels: re-running the provisioning
  is a no-op, and the Playwright test preserves biber's starting
  group membership across a full run.
- The "serial" constraint applies only to the test: the three role
  sequences (subscriber → editor → administrator) run one after
  another so a regression in one mapping does not mask another.
- Iteration follows the Role Loop + Playwright Spec Loop with
  SERVICES_DISABLED="matomo,email".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 00:27:22 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2dbb2aaa1a feat(email): introduce lookup('email') plugin, migrate all consumers
Moves the SYSTEM_EMAIL_* resolution logic out of group_vars/all/03_email.yml
into a dedicated Ansible lookup plugin, so every role template reads outgoing
mail settings via lookup('email') (global) or lookup('email', application_id)
(per-app). Per-app compose.services.email is merged on top of the resolved
defaults; SYSTEM_EMAIL_* variables remain usable as an inventory escape hatch.

Also splits the email docs: administrator-facing configuration stays under
docs/administration/configuration/, plugin contract and extension rules move
to docs/contributing/design/services/email.md. services.md is relocated to
services/base.md so the services/ subtree can host per-service design pages.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 22:47:40 +02:00
kevinveenbirkenbachandClaude Opus 4.7 424b60853d docs: update cross-file links to current paths
- iam/{common,ldap,oidc}.md: point at 11_oidc.yml and 12_ldap.yml
  (renamed from 13_/14_ in group_vars numbering).
- requirements/003-*.md: repoint iteration.md links to
  docs/agents/action/iteration/{role,workflow}.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 23:27:00 +02:00
kevinveenbirkenbach b5c4b8a3d9 Merge branch 'main' into fix/users-applications.yml 2026-04-19 22:21:16 +02:00
kevinveenbirkenbach da140094a1 docs(agents): split iteration loops and refactor Playwright SPOT
Split docs/agents/action/iteration.md into role.md, playwright.md,
workflow.md, and a scope-focused README. Move Certificate Authority
and Inspect rules into role.md. Rewrite playwright.md as a machine-
oriented state machine with explicit preconditions, procedure, exit
gates, and escape path.

Extract Playwright spec content MUSTs into the new contrib SPOT
docs/contributing/artefact/files/role/playwright.specs.js.md. Reduce
the agent-side spec and env.j2 pages to procedure only. Align the
framework SPOT and the test-e2e-playwright role README with the
central templates/package.json.j2 + files/playwright.config.js model
and images.playwright.version as single source of truth.

Sweep em dashes and sentence-case role headings across all touched
docs per documentation.md and readme_md.md rules.
2026-04-19 22:04:24 +02:00
kevinveenbirkenbach 44c31aac18 Merge branch 'main' into fix/users-applications.yml 2026-04-19 19:46:04 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0b25c6646f docs(iam): add OIDC/LDAP design guide, move services.md to design/
Introduce docs/contributing/design/iam/ with a docs README, common.md
for cross-cutting principles, and per-provider pages oidc.md and ldap.md.
The OIDC page documents the OIDC.* variable tree, claim attribute SPOT
(preferred_username), and the rule that app-specific Keycloak protocol
mappers belong in per-client scopes, not the shared default.json.j2.
The LDAP page documents the LDAP.* variable tree, attribute SPOT, and
the paired-identifier rule with OIDC.

Move services.md from docs/contributing/code/ to docs/contributing/design/
so the design/ folder is the SPOT for cross-cutting platform conventions.
Update the two architecture.md cross-links and the requirement doc.

Cross-link the new guides from 13_oidc.yml, 14_ldap.yml, and the
web-app-keycloak and svc-db-openldap role READMEs. Fix nearby README
compliance issues per readme_md.md (svc-db-openldap H1, missing
Credits section in web-app-jira, Credits formatting drift).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 17:18:41 +02:00
kevinveenbirkenbachandClaude Opus 4.7 fa10c3c9bc test(e2e): add playwright specs for keycloak, matomo, openwebui, discourse + extend dashboard
Closes Verification checkboxes in requirement 003. Each spec exercises
the `lookup('applications')` + `lookup('users')` pathway end-to-end:
login via lookup('users'), OIDC round-trip from dashboard (where the
app supports it), canonical-domain DOM assertion, and a full CSP
assertion covering every directive emitted by csp_filters.build_csp_header
(15 directives), meta-parity, enforced policy (not Report-Only), and
zero securitypolicyviolation events.

The dashboard spec is extended in place with the CSP + canonical-domain
assertions while keeping its existing login/logout coverage.

rerun-spec.sh is pointed at the Jinja template + the staged copy of
package.json (the repo has templates/package.json.j2, not files/package.json),
so the inner spec-only iteration loop works without re-deploying.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 08:53:47 +02:00
kevinveenbirkenbachandClaude Opus 4.7 4dfd30d142 docs(requirements/003): add Clarifications section and refine CSP scope
Narrow CSP directive list to what csp_filters.build_csp_header actually
emits today, and record the binding implementation decisions (personas,
realms, iteration mode, batching, credential sources) under a new
Clarifications subsection so they supersede ambiguous checkbox reads.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 02:41:17 +02:00
kevinveenbirkenbachandClaude Opus 4.7 4f7dfddb27 docs(requirements/003): extend Verification with E2E flow, CSP, and CI gate specs
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 01:30:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f8829c7129 docs(requirements/003): add lookup-only contributor docs and tick off criteria
Create contributor rules for the applications and users lookup plugins so
contributors understand the lookup-only access pattern and are not tempted
to reintroduce inline data in the removed group_vars files. Mark all 34
acceptance criteria in requirement 003 as satisfied.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 00:36:13 +02:00
kevinveenbirkenbach 34ddc4d2d2 Merge branch 'main' into fix/users-applications.yml
# Conflicts:
#	group_vars/all/00_general.yml
#	roles/web-app-dashboard/templates/playwright.env.j2
#	roles/web-app-nextcloud/config/main.yml
#	roles/web-app-nextcloud/templates/playwright.env.j2
2026-04-16 11:54:20 +02:00
kevinveenbirkenbach 262e1b43fc Merge branch 'main' into fix/users-applications.yml
# Conflicts:
#	docs/contributing/design/services.md
#	plugins/lookup/applications_current_play.py
#	plugins/lookup/service.py
#	roles/sys-stk-backend/lookup_plugins/service_should_load.py
#	roles/sys-stk-backend/tasks/_load_service.yml
#	roles/sys-stk-backend/vars/main.yml
#	roles/sys-svc-compose/templates/base.yml.j2
#	roles/sys-svc-mail/tasks/01_core.yml
#	roles/sys-utils-service-loader/tasks/load_frontend_service.yml
#	tests/unit/plugins/lookup/test_applications_current_play.py
#	tests/unit/plugins/lookup/test_service.py
#	tests/unit/roles/sys-stk-backend/lookup_plugins/test_service_should_load.py
2026-04-15 23:59:54 +02:00
Environment TestandClaude Opus 4.6 a2275e6991 docs(make): pass variables after target to match Bash(make*) permission
Reorders all documented and scripted `make` invocations from the
`VAR=value make target` shell-env prefix form to the make-native
`make target VAR=value` form. GNU Make auto-exports both command-line
and inherited shell-env variables to recipe shells, so the change is
purely about command-line shape: only the make-suffix form starts with
`make` and is therefore matched by the `Bash(make*)` permission entry
in .claude/settings.json, avoiding unnecessary approval prompts for
agent runs.

Adds an explicit CLAUDE.md rule with ✅/❌ examples and the underlying
reason (permission prefix match, not Makefile semantics).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 12:28:44 +02:00
kevinveenbirkenbach e9f95318e7 Merge branch 'infinito-nexus:main' into fix/nextcloud-talk 2026-04-15 01:46:38 +02:00
kevinveenbirkenbach f226828117 docs(requirements): refine lookup migration spec 2026-04-15 00:10:39 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 bb7aeae9d6 docs(requirements): add 003 for lookup-only applications and users access
Introduces a requirements document that mandates deleting
group_vars/all/04_users.yml and group_vars/all/05_applications.yml,
building applications and users as facts in the constructor via two new
lookup plugins backed by roles/*/config/main.yml and roles/*/users/main.yml,
removing the deprecated make setup generators, rewriting every
applications / users reference to the new lookups, and caching the
aggregated defaults at module level for performance.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 23:38:56 +02:00
Environment Test 83cd881dee feat(nextcloud): wire standalone coturn TURN + admin identity tasks
- Add 07_admin_identities.yml: resolves LDAP-backed admin identities
  for OCC calls that require the actual NC username, not the LDAP DN
- vars: add NEXTCLOUD_HPB_TURN/STUN_ACTIVE_CONFIG selecting standalone
  or onboard TURN based on enabled flags; add TALK_ADMIN_URL
- config: talk container gated on spreed plugin flag
- env.j2 + playwright.env.j2: expose active TURN/STUN config and
  Talk admin URL for container env and E2E tests
- playwright.spec.js: extend with Talk TURN/STUN E2E assertions
- schema: add spreed.enabled validation
- coturn compose: move relay port range before TLS block so ports are
  always set regardless of TLS mode
- docs/requirements: document TURN relay requirement
- test_resolver: cover new resolution paths
2026-04-14 02:29:29 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 1bcb7cfbb0 Merge branch 'main' into feature/service-refactoring
Resolve conflicts:
- docs/agents/action/iteration.md: keep both new bullets (SERVICES_DISABLED
  proposal + make test requirement)
- docs/contributing/design/services.md: deleted (renamed to
  docs/contributing/code/services.md in this branch)
- group_vars/all/20_services.yml: keep deletion from this branch (req-001)
- tests/lint/ansible/roles/test_service_core_first_task_run_once.py:
  use build_service_registry_from_roles_dir (req-001) with repo_root()
  from main for robust path resolution

Fix broken relative links caused by rename and deletion of files:
- docs/contributing/design/architecture.md: services.md → ../code/services.md
- docs/contributing/design/exceptions.md: load_frontend_service.yml →
  load_service.yml
- docs/requirements/001-service-registry-refactoring.md: fix paths for
  unit.md, integration.md, principles.md, documentation.md, AGENTS.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:58:51 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 be6a310d59 refactor(services): implement role-local service registry (req-001)
- Delete group_vars/all/20_services.yml; all service metadata now lives
  in each role's config/main.yml under compose.services.<name>
- Rename generic 'database' service key to explicit 'mariadb'/'postgres'
  across all app roles; remove legacy type: field
- Add utils/service_registry.py and plugins/lookup/service_registry.py
  as the single source of truth for service discovery
- Add utils/database_service.py to centralize DB service resolution
- Delete service_load_enabled filter and service_should_load lookup;
  replaced by service lookup plugin
- Update sys-utils-service-loader to run in constructor stage and use
  generic load_service.yml; remove load_frontend_service.yml
- Add _service_loader_preloading guard to sys-svc-compose,
  sys-svc-webserver-core and sys-ctl-alm-compose to prevent
  health-checks/alarms from firing during preload phase
- Replace manual load_app.yml calls for shared DB/Mailu with assert-
  based preload validation in sys-svc-rdbms and sys-svc-mail
- Update networks.yml.j2 templates to resolve LDAP/AI networks via
  service_registry instead of hardcoded role names
- Migrate all tests to new service discovery API; delete obsolete tests
- Update docs/contributing/code/services.md to reflect role-local model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:33:20 +02:00
Environment Test 8c19808a84 docs(requirements): add turn relay req 2026-04-13 11:24:51 +02:00
Environment Test ad280a891b docs(requirements): add turn relay req 2026-04-11 14:57:11 +02:00
Environment Test 80fe5e3846 docs: add scope README docs under docs
- document docs README purpose and scope in a dedicated SPOT
- add scope-focused README.md files across docs directories
- remove lint rule that banned README.md files under docs
2026-04-10 17:23:10 +02:00
kevinveenbirkenbach e5a30262aa docs(requirements): clarify service registry refactoring rules 2026-04-02 01:12:29 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 bc20b295dd docs(requirements): require simplification of all touched files per principles.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 01:05:51 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 d5fbd741bc docs(requirements): add service registry refactoring requirement
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-02 01:02:43 +02:00
kevinveenbirkenbach 0928e0ca37 docs(requirements): add contribution and agent requirement guides 2026-04-01 23:23:30 +02:00
kevinveenbirkenbach 36bf5af288 Decoupeled Sphinx 2025-04-09 16:56:41 +02:00
kevinveenbirkenbach c402583f2b Redrafted Sphinx for CyMaIS 2025-03-21 13:48:42 +01:00