Commit Graph
125 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.7 b3de2a4d7c refactor(meta): land req-011 meta/info.yml + utils/cache cleanup
req-011 — descriptive role-level metadata
  * Move logo / homepage / video / display out of
    meta/main.yml.galaxy_info into a new project-owned
    roles/<role>/meta/info.yml (file-root convention, no `info:` wrapper).
  * 78 role meta/main.yml files migrated via ruamel round-trip
    (key-order / quoting / inline-comments preserved). Roles with none
    of the four fields are left untouched (no empty info.yml created).
  * web-app-dashboard/lookup_plugins/docker_cards.py reads logo +
    display from meta/info.yml; description / galaxy_tags continue to
    come from galaxy_info (Galaxy-spec fields).
  * docker_cards.py now routes ALL its file reads through the
    process-wide caches: load_yaml_any for vars/main.yml,
    meta/main.yml, meta/info.yml; read_text from utils.cache.files for
    README.md. The orphan `import yaml` is gone.

New utils/cache/files (renamed from tests/utils/fs)
  * The process-wide filesystem cache (read_text, iter_project_files,
    iter_project_files_with_content, _all_project_files) is not
    test-specific; it now lives under utils/cache/files.py next to the
    other process-wide caches. 34 consumer imports rewritten across
    tests/ and docs/. The empty tests/utils/ package is removed.
  * New unit suite tests/unit/utils/cache/test_files.py — 12 cases
    covering read_text, iter_project_files (skip-dirs, extensions,
    exclude_tests, exclude_dirs, walk-cache invariants), the combined
    iterator, non-UTF-8 silent skip, and _reset.

utils/cache redundancy cleanup
  * applications.py + users.py + cli/create/inventory/services_disabler.py
    drop their direct yaml.safe_load calls and their private
    _load_yaml_mapping/_load_yaml_variant_list copies; everything routes
    through utils.cache.yaml.load_yaml / load_yaml_any. The dead
    utils.cache.base._load_yaml_mapping is removed.
  * users.py:_load_store_users (tokens.yml read) likewise migrated.
  * _load_yaml_variant_list moves out of base.py to applications.py
    (its only consumer) and is renamed _load_variants_overrides — also
    routes through the cached loader now.
  * utils.cache.yaml._reset_cache_for_tests is renamed _reset (kept as
    alias) for parity with the per-domain reset helpers.
  * utils.cache.files grows a _reset() that clears its two LRU caches.
  * utils.cache.__init__._reset_cache_for_tests now orchestrates ALL
    six caches (applications, users, domains, base, yaml, files) so
    tests get clean state across cache layers — needed because
    applications/users now share the yaml cache and would otherwise
    serve stale parses across reset boundaries.
  * utils.cache.files imports PROJECT_ROOT from base instead of
    redefining it.
  * tests/unit/utils/cache/test_data.py + test_base.py: obsolete
    TestLoadYamlMapping / TestLoadYamlVariantList classes deleted
    (covered by test_yaml.py + test_variants.py).
  * README.md updated for the new module list (files, yaml) and the
    six-module orchestrator.

Galaxy schema lint
  * tests/lint/repository/test_role_meta_main_galaxy_schema.py: split
    galaxy_info field requirements into hard-required {author,
    description, license} and soft-recommended {min_ansible_version,
    platforms}. Soft-recommended -> emit a GitHub Actions ::warning::
    annotation via utils.annotations.message.warning (titled
    "meta/main.yml galaxy_info" with file= pointing at the offender),
    no test failure. Rationale: min_ansible_version + platforms are
    only enforced by Galaxy when *publishing*, which Infinito.Nexus
    does not do.
  * Galaxy lint goes green (was 226 offenders before req-011 +
    license_url/repository/documentation strip; was 62 before this
    relaxation; now 0 hard offenders, 62 soft warnings).
  * roles/svc-opt-keyboard-color/meta/main.yml: backfill `license`
    (only role missing the hard-required field).

New lint
  * tests/lint/repository/test_role_meta_info.py — enforces meta/info.yml
    file-root convention, the four-field allowlist (logo, homepage,
    video, display), and shape (logo.class string, homepage/video
    non-empty strings, display bool). Also fails if any
    meta/main.yml.galaxy_info still carries one of the four migrated
    fields.

Docs
  * docs/contributing/design/services/layout.md: new section
    "Descriptive Role-Level Metadata — meta/info.yml" with allowed-
    fields table, optionality rule, materialised path, and consumer
    note. File-Layout table grows the meta/info.yml row;
    meta/main.yml row gets the explicit "no logo/homepage/video/display"
    note. req-011 added to header and Related Pages.
  * utils/cache/README.md: files.py + yaml.py rows added; six-module
    reset orchestrator documented.

Validation
  * Unit:        1573 passed.
  * Lint:        55 passed.
  * Integration: 161 passed (1 skipped) via pre-commit hook.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 12:27:17 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9c35909752 refactor(utils/cache): split data.py into base/applications/users/domains
The 934-line `utils/cache/data.py` monolith mixed cross-cutting helpers
(constants, yaml-loaders, deep-merge, signatures, render machinery)
with three independent domain caches (applications, users, domains).
Per the KISS/SPOT/DRY guidelines, split it along domain boundaries so
each file has a single, well-scoped responsibility:

  utils/cache/__init__.py     — package; owns _reset_cache_for_tests()
                                orchestrator
  utils/cache/base.py         — cross-cutting helpers: constants,
                                yaml-loaders, _deep_merge, cache-key +
                                content-fingerprint signatures, the
                                cross-domain re-entry guard, and the
                                templar-render machinery (the only
                                ansible-coupled symbol, lazy-imports
                                its dependency)
  utils/cache/applications.py — variants + defaults +
                                get_merged_applications. Strictly
                                ansible-free at import time so the
                                GitHub Actions runner-host CLI path
                                (cli.deploy.development.init →
                                plan_dev_inventory_matrix →
                                get_variants) keeps working without
                                ansible installed
  utils/cache/users.py        — user defs, token store hydration,
                                alias materialization,
                                get_user_defaults, get_merged_users
  utils/cache/domains.py      — get_merged_domains, derived from the
                                merged applications view
  utils/cache/data.py         — DELETED. Façade is gone; the package
                                exports nothing the domain modules
                                don't already

Backward-compat: the facade was deleted (per operator request) and all
36+ consumers — 18 plugins, 2 CLI entry points, 9 test files, 4 doc
files, plus a role-internal lookup_plugin — were rewired to import
directly from the owning domain module. `_reset_cache_for_tests` is
now exposed at `from utils.cache import _reset_cache_for_tests`.

Tests added (per-module, focused):
  tests/unit/utils/cache/test_base.py            (31 cases)
  tests/unit/utils/cache/test_applications.py    ( 9 cases)
  tests/unit/utils/cache/test_users_module.py    (21 cases; suffix
                                                  disambiguates from
                                                  the existing
                                                  tests/unit/plugins/
                                                  lookup/test_users.py)
  tests/unit/utils/cache/test_domains.py         ( 6 cases)

The ansible-block invariants from CI runs 24934007615 and 24935979190
are now pinned via subprocess isolation: each ansible-less import test
spawns a fresh `python3 -c '…'` with a meta_path filter denying every
`ansible*` import and asserts the import + invocation succeed in that
clean process. The earlier in-process sys.modules-manipulation shims
fragmented module identity and broke under Python 3.13 inside the
infinito container; subprocess isolation has neither problem.

Doc-style audit (per docs/contributing/documentation.md):
  - Removed em-dash from utils/cache/README.md (split into two
    sentences)
  - Updated stale `utils.runtime_data` reference in
    docs/contributing/design/matrix-deploy.md
  - Removed two SPOT self-declarations
    (docs/agents/files/group_vars/applications.md and
    docs/contributing/artefact/files/plugins/lookup/users.md) per
    the documentation rule that SPOT status is conferred by
    cross-links, not by the page asserting it about itself

Empirical proof:
  - All four domain modules import without ansible (subprocess
    isolation tests)
  - get_variants(roles_dir=ROLES_DIR) returns 99 apps in the
    ansible-blocked subprocess
  - 142 cache tests + 1085 wider-suite tests pass; the only
    remaining failures (test_host_vars free-function fixture bug,
    test_runtime_performance timing flake) are pre-existing and
    unrelated to this refactor (verified via `git stash`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-26 00:27:57 +02:00
kevinveenbirkenbach 12dba01156 refactor(utils): consolidate cache modules into utils/cache/ package
Both in-process cache layers — the cached YAML loader and the cached
runtime-data getters — now live next to each other under
`utils/cache/`. The package gains a README that documents the
"orthogonal layers, no third cache" rule so future contributors hit
the right primitive on the first look.

Moves:
- `utils/yaml_cache.py` -> `utils/cache/yaml.py`
- `utils/runtime_data.py` -> `utils/cache/data.py`
- `tests/unit/utils/test_yaml_cache.py` -> `tests/unit/utils/cache/test_yaml.py`
- `tests/unit/utils/test_runtime_data.py` -> `tests/unit/utils/cache/test_data.py`
- `tests/unit/utils/test_variants.py` -> `tests/unit/utils/cache/test_variants.py`

New:
- `utils/cache/__init__.py` (empty per project convention)
- `utils/cache/README.md` describing the two layers, when to reach for
  which, lifetime, test-only helpers, and design guidelines (RFC 2119
  keywords, emoji-after-headline, no self-SPOT declaration, no
  slash-in-link-text).
- `tests/unit/utils/cache/__init__.py`

Mechanical updates:
- All imports across cli/, plugins/lookup/, tests/, and roles/ rewritten
  from `utils.yaml_cache` -> `utils.cache.yaml` and
  `utils.runtime_data` -> `utils.cache.data` (40+ files).
- `REPO_ROOT` parents-index in the moved tests bumped from `parents[3]`
  to `parents[4]` to reflect the deeper nesting.
- Six broken markdown links in docs/ updated to the new module paths
  (test_markdown_links lint test stays green).
- Two test files that used the `from utils import runtime_data` form
  now use `from utils.cache import data as runtime_data` so the local
  alias remains.

Tests: 848 passed across unit + lint suites (1 pre-existing collection
error in test_host_vars.py unrelated to this change).
2026-04-25 17:17:14 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d97ed7a9e8 feat(deploy): per-role matrix-deploy variants with folder-per-round model
Introduces per-role `meta/variants.yml` for declaring multi-variant
deploys (e.g. WordPress Single-Site vs. Multisite) and reshapes the
development deploy CLI to materialise one inventory folder per variant
and iterate them with co-located sync+async passes.

Why:
- The previous single-inventory + Pass-1-then-Pass-2 split deployed each
  variant twice and ran the async pass against a host left in a
  different variant's state, masking real bugs.
- Inventory and runtime concerns were tangled: variant selection lived
  as a runtime extra-var and required cross-stage coordination.

How:
- `roles/<role>/meta/variants.yml` (file format docs at
  docs/contributing/artefact/files/role/variants.md): YAML list of
  deep-merge overrides on top of `config/main.yml`. Empty entry `{}` is
  variant 0 (= legacy payload). 6 roles get their dev overrides moved
  here from inventories/dev.yml; WordPress declares 2 variants.
- `utils/runtime_data.py` gains `_build_variants` / `get_variants` and
  drops the retired `_active_variants` runtime selector. Loader always
  returns variant 0 as the default; non-zero variants reach
  `get_merged_applications` as inventory-level `applications.<app>`
  overrides baked at init time.
- New `cli/deploy/development/inventory.py`: `DevInventorySpec`,
  `plan_dev_inventory_matrix` (pure planner shared between init and
  deploy), `build_dev_inventory_matrix` (materialises one folder per
  round at `<dir>-<R>`, or `<dir>` when single-variant). Single-folder
  paths preserve full backward compat.
- `cli/deploy/development/deploy.py`: matrix loop with per-round
  cleanup (only apps whose variant changed AND that are about to
  re-deploy), `--variant <idx>` (or `VARIANT=` env) to pin a single
  round, and `--full-cycle` (or `FULL_CYCLE=true`) which interleaves
  sync + async per round so the async re-deploy targets exactly the
  host state the matching sync deploy just produced.
- `cli/deploy/development/exec.py` gains repeatable `--env KEY=VAL` so
  bash callers can inject per-run context into in-container helpers
  without inlining heredocs.
- `scripts/meta/env/inventory.sh` exports `INVENTORY_VARS_FILE`
  (Python mirror in `cli.deploy.development.common.DEV_INVENTORY_VARS_FILE`;
  test_inventory_vars_file_spot.py locks them against drift).
- `make deploy-fresh-purged-apps` and `scripts/tests/deploy/ci/dedicated.sh`
  switch to a single init + single deploy `--full-cycle`. The
  `make deploy-reuse-*` bash scripts honour `VARIANT=` by suffixing
  `INVENTORY_DIR`/`INVENTORY_FILE`.
- The 5 `scripts/tests/deploy/local/deploy/*.sh` scripts now delegate
  their in-container heredocs to dedicated helpers in
  `scripts/tests/deploy/local/utils/`, removing the bash-in-bash
  quoting hell.

Tests:
- `tests/unit/utils/test_variants.py` (renamed from
  `test_application_variants.py`): loader edge cases, variant-zero
  default, retired-`_active_variants` guard.
- `tests/unit/cli/deploy/development/test_inventory.py` (new): planner,
  variant baking, builder, `filter_plan_to_variant`, `DevInventorySpec`.
- `tests/unit/cli/deploy/development/test_matrix_deploy.py` (new):
  per-round deploy filter, cleanup decisions, `--variant` pinning,
  `--full-cycle` per-round interleave.
- `tests/unit/cli/deploy/development/test_inventory_vars_file_spot.py`
  (new): drift guard for the bash <-> Python literal pair.
- `tests/unit/cli/deploy/development/test_exec.py` extended for `--env`.

Docs:
- `docs/contributing/artefact/files/role/variants.md` (new): file format.
- `docs/contributing/design/matrix-deploy.md` (new): runtime mechanism
  (folder layout, --variant, --full-cycle, cleanup logic).
- `docs/contributing/actions/deploy.md` (new): contributor dev-deploy
  flow (make targets, --variant pinning).
- `docs/administration/deploy.md` slimmed to admin-only (was mixed
  admin+dev), cross-links to the new contributor doc.
- `docs/agents/action/iteration/role.md` keeps only agent-iteration
  recipes; mechanism explanation moved out and linked.
- `docs/contributing/documentation.md` adds the "no self-SPOT
  declaration" rule; `applications.md` renamed its SoT header for
  consistency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 13:47:48 +02:00
kevinveenbirkenbachandClaude Opus 4.7 48766f0bf9 docs: align rbac/req docs with documentation.md and readme_md.md guidelines
- Remove every em-dash thought break (4 in 004, 24 in 006, 2 in
  the WordPress role README) per the "no em-dashes" rule in
  `docs/contributing/documentation.md`.
- Replace "is forbidden" with the RFC 2119 keyword "MUST NOT" in
  005 and 006.
- Disambiguate the ambiguous `[rbac.md](...)` links in the role
  README, the administration SPOT, and the design SPOT by giving
  each link a descriptive label ("Administration RBAC" /
  "RBAC design"), per the link-text rules in `documentation.md`.
- Move the emoji decorations on `roles/web-app-wordpress/tasks/
  plugins/README.md` headings from before to after the heading
  text and turn the U+2192 arrow into prose so the document
  matches the heading-emoji rule.
- Update `docs/contributing/design/iam/rbac.md` so the LDAP
  layout, Keycloak mapper, and `path_join` paragraphs reflect
  the flat-CN + per-app mapper implementation that requirement
  005 actually ships, instead of the hierarchical-OU vision the
  page was originally drafted against.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 09:34:03 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d009a35116 feat(rbac): hierarchical /roles/<app>/<role> + service-gated Playwright + Discourse round-trip
Implements requirements 005, 006, and 007.

005: per-application Keycloak `group-ldap-mapper`s anchor each app's
role groups under `/roles/<application_id>/<role_name>` (and, for
tenant-aware apps, `/roles/<application_id>/<tenant>/<role_name>`).
The flat-CN LDAP layout keeps each CN globally unique so Keycloak's
group-tree resolver never collides; per-app mappers use the LDAP
`description` as the visible Keycloak group name to keep the path
free of redundant prefixes. Migration purges legacy pre-005 entries
and forces a Keycloak `logout-all` so stale tokens drop pre-005
group paths.

006: per-service `<SERVICE>_SERVICE_ENABLED` env flags and the
`service-gating.js` helper let role specs `skipUnlessServiceEnabled`
cleanly when a shared service is missing or `SERVICES_DISABLED` is
set; `MODE_CI` markers (`INFINITO_MAKE_DEPLOY`, `GITHUB_ACTIONS`,
`ACT`) propagate through the deploy CLI into the test stage.

007: WordPress->Discourse round-trip Playwright scenario gated on the
`web-app-discourse` deployment.

WordPress Playwright runs 5/5 green on the single-site flow; the
multisite + Discourse scenarios are correctly skipped when their
service prerequisites are absent. The `rbac_group_path` lookup is the
single sanctioned producer for OIDC group paths and is wired into
every consumer (yourls, lam, minio, openwebui, pgadmin, phpldapadmin,
phpmyadmin, prometheus, xwiki).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 08:51:01 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f722e676ca refactor(service-lookup): rename 'needed' flag to 'required'
Rename the third flag returned by lookup('service', ...) from 'needed'
to 'required' across code, tests and design docs. Same semantics: True
when some deployed app has the service with both enabled: true AND
shared: true, directly or transitively through its own enabled service
dependencies.

The new name reads as a contract ("this service is required by a real
consumer") rather than a suggestion ("something needs it"), which
matches the stricter-than-enabled semantic the flag already carries.

Touchpoints:
- plugins/lookup/service.py: _is_service_needed → _is_service_required,
  any_needed var, dict key in _compute_flags, docstring rationale.
- roles/sys-utils-service-loader/tasks/main.yml: caller switched to
  .required.
- roles/web-app-wordpress/config/main.yml + tasks/plugins/wp-discourse.yml:
  wp-discourse plugin gate uses lookup('service', 'discourse').required;
  compose.services.discourse declared as shared-service dependency so
  the service loader auto-loads web-app-discourse when inventory sets
  enabled: true. The legacy include_tasks: utils/load_app.yml in
  plugins/wp-discourse.yml is removed — the service loader now handles
  it.
- tests/unit/plugins/lookup/test_service.py: 25 assertions and 6 test
  method names renamed (test_needed_* → test_required_*,
  test_transitive_needed_resolved,
  test_direct_provider_is_not_needed_without_shared_flag).
- tests/integration/services/test_transitive_dependencies.py: one
  assertion renamed.
- docs/contributing/design/services/base.md: Lookup-Plugins section
  replaces the 'needed' bullet with an expanded 'required' entry that
  documents when consumers should gate on it and why plain enabled is
  not enough.

Also moves wordpress-app-specific keys (title, max_upload_size,
plugins) under compose.services.wordpress to match the convention
the other role keys already follow, and updates vars/main.yml lookup
paths accordingly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 00:06:47 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2dbb2aaa1a feat(email): introduce lookup('email') plugin, migrate all consumers
Moves the SYSTEM_EMAIL_* resolution logic out of group_vars/all/03_email.yml
into a dedicated Ansible lookup plugin, so every role template reads outgoing
mail settings via lookup('email') (global) or lookup('email', application_id)
(per-app). Per-app compose.services.email is merged on top of the resolved
defaults; SYSTEM_EMAIL_* variables remain usable as an inventory escape hatch.

Also splits the email docs: administrator-facing configuration stays under
docs/administration/configuration/, plugin contract and extension rules move
to docs/contributing/design/services/email.md. services.md is relocated to
services/base.md so the services/ subtree can host per-service design pages.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 22:47:40 +02:00
kevinveenbirkenbachandClaude Opus 4.7 bc9d10f085 refactor(tests/integration): reorganize tests into topical cluster subdirs
Move flat tests/integration/test_*.py files into thematic subfolders
(action_plugins, application_id, categories, compose, domains, filters,
handlers, jinja, linting, oauth2_oidc, passwords, ports, roles, run_once,
services, workflows), strip redundant prefixes from filenames, and
adjust path-depth references by +1 level. Add __init__.py and a scope-
focused README.md per cluster so the directory purpose is discoverable
without duplicating the integration.md SPOT. Update design/services.md
to point at the new test locations.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 11:04:35 +02:00
kevinveenbirkenbachandClaude Opus 4.7 424b60853d docs: update cross-file links to current paths
- iam/{common,ldap,oidc}.md: point at 11_oidc.yml and 12_ldap.yml
  (renamed from 13_/14_ in group_vars numbering).
- requirements/003-*.md: repoint iteration.md links to
  docs/agents/action/iteration/{role,workflow}.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 23:27:00 +02:00
kevinveenbirkenbach 44c31aac18 Merge branch 'main' into fix/users-applications.yml 2026-04-19 19:46:04 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0b25c6646f docs(iam): add OIDC/LDAP design guide, move services.md to design/
Introduce docs/contributing/design/iam/ with a docs README, common.md
for cross-cutting principles, and per-provider pages oidc.md and ldap.md.
The OIDC page documents the OIDC.* variable tree, claim attribute SPOT
(preferred_username), and the rule that app-specific Keycloak protocol
mappers belong in per-client scopes, not the shared default.json.j2.
The LDAP page documents the LDAP.* variable tree, attribute SPOT, and
the paired-identifier rule with OIDC.

Move services.md from docs/contributing/code/ to docs/contributing/design/
so the design/ folder is the SPOT for cross-cutting platform conventions.
Update the two architecture.md cross-links and the requirement doc.

Cross-link the new guides from 13_oidc.yml, 14_ldap.yml, and the
web-app-keycloak and svc-db-openldap role READMEs. Fix nearby README
compliance issues per readme_md.md (svc-db-openldap H1, missing
Credits section in web-app-jira, Credits formatting drift).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 17:18:41 +02:00
kevinveenbirkenbach 59173f7274 Merge branch 'main' into fix/users-applications.yml 2026-04-17 00:02:48 +02:00
kevinveenbirkenbach cdcf431b40 Merge pull request #146 from infinito-nexus/dependabot/npm_and_yarn/roles/test-e2e-playwright/files/playwright/test-1.59.1
build(deps-dev): bump @playwright/test from 1.58.2 to 1.59.1 in /roles/test-e2e-playwright/files
2026-04-16 16:52:09 +02:00
kevinveenbirkenbach 34ddc4d2d2 Merge branch 'main' into fix/users-applications.yml
# Conflicts:
#	group_vars/all/00_general.yml
#	roles/web-app-dashboard/templates/playwright.env.j2
#	roles/web-app-nextcloud/config/main.yml
#	roles/web-app-nextcloud/templates/playwright.env.j2
2026-04-16 11:54:20 +02:00
kevinveenbirkenbach 262e1b43fc Merge branch 'main' into fix/users-applications.yml
# Conflicts:
#	docs/contributing/design/services.md
#	plugins/lookup/applications_current_play.py
#	plugins/lookup/service.py
#	roles/sys-stk-backend/lookup_plugins/service_should_load.py
#	roles/sys-stk-backend/tasks/_load_service.yml
#	roles/sys-stk-backend/vars/main.yml
#	roles/sys-svc-compose/templates/base.yml.j2
#	roles/sys-svc-mail/tasks/01_core.yml
#	roles/sys-utils-service-loader/tasks/load_frontend_service.yml
#	tests/unit/plugins/lookup/test_applications_current_play.py
#	tests/unit/plugins/lookup/test_service.py
#	tests/unit/roles/sys-stk-backend/lookup_plugins/test_service_should_load.py
2026-04-15 23:59:54 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbach e9f95318e7 Merge branch 'infinito-nexus:main' into fix/nextcloud-talk 2026-04-15 01:46:38 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 1bcb7cfbb0 Merge branch 'main' into feature/service-refactoring
Resolve conflicts:
- docs/agents/action/iteration.md: keep both new bullets (SERVICES_DISABLED
  proposal + make test requirement)
- docs/contributing/design/services.md: deleted (renamed to
  docs/contributing/code/services.md in this branch)
- group_vars/all/20_services.yml: keep deletion from this branch (req-001)
- tests/lint/ansible/roles/test_service_core_first_task_run_once.py:
  use build_service_registry_from_roles_dir (req-001) with repo_root()
  from main for robust path resolution

Fix broken relative links caused by rename and deletion of files:
- docs/contributing/design/architecture.md: services.md → ../code/services.md
- docs/contributing/design/exceptions.md: load_frontend_service.yml →
  load_service.yml
- docs/requirements/001-service-registry-refactoring.md: fix paths for
  unit.md, integration.md, principles.md, documentation.md, AGENTS.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:58:51 +02:00
Environment TestandClaude Sonnet 4.6 8946ea0461 docs(principles): add YAGNI to code design principles
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 12:52:11 +02:00
Environment TestandClaude Sonnet 4.6 5c59fe23dd docs(principles): add YAGNI to code design principles
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 12:51:23 +02:00
Environment TestandClaude Sonnet 4.6 6d1f7781c1 docs(contributing): add organisation dir and split principles
Move Agile Manifesto principles out of design/principles.md into a
dedicated organisation/principles.md so code and process concerns live
in separate files. Move communication.md into the same directory for
consistency. Add SRP to design/principles.md. Add cross-references
between the two principles files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 11:32:44 +02:00
Environment TestandClaude Sonnet 4.6 7fa51003a9 docs(contributing): add organisation dir and split principles
Move Agile Manifesto principles out of design/principles.md into a
dedicated organisation/principles.md so code and process concerns live
in separate files. Move communication.md into the same directory for
consistency. Add SRP to design/principles.md. Add cross-references
between the two principles files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 11:31:42 +02:00
Environment TestandClaude Sonnet 4.6 83262816a3 docs(contributing): apply documentation.md rules across all staged files
- Move emojis after headline text (CONTRIBUTING.md, actions/, design/, tools/, tests/)
- Replace em dashes with colons, semicolons, or complete sentences throughout
- Add missing emojis to all headlines in exceptions.md, principles.md, services.md, shell.md, setup.md, pull-request.md, commit.md, act.md, docker.md, pr/README.md
- Update H1 titles to reflect renamed directories (actions/, design/)
- Fix broken relative links in workflow.md after directory moves
- Fix full-path link texts in services.md (style.css.md, javascript.js.md)
- Fix broken principles.md link to lint.md after testing/ → actions/testing/ move

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 21:06:17 +02:00