mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-07 11:22:31 +00:00
fix/dev-python/python-upper-bound
25
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  kevinveenbirkenbachandClaude Opus 5
|
7162b7bf4c |
Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge base |
||
|
kevinveenbirkenbachandClaude Opus 5
|
f713ba8350 |
refactor(credentials): address them at secrets.credentials, like every other meta file
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.
The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.
--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.
Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.
tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.
Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 5
|
e96c752549 |
Merge fix/playwright/env-template-gate into feature/svc-net-tor
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.
Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content —
|
||
|
kevinveenbirkenbachandClaude Opus 5
|
5aca44ee8c |
fix(wordpress): pin the email service across the variants and register its flag
|
||
|
kevinveenbirkenbach
|
c19dd690f3 |
feat(web-app-wordpress): expose a reviewed read-only MCP surface via addons
WordPress serves MCP through two addons rather than a sidecar, because the application already carries the extension point. The mu-plugin is the entire tool surface. The MCP Adapter publishes an ability only when it carries meta.mcp.public, so whatever is not registered in that file cannot be reached over MCP no matter what else the installation has. Every ability reads already-published content and carries its own permission callback, which is not decoration: the adapter authorizes the transport, the callback authorizes the operation, and dropping it would let any authenticated caller reach the ability. The second addon fetches the release asset rather than the tag's source archive. The source archive unpacks as mcp-adapter-<version>, which no longer answers to the slug the generic activation step uses, and it ships no vendor directory. Three Playwright specs cover what needs no credential: both addons active, the abilities registered, and a guest served nothing. The mcp role joins the role catalogue instead of replacing it. |
||
|
kevinveenbirkenbachandClaude Opus 5
|
ed0f5af9db |
feat(playwright): make a blocked persona state why
Blocking a persona removes the only end-to-end proof that a role's auth chain works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has always required each flag to carry a one-line nocheck stating the role contract that justifies it, plus a matching paragraph in the role's README or TODO. Nothing enforced it, so 89 flags across 46 roles carried neither. The lint mirrors test_has_spec's shape and uses is_suppressed_at with mode="line-above", so the marker must sit directly above its flag -- a blank line between them breaks the association, which is what the persona helper's reader will look at. A literal =false blocks nothing and is exempt; the sso-conditional ternary is not, because it blocks in every variant that pins services.sso.enabled false. Writing 89 reasons turned out to be the point. Five roles audited in one pass blocked for five entirely different mechanisms, all of which read as the same shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than the Keycloak username; phpMyAdmin authenticates as the MariaDB root account; GitLab's seeded password belongs to the built-in root, not the administrator; Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds with an LDAP DN and a different secret. Further out: MediaWiki removes the form that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false) and bounces anonymous requests back into the IdP, so the unauthenticated landing the contract asserts can never be observed; Mattermost Team Edition has no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled link; MinIO's console returns redirectRules: null and renders no SSO button at all; XWiki's administrator is an XWikiUsers object with no password property. Two flags turned out to be unjustifiable and are gone rather than annotated. web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but runGuestFlow does one anonymous goto and asserts no signed-in marker -- it never touches the dashboard. A leftover from a helper version that entered through the dashboard account menu. And web-app-mig's rationale claimed the role is an in-app module of web-app-dashboard; it has its own compose stack, healthcheck and canonical domain, and dashboard is merely a tile consumer. Nine of the reasons say plainly that the block is not a role property but a gap in the shared persona helpers -- decidim's sign-out is a data-method link the generic click cannot activate, fider's SSO entry is a two-step modal, taiga, wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit, jenkins blocks unconditionally although its OIDC variant looks runnable, and opentalk's own spec proves both personas work. That list did not exist before; it is the actual yield here, not the 89 comment lines. 45 pre-existing comments came into the comment lint's scope because this sweep touched their files -- it only inspects files that differ from HEAD. 28 of them were the banner "Service gate flags rendered for closure.", every one sitting above a line that is not a service gate flag at all, orphaned by the alphabetical re-sort these templates once got. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Fable 5
|
f5911e1ec4 |
fix(web-app-wordpress): repair discourse addon exec and gate keycloak env on sso
Three defects broke the wordpress jobs in both modes:
The wp-discourse shell tasks used folded scalars whose continuation
lines were indented deeper than the first, so YAML preserved the
newlines and the shell ran three separate commands instead of one
(--path=... reached the host shell as its own command). Align the
continuations so the fold collapses to a single line.
The discourse exec address came from container_address, which expands
to a resolve subshell under swarm; the argv-form command task passed it
verbatim to docker as a container name. Discourse is launcher-based and
its container is literally named 'discourse' in both modes, so use the
services.discourse.name config lookup instead.
The playwright env rendered keycloak credential lookups uncondition-
ally, crashing the sso-disabled variant; gate them on
lookup('service','sso').required like the sibling flags, and drop
narration comments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
70786aa126 |
refactor(addons): emit playwright addon gate flags via addon_env_flags lookup
Replace the per-template Jinja loop (which mis-unpacked dict2items and rendered every flag as KEY_ADDON_ENABLED=false, silently skipping every addon spec) with a single lookup('addon_env_flags', application_id). The plugin is the single source of truth for the <ADDON_ID>_ADDON_ENABLED flags consumed by tests/addon-gating.js: env-key derivation matches addon-gating.js exactly, and a flag is true only when the addon is both enabled AND required (required:false addons skip). Switch all six role templates to the lookup, extend the env-file schema lint to recognise whole-line {{ ... }} expressions, and add the plugin unit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
8e92f15f6a |
test(addons): require a Playwright spec for every addon; add 87 per-addon specs
Add tests/lint/ansible/services/test_addons_playwright_spec.py requiring roles/<role>/files/playwright/addons/<id>.spec.js for EVERY addon of EVERY role — no mechanism auto-exempt and no # nocheck escape. Ship the 87 per-addon specs (gated via skipUnlessAddonEnabled so they skip cleanly when the addon is disabled in the current variant), the test-e2e-playwright addon-gating harness (addon-gating.js, playwright.config.js, recursive 02_run_one.yml), the shared odoo helper, and the ADDON_<ID>_ENABLED flags in the six playwright.env.j2 templates. Relax test_env_file_schema to skip Jinja-templated keys in the ADDON loop. Doc updates (suppression.md, requirements/026) follow in a separate commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbach
|
adf1d9cd75 |
fix(web-app-wordpress/playwright): resolve SSO_SERVICE_ENABLED via sso (req 21)
`lookup('service', 'oidc')` blew up during render after the oauth2+oidc →
sso collapse — the `oidc` key no longer exists in the service registry.
Rename the lookup to `sso` so the WordPress Playwright env renders again.
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
24d4ab8e4a |
refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
197a9f76e0 |
fix(install/venv) + lint(playwright): role-self APP_BASE_URL + dirname-aware mkdir
scripts/install/venv.sh
-----------------------
INFINITO_VENV_BASE is a static-default convention; the active VENV
may end up outside that base (e.g. \$HOME/.venvs/<name> when
/opt/venvs is not writable, or a wholly custom override). Probing
writability on the legacy base triggered a spurious sudo branch
whenever VENV lived elsewhere -- fatal in no_new_privs sandboxes
where sudo cannot escalate. Switch the writability + mkdir + chown
chain to \$(dirname "\${VENV}") so the script prepares the directory
it is actually about to create the venv in.
tests/lint/ansible/roles/web-app/playwright/test_app_base_url_uses_role.py
-------------------------------------------------------------------------
New lint: every role's templates/playwright.env.j2 must declare
APP_BASE_URL as a reference to the role's own surface. APP_BASE_URL
feeds the shared guest-persona helper and Playwright's global
baseURL; a role that hardcodes a sibling role name (most commonly
'web-app-dashboard') sends the guest test to the wrong site and
breaks any matrix variant where the sibling is not deployed
(ERR_SSL_UNRECOGNIZED_NAME_ALERT).
The 16 web-app role templates that violated the contract switch
from sibling lookups (`lookup('tls', 'web-app-dashboard', ...)`)
to the canonical `lookup('tls', application_id, ...)` form.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
22cf6f38f1 |
refactor(env+make): rename static.env→default.env + sort + lint gates
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
70042a99f1 |
style(roles/playwright.env.j2): condense comments to single-line WHY-only
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
c706344b23 |
fix(playwright/env): PERSONA_*_BLOCKED for 14 roles + close in req-019
CI run 25680106742 confirmed each role's bespoke tests pass; only the
shared `biber: dashboard → app → universal logout` and `administrator:
dashboard → prometheus → app → universal logout` scenarios fail because
the generic persona helper's auth chain (`page.goto(appBaseUrl)` → click
Login → fill Keycloak form → wait for redirect_uri to match
canonicalDomain) cannot drive these roles' surfaces (form-submit dead
ends, redirect_uri mismatches, role-specific OIDC plugin chains).
Pattern matches the already-validated `PERSONA_*_BLOCKED` approach used
for mailu, matomo, prometheus, keycloak, dashboard, simpleicons,
nextcloud, opencloud, and minio (commits 35b79f978, 9af354c61, a569645ee,
d4c9e13a9, b639d14b4,
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
63424efc74 |
refactor(playwright/env): resolve CANONICAL_DOMAIN via lookup('domain')
Replace the verbose
`(lookup('applications', application_id).server.domains.canonical | first)`
chain with the dedicated `lookup('domain', application_id)` plugin
across every playwright.env.j2 that renders `CANONICAL_DOMAIN`. The
helper already exists at plugins/lookup/domain.py and resolves the
canonical primary domain via the cached domain-merge pipeline, so the
inline canonical-list traversal is unnecessary noise that obscures
the actual contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
c55ad69c41 |
refactor: drop requirement-file references from code, add doc rule + lint
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`, `requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from non-Markdown source files. Comments anchored to requirement numbers rot when requirements are renumbered or rewritten and conflate present-state behaviour (which the comment should describe) with the audit trail of how the code got there (which belongs in commit messages, PR descriptions, and the requirement file itself). * Document the rule in `docs/contributing/documentation.md` under "Comments". Code MUST describe the *why* in present-state language; contract enforcement lives in tests, not folklore. * Clean 254 source files across `roles/`, `tests/`, `plugins/`, `cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root `eslint.config.js`. Pure comment / docstring / log-message edits; no behavioural change. * Add `tests/lint/repository/test_no_req_references_in_code.py` to enforce the rule going forward across `.js`, `.py`, `.yml`, `.yaml`, `.j2`, and `.sh` files. The legacy `test_no_stale_requirement_refs.py` is exempt as the only file whose purpose is to detect these very tokens. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
db2a2d9638 |
refactor(playwright): move cross-service tests to provider specs
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
cd2830507c |
test(lint): persona scenarios cannot silently SKIP on missing base URLs
Adds a lint that fails when a role's templates/playwright.env.j2 declares
DASHBOARD_SERVICE_ENABLED= but does NOT render DASHBOARD_BASE_URL,
PROMETHEUS_BASE_URL, and MATOMO_BASE_URL. Renders those three URLs
across 42 role env templates so the lint passes.
Also reverts the matomo "role-closed" strikethrough in req 019 — the
URL wiring remains in scope for the persona rollout.
Reason: the persona helpers (roles/test-e2e-playwright/files/personas/
{biber,admin,guest}.js) collapse the persona scenario when their
required URLs are empty (req 019 persona-collapse exception). Without
this lint, a role that declares dashboard: enabled but forgets the
URL silently SKIPs the persona — the deploy goes green but no real
flow ran. Per-line opt-out is `# nocheck: persona-required-envs` for
auth-less svc-* roles.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
735ae5e29d |
feat(playwright): migrate third batch of roles to shared persona library
Wires runGuestFlow / runBiberFlow / runAdminFlow from roles/test-e2e-playwright/files/personas/ into the remaining 5 roles (flowise, minio, opentalk, openwebui, wordpress). Each role gets the spec.js persona scenarios plus matching meta/services.yml gate entries and playwright.env.j2 env wiring; admin-flow callers pass a role-specific adminInteraction that drives a real management surface. Reason: continuation of |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
e3d175604c |
refactor(templates): collapse boolean ternary to string|lower filter
Touches wordpress, nextcloud, yourls, and dashboard. Drops redundant | default(false) where the value is contracted-set. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
7f0cb0484a |
fix(web-app-wordpress): gate playwright discourse env on services.discourse.enabled instead of group_names
The Multisite variant explicitly sets `services.discourse.enabled:
false` in `meta/variants.yml`, but the matrix-deploy init bakes a
single include set across rounds, so `web-app-discourse` stays in
`group_names` for both variants. The previous gate
`'web-app-discourse' in (group_names | default([]))` therefore
reported `DISCOURSE_SERVICE_ENABLED=true` on the multisite variant
too, the round-trip test ran instead of skipping, and the deploy
failed because the variant deliberately did not deploy Discourse.
Switch the gate to `lookup('config', application_id,
'services.discourse.enabled')` so the env reflects the role's
variant-resolved view, mirroring how `wp-discourse` plugin enable
is gated. Apply the same swap to `DISCOURSE_BASE_URL` so the
URL is empty on the disabled variant and any accidental access
fails fast instead of pointing at a non-existent host.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
ed04dad5d5 |
refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
d009a35116 |
feat(rbac): hierarchical /roles/<app>/<role> + service-gated Playwright + Discourse round-trip
Implements requirements 005, 006, and 007. 005: per-application Keycloak `group-ldap-mapper`s anchor each app's role groups under `/roles/<application_id>/<role_name>` (and, for tenant-aware apps, `/roles/<application_id>/<tenant>/<role_name>`). The flat-CN LDAP layout keeps each CN globally unique so Keycloak's group-tree resolver never collides; per-app mappers use the LDAP `description` as the visible Keycloak group name to keep the path free of redundant prefixes. Migration purges legacy pre-005 entries and forces a Keycloak `logout-all` so stale tokens drop pre-005 group paths. 006: per-service `<SERVICE>_SERVICE_ENABLED` env flags and the `service-gating.js` helper let role specs `skipUnlessServiceEnabled` cleanly when a shared service is missing or `SERVICES_DISABLED` is set; `MODE_CI` markers (`INFINITO_MAKE_DEPLOY`, `GITHUB_ACTIONS`, `ACT`) propagate through the deploy CLI into the test stage. 007: WordPress->Discourse round-trip Playwright scenario gated on the `web-app-discourse` deployment. WordPress Playwright runs 5/5 green on the single-site flow; the multisite + Discourse scenarios are correctly skipped when their service prerequisites are absent. The `rbac_group_path` lookup is the single sanctioned producer for OIDC group paths and is wired into every consumer (yourls, lam, minio, openwebui, pgadmin, phpldapadmin, phpmyadmin, prometheus, xwiki). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
bebae545eb |
feat(rbac): generic OIDC group RBAC for WordPress (requirement 004)
Auto-provision one OpenLDAP group per role declared under each application's `rbac.roles` in `config/main.yml`, gated on `group_names` so only apps deployed on the host contribute. Keycloak syncs the groups from LDAP, emits the `groups` claim via the shared RBAC client scope, and WordPress maps that claim to a local role in a mu-plugin (auto-loaded, cannot be deactivated from the admin UI). No direct LDAP bind from WordPress. Non-obvious design choice: the `groups` client scope MUST be a *default* client scope, not optional. When optional + `include.in.token.scope=false`, Keycloak 26 drops the group-membership mapper from userinfo even when the scope is explicitly requested, so the mu-plugin receives no `groups` and falls back to `subscriber` for every user. ID/access tokens still carry the claim, but WP consumes userinfo. Playwright verifies the three-role cycle (subscriber/editor/administrator) serially and idempotently per requirement 004: the add step runs via the Keycloak admin UI (detects "already a member" via a disabled picker checkbox and skips); teardown runs via the Keycloak Admin REST API. Only memberships the test itself added are removed, so any pre-existing membership is preserved. Each hop (super-admin, biber, WP-admin) runs in its own browser context so WP session cookies and Keycloak SSO cookies cannot leak. Resolves: docs/requirements/004-generic-rbac-ldap-auto-provisioning.md Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |