mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-09 12:17:13 +00:00
fix/dev-python/python-upper-bound
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  kevinveenbirkenbachandClaude Opus 5
|
ed0f5af9db |
feat(playwright): make a blocked persona state why
Blocking a persona removes the only end-to-end proof that a role's auth chain works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has always required each flag to carry a one-line nocheck stating the role contract that justifies it, plus a matching paragraph in the role's README or TODO. Nothing enforced it, so 89 flags across 46 roles carried neither. The lint mirrors test_has_spec's shape and uses is_suppressed_at with mode="line-above", so the marker must sit directly above its flag -- a blank line between them breaks the association, which is what the persona helper's reader will look at. A literal =false blocks nothing and is exempt; the sso-conditional ternary is not, because it blocks in every variant that pins services.sso.enabled false. Writing 89 reasons turned out to be the point. Five roles audited in one pass blocked for five entirely different mechanisms, all of which read as the same shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than the Keycloak username; phpMyAdmin authenticates as the MariaDB root account; GitLab's seeded password belongs to the built-in root, not the administrator; Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds with an LDAP DN and a different secret. Further out: MediaWiki removes the form that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false) and bounces anonymous requests back into the IdP, so the unauthenticated landing the contract asserts can never be observed; Mattermost Team Edition has no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled link; MinIO's console returns redirectRules: null and renders no SSO button at all; XWiki's administrator is an XWikiUsers object with no password property. Two flags turned out to be unjustifiable and are gone rather than annotated. web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but runGuestFlow does one anonymous goto and asserts no signed-in marker -- it never touches the dashboard. A leftover from a helper version that entered through the dashboard account menu. And web-app-mig's rationale claimed the role is an in-app module of web-app-dashboard; it has its own compose stack, healthcheck and canonical domain, and dashboard is merely a tile consumer. Nine of the reasons say plainly that the block is not a role property but a gap in the shared persona helpers -- decidim's sign-out is a data-method link the generic click cannot activate, fider's SSO entry is a two-step modal, taiga, wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit, jenkins blocks unconditionally although its OIDC variant looks runnable, and opentalk's own spec proves both personas work. That list did not exist before; it is the actual yield here, not the 89 comment lines. 45 pre-existing comments came into the comment lint's scope because this sweep touched their files -- it only inspects files that differ from HEAD. 28 of them were the banner "Service gate flags rendered for closure.", every one sitting above a line that is not a service gate flag at all, orphaned by the alphabetical re-sort these templates once got. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
24d4ab8e4a |
refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
22cf6f38f1 |
refactor(env+make): rename static.env→default.env + sort + lint gates
Rename env/static.env → env/default.env across the entire repo
(string label, doc links, Python file-path constants) — the
`parse_static_env*` function names stay as a stable API surface.
Sort all `*.env` / `*.env.j2` entries alphabetically (52 in
default.env, 4 in ci.env, 46 of 49 j2 templates; 3 templates
with Jinja2 `{% %}` control flow are skipped because conditional
blocks would tear apart under a naive sort). Sort all 83
Makefile target blocks alphabetically; .PHONY group headers
stay in place.
Collapse the two remaining multi-line Makefile recipe comments
(dotenv-force, test-signed) to single-line WHY comments.
Add four lint tests under tests/lint/filesystem/ to keep the
invariants enforced going forward:
- env/test_alphabetical_sort.py
- env/test_single_line_comments.py (# nocheck: markers exempt)
- makefile/test_alphabetical_targets.py
- makefile/test_single_line_comments.py (# nocheck: markers exempt)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
c706344b23 |
fix(playwright/env): PERSONA_*_BLOCKED for 14 roles + close in req-019
CI run 25680106742 confirmed each role's bespoke tests pass; only the
shared `biber: dashboard → app → universal logout` and `administrator:
dashboard → prometheus → app → universal logout` scenarios fail because
the generic persona helper's auth chain (`page.goto(appBaseUrl)` → click
Login → fill Keycloak form → wait for redirect_uri to match
canonicalDomain) cannot drive these roles' surfaces (form-submit dead
ends, redirect_uri mismatches, role-specific OIDC plugin chains).
Pattern matches the already-validated `PERSONA_*_BLOCKED` approach used
for mailu, matomo, prometheus, keycloak, dashboard, simpleicons,
nextcloud, opencloud, and minio (commits 35b79f978, 9af354c61, a569645ee,
d4c9e13a9, b639d14b4,
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
63424efc74 |
refactor(playwright/env): resolve CANONICAL_DOMAIN via lookup('domain')
Replace the verbose
`(lookup('applications', application_id).server.domains.canonical | first)`
chain with the dedicated `lookup('domain', application_id)` plugin
across every playwright.env.j2 that renders `CANONICAL_DOMAIN`. The
helper already exists at plugins/lookup/domain.py and resolves the
canonical primary domain via the cached domain-merge pipeline, so the
inline canonical-list traversal is unnecessary noise that obscures
the actual contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
c55ad69c41 |
refactor: drop requirement-file references from code, add doc rule + lint
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`, `requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from non-Markdown source files. Comments anchored to requirement numbers rot when requirements are renumbered or rewritten and conflate present-state behaviour (which the comment should describe) with the audit trail of how the code got there (which belongs in commit messages, PR descriptions, and the requirement file itself). * Document the rule in `docs/contributing/documentation.md` under "Comments". Code MUST describe the *why* in present-state language; contract enforcement lives in tests, not folklore. * Clean 254 source files across `roles/`, `tests/`, `plugins/`, `cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root `eslint.config.js`. Pure comment / docstring / log-message edits; no behavioural change. * Add `tests/lint/repository/test_no_req_references_in_code.py` to enforce the rule going forward across `.js`, `.py`, `.yml`, `.yaml`, `.j2`, and `.sh` files. The legacy `test_no_stale_requirement_refs.py` is exempt as the only file whose purpose is to detect these very tokens. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
db2a2d9638 |
refactor(playwright): move cross-service tests to provider specs
Cross-service surface assertions for dashboard, prometheus and matomo
no longer run in every consumer role's persona. They are owned by the
provider's own spec, parameterised once over a `<NAME>_TARGET_ROLES_JSON`
manifest rendered into the provider's `templates/playwright.env.j2` via
the new `lookup('roles_with_service', '<svc>')` Ansible lookup.
Changes
-------
- New `plugins/lookup/roles_with_service.py` enumerates roles that
declare `services.<svc>.{enabled, shared}` truthy in the merged
applications config, returning `[{id, canonical_domain}, …]`.
- `roles/web-app-{dashboard,prometheus,matomo}/files/playwright.spec.js`
parse `<NAME>_TARGET_ROLES_JSON` and parameterise the per-consumer
assertion (dashboard tile reachability, prometheus `up=1`, matomo
SitesManager registration). The matomo spec gains an explicit
biber-denial test (request body validated against admin DOM markers
per the body-marker rule); the prometheus admin-reach test drops
the dashboard-iframe path in favour of direct goto.
- Persona helpers `roles/test-e2e-playwright/files/personas/{admin,
biber,guest}.js` enter at `${APP_BASE_URL}/` directly. The
`clickRoleTileFromDashboard` step and the `assertPrometheus*` /
`assertMatomo*` cross-service probes are gone; `utils/{dashboard,
prometheus,matomo}.js` are deleted. `utils/csp.js` switches to
literal `process.env.<KEY>` reads so the env-keys-used parity guard
recognises matomo's CSP injector consumption.
- 42 consumer-role `templates/playwright.env.j2` templates drop
`DASHBOARD_BASE_URL`, `DASHBOARD_SERVICE_ENABLED`, `PROMETHEUS_BASE_URL`,
`PROMETHEUS_SERVICE_ENABLED`. `MATOMO_BASE_URL` and
`MATOMO_SERVICE_ENABLED` stay because the per-role CSP injection
helper still gates on matomo's tracker host.
- `tests/integration/roles/test_playwright_env_services_match.py`
globally exempts `dashboard` and `prometheus` from the
services.yml-to-env parity rule.
- `tests/unit/plugins/lookup/test_roles_with_service.py` adds 14
unit tests for the lookup, including an explicit-contract test
that pins the over-inclusive behaviour on raw Jinja strings (the
lookup expects a templar-rendered payload).
- Docs: `docs/requirements/019-playwright-meta-services-parity.md`
rewords Rules 1, 3, 12 and adds Rule 13 describing the SPOT shift.
`docs/contributing/artefact/files/role/playwright.specs.js.md`
reflects the new persona shape and per-service catalogue, and is
cleaned per `docs/contributing/documentation.md` (no SPOT self-
labels, no req-references, em dashes replaced with complete
sentences, multi-clause lines split onto separate lines, broken
anchor `#two-personas-fixed-flow-` corrected).
Verification
------------
- 418 passed / 1 skipped / 0 failed across `tests/lint/`,
`tests/integration/roles/`, `tests/unit/plugins/lookup/`.
- `make lint-javascript`: 0 errors.
- `markdownlint-cli2`: 0 errors on all touched docs.
- `make autoformat`: clean (FURB171 single-item-set inequality fixed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
44d51744a7 |
feat(playwright): migrate second batch of roles to shared persona library
Wires runGuestFlow / runBiberFlow / runAdminFlow from
roles/test-e2e-playwright/files/personas/ into another ~28 roles
(akaunting, baserow, bluesky, bookwyrm, bridgy-fed, decidim, discourse,
fider, friendica, fusiondirectory, gitea, hugo, jenkins, joomla, kix,
matrix, mattermost, mig, moodle, odoo, opencloud, peertube, pixelfed,
postmarks, sphinx, taiga, yourls, web-svc-libretranslate, web-svc-xmpp).
Each role gets the spec.js persona scenarios plus matching
meta/services.yml gate entries and playwright.env.j2 env wiring;
admin-flow callers pass a role-specific adminInteraction that drives
an actual management surface so the body is not a stub.
Reason: continuation of
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
7615ad4e89 |
chore(roles): prune unused playwright.env.j2 keys (lint cleanup)
Drop env declarations that were rendered but never consumed by the matching playwright.spec.js, surfaced by tests/lint/ansible/roles/test_playwright_env_keys_used.py: - web-app-friendica: OIDC_ISSUER_URL - web-app-mailu: MAILU_DOMAIN - web-app-opentalk: OPENTALK_OPENCLOUD_URL - web-svc-xmpp: OIDC_ISSUER_URL, ADMIN_USERNAME, ADMIN_PASSWORD The keys will land again only when an actual scenario consumes them. For the deeper coverage gap that some of these env files imply (xmpp in particular has no auth scenarios despite declaring admin creds), see requirements 017 (biber RBAC coverage) and 018 (LDAP authentication coverage). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
91c57e0471 |
feat(req-013): alpha-to-beta promotion for 13 in-scope roles
Promote akaunting, baserow, bluesky, bookwyrm, bridgy-fed, flowise, fusiondirectory, jenkins, joomla, minio, postmarks, libretranslate, xmpp from `lifecycle: alpha` to `lifecycle: beta` per requirement 013. socialhome stays alpha (operator-excluded). SSO/LDAP integration per role: - minio + jenkins + joomla + fusiondirectory: native first-party paths (MINIO_IDENTITY_OPENID/_LDAP, oic-auth + ldap + role-strategy JCasC, plg_system_keycloak, LDAP-as-storage). All V0+V1 verified. - akaunting, baserow, bookwyrm, flowise, postmarks, libretranslate: oauth2-proxy perimeter gate as universal glue path. Native in-app OIDC integrations deferred to follow-up issues #219 (Baserow), #220 (BookWyrm), #221 (Akaunting). - bluesky: variant A+ login-broker (oauth2-proxy → broker → social-app) with AES-256-GCM-encrypted PDS app-passwords cached in-process (Keycloak user-attribute storage caused LDAP federation conflicts that broke other realm-clients' OIDC logins). - xmpp: native ejabberd LDAP backend for both variants (mod_oauth2_client is contrib-only and deferred). - bridgy-fed: documented SSO/LDAP/RBAC exception (federated source- platform auth, no local user table). Project-wide bugfixes: - web-app-oauth2-proxy: propagate X-Forwarded-User / -Preferred-Username / -Email / -Groups via auth_request_set bridge (set_xauthrequest + pass_user_headers in cfg + nginx headers in following_directives.conf). Benefits all 8 oauth2-proxy consumers. - scripts/tests/deploy/local/deploy/fresh-kept-app.sh: bake `RUNTIME=dev` into the matrix-init host vars so the Playwright E2E gate fires (was silently skipped on kept-deploys). From-scratch role implementations: jenkins (JCasC + plugin pre-bake + JVM cacerts CA-trust entrypoint), fusiondirectory (LDAP-as-storage compose), xmpp (ejabberd LDAP backend ejabberd.yml). Verification: - Per-role baseline: each in-scope role passed standalone V0/V1 Playwright suites green. - Multi-app fresh-purged-apps over all 13 in-scope roles (FULL_CYCLE=true): Pass 1 sync + Pass 2 async both `failed=0`, every role's Playwright suite green. Doc 013 acceptance section reflects the divergences (oauth2-proxy glue path, in-memory bluesky cache, deferred contrib modules) with explicit follow-up issue references. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |