mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 10:59:59 +00:00
fix/dev-python/python-upper-bound
16
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 kevinveenbirkenbach
|
f45df04bf1 |
test(docker): forbid ARG defaults in every Dockerfile
A defaulted build argument answers a question the caller failed to answer. The build then succeeds against a base image, version or path nobody chose, and the mistake surfaces as wrong behaviour at runtime instead of a failed build. web-app-bridgy-fed shows the cost: its metadata declared python 3.14 while the Dockerfile default silently built against 3.12. Both look plausible, so nothing pointed at the drift. It and web-app-postmarks now take the base image from the same services declaration the rest of the role reads. The lint walks every tracked Dockerfile, template variants and paths outside roles/ included, which the existing ARG-scope check does not. A second assertion fails when the scan finds no Dockerfile at all, so the rule cannot pass vacuously if the walk ever breaks. Touching the two Dockerfiles brought their pre-existing narration under the comment lint; what carried substance moved into the file header. |
||
 kevinveenbirkenbachandClaude Opus 5
|
acc9e25c0f |
refactor(volumes): one canonical mount surface, context-wired lookups
The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.
As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).
The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.
Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).
Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.
Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 5
|
e3e81e816f |
refactor(healthcheck): declare probes in services.yml, render them from one lookup
Seven include templates under sys-svc-container decided how a container was
probed, and they read their inputs from template-local `{% set %}` scope.
That scope leaks: a value set for one service silently reached every service
below it, which is how taiga's front and gateway ended up probing a port
declared for a different service, and how bluesky's frontend probed a PDS
endpoint.
Probes now live in utils/docker/healthcheck.py, one class per flavor behind a
PROBES registry, and each service declares its own:
healthcheck:
flavor: curl
path: health/ready
start_period: 20m
`port_key` picks another entry under ports.internal for the services that
expose their probe elsewhere (keycloak on management, nextcloud talk on
signaling). Without a flavor the entry must carry an explicit test argv, so a
missing declaration fails loudly instead of rendering a probe against nothing.
Rendering HEAD's templates against the new output confirms 49 of 56 call
sites are byte-identical; the remaining seven are covered by an Ansible run.
Three probes change on purpose: erpnext's backend gains a check it never had,
bluesky's frontend drops the leaked PDS path, and msmtp_curl moves from a
hardcoded localhost:80 to the service's own port on 127.0.0.1 like every
other flavor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
d4ee9a85ba |
refactor(compose): align templates to the service_name skeleton
Every compose.yml.j2 now declares services as '{{ service_name }}:' with
its '{% set %}' knobs on two-space lines above the key, includes at four
spaces and if/for tags aligned to the content they wrap. The layout
renders correctly because stack_host_template enables lstrip_blocks for
compose.yml.j2; indent(4, True) in the container base and '{%+' guards
in whitespace-sensitive scalars keep the shared fragments identical
under both render modes. A new lint test enforces the skeleton.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
594109018a |
refactor(compose): publish every container port through the container_ports lookup
Take the host IP as an optional third element of each term ([service, protocol, host_ip]) instead of an ip= kwarg, and build the whole ports: block from declared ports.local/public (host) + ports.internal (container). Convert every compose template across 84 roles, folding the remaining raw var/numeric/specific-ip mappings (including mailu's per-interface mail ports bound to MAILU_IP4_PUBLIC). Multi-term calls render one term per line. Add the compose-ports-must-use-container-ports lint: it forbids any raw "<host>:<published>:<container>" publish item under a ports: block and requires every term to declare ports.internal. Remove tests/lint/roles/test_swarm_internal_ports.py, now subsumed. Fix a container_depends_on rendering regression. The lookup self-indented (default indent=4), so the added 4-space placement + | indent(4) triple-indented the block. Flip the lookup default to column 0 (consistent with container_networks), make every call use the 4-space + | indent(4) form, and add a lint enforcing it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
25f286d2bf |
feat(swarm): per-service internal ports and entity-named services
Give each reverse-proxied app a swarm-resolvable upstream: add services.<entity>.ports.internal.http to the role metadata and rename the main compose service from the generic 'application' to the entity name, so its short DNS alias on the shared overlay is unique. Thread the service name through compose, vars (*_SERVICE_NAME), volume mounts and the oauth2 origin host. Consolidate the split image/ports blocks in bridgy-fed and postmarks into the entity block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
cb27ff3756 |
refactor(networks): replace Jinja includes with compose_networks/container_networks lookup plugins
Convert the two registry-driven networks templates into pure-Python
rendering functions plus thin Ansible lookup plugins:
roles/sys-svc-compose/templates/networks.yml.j2 -> {{ lookup('compose_networks') }}
roles/sys-svc-container/templates/networks.yml.j2 -> {{ lookup('container_networks') | indent(4) }}
The rendering logic lives in utils/networks/render.py as pure-Python
functions that take a registry + closure-style config/database lookups,
making the logic directly unit-testable without a Jinja stub harness.
The lookup plugins are thin wrappers that resolve application_id,
DEPLOYMENT_MODE, and swarm.network.encryption from the templar, build
the service_registry, and forward to the render functions.
Migrate 233 include lines across 85 production templates. Convert
web-app-matrix's hand-coded MDAD networks block (default + conditional
postgres + conditional openldap) to the generic lookup; this adds an
openresty attachment when services.sso.enabled (harmless extra
interface, pinned in test_matrix_mdad_consumer_attaches_postgres_ldap_sso).
Mark local-DB / redis sidecars (sys-svc-rdbms/templates/services/,
svc-db-redis/templates/service.yml.j2) with `# nocheck: networks-literal`
because they intentionally attach to default-only.
Extend the lint to scan ALL roles/*/templates/**/*.yml.j2 (not only
compose.yml.j2) and to forbid the legacy include form; the marker
above is the documented escape hatch.
Coverage:
- tests/unit/utils/networks/test_render.py (30 cases incl. encrypted=false literal pin)
- tests/unit/plugins/lookup/test_compose_networks.py (5 cases incl. lookup_loader arg-schema)
- tests/unit/plugins/lookup/test_container_networks.py (5 cases incl. lookup_loader arg-schema)
- tests/integration/infrastructure/compose/test_networks_render.py (10 snapshot scenarios)
- tests/integration/infrastructure/compose/test_network_includes.py (3 lint rules)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
0ffbb73d72 |
refactor(images): central container_image lookup + migrate all roles
Add plugins/lookup/container_image.py: returns the full `image: "<ref>"`
line so templates drop the redundant image: prefix and surrounding
quotes. Supports `custom=True` to auto-construct `<entity_name>_custom`
for locally-built images, plus explicit `image=`/`version=` overrides
for exotic cases (taiga back-auth/front-oidc variants, decidim infinito
suffix, etc.).
* 25 unit tests for the lookup cover compose vs swarm prefix, custom
flag, overrides, tag_only=True, missing version warnings.
* New strict lint (tests/lint/.../test_no_raw_image_in_compose_templates)
fails on any raw `image:` line in roles/*/templates/*.j2 unless
nocheck-suppressed. Legacy `image: "{{ lookup(...) }}"` form no
longer accepted.
* Migrate 159 image refs across ~80 roles to the standalone
`{{ lookup('container_image', application_id, '<service_key>') }}`
form. 26 roles use `custom=True` (mariadb, postgres, openldap,
jenkins, jira, gitlab, nextcloud, mediawiki, shopware, wordpress,
odoo, joomla, openproject, bookwyrm, espocrm, hugo, littlejs,
fediwall, roulette-wheel, openresty, confluence and more).
* Fix svc-db-redis application_id leak: service.yml.j2 is included
into consuming roles where application_id evaluates to the outer
role, so the image lookup needs the hardcoded 'svc-db-redis'.
* Integration tests (test_templates_require_image,
test_build_requires_image) recognise the standalone lookup as an
implicit `image:` at the same indent.
* Mediawiki swarm tasks: add explicit `:{{ MEDIAWIKI_VERSION }}` to
the registry-prefixed image string because the custom var is now
bare.
Only 3 legitimate exotic nocheck-suppressed image lines remain:
matrix-mdad runner (nested services.matrix.runner.image path),
matrix mautrix bridges (per-iteration dynamic) and svc-registry-docker
(chicken-and-egg self-reference).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
1d322f4e10 |
refactor(compose): replace lookup('applications') | compose_volumes(...) pipe with lookup('compose_volumes', ...)
Single SPOT lookup `plugins/lookup/compose_volumes.py` absorbs the applications-lookup pipe step and auto-wires `DEPLOYMENT_MODE` plus the `storage` group var from the templating context. Callers only pass what genuinely varies per service (`application_id` and an optional `extra_volumes` dict). Behavior change (bug fix): the filter was always invoked with the default `deployment_mode='compose'` because no caller passed it explicitly, which silently skipped the swarm-NFS driver-opts injection path. The lookup reads `DEPLOYMENT_MODE` from globals, so swarm-NFS volumes now render the `driver: local` / `driver_opts: type: nfs` block correctly. Callers can still override via `deployment_mode=` / `storage=` kwargs when a template needs non-default rendering. Other coordinated changes: - 81 template call sites migrated from pipe form to lookup form (single-line and multi-line shapes). - `FilterModule` registration in plugins/filter/compose_volumes.py removed so the deprecated pipe form fails fast at render time. The underlying `compose_volumes()` function stays exported and is imported by the new lookup plus the existing 21-test filter unit test module. - 10 new lookup unit tests pin the contract (term shape, kwargs pass-through, auto-wiring, override precedence, error cases). - Integration test `test_compose_volumes_call_required.py` updated: asserts the new lookup form is present in every `compose.yml.j2` and adds a second test forbidding the deprecated pipe form anywhere under `roles/**/templates/`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
587a3a1796 |
feat(swarm): consolidate container_name + pull_policy into generic compose_only lookup
Replace per-key `compose_container_name` lookup with a generic
`compose_only(key, value)` SPOT that emits `key: "<value>"` only in
compose mode and nothing in swarm. Same gating logic now serves
container_name (157 call sites) and pull_policy (17 call sites)
without duplicate plugin code.
Why this consolidation:
- The earlier `build-key` lint rule was misdirected. Swarm tolerates
`build:` with a harmless warning when `image:` is pre-pushed (via
the existing handlers/swarm_registry_push.py flow in
roles/sys-svc-compose). The actual swarm-rejected key is
`pull_policy:` ("Additional property pull_policy is not allowed"),
and 10 templates were silently broken because they declared inline
`pull_policy: never` outside the existing `{% if DEPLOYMENT_MODE
\!= 'swarm' %}` gate in roles/sys-svc-container/templates/build.yml.j2.
- A single generic lookup is cleaner than N per-key wrappers when
the gating logic is identical (emit-key-or-nothing).
Lint changes in tests/lint/ansible/jinja/test_mode_incompatible_keys.py:
- New `pull-policy-key` rule, regex `^\s*pull_policy\s*:`, remediation
points at compose_only.
- Existing `container-name` rule keeps its raw-key regex; remediation
now also points at compose_only.
- Broken-split / broken-embed regexes rekeyed on the `compose_only`
lookup name (was `compose_container_name`).
- `build-key` rule and its 12 nocheck annotations across templates
removed.
Coordinated changes:
- 86 templates rewritten (157 container_name calls migrated;
17 inline pull_policy lines replaced).
- 1 redundant `{% if DEPLOYMENT_MODE \!= 'swarm' %}` guard dropped in
build.yml.j2 (the lookup self-gates).
- 12 unit tests pin the compose_only contract.
- Catalog (suppression.md) updated: `pull-policy-key` row added,
`container-name` row remediation updated.
- Old `plugins/lookup/compose_container_name.py` and its test module
deleted.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
a0f0e3ac03 |
feat(swarm): SPOT lookup gates container_name emission per deploy mode
container_name is illegal in swarm mode when replicas > 1 (docker rejects
with "services.deploy.replicas: can't set container_name and X as container
name must be unique"). All compose.yml.j2 templates now route through a
single lookup that suppresses the key in swarm mode and emits it in compose
mode.
SPOT: plugins/lookup/compose_container_name.py
Reads DEPLOYMENT_MODE from the templating context, returns
'container_name: "<name>"' under compose and an empty string under swarm.
Call sites read:
{{ lookup('compose_container_name', MY_CONTAINER) }}
No explicit DEPLOYMENT_MODE pass-through needed.
Sweep: 82 templates rewritten across roles/web-app-*, roles/web-svc-*,
roles/svc-*, plus 4 partial includes (sso_proxy, redis, mariadb, postgres).
For container names assembled from multiple Jinja fragments the args use
tilde concat (X ~ '-' ~ Y) so the lookup receives a single resolved string
and the call stays a single well-formed Jinja expression.
Lint: tests/lint/ansible/jinja/test_no_raw_container_name.py blocks
(a) raw 'container_name:' YAML keys in any roles/*/templates/**/*.j2,
(b) split-Jinja regression where the outer braces close mid-call:
{{ lookup('compose_container_name', A }}-{{ B) }},
(c) embedded-Jinja regression inside a string argument:
lookup(..., "{{ X }}_suffix") (removed in ansible-core 2.23).
The (b) and (c) regexes accept either single- or double-quoted lookup
name and scan the full file so multi-line shapes also surface. Suppress
per-site with '# nocheck: container-name' on the same or preceding line.
Tests: tests/unit/plugins/lookup/test_compose_container_name.py pins the
lookup contract: compose vs swarm output, whitespace tolerance, default-to
compose when DEPLOYMENT_MODE is absent, AnsibleError on missing or excess
terms, non-string name coercion.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
24d4ab8e4a |
refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
7b15c61dfc |
refactor(roles): migrate version/branch git-ref keys to ref + add integration guard
Unify the source-repo half of the entity naming convention introduced by `0cc37d0fd`. Every entity that declares `repository:` MUST now also declare `ref:` (git ref — tag, branch, or commit SHA). The overloaded `version:` (when used as git ref) and the alternative `branch:` key are gone. Roles migrated: * web-app-bigbluebutton: `version: main` → `ref: main` (consumed in tasks/01_core.yml as `docker_git_repository_version`). * web-app-discourse: `version: main` → `ref: main`; `DISCOURSE_VERSION` renamed to `DISCOURSE_REF`. * web-app-bridgy-fed: `branch: main` → `ref: main`; `BRIDGY_REPO_BRANCH` renamed to `BRIDGY_REF` across vars, the Dockerfile ARG, and the compose build-args. * web-app-postmarks: identical migration to bridgy-fed. New guard `tests/integration/roles/meta/services/test_repository_requires_ref.py` walks every `meta/services.yml` and fails when an entity declares `repository` without `ref` (or vice versa), so the next role that forks the source-build pattern cannot regress to either half- declaration without `make test` catching it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
c0ef883fc1 |
refactor(compose_volumes): auto-resolve database volume, drop manual override
Filter no longer takes a `database_volume=` parameter. The volume name is derived from the central provider's `services.<dbtype>.name` (shared) or `<entity>_database` (dedicated) — mirroring `plugins/lookup/database.py`'s `volume_prefix + host` construction. Simultaneous postgres + mariadb on the same role now raises an AnsibleFilterError with a clear message; support is deferred per the docstring TODO. The integration test (renamed from `test_shared_service_volumes.py` to `test_compose_volumes_call_required.py`) now requires every `roles/*/templates/compose.yml.j2` to call `compose_volumes`, so the top-level `volumes:` block is always rendered. Without it, flipping a shared service to dedicated mode adds a named-volume reference that Compose rejects with `service "<name>" refers to undefined volume`. Adds the call to 14 templates that were missing it (dashboard, hugo, sphinx, bridgy-fed, fediwall, littlejs, mig, mini-qr, navigator, roulette-wheel, simpleicons, collabora, logout, openresty) and strips the `database_volume=...` argument from every other role plus Taiga's now-unused `TAIGA_DATABASE_VOLUME` var. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
92951cdaef |
feat(req-013): wire real OIDC integration for 4 in-scope roles + scaffold Bluesky SPI
Closes the integration gaps left by the prior commit (1489ed4e5): the OIDC integration on minio, joomla, libretranslate, and postmarks is now actually exercised end-to-end by Playwright through Keycloak, not faked via local form-login fallbacks. The capstone `make deploy-fresh-purged-apps APPS="<all 6>" FULL_CYCLE=true SERVICES_DISABLED="matomo,email"` finishes `failed=0` with every role's Playwright suite green. Per-role: * MinIO (✅/✅/✅). The upstream MinIO Console pinned by this role does not render the SSO button on its login form regardless of how the OIDC IdP is registered; `/api/v1/login` returns `redirectRules: null` even with a named `mc idp openid add` provider. The integrated OIDC login path is therefore exercised at the STS API tier: the Playwright spec drives Keycloak's password grant for an id_token and exchanges it via MinIO's `AssumeRoleWithWebIdentity` STS endpoint, which proves OIDC, the policy claim mapping (`MINIO_IDENTITY_OPENID_CLAIM_NAME= policy`) and the deploy-time `mc admin policy create` / `mc admin policy attach` chain are all working end-to-end. * Joomla (✅/✅/🛠️). Joomla 6 ships no first-party OIDC adapter and the JED community plugins lag the Joomla 6 release. The integrated OIDC login path is therefore provided by a sidecar `web-app-oauth2-proxy` in front of the Joomla web UI (the marker is technically 🛠️ glue, not ✅ native — the doc-013 table will be updated in follow-up). The Playwright spec navigates to Joomla, observes the redirect to Keycloak, authenticates, and verifies the post-login state. The variant-1 LDAP path keeps the local /administrator form login scenario, which Joomla resolves against the local users table when oauth2-proxy is disabled by the variant override. * LibreTranslate (🛠️/❌/❌). Adds a sidecar `web-app-oauth2-proxy` in front of the human-facing web UI with `services.oauth2.acl.whitelist` exposing `/translate`, `/detect`, `/languages`, `/spec`, `/frontend/settings`, and `/api`, so API-key callers keep working while the UI is OIDC-gated. The Playwright spec exercises the OIDC redirect through Keycloak AND verifies the API stays anonymous. * Postmarks (🛠️/🛠️/❌). Adds a sidecar `web-app-oauth2-proxy` in front of the web UI. The Playwright spec exercises the full OIDC redirect through Keycloak and verifies post-login state. * Bluesky (🛠️/🛠️/❌). The role now ships the source for the Keycloak event-listener SPI bridge that auto-provisions Bluesky PDS accounts on REGISTER / LOGIN events and stores the synthesised app-password back as a Keycloak user attribute (per the requirement-013 per-role notes): roles/web-app-bluesky/files/keycloak-bluesky-bridge/ pom.xml README.md src/main/java/com/infinito/bluesky/ BlueskyBridgeEventListener.java BlueskyBridgeProviderFactory.java src/main/resources/META-INF/services/ org.keycloak.events.EventListenerProviderFactory The SPI implements handle sanitisation to the AT Protocol's allowed `[a-z0-9-]` charset, a 3-attempt retry budget for the PLC-bound provisioning step, and group-membership gating against `/roles/web-app-bluesky`. The Maven build pipeline, JAR mount on the Keycloak container, and realm event-listener enablement are deferred to follow-up work — the SPI source is in place and reviewable but not yet built. * Bridgy-fed (❌/❌/❌). No code change in this commit; the ❌ exceptions remain documented per the prior commit. Shared infrastructure / tooling: * `.github/dependabot.yml`: enable Maven coverage for the new Bluesky SPI pom.xml. * `.github/workflows/security-codeql.yml`: enable `java-kotlin` in the CodeQL matrix to scan the new Java source. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
07ada34155 |
feat(req-013): promote 6 web-* roles to beta with full per-role baselines
Implements requirement docs/requirements/013-alpha-to-beta-promotion.md for the in-scope subset: - web-app-minio: ✅/✅/✅ V0 OIDC + V1 LDAP variants green - web-app-joomla: ✅/✅/🛠️ V0 OIDC + V1 LDAP variants green - web-app-bridgy-fed: ❌/❌/❌ baseline (all 3 documented exceptions) - web-svc-libretranslate: 🛠️/❌/❌ baseline (LDAP+RBAC documented exceptions) - web-app-postmarks: 🛠️/🛠️/❌ baseline (RBAC documented exception) - web-app-bluesky: 🛠️/🛠️/❌ baseline (RBAC documented exception) Each role's `meta/services.yml.<entity>.lifecycle` is now `beta`. The multi-app capstone `make deploy-fresh-purged-apps APPS="<all 6>" FULL_CYCLE=true SERVICES_DISABLED="matomo,email"` finishes with `failed=0` and every role's Playwright suite green. Per-role highlights: * MinIO. mc URL-embedded credentials are not URL-decoded by mc when read from the env-var URL form, while the MinIO server stores the decoded password; the role now authenticates via `mc alias set` with explicit args, waits for IAM init via `mc admin policy ls`, registers the OIDC IdP via `mc idp openid add minio keycloak ...` (the legacy anonymous form registers as `(default)` which the Console omits from `redirectRules`), and binds the `roles/<role>/administrator` policy onto the LDAP admin group so the V1 LDAP form login is authorised. * Bridgy-fed. Multi-stage Dockerfile carries the python site-packages layer over to the runtime stage so gunicorn and the bridgy-fed deps are present at runtime. Removes the colon from `meta/main.yml.galaxy_info.description` so the dashboard config rendering does not produce invalid YAML. * Joomla. Adds `oidc:` enable + `ldap:` block in meta/services.yml, matrix-deploy variants for the two auth flows, and an extended Playwright env exposing OIDC issuer, biber persona, canonical domain, and the `<service>_SERVICE_ENABLED` gate flags. * LibreTranslate. Adds `container_port=5000`, drops the `dmbs_excl.yml.j2` depends_on partial that produced an empty `'': condition: service_healthy` map, and pins `LT_LOAD_ONLY=en,de` so the deploy gate's readiness probe does not race the upstream argos-translate model mirror. * Postmarks. Built from scratch as a full role (was a stub): tasks, Node-based Dockerfile that clones ckolderup/postmarks, compose template, env with SESSION_SECRET / ADMIN_KEY pulled from meta/schema.yml credentials, server.yml with subnet, Playwright env + spec. * Bluesky. Adds `files/playwright.spec.js` and `templates/playwright.env.j2` for the deploy gate. The Keycloak event-listener bridge to PDS `com.atproto.server.createAccount` remains to be built in follow-up work; until then the spec exercises baseline reachability and the OIDC/LDAP/RBAC gaps stay documented in `roles/web-app-bluesky/README.md` per lifecycle.md. Shared infrastructure: * `roles/test-e2e-playwright/tasks/run_one.yml`. The wait-for-app probe now accepts 4xx as proof of life; the prior `<400` boundary rejected legitimate 404 landing surfaces (bridgy-fed serves federation endpoints under specific paths, not `/`). 5xx remains the only signal of an unhealthy app. README.md SSO/RBAC exception sections per lifecycle.md (`docs/contributing/design/services/lifecycle.md`): * web-app-bluesky — RBAC exception * web-app-bridgy-fed — full SSO/RBAC exception * web-app-postmarks — RBAC exception * web-svc-libretranslate — LDAP + RBAC exception Out of scope for this commit (deferred to follow-up requirements): * Bluesky Keycloak event-listener SPI for PDS account auto- provisioning (needs Java SPI build pipeline + JAR mount + realm wiring). * LibreTranslate sidecar oauth2-proxy on the UI subpath only (must not gate the API endpoints). * Postmarks sidecar oauth2-proxy in front of the web UI. * Joomla community OIDC plugin install + group-mapping config. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |