`#user-menu button` is non-strict on Nextcloud 32+: the wrapper
`#user-menu` contains both the trigger button (aria-label "Settings
menu") AND the submenu's own buttons once it has been opened. The
biber OIDC logout test in roles/web-app-nextcloud/files/playwright.spec.js
then strict-mode-violated with:
locator.click: Error: strict mode violation:
locator('#user-menu button') resolved to 2 elements:
1) <button ... aria-label="Settings menu" ...>
2) <button ... class="button-vue button-vue--size-normal
button-vue--icon-only button-vue--vue-secondary">
Pin the locator to the trigger via its aria-label, with `> button` and
`.first()` fallbacks so localized aria-labels still work.
Surfaced once the metadata-plugin failure stopped aborting the deploy
before the test phase ran.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The package-cache active probe in
scripts/tests/environment/utils/cache.sh had `releases/40/...`
hardcoded as the upstream path for the `yum-fedora` Nexus proxy.
Fedora 40 went EOL in May 2025 and dl.fedoraproject.org stopped
serving its repodata, so the probe failed every Test Environment
(fedora:latest) CI job with:
[probe] package-cache: GET /repository/yum-fedora/releases/40/Everything/x86_64/os/repodata/repomd.xml
curl: (22) The requested URL returned error: 404
[FAIL] package-cache: proxy 'yum-fedora' did not serve
'releases/40/Everything/x86_64/os/repodata/repomd.xml'
Resolve the version dynamically from the runtime image's
`/etc/os-release` instead. The new `_resolve_fedora_release` helper
runs `docker run --rm --entrypoint /bin/cat <image> /etc/os-release`
and lifts `VERSION_ID` via awk, so `fedora:latest` is tracked
automatically and future EOL transitions no longer require a code
change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related fixes for the Deploy web-app-joomla CI job:
1. Wait for the plugin XML, not just `index.php`, before installing.
The upstream image (joomla:5.x-php8.3-apache) extracts plugin
manifests asynchronously after the front controller, so the CLI
install fired before the plugin tarball had finished extracting and
crashed with:
In Installer.php line 2120:
Warning: simplexml_load_file(): I/O warning: failed to load
external entity
"/var/www/html/plugins/actionlog/joomla/joomla.xml"
At that point the install had already created the schema (Creating
and populating the database...OK) but had NOT inserted the admin
user yet — so the retry loop saw existing tables, skipped admin
creation, and every Playwright login then failed with "Username and
password do not match". Waiting for that specific manifest is the
cheapest end-of-extract marker; switching the wait to it makes the
first install attempt clean and the admin user always lands.
2. Harden the spec so credentials problems fail fast and Joomla 5.x
template tweaks no longer false-fail. The body class on Joomla 5.x
admin uses `option-com_*`, not `com_cpanel`, so add `body[class*=
'option-com_']` to the cpanel-marker selector. Bump the wait timeout
60s → 120s for CI headroom. Race the success marker against
Joomla's "Username and password do not match" alert via
`Promise.race`, so a credentials problem now fails in seconds with
a clear `Joomla rejected the admin login: <text>` instead of timing
out 1.1 min × 3 retries on the cpanel locator.
Verified locally: `dashboard to joomla: admin login, verify control
panel, logout` passes in 5.1s.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The infinito container runs as `nobody`, so any pyc artefacts it writes
under `tests/`, `cli/`, `plugins/`, `roles/*/filter_plugins/` etc. land
owned by an in-container UID the host runner cannot delete. The host
side's `git clean -fdX` then errors out with:
warning: failed to remove plugins/filter/__pycache__/: Directory
not empty
make: *** [Makefile💯 clean] Error 1
That cleanup runs from `cli/deploy/dedicated/runner.py` between PASS 1
and PASS 2 of the matrix-deploy, so even a fully successful PASS 1
(failed=0, rescued=1) aborted the Deploy web-app-discourse CI job
purely because of the post-pass GC step.
Now the target retries with `sudo -n` on failure (GitHub Actions
runners and most desktop Linux setups have passwordless sudo) and
treats a remaining sudo-less failure as a warning + `exit 0` so the
surrounding deploy/test pipeline never aborts on a pure GC step.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The upstream Nextcloud "Metadata" app is not compatible with the
currently pinned Nextcloud server version, so `occ app:enable metadata`
failed every Deploy web-app-nextcloud CI job with:
App "Metadata" cannot be installed because it is not compatible
with this version of the server.
Disable the entry in roles/web-app-nextcloud/meta/services.yml until a
compatible release ships, with a comment so a future maintainer knows
why and when to flip it back on.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-up lint fixes after the bluesky bridge SPI deletion +
WebFetch domain auto-additions during the iteration session:
- Run `bash scripts/lint/sort_claude_settings.sh` (via `make exec`
because the host sandbox denies writes to `.claude/settings.json`).
Sorts `permissions.allow` (265 entries) — Claude Code had appended
new WebFetch domain grants to the end as the iteration ran, which
tripped `tests.lint.repository.test_claude_settings_sorted`.
- Comment out `language: java-kotlin` in
`.github/workflows/security-codeql.yml`. The previous commit
removed the only Java source in the repo (the unused
`roles/web-app-bluesky/files/keycloak-bluesky-bridge/` SPI
scaffold), and `tests.integration.workflows.test_security_coverage`
fails when CodeQL has a language enabled with no matching files.
Re-enable once a real Java component lands again.
`make test` now green (1833/1833). External URL test still green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
URL test fixes (tests.external.repository.test_urls_reachable):
- Delete unused `roles/web-app-bluesky/files/keycloak-bluesky-bridge/`
Java SPI scaffold (pom.xml carried two unreachable Maven namespace
URIs `http://maven.apache.org/POM/4.0.0`). The scaffold was never
built (no Maven build pipeline) and never registered with Keycloak
as an event listener; runtime impact is zero — the login-broker
is the actual provisioning agent. The doc 013 Future hardening
note now describes a fresh design that starts from external
secrets storage instead of the SPI's old Keycloak-attribute model.
- Add `# nocheck: url` markers on two `scripts/docker/cache/package.sh`
Nexus proxy `remoteUrl` config strings (`deb.debian.org/debian-security`,
`repo.packagist.org`) — both return 404 on bare-root GET because
apt/composer clients always append paths; downstream consumers
verify reachability of the real paths.
Bluesky V1 cross-variant recovery (deploy-fresh-kept then deploy-reuse-kept):
- Broker now recovers from PDS `Handle already taken` when its
in-memory cache is empty but the PDS still holds the account from
a previous broker session. Implementation: catch the createAccount
error, `pdsResolveHandle()` to get the existing DID, then
`pdsAdminUpdatePassword()` via the PDS admin Basic-auth API to
reset the password to a fresh value the broker controls. This
fixes the V0->V1 cross-variant deploy where the persistent
`pds_data` volume carries the V0 account into V1.
- New `PDS_ADMIN_PASSWORD` env on the bluesky-broker container,
sourced from the same `BLUESKY_ADMIN_PASSWORD` the PDS itself
uses. The value passes through Jinja `replace('$', '$$')` because
docker-compose interpolates the `environment:` map at parse time
and would otherwise eat any `$word` substring (silently truncating
the password to whatever survives interpolation; PDS rejects the
resulting Basic-auth header with HTTP 401).
Local validation:
- `make test`: 56/57 pass; the one failure is unrelated
(`test_arrays_are_ascending` on `.claude/settings.json` —
Claude Code auto-appended WebFetch domains during the iteration
session, the file is sandbox-write-denied so a manual sort by
the operator is required to clear that lint).
- `tests.external.repository.test_urls_reachable`: green (was
failing on the 3 URLs above).
- `VARIANT=0 make deploy-fresh-purged-apps APPS=web-app-bluesky
FULL_CYCLE=true`: PASS 1 + PASS 2 both `failed=0`, OIDC + baseline
Playwright tests green.
- `VARIANT=1 make deploy-reuse-kept-apps APPS=web-app-bluesky`:
`failed=0`, all 3 Playwright tests green (baseline + OIDC variant
A+ recovery + LDAP).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CLI install in roles/web-app-joomla/tasks/02_install.yml runs
through `bash -lc` with the admin password Jinja-substituted into a
double-quoted shell argument. The default admin password
(`'$tr0ng€xampl3PW\!`) contains `$tr0ng`, which bash expands as an
undefined parameter — the installed secret silently truncates to
`'€xampl3PW\!`. Joomla then rejects every subsequent login (admin
Playwright test, password reset, etc.).
Pass admin and DB passwords through the task's `environment:` and
reference them as bash env vars (`${J_ADMIN_PASSWORD}`,
`${J_DB_PASSWORD}`) so bash expands the env var (which holds the
literal secret) instead of re-parsing the secret as shell input.
Verified via `SERVICES_DISABLED='matomo,email' make deploy-fresh-purged-apps APPS=web-app-joomla FULL_CYCLE=true` — both sync and async passes report `failed=0`, Playwright `dashboard to joomla: admin login, verify control panel, logout` passes in 5.0s and 4.0s.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
probe_did_inner_build() in scripts/tests/environment/utils/cache.sh
hardcoded the compose network as `infinito-nexus_default`. Compose
derives the network name from the project name, which defaults to the
host CWD basename — `infinito-nexus` locally, `infinito-nexus-core` in
the CI fork, something else again in Act runners and worktrees. CI
therefore failed the test-environment matrix on every distro with
`network infinito-nexus_default not found` even though the cache
stack itself was healthy.
Look up the network the running `infinito-package-cache-frontend`
container is actually attached to via `docker inspect` and use that.
Fail loudly if discovery returns empty rather than silently skipping
the probe.
Verified via `make act-workflow ACT_WORKFLOW=.github/workflows/test-environment.yml ACT_JOB=test-environment ACT_MATRIX='dev_runtime_image:debian:bookworm'` — Job succeeded, probe step reports `[OK] DiD inner-build flowed through cache frontend`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add requirement 015 for replacing the discontinued
``bitnamilegacy/moodle`` image with a self-built one based on
the official ``php:8.3-fpm`` and the floating Moodle stable
tarball, sourced via the ``moodlehq/moodle-docker`` reference
patterns where applicable.
Identity integration is laid out as two role-variant slots
running side-by-side in CI:
* Variant 0 — ``auth_oidc`` (Keycloak SSO) + ``auth_ldap`` in
sync-only mode for cron-driven attribute propagation.
* Variant 1 — ``auth_ldap`` as the primary login plugin (no
OIDC), with the same cron-driven sync.
The requirement nails down the source-of-truth hierarchy
(LDAP = canonical store, Keycloak federation mapper = SPOT for
Keycloak ↔ LDAP attribute pairing, Keycloak Account Console =
sole edit surface, Moodle = read-only via locked field-map),
the Keycloak ↔ LDAP federation mode (WRITABLE), the
``auth_ldap`` ``field_lock_<col>=locked`` enforcement, the
custom LDAP schema for the three attributes that have no
core ``inetOrgPerson`` equivalent
(``infinitoNamePhoneticGiven``, ``infinitoNamePhoneticFamily``,
``infinitoTimezone`` under the project's existing
``1.3.6.1.4.1.99999.x`` placeholder OID prefix), and the cron
sidecar that runs Moodle's scheduled tasks on a ≤ 60s loop.
End-to-end coverage is via a Playwright spec that exercises
all three propagation paths per variant: OIDC at login (variant
0 only), ``auth_ldap`` sync without login (both variants), and
direct LDAP-bind login (variant 1 only). Test seeds biber's
profile through the Keycloak Admin REST API in variant 0 or
directly via ``ldapmodify`` in variant 1, mutates ``phone1`` /
``phone2`` server-side, and asserts propagation into Moodle
without an additional login round-trip. Also asserts
read-only enforcement on Moodle's profile-edit form.
The role MUST iterate per the matrix-variant guidance in
``docs/contributing/design/variants.md``: a FULL-matrix baseline
first, then ``VARIANT=<idx>`` pinned reuse-deploy iterations,
``make test`` green before every deploy, ``SERVICES_DISABLED``
``matomo,email`` persisted from the start. The requirement
explicitly forbids intermediate commits; the implementing
agent works the full set of acceptance criteria to completion
and lands the migration as a single commit on
``feature/moodle-self-built``.
Adopt the version bumps from #222 against the current
``meta/services.yml`` layout instead of the deprecated
``config/main.yml`` files (post-req-008):
- roles/web-app-funkwhale/meta/services.yml:
funkwhale (front): '2' -> 2.0.2
api: '2' -> 2.0.2
- roles/web-app-prometheus/meta/services.yml:
alertmanager: v0.32.0 -> v0.32.1
Other version bumps from #222 were already applied on this branch
(baserow 2.2.2, prometheus v3.11.3). The three legacy
``config/main.yml`` files restored from the incoming side are
re-deleted because the source-of-truth has moved to ``meta/`` here.
Also picks up:
- docs/requirements/014-svc-runner-ci.md (new)
- skills-lock.json (latest hashes)
Replace the previous OR-based assertion in scripts/tests/environment
(which trivially passed because the Nexus log always grows from
health probes) with a richer set of helpers split across two files
under utils/ for SRP/SPOT.
scripts/tests/environment/utils/cache.sh (new):
* expected_system_proxy_for_runtime maps DEV_RUNTIME_IMAGE to the
Nexus proxy that handles its system package manager (apt-debian,
apt-ubuntu, yum-fedora, yum-rocky; empty for Manjaro/Arch where
pacman is not cached).
* nexus_repo_requests counts only /repository/<name>/... entries in
the Nexus request log, optionally scoped to one proxy. Filters out
health probes, admin REST calls, and Web-UI traffic.
* registry_cache_flows counts every cache-flow status (HIT, MISS,
REVALIDATED, UPDATING, STALE) so cold deploys also count.
* assert_caches_used fails with a per-cache breakdown when any of
three independent conditions is violated: total Nexus repo
traffic = 0, the runtime's expected system proxy has 0 cached
requests, or the deploy itself produced 0 registry-cache flows.
* probe_caches actively fetches a known upstream resource through
each cache (apt-debian/dists/bookworm/Release for the package
cache, /ca.crt for the registry cache).
* probe_did_inner_build drives a throwaway docker build whose
apt-get update runs against deb.debian.org. The build container
is placed on the compose network and pinned via --add-host to the
frontend IP, exactly like the runtime override that
roles/sys-svc-compose/files/compose.py injects for app builds.
After the build the test snapshots
``docker logs infinito-package-cache-frontend`` and asserts the
frontend recorded new requests, proving DiD apt traffic flows
through nginx -> Nexus apt-debian and not directly to upstream.
DOCKER_BUILDKIT=0 because BuildKit refuses --network=<compose-net>.
scripts/tests/environment/utils/common.sh (renamed from lib.sh):
keeps the bootstrap, constants, inventory inspect, and
assert_http_status; cache helpers moved out so the file is
single-concern.
scripts/tests/environment/07_deploy_minimal.sh sources both files;
order is snapshot, deploy, probe_caches, probe_did_inner_build,
snapshot, assert_caches_used. Active probes contribute traffic that
the AND assertion verifies, so the assertion is robust on deploys
that do not naturally trigger inner-build apt traffic.
.github/workflows/test-environment.yml: propagate DEV_RUNTIME_IMAGE
into the inner ``docker run`` so per-distro probe target and proxy
expectation resolve correctly inside the orchestrator.
.gitignore + .dockerignore: also ignore .claude/settings.local.json,
which compose mounts as /dev/null and which the classic docker
builder would otherwise try (and fail) to read into the build
context.
Verified end-to-end with five iterations of
``make act-workflow ACT_WORKFLOW=.github/workflows/test-environment.yml
ACT_JOB=test-environment ACT_MATRIX='dev_runtime_image:debian:bookworm'``.
The final iteration reports +81 apt-debian, +235 registry-cache
flows, +13 frontend log lines from the DiD probe, and "[OK] both
cache stacks observed activity" plus "[OK] DiD inner-build flowed
through cache frontend".
`docs/requirements/014-svc-runner-ci.md` referenced three paths
(`cli/deploy/runner/`, `roles/svc-runner/`) that the document itself
defines as future acceptance criteria — they don't exist yet, so the
markdown-link-resolver test (`test_markdown_relative_links_resolve`)
hard-failed `make test` for the whole repo.
Drop the link syntax, keep the path as inline code. Same meaning, no
broken-link false positives.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promotes web-app-bluesky's OIDC integration to a working Keycloak SSO
flow that ends up in the official Bluesky social-app web client
WITHOUT exposing the synthesised PDS app-password to the browser.
* New `login-broker` Node sidecar under
`roles/web-app-bluesky/files/login-broker/`. The broker sits behind
`web-app-oauth2-proxy` and in front of the official
`@bluesky-social/social-app` web client. On a user's first
OIDC-authenticated visit it:
1. Looks up the Keycloak user via the Admin REST API.
2. Provisions a Bluesky PDS account via createAccount if no
encrypted app-password is recorded yet, encrypts the
synthesised app-password with AES-256-GCM, and stores the
ciphertext as the `bluesky_app_password_enc` user attribute.
3. Decrypts the password and exchanges it for a PDS session via
createSession.
4. Renders an HTML handoff page that drops the session JWTs
into localStorage["BSKY_STORAGE"] (canonical social-app key)
and redirects to /.
For subsequent requests carrying the handoff cookie, the broker
reverse-proxies to the social-app upstream.
* `social-app` is built from a pinned upstream tag (1.121.0) so the
broker's BSKY_STORAGE schema stays in sync with the upstream
persistence layer.
* `roles/web-app-bluesky/tasks/05_keycloak_admin_client.yml` adds a
service-account Keycloak client for the broker (deterministic
secret from the credentials vault, view-users + manage-users
realm-management roles). kcadm output is filtered for the JVM
cgroup-detection warnings on cgroup-v2 hosts.
* `roles/web-app-bluesky/tasks/01_core.yml` switches from
sys-svc-compose to sys-stk-backend so the oauth2-proxy sidecar is
auto-included when services.oauth2.enabled is true. A pre-task
heals the oauth2-proxy bind-mount when a previous deploy left it
as an empty directory.
* `meta/schema.yml` adds `bridge_encryption_key` (AES-256-GCM key)
and `kc_admin_client_secret` (broker's KC service-account secret).
* `meta/services.yml` enables oauth2 with origin `bluesky-broker:8080`,
whitelist for /xrpc/ and /.well-known/ so federation traffic to
api.bluesky.<domain> stays open.
* `meta/variants.yml` adds the LDAP variant so the matrix-deploy
exercises both auth-flow legs.
* `templates/extra_locations.conf.j2` uses the new
NETWORK_PUBLIC_DNS_RESOLVERS SPOT and the on_off filter for the
nginx resolver directive in front of the public ip.bsky.app /
bsky.app upstreams.
* Documentation:
- `roles/web-app-bluesky/README.md` rewritten per documentation.md
and readme_md.md (software-focused description, current SSO
flow, no em-dashes, canonical Credits format).
- `roles/web-app-bluesky/files/login-broker/README.md` covers the
handoff sequence, env contract, and failure modes.
- `roles/web-app-bluesky/files/keycloak-bluesky-bridge/README.md`
cleaned up per documentation.md.
- `docs/requirements/013-alpha-to-beta-promotion.md` bluesky
per-role-notes updated to reference the variant A+ broker; new
"Future hardening" section tracks key rotation, an external
secrets store, the Joomla `?fallback=local` IP allowlist, and
the still-source-only Keycloak event-listener SPI.
* `.github/dependabot.yml` adds the npm ecosystem entry that covers
the broker's package.json metadata stub.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promotes web-app-joomla's OIDC integration from "documented future
work" to a working native plugin shipped inside the role.
* New `plg_system_keycloak` Joomla 6 plugin under
`roles/web-app-joomla/files/joomla-oidc-plugin/`. The plugin's PHP
dependency `jumbojett/openid-connect-php` is vendored at deploy time
inside the Joomla container (Composer is baked into the role's
custom image), then packaged as a ZIP and installed via the Joomla
CLI. The repo working tree carries no `vendor/` tree.
* Modus 3 per requirement 013 (Force-Frontend, Local-Backup-Backend):
every visit to `/` and `/administrator` redirects to Keycloak
unless the request explicitly carries `?fallback=local` AND the
env-var `JOOMLA_OIDC_FALLBACK_ENABLED` is `true`. The fallback hatch
is env-toggleable so high-security deployments can flip to Modus 1
(no local form).
* RBAC group mapping: the plugin maps the Keycloak `groups` claim
onto Joomla's standard usergroup IDs:
/roles/web-app-joomla/administrator -> Super Users (id 8)
/roles/web-app-joomla/editor -> Editor (id 4)
/roles/web-app-joomla -> Registered (id 2)
Users whose Keycloak groups match none of these paths are refused.
* Matrix-deploy variants flip the auth-flow gate: variant 0 is the
canonical OIDC variant (this commit's primary path), variant 1
flips OIDC off and LDAP on so Joomla's core LDAP authentication
plugin against svc-db-openldap is the exercised auth path.
* Role wiring:
- `tasks/07_oidc_plugin.yml` builds + installs the plugin idempotently.
- `templates/env.j2` propagates JOOMLA_OIDC_* env into the container.
- `meta/services.yml` enables `oidc` with `fallback_enabled: true`.
- `meta/rbac.yml` declares the `editor` role.
- `templates/playwright.env.j2` keys `OIDC_SERVICE_ENABLED` off
`services.oidc.enabled` for the Playwright test gate.
* Playwright e2e covers three scenarios:
- native OIDC redirect from `/` to Keycloak and back to Joomla,
- `/administrator?fallback=local` hatch bypassing Keycloak,
- LDAP variant against the core Joomla LDAP plugin.
* Documentation:
- `roles/web-app-joomla/README.md` rewritten per documentation.md
and readme_md.md (software-focused description, no em-dashes,
canonical Credits format).
- `roles/web-app-joomla/files/joomla-oidc-plugin/README.md` covers
plugin build, env contract, claimed routes, and watch points.
- `docs/requirements/013-alpha-to-beta-promotion.md` joomla
per-role-notes updated to reference the in-role plugin and the
Modus 3 fallback contract.
* `.github/dependabot.yml` adds the composer ecosystem entry that
covers the plugin's `composer.json`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related cleanups extracted from the in-flight req-013 work because
they touch shared config layers (filter plugins, group_vars/all,
unrelated role templates) and deserve their own atomic commit so the
req-013 PR stays focused on the joomla/bluesky integration.
1. `on_off` filter (plugins/filter/on_off.py + unit tests)
Replaces the verbose `{{ x | bool | ternary('on', 'off') }}` idiom
that nginx, msmtp, unbound etc. need for their config syntax.
Coercion mirrors Ansible's `bool` filter; unknown strings raise
so templates can't silently mis-render. Migrated call sites:
- roles/sys-svc-mail-msmtp/templates/msmtprc.conf.j2 (4x)
- roles/sys-svc-proxy/templates/location/html.conf.j2
2. `NETWORK_PUBLIC_DNS_RESOLVERS` SPOT (group_vars/all/08_networks.yml)
New canonical list of public DNS resolvers used wherever a role
needs runtime external-hostname lookups (nginx `resolver`
directives, unbound `forward-addr`, etc.). Default
`[1.1.1.1, 8.8.8.8]`. Migrated call sites:
- roles/web-app-mailu/templates/unbound.conf.j2 (forward-zone loop)
The IPv6 toggle in any nginx `resolver` directive comes from the
existing `NETWORK_IPV6_ENABLED` flag in the same file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`docs/requirements/014-svc-runner-ci.md` referenced three paths
(`cli/deploy/runner/`, `roles/svc-runner/`) that the document itself
defines as future acceptance criteria — they don't exist yet, so the
markdown-link-resolver test (`test_markdown_relative_links_resolve`)
hard-failed `make test` for the whole repo.
Drop the link syntax, keep the path as inline code. Same meaning, no
broken-link false positives.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mandates iteration via docs/agents/action/iteration/workflow.md +
role.md, batched clarifying questions only at the start, no mid-run
ask prompts, no interruptions, and a single end-of-run commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New requirements file 014-svc-runner-ci.md scoping a dedicated CI runner
role plus a cli/deploy/runner/ entry point (argparse-based) so CI can
run on a dedicated server instead of being bound by local hardware.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New requirements file 014-svc-runner-ci.md scoping a dedicated CI runner
role plus a cli/deploy/runner/ entry point (argparse-based) so CI can
run on a dedicated server instead of being bound by local hardware.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restore the assertion lost in ed04dad5d when the centralised
`group_vars/all/09_ports.yml` and per-role `config/main.yml` went away:
every role with `oauth2.enabled: true` in `meta/services.yml` MUST
allocate an oauth2-proxy listen port via
`<service>.ports.local.oauth2: <port>` in the same file. The contract
itself is unchanged; only the storage location moved into the role's
own meta/services.yml. Use `utils.cache.yaml.load_yaml` so the file is
parsed once per process and stays consistent with the loader policy
enforced by `test_no_direct_yaml_calls.py`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The wrapper convention applies inside Ansible roles. Bootstrap shell
scripts and CI workflows live outside `roles/` and either pre-date the
wrapper's installation or run on hosts that never see it; flagging them
adds noise without protecting the contract. Add a roles/ path-prefix
filter, rename the test method, drop the now-redundant `nocheck:
raw-docker` headers from the 6 GitHub workflows, and rewrite the two
suppression-doc examples on realistic in-scope scenarios (a per-line
marker on an Ansible task, a file-level marker on a role-bundled .sh).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop the local `# no-raw-docker: skip` regex in favour of
utils.annotations.suppress.is_suppressed_at / is_suppressed_in_head
under rule key `raw-docker`, gaining per-line opt-outs (`# noqa:
raw-docker` / `# nocheck: raw-docker`) on top of the existing
file-level head suppression. Migrate the 6 GitHub-hosted-runner
workflow markers to the new grammar, lift the head scan to the catalog
default of 30 lines, and add the rule to the suppression catalog with
two examples.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduce a single grammar for per-line, per-block, and per-file
suppression markers across the test suite, replacing seven ad-hoc
syntaxes that had drifted apart.
Helper
utils/annotations/suppress.py exposes is_suppressed_at,
is_suppressed_in_head, is_suppressed_anywhere, line_has_rule, and
suppressed_line_numbers. Markers may use either `noqa` or `nocheck`
(case-insensitive synonyms), live inside `#`, `//`, `{# ... #}`, or
`<\!-- ... -->` comments, and combine multiple kebab-case rules
comma-separated. 17 unit tests cover the grammar in isolation.
Refactored consumers
- tests/external/repository/test_urls_reachable.py (rule: url)
- tests/external/docker/test_image_versions.py via
utils/docker/version_updater.py (rule: docker-version)
- tests/lint/repository/test_no_direct_yaml_calls.py (rule:
direct-yaml)
- tests/lint/ansible/test_service_shared_consistency.py (rule:
shared)
- tests/lint/ansible/roles/test_web_app_email_integration.py (rule:
email)
- tests/lint/repository/test_python_file_size.py (rule: file-size)
- tests/lint/ansible/test_run_once_tags.py +
tests/integration/roles/run_once/test_schema.py (rules: run-once,
run-once-suffix)
Marker migration
12 role tasks/main.yml files: `# run_once_<key>: deactivated`
-> `# nocheck: run-once`. roles/web-app-suitecrm/meta/services.yml
drops a malformed inline `noqa: intentional legacy pin` from a TODO
comment so the new parser does not pick it up as a stray rule.
Caching wins
Test files that previously loaded YAML / read files via raw
yaml.safe_load + Path.read_text now route through utils.cache.yaml
(load_yaml_any / load_yaml_str) and utils.cache.files.read_text:
- test_service_shared_consistency.py
- test_web_app_email_integration.py
- tests/integration/roles/run_once/test_schema.py
- utils/docker/version_updater.py
This both speeds up repeated reads and stays consistent with the
direct-yaml lint policy.
Documentation
New SPOT page at docs/contributing/actions/testing/suppression.md
with grammar, position semantics, the catalog of rules, examples,
and an "adding a new rule" checklist. lint.md and external.md drop
their per-rule tables and link to the new page. All touched docs
conform to docs/contributing/documentation.md (no em-dash thought
breaks, no SPOT self-labels, RFC-2119 keywords used consistently).
Settings sort tooling
scripts/lint/sort_claude_settings.sh wraps utils/claude/settings_sort.py
and is wired into scripts/lint/autoformat.sh. The tooling fixes the
ASCII-ascending requirement enforced by
tests/lint/repository/test_claude_settings_sorted.py; the test's
failure message now points operators at the script. .claude/settings.json
is committed in sorted form.
Reintroduce the no_raw_docker integration test that was lost in the
flat-to-clustered tests/integration reorg, scoped to .yml/.yaml/.j2/.sh
and powered by utils.cache.files.iter_non_ignored_files plus read_text
so file enumeration respects .gitignore and content reads are
lru-cached. Catch two argv-/inline-form blind spots that the original
single-line regex missed (argv: - docker / shell: "docker exec ...")
and add a `no-raw-docker: skip` header marker for files where the
container wrapper is genuinely unavailable.
Apply the marker to the 6 GitHub-hosted-runner workflows that cannot
reach the wrapper, and rewrite the 7 role-task argv invocations
(BigBlueButton, Discourse, Matomo, Joomla 03/04/06) onto
'container' so the engine stays swappable.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Group the prometheus integration test next to the other config-scoped
tests under tests/integration/config/, drop the redundant _config
suffix, and bump the __file__-rooted path computations by one .parent
to compensate for the new depth.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Catch the runtime "role 'X' is not declared under
applications[Y].rbac.roles" failure of the rbac_group_path lookup at CI
time by cross-checking every callsite under roles/ against each
application's meta/rbac.yml. Move both rbac-scoped guards into a
dedicated oauth2_oidc/rbac/ subdirectory with its own README and drop
the now-redundant rbac prefix from the test filenames.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduce a single grammar for per-line, per-block, and per-file
suppression markers across the test suite, replacing seven ad-hoc
syntaxes that had drifted apart.
Helper
utils/annotations/suppress.py exposes is_suppressed_at,
is_suppressed_in_head, is_suppressed_anywhere, line_has_rule, and
suppressed_line_numbers. Markers may use either `noqa` or `nocheck`
(case-insensitive synonyms), live inside `#`, `//`, `{# ... #}`, or
`<\!-- ... -->` comments, and combine multiple kebab-case rules
comma-separated. 17 unit tests cover the grammar in isolation.
Refactored consumers
- tests/external/repository/test_urls_reachable.py (rule: url)
- tests/external/docker/test_image_versions.py via
utils/docker/version_updater.py (rule: docker-version)
- tests/lint/repository/test_no_direct_yaml_calls.py (rule:
direct-yaml)
- tests/lint/ansible/test_service_shared_consistency.py (rule:
shared)
- tests/lint/ansible/roles/test_web_app_email_integration.py (rule:
email)
- tests/lint/repository/test_python_file_size.py (rule: file-size)
- tests/lint/ansible/test_run_once_tags.py +
tests/integration/roles/run_once/test_schema.py (rules: run-once,
run-once-suffix)
Marker migration
12 role tasks/main.yml files: `# run_once_<key>: deactivated`
-> `# nocheck: run-once`. roles/web-app-suitecrm/meta/services.yml
drops a malformed inline `noqa: intentional legacy pin` from a TODO
comment so the new parser does not pick it up as a stray rule.
Caching wins
Test files that previously loaded YAML / read files via raw
yaml.safe_load + Path.read_text now route through utils.cache.yaml
(load_yaml_any / load_yaml_str) and utils.cache.files.read_text:
- test_service_shared_consistency.py
- test_web_app_email_integration.py
- tests/integration/roles/run_once/test_schema.py
- utils/docker/version_updater.py
This both speeds up repeated reads and stays consistent with the
direct-yaml lint policy.
Documentation
New SPOT page at docs/contributing/actions/testing/suppression.md
with grammar, position semantics, the catalog of rules, examples,
and an "adding a new rule" checklist. lint.md and external.md drop
their per-rule tables and link to the new page. All touched docs
conform to docs/contributing/documentation.md (no em-dash thought
breaks, no SPOT self-labels, RFC-2119 keywords used consistently).
Settings sort tooling
scripts/lint/sort_claude_settings.sh wraps utils/claude/settings_sort.py
and is wired into scripts/lint/autoformat.sh. The tooling fixes the
ASCII-ascending requirement enforced by
tests/lint/repository/test_claude_settings_sorted.py; the test's
failure message now points operators at the script. .claude/settings.json
is committed in sorted form.
The Nexus 3 raw-packagist proxy points at https://repo.packagist.org/
as the upstream root. That root returns HTTP 404 (Packagist serves
content under /packages/, /p2/, etc., not at /), so the external URL
reachability test fails the run even though the host is healthy and
the proxy works. Mark the line with `# nocheck: url` to skip the
literal-URL probe; the upstream is still validated indirectly when
the cache stack pulls real packages.
Closes the integration gaps left by the prior commit (1489ed4e5):
the OIDC integration on minio, joomla, libretranslate, and postmarks
is now actually exercised end-to-end by Playwright through Keycloak,
not faked via local form-login fallbacks. The capstone
`make deploy-fresh-purged-apps APPS="<all 6>" FULL_CYCLE=true
SERVICES_DISABLED="matomo,email"` finishes `failed=0` with every
role's Playwright suite green.
Per-role:
* MinIO (✅/✅/✅). The upstream MinIO Console pinned by this role
does not render the SSO button on its login form regardless of
how the OIDC IdP is registered; `/api/v1/login` returns
`redirectRules: null` even with a named `mc idp openid add`
provider. The integrated OIDC login path is therefore exercised
at the STS API tier: the Playwright spec drives Keycloak's
password grant for an id_token and exchanges it via MinIO's
`AssumeRoleWithWebIdentity` STS endpoint, which proves OIDC,
the policy claim mapping (`MINIO_IDENTITY_OPENID_CLAIM_NAME=
policy`) and the deploy-time `mc admin policy create` /
`mc admin policy attach` chain are all working end-to-end.
* Joomla (✅/✅/🛠️). Joomla 6 ships no first-party OIDC adapter
and the JED community plugins lag the Joomla 6 release. The
integrated OIDC login path is therefore provided by a sidecar
`web-app-oauth2-proxy` in front of the Joomla web UI (the
marker is technically 🛠️ glue, not ✅ native — the doc-013
table will be updated in follow-up). The Playwright spec
navigates to Joomla, observes the redirect to Keycloak,
authenticates, and verifies the post-login state. The
variant-1 LDAP path keeps the local /administrator form login
scenario, which Joomla resolves against the local users
table when oauth2-proxy is disabled by the variant override.
* LibreTranslate (🛠️/❌/❌). Adds a sidecar
`web-app-oauth2-proxy` in front of the human-facing web UI
with `services.oauth2.acl.whitelist` exposing `/translate`,
`/detect`, `/languages`, `/spec`, `/frontend/settings`, and
`/api`, so API-key callers keep working while the UI is
OIDC-gated. The Playwright spec exercises the OIDC redirect
through Keycloak AND verifies the API stays anonymous.
* Postmarks (🛠️/🛠️/❌). Adds a sidecar
`web-app-oauth2-proxy` in front of the web UI. The Playwright
spec exercises the full OIDC redirect through Keycloak and
verifies post-login state.
* Bluesky (🛠️/🛠️/❌). The role now ships the source for the
Keycloak event-listener SPI bridge that auto-provisions
Bluesky PDS accounts on REGISTER / LOGIN events and stores
the synthesised app-password back as a Keycloak user
attribute (per the requirement-013 per-role notes):
roles/web-app-bluesky/files/keycloak-bluesky-bridge/
pom.xml
README.md
src/main/java/com/infinito/bluesky/
BlueskyBridgeEventListener.java
BlueskyBridgeProviderFactory.java
src/main/resources/META-INF/services/
org.keycloak.events.EventListenerProviderFactory
The SPI implements handle sanitisation to the AT Protocol's
allowed `[a-z0-9-]` charset, a 3-attempt retry budget for the
PLC-bound provisioning step, and group-membership gating
against `/roles/web-app-bluesky`. The Maven build pipeline,
JAR mount on the Keycloak container, and realm event-listener
enablement are deferred to follow-up work — the SPI source
is in place and reviewable but not yet built.
* Bridgy-fed (❌/❌/❌). No code change in this commit; the
❌ exceptions remain documented per the prior commit.
Shared infrastructure / tooling:
* `.github/dependabot.yml`: enable Maven coverage for the new
Bluesky SPI pom.xml.
* `.github/workflows/security-codeql.yml`: enable
`java-kotlin` in the CodeQL matrix to scan the new Java source.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`07_deploy_minimal.sh` now snapshots Nexus request-log line count and
registry-cache HIT count before the deploy, runs the deploy, snapshots
again, and fails if neither counter grew. Catches regressions where
the runner-side wiring stops routing through the cache stack.
Helpers `cache_snapshot` and `assert_cache_used` live in `lib.sh` so
other environment scripts can reuse the same check.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Extend the local cache stack with three independent capabilities:
1. Inner-`dockerd` build coverage for HTTP-only package mirrors.
`roles/sys-svc-compose/files/compose.py` (the runner's `compose`
wrapper) auto-generates a transient `compose.cache.override.yml`
per app when `INFINITO_PACKAGE_CACHE_FRONTEND_IP` is exported. The
override pins `build.extra_hosts` for `deb.debian.org`,
`archive.ubuntu.com`, and `security.ubuntu.com` to the frontend.
`pull.py` now delegates to the wrapper so build/pull operations see
the same `-f` set the wrapper auto-detects. nginx in the frontend
gains a port-80 listener with path rewrites for those hostnames,
and `package.sh` registers two new Nexus proxies
(`apt-debian-security`, `apt-ubuntu-security`) for the security
suites whose upstream URL prefix differs from the main archive.
Alpine is intentionally excluded from the inner-build extra_hosts:
3.18+ defaults to HTTPS, and inner-`dockerd` build containers do
not import the frontend CA. Routing apk through the frontend would
trip TLS verification.
2. General cache freshness env-var. `scripts/meta/env/cache/package.sh`
exports `INFINITO_PACKAGE_CACHE_MAX_AGE_MIN` (default `129600`,
i.e. 90 days). The bootstrap helper consumes it for every Nexus
proxy's `contentMaxAge`, `metadataMaxAge`, and
`negativeCache.timeToLive`, replacing the previous 1440-minute
literal. Lower the value when chasing fast-moving upstreams.
3. SPOT cache documentation at `docs/contributing/environment/cache.md`.
`compose.yml.md`, `variables.md`, the per-service READMEs, and
`requirements/012-...` now link to it instead of restating the
stack's behaviour. Net documentation lines drop substantially while
the cache page becomes the single user-facing reference.
Also: code-comment cleanup per `docs/contributing/documentation.md`
guidelines across all touched .py / .sh / .yml files. Multi-paragraph
docstrings, banner separators, and restate-the-obvious comments are
removed; one-line WHY comments remain where the rationale is not
already visible in the code.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements requirement docs/requirements/013-alpha-to-beta-promotion.md
for the in-scope subset:
- web-app-minio: ✅/✅/✅ V0 OIDC + V1 LDAP variants green
- web-app-joomla: ✅/✅/🛠️ V0 OIDC + V1 LDAP variants green
- web-app-bridgy-fed: ❌/❌/❌ baseline (all 3 documented exceptions)
- web-svc-libretranslate: 🛠️/❌/❌ baseline (LDAP+RBAC documented exceptions)
- web-app-postmarks: 🛠️/🛠️/❌ baseline (RBAC documented exception)
- web-app-bluesky: 🛠️/🛠️/❌ baseline (RBAC documented exception)
Each role's `meta/services.yml.<entity>.lifecycle` is now `beta`. The
multi-app capstone `make deploy-fresh-purged-apps APPS="<all 6>"
FULL_CYCLE=true SERVICES_DISABLED="matomo,email"` finishes with
`failed=0` and every role's Playwright suite green.
Per-role highlights:
* MinIO. mc URL-embedded credentials are not URL-decoded by mc when
read from the env-var URL form, while the MinIO server stores the
decoded password; the role now authenticates via `mc alias set`
with explicit args, waits for IAM init via `mc admin policy ls`,
registers the OIDC IdP via `mc idp openid add minio keycloak ...`
(the legacy anonymous form registers as `(default)` which the
Console omits from `redirectRules`), and binds the
`roles/<role>/administrator` policy onto the LDAP admin group so
the V1 LDAP form login is authorised.
* Bridgy-fed. Multi-stage Dockerfile carries the python site-packages
layer over to the runtime stage so gunicorn and the bridgy-fed deps
are present at runtime. Removes the colon from
`meta/main.yml.galaxy_info.description` so the dashboard config
rendering does not produce invalid YAML.
* Joomla. Adds `oidc:` enable + `ldap:` block in meta/services.yml,
matrix-deploy variants for the two auth flows, and an extended
Playwright env exposing OIDC issuer, biber persona, canonical
domain, and the `<service>_SERVICE_ENABLED` gate flags.
* LibreTranslate. Adds `container_port=5000`, drops the
`dmbs_excl.yml.j2` depends_on partial that produced an empty
`'': condition: service_healthy` map, and pins
`LT_LOAD_ONLY=en,de` so the deploy gate's readiness probe does not
race the upstream argos-translate model mirror.
* Postmarks. Built from scratch as a full role (was a stub): tasks,
Node-based Dockerfile that clones ckolderup/postmarks, compose
template, env with SESSION_SECRET / ADMIN_KEY pulled from
meta/schema.yml credentials, server.yml with subnet, Playwright
env + spec.
* Bluesky. Adds `files/playwright.spec.js` and
`templates/playwright.env.j2` for the deploy gate. The Keycloak
event-listener bridge to PDS `com.atproto.server.createAccount`
remains to be built in follow-up work; until then the spec
exercises baseline reachability and the OIDC/LDAP/RBAC gaps stay
documented in `roles/web-app-bluesky/README.md` per
lifecycle.md.
Shared infrastructure:
* `roles/test-e2e-playwright/tasks/run_one.yml`. The wait-for-app
probe now accepts 4xx as proof of life; the prior `<400` boundary
rejected legitimate 404 landing surfaces (bridgy-fed serves
federation endpoints under specific paths, not `/`). 5xx remains
the only signal of an unhealthy app.
README.md SSO/RBAC exception sections per
lifecycle.md (`docs/contributing/design/services/lifecycle.md`):
* web-app-bluesky — RBAC exception
* web-app-bridgy-fed — full SSO/RBAC exception
* web-app-postmarks — RBAC exception
* web-svc-libretranslate — LDAP + RBAC exception
Out of scope for this commit (deferred to follow-up requirements):
* Bluesky Keycloak event-listener SPI for PDS account auto-
provisioning (needs Java SPI build pipeline + JAR mount + realm
wiring).
* LibreTranslate sidecar oauth2-proxy on the UI subpath only (must
not gate the API endpoints).
* Postmarks sidecar oauth2-proxy in front of the web UI.
* Joomla community OIDC plugin install + group-mapping config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the `docker compose -f ...` flag list and the cache-override
env-var setup into single helpers in `cli/deploy/development/common.py`:
* `compose_file_args()` — decides whether to layer
`compose/cache.override.yml` on top of `compose.yml`.
* `cache_env_overrides()` — produces the env-var dict the
override file requires strictly via `${VAR:?…}`.
Both `cli.deploy.development.compose` (up flow) and
`cli.deploy.development.down` (down flow) now call these helpers, so
up and down resolve to the same compose-file set and the same env
vars. Previously the cache env-var setup was duplicated inline in
compose.py and missing entirely from down.py, which made `make down`
fail under the `cache` profile with a confusing "required variable
missing" error before even reaching the orphan-cleanup step.
Single SPOT discipline: the host CA directory default lives only in
`scripts/meta/env/cache/package.sh` (sourced via BASH_ENV by every
Makefile recipe). `cache_env_overrides()` refuses to fall back to a
duplicate literal — a missing env var raises SystemExit pointing at
the env script. Tests that bypass the env layer set the var
explicitly so the SPOT-strict guard does not raise.
Containerize cert-gen: the original
`scripts/docker/cache/package-frontend-certs.sh` ran `mkdir`/`openssl`
host-side, which fails when `/var/cache/infinito` is not writable from
the invoking shell (sandboxed tooling, non-root operators). Move the
openssl logic into `compose/package-cache-frontend/cert-gen.sh` and
have the host wrapper docker-run it inside a throw-away alpine
container so the docker daemon (root) creates the bind-mount source
paths. Same pattern as how rpardini's registry-cache generates its CA
in-container.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add a TLS-terminating reverse proxy in front of the Sonatype Nexus 3 OSS
pull-through cache so the `infinito` runner can hit upstream
package-manager URLs (pypi.org, registry.npmjs.org, rubygems.org,
repo.packagist.org, proxy.golang.org, dl-cdn.alpinelinux.org,
files.pythonhosted.org, raw.githubusercontent.com, codeload.github.com)
unchanged and still flow through Nexus.
Mechanism:
* scripts/docker/cache/package-frontend-certs.sh — host-side helper,
idempotently generates a dedicated CA + per-hostname leaf certs.
* compose/package-cache-frontend/upstreams.conf — nginx server-blocks
that match SNI per hostname and reverse-proxy onto the matching
Nexus repo path.
* compose/cache.override.yml `infinito.extra_hosts` — DNS-hijack so
the runner's clients resolve each upstream onto the frontend's
static IP.
* scripts/docker/cache/package-frontend-ca.sh — runner-side helper,
drops the CA into the system trust store post-startup so apt/pip/
curl/etc. accept TLS against the hijacked hostnames.
* cli/deploy/development/compose.py — orchestrates cert gen before
compose-up, CA install after health.
Scope today: runner-only. App-service Dockerfile builds executed by
the inner dockerd are NOT routed through the frontend; that requires
build-time CA injection and is deferred.
Also extends scripts/docker/cache/package.sh with five additional
Nexus proxies registered for transparent use: gem-proxy, go-proxy,
yum-rocky, yum-fedora, raw-packagist, raw-alpine, raw-codeload-github.
Refactor: move all cache services + runner-side cache mounts/depends_on/
extra_hosts from compose.yml into compose/cache.override.yml. Base
compose.yml now describes only the bare CI/runtime stack (coredns +
infinito). The override file is the single gate for the cache stack;
`profiles: ["cache"]` attributes and `--profile cache` flag emission
were redundant once file inclusion controls existence and removed
accordingly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
