Add a per-deployment Matomo tracking scope, defaulting to per-subdomain:
- services.matomo.tracking.scope 'sub' (default) registers one Matomo site per
consumer subdomain with an isolated cookie; 'root' keeps the prior behaviour
of one shared site per registrable base domain with cross-subdomain linking.
- sys-front-inj-matomo derives matomo_site_domain (full host vs registrable
base) and the shared matomo_site_url from it; getSitesId / addSite and the
tracking snippet (setCookieDomain / setDomains / enableCrossDomainLinking) all
follow the scope.
- The matomo Playwright env emits MATOMO_TRACKING_SCOPE; the SitesManager test
matches the consumer's full host in 'sub' and its base domain in 'root'.
- variants.yml variant 0 sets scope=root so the matrix exercises both flavours
(every other deploy runs the sub default).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Untrusted fork PRs mirrored anonymously after the credential hardening (empty Docker Hub creds -> Login to Docker Hub skipped -> anonymous pulls), which got rate-limited and failed images-mirror-missing-fork. Restore secrets: inherit on the fork mirror job and the GHCR login token so the mirror authenticates again. The build keeps its hardening (it pulls its parent from GHCR and needs no Docker Hub creds).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
crs-k/stale-branches@v9.0.1 rejects days-before-stale >= days-before-delete (the v8 'delete immediately' behaviour with days-before-delete: 0 is gone), so the daily cleanup failed with 'Invalid inputs'. Set days-before-stale: 350 and days-before-delete: 360 - deletion still triggers at 360 days of inactivity, with a 10-day stale warning before it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The per-round swarm purge ran `docker volume prune -f`, which on Docker 23+
removes only anonymous volumes. A named DB volume such as mariadb_data survives
`docker stack rm`, so the next round booted the prior round's database, whose
root password no longer matched the freshly regenerated credentials, and mariadb
failed with "Access denied for root". Use `docker volume prune -a -f` so named
unused volumes are wiped too and each round re-initialises from clean state.
Swarm-test-harness only; never runs in compose.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
make test materialises a gitignored build artifact at build/install.stamp;
ansible-lint then walked the build/ tree and aborted with
load-failure[not-found] on it. Add build/ to exclude_paths so the lint pass
ignores the build tree.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The swarm convergence check flagged every service whose running replicas did not
equal its desired count, but a completed run-once job (restart-condition none,
such as the matomo bootstrap service) sits at 0/N with all tasks Complete and is
healthy, not stuck. That false flag failed the deploy health gate.
Inspect each candidate's task states and skip it only when a task is Complete and
none is in an active or failed state (Running/Ready/Starting/.../Failed/...); a
crashing or still-converging service keeps such a task and stays flagged. The
change lives entirely inside the swarm-manager branch, so compose is unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Break the single 457-line playwright.spec.js into topic files discovered by the
harness testMatch "**/*.@(spec|test).js", mirroring the pihole / gitea layout:
- _shared.js env vars, the setup hook, loginAsAdmin, attachDiagnostics,
hostOf / baseDomainOf and the MATOMO_TARGET_ROLES_JSON list
- playwright.spec.js CSP + canonical-domain smoke (filename kept; harness asserts it)
- auth.spec.js admin login/logout and biber denied at the admin surface
- tracking.spec.js SitesManager site registration and per-consumer _paq injection
- persona-flows.spec.js guest / biber / admin universal-logout flows
Test logic is unchanged: same titles, selectors, regexes, timeouts and
assertions (verified by title parity and an adversarial diff against HEAD). The
only behavioural delta is that the extracted loginAsAdmin waits for the username
field to be visible before filling, which the SitesManager login previously
skipped (hardening only). The duplicated admin-login form-fill and the old
beforeEach are now the shared loginAsAdmin and setupMatomoPage.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The matomo injection registers one shared SitesManager site per registrable base
domain (addSite urls=proto://base_domain), so a subdomained consumer such as
logout.infinito.example is tracked under the infinito.example site, not its own.
The SitesManager test asserted the full canonical host as a substring of a site
URL, which never matched the base-domain site and failed in swarm.
Reduce the consumer's canonical_domain to its base domain (mirroring the regex in
sys-front-inj-matomo/vars/main.yml) and match it against each site's hostname
exactly. The exact match also drops the substring false positive, where a site at
evil-infinito.example would have satisfied infinito.example. Per-consumer tracker
presence stays covered by the separate per-role injection test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The matomo bootstrap re-injects early-rendered web-svc consumers (such as
web-svc-logout) after obtaining the admin token, but in swarm this never ran:
roles_with_service filters by the manager's group_names, which omits
worker-placed web-svc services, so the re-injection loop was empty and the
logout page shipped without the matomo tracker.
- roles_with_service: add a scope kwarg; scope='all' bypasses the per-host
group_names filter (the merged applications config is already
deployment-scoped, so iterating all of it equals the deployment's consumers).
- 05_bootstrap: enumerate consumers with scope='all', stat each compose project
dir, and re-inject only deployed ones. Avoids a fatal chdir into an undeployed
consumer's missing /opt/compose/<entity>/ that scope='all' would otherwise hit.
- sys-front-inj-matomo: resolve matomo_effective_token (override else
lookup('users')); the disable-gate and the SitesManager getSitesId/addSite
calls use it, so the re-injection passes the fresh bootstrap token and bypasses
the play-start-stale controller-side users cache that left matomo disabled.
scope defaults to 'host' and the override defaults to empty, so every existing
caller and the per-consumer injection path are unchanged. Validated green in
both compose and swarm round 1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_comments_valid.py scopes to the working-tree diff via 'git diff --name-only HEAD', but the make-test container mounts the repo as a different uid, so git aborted with 'dubious ownership' and the linter silently scanned zero files. Pass '-c safe.directory=*' so it runs. Also drop 'gotcha' from the accepted exception markers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the pre-redeploy confidence gate to the compose and swarm loops and
allow bringing up the other deploy mode in parallel for comparison.
Sync the roundtrip cheatsheet prompt to the same rule.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
odoo: sale_management.spec.js asserted `breadcrumb.or(appTitle)` without
`.first()`; when the Sales module installs correctly BOTH the menu brand
and the breadcrumb render, so `.or()` resolved to 2 elements and
`toBeVisible()` raised a strict-mode violation — failing the success case.
Add `.first()` (matching the same file's createQuotation locator).
nextcloud: the GitLab OAuth provisioning hard-`raise`d when no Organization
existed yet (default org is seeded asynchronously). Resolve/create it
defensively and add `retries/until` so a transient not-yet-seeded window no
longer fails the whole deploy.
jenkins: the deploy-lockdown service stop had no retry; under load systemd
intermittently refused with "Transaction is destructive". Add `retries/until`
so the transient contention is ridden out.
moodle: the shared `compose up` handler had no retry; a transient missing
external network (central DB stack still creating it) failed the deploy. Add
`retries/until`, matching the handler's existing compose-pull pattern.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Untrusted fork PRs mirrored anonymously after the credential hardening (empty Docker Hub creds -> Login to Docker Hub skipped -> anonymous pulls), which got rate-limited and failed images-mirror-missing-fork. Restore secrets: inherit on the fork mirror job and the GHCR login token so the mirror authenticates again. The build keeps its hardening (it pulls its parent from GHCR and needs no Docker Hub creds).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
crs-k/stale-branches@v9.0.1 rejects days-before-stale >= days-before-delete (the v8 'delete immediately' behaviour with days-before-delete: 0 is gone), so the daily cleanup failed with 'Invalid inputs'. Set days-before-stale: 350 and days-before-delete: 360 - deletion still triggers at 360 days of inactivity, with a 10-day stale warning before it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements requirement 030: web-app-jellyfin deploying jellyfin/jellyfin
10.11.11 behind sys-svc-proxy. No external DB (internal SQLite); /config,
/cache, /media named volumes; CPU transcoding. Jellyfin has no native SSO, so
auth is plugin-based (operator chose BOTH): jellyfin-plugin-ldapauth (central
OpenLDAP, all clients incl. native apps) + 9p4/jellyfin-plugin-sso (Keycloak
OIDC, web one-click). files/configure-auth.sh completes the first-run /Startup
wizard (seeds break-glass admin), installs both plugins via Jellyfin's own
/Packages installer, writes the plugin configs, and restarts.
Schemas source-verified against the pinned upstream: Jellyfin v10.11.11, LDAP
plugin v23 (GUID 958aad66-..., flat XmlSerializer config), SSO plugin v4.0.0.3
(OidConfig fields). Honest caveats: the SSO OidConfigs is a SerializableDictionary
(XML shape best-effort), plus plugin ABI compat + the SPA Playwright selectors
are live-deploy-validation items (env-blocked). Each config file is XML-well-
formedness checked before reload.
Validation: make autoformat clean; make test green (lint, test-lint, test-unit,
test-integration 190, test-external).
Refs: docs/requirements/030-web-app-jellyfin.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements requirement 029: web-app-checkmk deploying checkmk/check-mk-raw
2.4.0p32 (Raw/Community) behind sys-svc-proxy. No native OIDC, so SSO uses the
platform oauth2-proxy gate + Checkmk header-auth (auth_by_http_header =
X-Remote-User, injected by the role proxy.conf.j2 from $authrequest_username =
Keycloak preferred_username), plus a native LDAP connection for user records.
No external DB. Agent receiver (8000) host-port mapped via a new
PORT_BANDS.local.agent band.
Auth/LDAP/proxy mechanisms source-verified against Checkmk v2.4.0p32
(cmk/gui/auth.py, userdb/ldap_connector.py, userdb/_connections.py) and the
infinito sso_proxy nginx chain: fixed the LDAP file path (user_connections.mk)
and directory_type ("openldap"). LDAP sync timing and classic-GUI Playwright
selectors remain live-deploy-validation items (env-blocked).
Validation: make autoformat clean; make test green (lint, test-lint, test-unit,
test-integration 190, test-external).
Refs: docs/requirements/029-web-app-checkmk.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Block-sequence items under canonical/connect-src reindented two spaces to the autoformat-canonical style after the server.yml domains/csp split. Whitespace-only; git diff -w is empty, parsed YAML unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Static-file (autoindex) and 301-redirect/alias web-svc vhosts route no request through lua-resty-prometheus and carry no user-facing HTML, so they have no per-vhost app=<role> scrape target and no _paq tracker page. scrape:false (asset, cdn, css, html, file, collabora) drops them from PROMETHEUS_TARGET_ROLES_JSON; track:false (asset, cdn, file) drops them from MATOMO_TARGET_ROLES_JSON. suppression.md catalogs both markers. Real apps stay scrape/track targets so monitoring is default-on.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tests/lint/repository/test_comments_valid.py: across .py/.sh/.yml/.rb/
.php/.css, a comment is valid only as a file header, a doc comment directly
above a class/function/rule, a tool directive, or a mid-code EXCEPTION that
flags a real trip-wire (text starts with Warning/Exception/Caution/Gotcha/
Workaround/Security/Important/...). Plain narration, banners, commented-out
code and neutral notes are rejected.
Scope is the working-tree diff (staged + unstaged + untracked vs HEAD): the
legacy tree is grandfathered in and never breaks the build, while every file a
change touches must comply going forward. Python is parsed with tokenize
(string-aware); a multi-line trip-wire's marker on its first line validates the
whole block.
Also fold in autoformat normalization of 11 meta/{csp,domains,server}.yml files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract the csp, domains, and networks blocks out of roles/*/meta/server.yml
into dedicated meta files, each mapping to its own top-level applications key
via the recursive meta-walk:
csp: -> meta/csp.yml -> applications.<app>.csp
domains: -> meta/domains.yml -> applications.<app>.domains
networks: -> meta/networks.yml -> applications.<app>.networks
Residual nginx/proxy keys (status_codes, client_max_body_size, locations,
ip6_enabled, config_upstream_url) stay in server.yml; an emptied server.yml is
deleted. 98 server.yml -> 249 new files, 74 deleted.
- Consumers repointed applications.<app>.server.{csp,domains,networks} ->
applications.<app>.{csp,domains,networks} across templates, vars, tasks,
lookups and filters.
- Infra readers migrated: canonical_domains_map, application_domain_index,
build_ldap_role_entries, service-registry overlay discovery,
roles_with_service, and the file-based readers utils/domains/list.py +
utils/meta/scan.py (read the now-deleted server.yml directly).
- Detection + mapping: _META_TOPICS gains csp/domains/networks;
ROLE_FILE_META_{CSP,DOMAINS,NETWORKS} constants + ROLE_FILES entries.
- New self-expiring lint guard (test_server_meta_no_split_fields) fails if a
server.yml still carries a split field; auto-fails from 2026-12-28 as a
delete reminder.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements requirement 028: web-app-semaphore deploying semaphoreui/semaphore
v2.18.13 behind sys-svc-proxy with central PostgreSQL, native Keycloak OIDC
SSO, and LDAP login, plus guest/biber/administrator Playwright personas.
Auth/admin model verified against the upstream v2.18.13 source (config.go,
api/login.go, api/router.go, web/src/views/Auth.vue, config.schema.yaml,
cli/cmd/user_add.go): the OIDC callback matches by email and rejects local
users, the login form is LDAP-only when ldap_enable=true, and OIDC users are
never admin from a claim. The role therefore seeds a distinct break-glass
local admin and creates the Keycloak administrator as an external admin at
deploy time (semaphore user add --admin --external). Correct v2.x env names
(SEMAPHORE_LDAP_ENABLE/_SERVER/_BIND_DN/_BIND_PASSWORD/_SEARCH_DN, OIDC
provider JSON) confirmed against the source.
Validation: make autoformat clean; make test green (lint, test-lint,
test-unit, test-integration 190, test-external). Live full-matrix deploy +
Playwright still pending (host container runtime fault, environment-only).
Refs: docs/requirements/028-web-app-semaphore.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add tests/lint/repository/naming/test_branch_name.py validating the checked-out branch against the naming policy in branch.md. Rejects unknown prefixes (e.g. feat/ instead of feature/), non-kebab-case descriptions, and fix/ branches missing a ticket id; exempts main, master, and detached HEAD. Document the local enforcement in branch.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Requirements for a web-app-semaphore role (Semaphore UI): native OIDC SSO via Keycloak, LDAP login, central PostgreSQL, single all-in-one container with embedded runner, always-on local-admin bootstrap plus OIDC claim->admin mapping, and Playwright persona coverage. Includes upstream research with cited sources and acceptance criteria.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The roles updaters (Docker image versions, repository refs) opened a
single combined PR for all role changes, and the jobs checked out the
trigger commit (actions/checkout default), so updates were computed
against a stale tree and could re-propose already-merged changes.
- Check out 'ref: main' in all update jobs so the diff is always
computed against the latest main tip at run time.
- Add open_role_prs.sh: split the working-tree changes per role and
open one isolated branch/PR per changed role, reusing open_pr.sh for
the dedup/push/PR logic (new UPDATE_DEDUPE_PREFIX hook scopes the
duplicate check per role; default preserves prior single-PR behavior
for the skills job).
- Wire the Docker and repository-refs jobs to open_role_prs.sh; skills
stays a single PR.
Branch: update/<type>-<role>-<date>-<fp>; commit/title gain a (<role>)
suffix and a Scope line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
