Commit Graph
343 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.8 e99d8a8813 feat(cdn): serve role frontend dependencies from the internal CDN
Add lookup('asset', app, pkg, path) and lookup('asset_host'): both resolve
the asset origin through one shared resolve_host, returning the internal
CDN domain when web-svc-cdn is deployed with flavor internal and
cdn.jsdelivr.net otherwise, so template URLs and CSP origins can never
diverge. Versions come from the role's committed files/package-lock.json.

Roles declare frontend dependencies in files/package.json; web-svc-cdn
aggregates every role lockfile, runs one npm ci per role in a node
container and mirrors the packages under /npm/<pkg>@<version>/ following
jsdelivr's path scheme, so internal and external URLs differ only by host.

Convert web-app-dashboard (keycloak-js 24.0.5 UMD) and web-app-littlejs
(bootstrap, fontawesome) to the asset lookup, drop the vendored keycloak
blob, and add a repository lint forbidding hardcoded external CDN hosts
in templates and CSP metas; pre-existing external CDN entries are checked
out with nocheck markers and a migration TODO.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Fable 5 2584de09ef Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Conflicts: keep both new passthrough env vars; adopt the swarm-nfs
ollama manager-ops split while porting our model-pull retry hardening
into it. Strip three narration comments the stricter comment lint
rejects.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 02:09:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e5e7ed1191 fix(matomo): own /var/www/html/tmp in the image so fresh node-local volumes populate writable
The node-local matomo_tmp volume populated as root on any node that had not run the first-install entrypoint: the matomo image ships /var/www/html empty (code lives in /usr/src), and the entrypoint's chown -R www-data runs only when matomo.php is absent, which on a rescheduled task is never the case because matomo_data is shared NFS. The rescheduled task then died on 'tmp/cache/tracker could not be created', pausing every forced service update. Build a custom image whose /var/www/html/tmp is www-data-owned: docker volume populate replicates the image path's ownership into every fresh volume on every node (proven via moby populateVolume/CopyDir and a local reproduction), so reschedules get a writable tmp deterministically.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 22:38:42 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7d21876d3a chore(quality): settle lint follow-ups of the swarm-topology and matomo commits
Sort the new INFINITO_SWARM_* IP/subnet keys alphabetically in default.env, wire them through the static passthrough handler so make dotenv materialises them, and shorten the matomo DB-reset task name under the 80-char limit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 20:50:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e46659ac4b fix(matomo): keep ephemeral cache off NFS and reset DB on bootstrap retry
Matomo's swarm bootstrap failed rc=1: the high-churn tmp/templates_c Twig cache lived on the soft NFS state share, so a single mid-write RPC timeout surfaced as EIO and stalled the installer 300s; the DB was already seeded, so retries took the eraseAllTables path that wedged Matomo on the CoreUpdater screen the bootstrap image cannot click through. Mount a node-local matomo_tmp volume (nfs: false) for /var/www/html/tmp so caches never traverse NFS, and drop+recreate the app's own database in the pre-retry cleanup (via a container-run reset_database.php over the app credentials, gated exactly like the config-aside task on attempt>0 + no installed config) so every retry takes the clean-install path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 20:50:17 +02:00
kevinveenbirkenbachandClaude Fable 5 62a46e472f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 16:27:25 +02:00
kevinveenbirkenbachandClaude Fable 5 3125c5ac03 docs: regenerate role catalogs with the tor onion wiring
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 15:49:50 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ea9ebe0de3 refactor(inventory): rename prod inventory to production
Standardize the private inventory dir on the fully spelled "production" (matching inventories/development). Update the role README template (SPOT) and regenerate 153 role READMEs plus the admin deploy guide; add inventories/production/ to .gitignore, propagated to .dockerignore via make fix-dockerignore.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:08:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9ad8fa2971 feat(guide): default self_signed and test the whole README block
The guide deploys to a .localhost target, where letsencrypt cannot issue (Cloudflare DNS-01 needs a real domain). Default TLS_MODE=self_signed in the generated Production block so the guide uses the working self-signed provider. Document every deploy parameter as a top-level shell var (APP, HOST, TLS_MODE, SSH_PUBLIC_KEY) instead of an inline placeholder.

The compose guide now extracts and runs the whole README block, sourcing params from it (no more -e injection); HOST is forced to localhost. Skip checkout/login under ACT so the guide workflow is iterable locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:23:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 972596a758 fix(guide): document the administrator SSH key step and seed it in CI via --vars
The README Production block went provision -> deploy with no admin-key step, so user-administrator failed early (authorized_keys empty) for both the Guide CI jobs and any real user copying the README. Add a visible 'provision --vars' step that sets users.administrator.authorized_keys (merged into host_vars) in both Production variants of the role README template, regenerate all role READMEs, and replace the invisible CI payload splice with a visible substitution of the documented <your-ssh-public-key> placeholder.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 05:45:26 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Fable 5 17cd53609a feat(readme): merge the Production deploy into one docker command
The role README Production section now runs provision and deploy in a
single docker call: APP and HOST are shell variables passed via -e,
INVENTORY is the in-container SPOT for the duplicated paths, and the
redundant --vars-file (optional, default None) and --id (the inventory
is already narrowed by --include) flags are gone. All 153 role READMEs
regenerated from the template; test-run.yml executes exactly this block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 13:03:25 +02:00
kevinveenbirkenbachandClaude Fable 5 194671d90f feat(readme): uniform description schema and clearer role titles
Rewrite all 254 role descriptions to a uniform noun-phrase schema
(function-first, no leading verb, no trailing period, <=120 chars) and
enforce it via the extended description lint. Rename 12 role README H1
titles for clarity (e.g. Backup Container to Local -> Backup Docker
Volumes, CSS Service -> Corporate Design, NGINX Legal -> Imprint) and
regenerate their Cosmos and Quick Setup sections. Overview table
regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 01:11:48 +02:00
kevinveenbirkenbachandClaude Fable 5 da598a2d67 feat(readme): roles overview table with homepage/video backfill
Add cli.build.readme.overview (make readme-index / make docs) rendering a
roles overview in the root README: Name, Lifecycle, Homepage, Video,
Description, Integrated with, Install. New info-media lint requires
homepage + video in every invokable role's meta/info.yml (opt out via
'# nocheck: info-media' in meta/main.yml); backfilled 147 roles with
researched, adversarially-verified homepages and official videos.

Retire the 'unsupported' lifecycle value: fold it into 'eol' as a valid
live end-of-life status (shipped but untested/unmaintained) across the
stage lists, symbol glossary, allow-list lint and docs. Retag confluence,
jira, minio, phpldapadmin to eol and magento to pre-alpha.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 19:10:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 dd3d83148e feat(readme): show Ansible role dependencies in Cosmos with a gear marker
Roles declared under meta/main.yml dependencies now appear on both the
Dependencies and Dependents side, labelled with the glossary gear
symbol and connected by a fixed 1:1 edge; the legend explains it. All
role READMEs regenerated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:44:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a9fd37907b feat(readme): cardinality labels and a symbol legend on Cosmos diagrams
Edges now carry their relation: solid 1:1 for fixed consumption, dashed
0..1 for conditional (group-membership gated) consumption. Below every
diagram the generator renders a reading key for the edge kinds and the
node markers, with the glyphs drawn from the shared symbol glossary.
All role READMEs regenerated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:44:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 90b3845998 feat(readme): role README generator with Cosmos diagrams, Quick Setup, and lint
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).

The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.

test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.

Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.

Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 14:44:33 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7a6268fa96 refactor(deploy): replace skip with services modes
Per-role deploy-mode opt-out moves from meta/tests.yml 'skip:' to meta/services.yml <entity>.modes.{compose,swarm}.enabled on the primary entity. get_role_skip now derives from it; get_role_mode_enabled added. A new lint requires every role shipping templates/*compose*.yml.j2 to declare both mode flags. Migrates 107 services.yml and drops 11 skip-only tests.yml. Also removes the variant_status deployability gate so swarm runs one job per variant unconditionally (variant_status.py and deployable_variant_indices deleted, complexity model simplified). Adds the i8matix alias.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 15e7647d2f refactor(env): central TZ+LANG base included by every role env.j2
New roles/sys-svc-container/templates/env.j2 sets TZ={{ HOST_TIMEZONE }} and LANG=C.UTF-8. Every role env.j2 includes it at the top; it renders via stack_host_template, the same engine that resolves the includes in compose.yml.j2, so it works in both compose and swarm. TZ/LANG are removed from the individual role env.j2 so each value stays single-source.

New lint tests/lint/ansible/test_env_base_include.py verifies every role env.j2 includes the base and never re-sets a base-owned key (keys derived from the base file).

mattermost and gitea meta/volumes.yml drop the /etc/timezone host-path bind: swarm hard-rejects a task whose bind source is missing on the node (Debian-trixie nodes have no /etc/timezone). TZ now comes from the base env. /etc/localtime bind also dropped; the container timezone follows TZ.

Non-directive comments were removed from the touched env.j2 (banners, restatements, reference URLs, narration) so the changed files pass the comment-lint; only tool directives remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07c6ad8e7c test(e2e): scale every nav click + timeout literal for onion targets
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.

Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).

Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:35:53 +02:00
kevinveenbirkenbachandClaude Fable 5 0d13895616 fix(matomo): treat a stub config.ini.php as not installed
The container healthcheck's first page views make the installer write a
stub config before the install check runs, so the check reported
'installed', the retry clean-slate step (gated on the missing-config rc)
never fired, and every bootstrap retry inherited the previous attempt's
partial state until Matomo wedged in the CoreUpdater redirect (3x420s,
whole lab red). Installed now means the config carries a [database]
section, and the retry move-aside fires on any non-zero pre-check rc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 21:42:13 +02:00
kevinveenbirkenbachandClaude Fable 5 c51c0de9ba refactor(diagnostics): recursive workflow-level rescue snapshot, no in-play rescue blocks
Replace scripts/system/diagnostics/rescue.sh with utils/diagnostics/
container.py: one workflow call on failure captures the current level
(containers with logs+inspect, swarm services, journal, host resources,
in-play role dumps) and recurses itself through every DiD boundary via
self-copy, from the runner down to the innermost runtime (RESCUE_MAX_DEPTH).
compose and swarm now share an identical failure step; the swarm per-node
collector wrapper is gone. All 17 script-only Ansible rescue: blocks are
deleted and their bare block: wrappers inlined (plus a wordpress leftover);
the three justified blocks (postgres pg_hba, xwiki extensions, keycloak
cross-node recovery) keep their evidence tasks and now end in a fail task.
Two lints enforce the state: rescue: is forbidden without a
'# nocheck: rescue-<reason>' first-task marker, and a block without
rescue/always/when or another scoping key must be inlined.
sys-service-terminator hoists become to the include apply. Also move
routine/08_cleanup.sh to utils/clean/teardown.sh and topology/ to
utils/topology/ so routine/ holds only the sequenced flow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 094c02a69d feat(diagnostics): single SPOT rescue script with CI-uploaded snapshots
Every Ansible rescue: block used to carry ad-hoc per-role failure dumps that
drifted apart and captured different subsets. Replace them all with one
diagnostics script (scripts/system/diagnostics/rescue.sh) that snapshots
containers (full logs plus inspect per container), swarm services (ps plus
logs per service), journal, host resources, runtime info and stats into a
per-run folder and always exits 1 so the deploy still fails. A new lint
enforces the shape: every rescue ends with the script, at most one
nocheck-justified extra task before it (postgres pg_hba dump, xwiki
extension-repo probe), and a nocheck rescue-recovery exemption for the one
block that must genuinely recover: keycloak object updates, where a
cross-node reschedule needs a swarm-wide node re-resolve plus a delegate_to
change that only a rescue can express; its recovery is kept and now also
captures diagnostics before the terminal failure.

The snapshot folder path is a group_vars SPOT: DIR_RESCUE_DIAGNOSTICS and
DIR_VAR_LIB live in group_vars/all/05_paths.yml as plain strings; new env
handlers derive INFINITO_RESCUE_DIAGNOSTICS_DIR and INFINITO_DIR_VAR_LIB
from it for the generated .env, replacing the default.env entries, and the
play-level environment forwards the value to every task host so rescue.sh
needs no second default. CI uploads the folder like the inventory and
playwright artifacts: compose copies it out of the deploy container in
dedicated.sh, swarm captures per node (manager and both workers) into
node-named subfolders.

Deleting the ad-hoc dumps orphaned helper files (postgres inspect_container,
terminator docker_diagnostics, peertube diagnostics, matomo bootstrap debug,
xwiki and openresty rescue includes) and 14 role vars; all removed. Generic
pieces of those dumps (runtime version and info, stats, dmesg oom evidence,
swapon) moved into the SPOT script instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:04:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c6d9479d1b fix(matomo): recover the swarm bootstrap from a wedged installer
A flaked installer that created the superuser and site but never reached
the finished step leaves Matomo redirecting every request to CoreUpdater,
so the sidecar probe never passes. The old until-retry only force-restarted
the sidecar between attempts and never reset that partial state, so all
three attempts failed identically and the job died after ~29 minutes
(everything depending on matomo hung with it).

Drive the attempts through a loop over swarm_attempt.yml that, before each
retry, moves the partial config aside (config.ini.php.wedged-<ts>, never
deleted) and clears the cache so the installer runs fresh; the eraseAllTables
step handles leftover tables. The move is gated to the config-absent-at-entry
path and skips the whole attempt once a token is obtained, so a healthy
install is never touched. bootstrap_swarm_poll.sh anchors its token scrape
with --since so a token still buffered from the previous task is not returned
before the restarted installer emits its own, and surfaces a failed move
instead of masking it. The swarm abort message reports the swarm attempt
count.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-11 03:04:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f76bfcf531 refactor(svc-bkp-volume-2-local): rename role from svc-bkp-container-2-local
Rename the role, its test tree, the primary entity key
(volume-2-local), the derived vars (BKP_VOLUME_2_LOCAL_*,
SYS_SERVICE_BACKUP_VOLUME_2_LOCAL), the run-once fact, and every
group_names consumer flag, include path, doc, and lint reference. The
container_backup service key and the on-disk backup-docker-to-local
tree keep their names: both are contracts with existing consumer
entries, snapshot trees, and the baudolo pull tooling.

Operators must rename the svc-bkp-container-2-local inventory group and
disable the previous versioned units on already-provisioned hosts; the
pre-deploy snapshot glob derives its base from the new role name and
will not find them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:52:02 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3875eb2cf9 feat(variants): pin container_backup per matrix variant
Variant 0 pins the backup consumer flag enabled so the matrix baseline exercises the in-group branch; every other variant pins it disabled (shared: false carries the static-override suppression since the base value stays literally true). The matrix and wordpress variant entries move their services mapping to the list-item head so the static-override line parser resolves the override paths; yaml semantics are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:04:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4473edafb2 feat(roles): declare the container_backup consumer flag on stack roles with volumes
Every invokable stack role that declares named volumes in meta/volumes.yml now carries the standard container_backup consumer entry: enabled resolves via 'svc-bkp-container-2-local' in group_names, shared stays literally true since there is no per-consumer backup instance. Playwright/dynamic-flag/bond-baseline suppressions ride on the block per the established idioms; stale explanatory comments in the touched files are dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:04:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4c91df5de7 test(e2e): forbid raw page.goto, migrate all specs to Tor-resilient gotoOnion
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:59:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 35dee215c8 test(playwright): route all timeouts through resolveTimeout for onion-compat
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:38:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fac2de5fd3 style: apply make autoformat (whitespace only)
Indentation/whitespace normalization from make autoformat across role meta files. No semantic changes (git diff --ignore-all-space is empty for every file). Kept separate from the mailu notls-over-onion and Tor-router idempotency fixes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 11:24:55 +02:00
kevinveenbirkenbachandClaude Opus 4.8 77be21f5fd feat(tor): mark tor as non-playwright-gateable in domain services.yml
Onion routing is transport-layer with no persona/UI surface, so tor cannot be gated from the per-persona Playwright specs. Mark the tor block with '# nocheck: playwright-service-flag' in the 60 domain roles that ship a playwright.env.j2, per test_env_services_match's documented exemption for services that legitimately cannot be Playwright-gated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1d9d355bc9 feat(tor): add onion on/off variant coverage to domain roles
Each role's meta/variants.yml now pins the tor onion gate both ways so the matrix exercises it: variant 0 pins services.tor.enabled: true, every non-baseline variant pins enabled: false + shared: false (# nocheck: variants-static-override, since tor.shared is a static literal). This gives real test_coverage / test_bond_baseline coverage without the removed per-key code exemptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0a7164af48 feat(tor): per-role onion service block in meta/services.yml
Domain-serving roles reference the svc-net-tor provider via a tor: block. Consumers declare enabled (dynamic group_names gate) and shared: true (static — onion is always the shared node model, inline nocheck: dynamic-flag). The provider svc-net-tor additionally declares exclusive/primary defaults that consumers inherit via resolve_service_config unless overridden.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:16 +02:00
kevinveenbirkenbachandClaude Fable 5 1072ee27cc test(lint): catch argv-form container exec + mark legitimate bare-name uses
The container-exec-resolver lint only saw the inline shell form, so the
`command: argv: [container, exec, <name>]` list form (two separate lines)
slipped through, which is how checkmk's bare name reached swarm. Extend the
scanner to parse argv sequences via yaml.compose_all with node line numbers,
and annotate the nine legitimate bare-name execs (mariadb recovery container,
matomo compose-only debug, probe-gated nextcloud partner provisioning, pixelfed
runtime-resolved CID, wordpress launcher-managed discourse) with
# nocheck: container-exec-resolver.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 20:35:44 +02:00
kevinveenbirkenbachandClaude Opus 4.8 945fbccc1e refactor(swarm): move service_replicas into services.yml
Replace the per-role service_replicas Jinja variable with a per-service
replicas: field in meta/services.yml. deploy.yml.j2 resolves it via
resource_filter (with entity fallback), the same path already used for
cpus/mem, so services.yml is the single source of truth for replica
overrides.

- 13 web roles migrated to per-service (or entity-level) replicas: 1
- matomo bootstrap -> replicas: 0, mastodon migrate -> replicas: 1
- minio/seaweedfs provider entries -> replicas: 1, which also pins the
  central single-writer object-store providers instead of relying on the
  topology default
- service_replicas removed entirely; dropped its dead reference from the
  compose-structure lint allowlist and updated requirement 023 wording

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f5412682b0 refactor(container-port): derive healthcheck port, drop redundant defs
The sys-svc-container healthcheck templates (curl/wget/nc/tcp/http) now
derive container_port from services.<service_name>.ports.internal.http,
so a role no longer needs to set container_port only to feed the shared
healthcheck.

Add the container-port-set-unused lint that fails a role setting
container_port without using it elsewhere, then resolve every flagged
role: drop the redundant assignment where the derivation reproduces the
value, or mark it with nocheck where the port is non-http, a constant, or
keyed on a service the derivation cannot reach.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f2af680a2f test(lint): require container_port to read the internal-port config lookup
Add a lint that rejects hardcoded container_port literals and the
host-published ports.local port; container_port is the port the container
listens on, so it must resolve from
lookup('config', application_id, 'services.<svc>.ports.internal.<name>')
or a constant built that way. Bring 42 roles into compliance: move each
literal into services.<svc>.ports.internal.<name> and reference it via the
lookup, using entity_name/service_name where they equal the service key.
Values are preserved (mattermost/prometheus pointed at ports.local with the
same value as internal). Drop the dead container_port from fider and
prometheus, and the narrative comments the edits pulled into the comment
lint's changed-file scope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 00:20:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d4ee9a85ba refactor(compose): align templates to the service_name skeleton
Every compose.yml.j2 now declares services as '{{ service_name }}:' with
its '{% set %}' knobs on two-space lines above the key, includes at four
spaces and if/for tags aligned to the content they wrap. The layout
renders correctly because stack_host_template enables lstrip_blocks for
compose.yml.j2; indent(4, True) in the container base and '{%+' guards
in whitespace-sensitive scalars keep the shared fragments identical
under both render modes. A new lint test enforces the skeleton.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 02:36:58 +02:00
kevinveenbirkenbachandClaude Opus 4.8 73f9939702 fix(web-app-matomo): re-render consumer injections after the final compose flush
The consumer re-render loop included sys-front-inj-all under web-svc-* application_ids in the middle of the matomo pipeline; the shared sys-svc-compose handlers rebind to the most recent role instantiation, so matomo's compose-up notifies from 06/07 flushed against a consumer's instance dir (/opt/compose/logout) that does not exist when that service is outside the play's whitelist. Move the loop into 10_reinject_consumers.yml as the last step after 08_config, behind the final flush.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 96aa7784b6 feat(sys-svc-compose): scale all task timeouts via TIMEOUT_FACTOR lookup
Add a 'timeout' lookup plugin that returns round(base * TIMEOUT_FACTOR)
(group_vars default 1, so value-identical to the old literals). One knob
in group_vars stretches every wait in the stack for a slow uplink:
TIMEOUT_FACTOR: 2 doubles them all.

- plugins/lookup/timeout.py + unit tests (identity, factor scaling,
  string coercion, error paths).
- group_vars/all/17_resource.yml: TIMEOUT_FACTOR: 1.
- Convert all 34 literal 'timeout:' task keywords / module args across 25
  task+handler files to "{{ lookup('timeout', <base>) | int }}".
- tests/lint/repository/yaml/test_timeout_uses_lookup.py enforces the
  lookup in every task/handler timeout (opt-out: # nocheck: timeout-lookup).
- swarm.yml pre-deploy build handler: add the missing timeout (same
  network-build hang vector as compose.yml's Build compose).
- Drop 13 pre-existing narration comments in the touched files (comment
  lint enforces whole-file on any edited file).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:57:30 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7c6c56df70 fix(web-app-matomo): gate web-svc re-injection on the rendered body_filter Lua
The re-injection loop stat-checked each web-svc consumer's compose instance dir,
which proxy-only vhosts (web-svc-html) never have, so they were skipped and kept
a placeholder matomo token, failing the Playwright tracker assertion. Stat the
per-consumer body_filter Lua that sys-front-inj-all renders (the identical
lookup used to write it), so every deployed+injected consumer is re-rendered
with the real token. compose-dir-exists implies lua-exists for all six
candidates, so no consumer that was re-injected before is now skipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 00:26:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 674d26de3a feat(lint/role-readme): derive Credits author from meta author SPOT
Role README Credits now name galaxy_info.author (plain bold or wrapped
in a Markdown link), matched byte-for-byte against the role's own
meta/main.yml, instead of a hard-coded maintainer name. Regenerate all
245 role README Credits blocks to the new 'Implemented by ...' format.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 16:47:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9c988999b6 fix(web-app-matomo): accept 2xx/3xx in the healthcheck
A fresh, un-installed Matomo serves 302 on /index.php (redirect to the web installer), but the healthcheck demanded exactly 200, so it never became healthy: swarm reschedule-looped the container (0/1) and the bootstrap replicas piled up and raced on the database, so the installer never completed (rc=1) across every matomo-dependent bundle. Accept any 2xx/3xx.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 03:45:58 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07f51ab81d fix(matomo): deploy swarm bootstrap unscheduled, scale it up in the poll
In swarm the bootstrap service now deploys at replicas: 0 (created but unscheduled); the poll script scales it 0->1 with --force --replicas 1 so exactly one fresh installer run prints the token while the poll is already scraping its logs, removing the deploy-time-vs-poll-time race. Compose is unaffected (bootstrap stays profile-gated, run via run --rm).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 02:02:22 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e391c92f41 fix(web-app-matomo): make swarm bootstrap converge
chown the config dir to www-data before the swarm installer run so the PHP writability checks pass, and raise the healthcheck start_period to 180s to cover the slower swarm/NFS first boot.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 17:22:14 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6cddce1db0 style(meta): normalize YAML indentation and wrap long scalar
Autoformat (mbake/yamllint) re-indented sequence items under their mapping key
across the regenerated meta files and wrapped one long URL scalar in the matomo
inject task. Whitespace/formatting only, no semantic change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:36:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 23d1447566 refactor(ansible): add path_absolute lookup, replace playbook_dir joins
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:

  [playbook_dir, '<rel>'] | path_join   -> lookup('path_absolute', '<rel>')
  playbook_dir ~ '/<rel>'               -> lookup('path_absolute', '<rel>')
  {{ playbook_dir }}/<rel>              -> lookup('path_absolute', '<rel>')

Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.

Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.

Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:36:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bd0c70af17 feat(matomo): configurable tracking scope (sub default / root)
Add a per-deployment Matomo tracking scope, defaulting to per-subdomain:

- services.matomo.tracking.scope 'sub' (default) registers one Matomo site per
  consumer subdomain with an isolated cookie; 'root' keeps the prior behaviour
  of one shared site per registrable base domain with cross-subdomain linking.
- sys-front-inj-matomo derives matomo_site_domain (full host vs registrable
  base) and the shared matomo_site_url from it; getSitesId / addSite and the
  tracking snippet (setCookieDomain / setDomains / enableCrossDomainLinking) all
  follow the scope.
- The matomo Playwright env emits MATOMO_TRACKING_SCOPE; the SitesManager test
  matches the consumer's full host in 'sub' and its base domain in 'root'.
- variants.yml variant 0 sets scope=root so the matrix exercises both flavours
  (every other deploy runs the sub default).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 17:32:51 +02:00
kevinveenbirkenbachandClaude Opus 4.8 edb52b3c51 refactor(matomo): split playwright spec into logical files + shared module
Break the single 457-line playwright.spec.js into topic files discovered by the
harness testMatch "**/*.@(spec|test).js", mirroring the pihole / gitea layout:

- _shared.js            env vars, the setup hook, loginAsAdmin, attachDiagnostics,
                        hostOf / baseDomainOf and the MATOMO_TARGET_ROLES_JSON list
- playwright.spec.js    CSP + canonical-domain smoke (filename kept; harness asserts it)
- auth.spec.js          admin login/logout and biber denied at the admin surface
- tracking.spec.js      SitesManager site registration and per-consumer _paq injection
- persona-flows.spec.js guest / biber / admin universal-logout flows

Test logic is unchanged: same titles, selectors, regexes, timeouts and
assertions (verified by title parity and an adversarial diff against HEAD). The
only behavioural delta is that the extracted loginAsAdmin waits for the username
field to be visible before filling, which the SitesManager login previously
skipped (hardening only). The duplicated admin-login form-fill and the old
beforeEach are now the shared loginAsAdmin and setupMatomoPage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:37:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 833db8fd81 test(matomo): match SitesManager sites by base domain, not full host
The matomo injection registers one shared SitesManager site per registrable base
domain (addSite urls=proto://base_domain), so a subdomained consumer such as
logout.infinito.example is tracked under the infinito.example site, not its own.
The SitesManager test asserted the full canonical host as a substring of a site
URL, which never matched the base-domain site and failed in swarm.

Reduce the consumer's canonical_domain to its base domain (mirroring the regex in
sys-front-inj-matomo/vars/main.yml) and match it against each site's hostname
exactly. The exact match also drops the substring false positive, where a site at
evil-infinito.example would have satisfied infinito.example. Per-consumer tracker
presence stays covered by the separate per-role injection test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 16:37:56 +02:00