Commit Graph
111 Commits
Author SHA1 Message Date
kevinveenbirkenbach 763987d262 fix(checkmk): set the administrator password on a site that already exists
CMK_PASSWORD only applies while the site is being created. The site
volume outlives an inventory re-initialization, so the site kept the
password it was born with while the deployment minted a new one, and
every API call answered 401. The password is now written onto the
running site each deploy, which is what the shipped cmk-passwd tool
exists for.

401 becomes a transient status for the identity script, because the site
serves its API before it writes the admin htpasswd: from outside, a
starting site and a wrong password answer identically, and only the
retry budget tells them apart. That distinction is what showed the
password was wrong rather than late.
2026-08-10 11:20:37 +02:00
kevinveenbirkenbach 77896fba12 refactor(mcp): expose every role's MCP work at tasks/utils/mcp.yml
The reconciler addresses a client with tasks_from: utils/mcp.yml, which
only works when every role offers its MCP work at the same path. The
files were named per role, and on the two agents the work sat inline in
the entry point, where nothing but a full role run could reach it.

The filter docstring pointed at the pre-rename path and now names the
file that exists.
2026-08-09 23:57:43 +02:00
kevinveenbirkenbach 5ededade34 fix(mcp): pass the adapter base digest at every provider build site
Each provider builds its own sidecar from the shared Dockerfile and
supplies the base image itself, so renaming the build argument had to
reach all ten call sites. Without this the argument is undefined, the
FROM line resolves to a bare image with no digest, and no adapter builds.
2026-08-09 23:45:07 +02:00
kevinveenbirkenbach 09d4e496cc fix(mcp): build the adapter against the base image digest it declares
The metadata named infinito/mcp-adapter:1.0.0 with an all-zero digest,
an image that does not exist, while the build actually used
python:3.13-slim. Declaration and artifact disagreed, so the declaration
proved nothing. Both now name the real base image and pin it by digest,
which a tag cannot guarantee across a rebuild.

The adapter base moves from the provider's service version to
mcp.adapter.digest, alongside the tool specification and its checksum.
2026-08-09 23:20:58 +02:00
kevinveenbirkenbach b48346212e fix(networks): admit only the MCP clients a provider named
Being an MCP client was enough to join a provider's overlay, so a single
admission anywhere reached every provider that opened one. Attachment now
requires both that the role speaks as a client and that the provider
lists it in mcp.allowed_consumers.

consumer.kind accepts a list so one overlay can carry both its
service-flag consumers and its MCP clients, which is what the vector
store needs.
2026-08-09 22:55:43 +02:00
kevinveenbirkenbach c8ff5e8605 feat(services): register every MCP provider as a shared service
Ten provider roles served an MCP surface that other roles consume, but
were never declared as shared services, so the registry did not know
them. Declaring them pulls in the recursion contract every registry
service owes: seven roles move their entry tasks into tasks/00_core.yml
behind the run_once flag, so the service loader cannot include them a
second time.

pretix already had a run_once block, but set the flag last; the flag now
comes first, before the body it protects.
2026-08-09 22:22:08 +02:00
kevinveenbirkenbachandClaude Opus 5 a423bde6cc refactor(mcp): unify client entrypoints and read provider tokens through one lookup
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.

Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.

The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.

The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.

The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 21:06:31 +02:00
kevinveenbirkenbach 1c03f638be feat(mcp): serve MCP from a locked-down sidecar for ten more applications
Ten applications with no MCP server of their own get one without a line of code
inside them: qdrant, checkmk, fider, jellyfin, listmonk, pretix, prometheus,
snipe-it, zammad and libretranslate.

The sidecar is confined rather than trusted. It runs read-only, drops every
capability, forbids privilege escalation, publishes no port and joins only its
own provider's network, carrying neither that provider's alias nor the consumer
networks the provider itself joined. Its tool surface is a JSON contract, not
code: which tools exist, which upstream path each maps to and what shape the
arguments take. Adding a tool is a line of JSON, not a new image.

The image is built locally from the audited adapter source with the base image
as a build argument, so all ten share one reviewed build instead of ten upstream
dependencies. Each role provisions its own read-only identity in the language
its application speaks, and every adapter include is guarded by the role's MCP
flag, so switching it off leaves no container behind.
2026-08-06 20:44:16 +02:00
kevinveenbirkenbachandClaude Opus 5 ed0f5af9db feat(playwright): make a blocked persona state why
Blocking a persona removes the only end-to-end proof that a role's auth chain
works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has
always required each flag to carry a one-line nocheck stating the role contract
that justifies it, plus a matching paragraph in the role's README or TODO.
Nothing enforced it, so 89 flags across 46 roles carried neither.

The lint mirrors test_has_spec's shape and uses is_suppressed_at with
mode="line-above", so the marker must sit directly above its flag -- a blank
line between them breaks the association, which is what the persona helper's
reader will look at. A literal =false blocks nothing and is exempt; the
sso-conditional ternary is not, because it blocks in every variant that pins
services.sso.enabled false.

Writing 89 reasons turned out to be the point. Five roles audited in one pass
blocked for five entirely different mechanisms, all of which read as the same
shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than
the Keycloak username; phpMyAdmin authenticates as the MariaDB root account;
GitLab's seeded password belongs to the built-in root, not the administrator;
Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds
with an LDAP DN and a different secret. Further out: MediaWiki removes the form
that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false)
and bounces anonymous requests back into the IdP, so the unauthenticated
landing the contract asserts can never be observed; Mattermost Team Edition has
no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled
link; MinIO's console returns redirectRules: null and renders no SSO button at
all; XWiki's administrator is an XWikiUsers object with no password property.

Two flags turned out to be unjustifiable and are gone rather than annotated.
web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but
runGuestFlow does one anonymous goto and asserts no signed-in marker -- it
never touches the dashboard. A leftover from a helper version that entered
through the dashboard account menu. And web-app-mig's rationale claimed the
role is an in-app module of web-app-dashboard; it has its own compose stack,
healthcheck and canonical domain, and dashboard is merely a tile consumer.

Nine of the reasons say plainly that the block is not a role property but a gap
in the shared persona helpers -- decidim's sign-out is a data-method link the
generic click cannot activate, fider's SSO entry is a two-step modal, taiga,
wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit,
jenkins blocks unconditionally although its OIDC variant looks runnable, and
opentalk's own spec proves both personas work. That list did not exist before;
it is the actual yield here, not the 89 comment lines.

45 pre-existing comments came into the comment lint's scope because this sweep
touched their files -- it only inspects files that differ from HEAD. 28 of them
were the banner "Service gate flags rendered for closure.", every one sitting
above a line that is not a service gate flag at all, orphaned by the
alphabetical re-sort these templates once got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 20:36:39 +02:00
kevinveenbirkenbach 06bc856c32 refactor(mcp): make MCP a role topic instead of a service entry
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.

Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.

The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.

Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
2026-08-06 20:36:00 +02:00
kevinveenbirkenbachandClaude Opus 5 946b3769cb feat(svc-net-tor): let CI deploy the role in compose and swarm
modes.compose.enabled and modes.swarm.enabled were both false, and a
stack role is forced host=false, so all three test_<mode> columns were
false and the CI discovery filter (test_<mode> == true) dropped the role
from every matrix. It was resolved into the priority whitelist as a new
role and then silently filtered out again — svc-net-tor was only ever
exercised indirectly, as a dependency other roles pull in.

The flag is read solely by the discovery model and the README generator;
nothing in the deploy path consults it, so this has no runtime effect.

meta/categories.yml pins the whole net category to compose/swarm false,
which fits svc-net-wireguard-{core,plain,firewalled}: none of them ships
a compose template. svc-net-tor does, and is the only role in the
category that does, so it takes the category-modes exemption the lint
provides rather than bending the category default for three host roles.

Regenerated READMEs follow: svc-net-tor now carries its mode symbols in
every cosmos diagram that names it as a dependency.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 19:02:24 +02:00
kevinveenbirkenbach 342de43111 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	roles/svc-ai-ollama/tasks/01_manager_ops.yml
#	roles/web-app-decidim/templates/compose.yml.j2
#	roles/web-app-joomla/templates/compose.yml.j2
#	roles/web-app-matrix/files/flavor/ansible/runner/bootstrap.service
#	roles/web-app-matrix/templates/flavor/compose/services.yml.j2
#	roles/web-app-nextcloud/files/playwright/_shared.js
#	roles/web-svc-cdn/tasks/01_assets.yml
#	scripts/tests/deploy/swarm/utils/trigger_units.sh
#	scripts/tests/deploy/utils/filesystem/docker_dataroot.sh
2026-08-05 02:32:58 +02:00
kevinveenbirkenbachandClaude Opus 5 e3e81e816f refactor(healthcheck): declare probes in services.yml, render them from one lookup
Seven include templates under sys-svc-container decided how a container was
probed, and they read their inputs from template-local `{% set %}` scope.
That scope leaks: a value set for one service silently reached every service
below it, which is how taiga's front and gateway ended up probing a port
declared for a different service, and how bluesky's frontend probed a PDS
endpoint.

Probes now live in utils/docker/healthcheck.py, one class per flavor behind a
PROBES registry, and each service declares its own:

    healthcheck:
      flavor: curl
      path: health/ready
      start_period: 20m

`port_key` picks another entry under ports.internal for the services that
expose their probe elsewhere (keycloak on management, nextcloud talk on
signaling). Without a flavor the entry must carry an explicit test argv, so a
missing declaration fails loudly instead of rendering a probe against nothing.

Rendering HEAD's templates against the new output confirms 49 of 56 call
sites are byte-identical; the remaining seven are covered by an Ansible run.
Three probes change on purpose: erpnext's backend gains a check it never had,
bluesky's frontend drops the leaked PDS path, and msmtp_curl moves from a
hardcoded localhost:80 to the service's own port on 127.0.0.1 like every
other flavor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 04:05:38 +02:00
kevinveenbirkenbach e04c448cdd Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Three conflicts, none resolved by picking a side wholesale:

- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
  instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
  rc now counts as failure; the previous default(0) let an aborted Playwright
  run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
  which this branch's lint enforces, and adopted the incoming retry-inside-poll
  for the OIDC button plus its networkidle wait.
2026-07-30 15:38:15 +02:00
kevinveenbirkenbachandClaude Opus 5 028f4b85f4 chore(storage): set every image-backed min_storage to its image size plus 200MB
The declared values were guesses that ran up to 60x the size of the image they belong to: seaweedfs asked for 30GB against a 104MB image, nextcloud for 20GB against 353MB. Because min_storage feeds the storage guard, the runner packer and the sufficient-storage filter, those numbers made hosts look too small and pruned caches that deploys did not need to lose.

Every service that declares min_storage together with image and version now carries the measured compressed transfer size of that image plus 200MB of headroom. Sizes were read from the registries through utils.docker.registry.manifest_transfer_size, the same path the headroom test uses, so the declarations and the gate agree by construction. Values are written in MB throughout, matching the existing 500MB / 100MB / 50MB style.

119 declarations across 89 roles. Range: 206MB (memcached) to 3473MB (ollama); four exceed 1GB. The 84 min_storage entries that name no image are untouched, since there is nothing to measure them against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:08:51 +02:00
Alejandro Roman 871b00670a Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-27 11:25:42 +02:00
kevinveenbirkenbach 1a0af0ce09 Merge remote-tracking branch 'fork/feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	roles/sys-ctl-mtn-cert-deploy/tasks/main.yml
#	roles/sys-svc-compose-ca/tasks/main.yml
#	roles/web-app-dashboard/templates/javascript/oidc.js.j2
#	roles/web-app-mastodon/tasks/01_overrides.yml
#	roles/web-app-peertube/templates/compose.yml.j2
#	scripts/github/guide/compose_deploy.sh
#	scripts/github/guide/host_deploy.sh
#	scripts/make/alias.sh
#	tests/lint/ansible/roles/test_service_core_first_task_run_once.py
2026-07-24 04:30:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4039313266 refactor(tasks): enforce and apply a single task-file numbering scheme
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.

Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Fable 5 62a46e472f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 16:27:25 +02:00
kevinveenbirkenbachandClaude Fable 5 3125c5ac03 docs: regenerate role catalogs with the tor onion wiring
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 15:49:50 +02:00
Alejandro Roman 1915ec115b Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-22 12:16:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ea9ebe0de3 refactor(inventory): rename prod inventory to production
Standardize the private inventory dir on the fully spelled "production" (matching inventories/development). Update the role README template (SPOT) and regenerate 153 role READMEs plus the admin deploy guide; add inventories/production/ to .gitignore, propagated to .dockerignore via make fix-dockerignore.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:08:08 +02:00
Alejandro Roman ac101d2efa Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-21 10:00:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9ad8fa2971 feat(guide): default self_signed and test the whole README block
The guide deploys to a .localhost target, where letsencrypt cannot issue (Cloudflare DNS-01 needs a real domain). Default TLS_MODE=self_signed in the generated Production block so the guide uses the working self-signed provider. Document every deploy parameter as a top-level shell var (APP, HOST, TLS_MODE, SSH_PUBLIC_KEY) instead of an inline placeholder.

The compose guide now extracts and runs the whole README block, sourcing params from it (no more -e injection); HOST is forced to localhost. Skip checkout/login under ACT so the guide workflow is iterable locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:23:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 972596a758 fix(guide): document the administrator SSH key step and seed it in CI via --vars
The README Production block went provision -> deploy with no admin-key step, so user-administrator failed early (authorized_keys empty) for both the Guide CI jobs and any real user copying the README. Add a visible 'provision --vars' step that sets users.administrator.authorized_keys (merged into host_vars) in both Production variants of the role README template, regenerate all role READMEs, and replace the invisible CI payload splice with a visible substitution of the documented <your-ssh-public-key> placeholder.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 05:45:26 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Fable 5 17cd53609a feat(readme): merge the Production deploy into one docker command
The role README Production section now runs provision and deploy in a
single docker call: APP and HOST are shell variables passed via -e,
INVENTORY is the in-container SPOT for the duplicated paths, and the
redundant --vars-file (optional, default None) and --id (the inventory
is already narrowed by --include) flags are gone. All 153 role READMEs
regenerated from the template; test-run.yml executes exactly this block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 13:03:25 +02:00
kevinveenbirkenbachandClaude Fable 5 194671d90f feat(readme): uniform description schema and clearer role titles
Rewrite all 254 role descriptions to a uniform noun-phrase schema
(function-first, no leading verb, no trailing period, <=120 chars) and
enforce it via the extended description lint. Rename 12 role README H1
titles for clarity (e.g. Backup Container to Local -> Backup Docker
Volumes, CSS Service -> Corporate Design, NGINX Legal -> Imprint) and
regenerate their Cosmos and Quick Setup sections. Overview table
regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 01:11:48 +02:00
kevinveenbirkenbachandClaude Fable 5 da598a2d67 feat(readme): roles overview table with homepage/video backfill
Add cli.build.readme.overview (make readme-index / make docs) rendering a
roles overview in the root README: Name, Lifecycle, Homepage, Video,
Description, Integrated with, Install. New info-media lint requires
homepage + video in every invokable role's meta/info.yml (opt out via
'# nocheck: info-media' in meta/main.yml); backfilled 147 roles with
researched, adversarially-verified homepages and official videos.

Retire the 'unsupported' lifecycle value: fold it into 'eol' as a valid
live end-of-life status (shipped but untested/unmaintained) across the
stage lists, symbol glossary, allow-list lint and docs. Retag confluence,
jira, minio, phpldapadmin to eol and magento to pre-alpha.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 19:10:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 dd3d83148e feat(readme): show Ansible role dependencies in Cosmos with a gear marker
Roles declared under meta/main.yml dependencies now appear on both the
Dependencies and Dependents side, labelled with the glossary gear
symbol and connected by a fixed 1:1 edge; the legend explains it. All
role READMEs regenerated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:44:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a9fd37907b feat(readme): cardinality labels and a symbol legend on Cosmos diagrams
Edges now carry their relation: solid 1:1 for fixed consumption, dashed
0..1 for conditional (group-membership gated) consumption. Below every
diagram the generator renders a reading key for the edge kinds and the
node markers, with the glyphs drawn from the shared symbol glossary.
All role READMEs regenerated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:44:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 90b3845998 feat(readme): role README generator with Cosmos diagrams, Quick Setup, and lint
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).

The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.

test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.

Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.

Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 14:44:33 +02:00
kevinveenbirkenbachandClaude Opus 4.8 687759572f feat(lifecycle): promote checkmk, jellyfin, n8n and both registries to beta
Moves web-app-checkmk, web-app-jellyfin, web-app-n8n,
svc-registry-cache and svc-registry-docker from alpha to beta, pulling
them into the default CI test-deploy lifecycle envelope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 05:20:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7a6268fa96 refactor(deploy): replace skip with services modes
Per-role deploy-mode opt-out moves from meta/tests.yml 'skip:' to meta/services.yml <entity>.modes.{compose,swarm}.enabled on the primary entity. get_role_skip now derives from it; get_role_mode_enabled added. A new lint requires every role shipping templates/*compose*.yml.j2 to declare both mode flags. Migrates 107 services.yml and drops 11 skip-only tests.yml. Also removes the variant_status deployability gate so swarm runs one job per variant unconditionally (variant_status.py and deployable_variant_indices deleted, complexity model simplified). Adds the i8matix alias.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 15e7647d2f refactor(env): central TZ+LANG base included by every role env.j2
New roles/sys-svc-container/templates/env.j2 sets TZ={{ HOST_TIMEZONE }} and LANG=C.UTF-8. Every role env.j2 includes it at the top; it renders via stack_host_template, the same engine that resolves the includes in compose.yml.j2, so it works in both compose and swarm. TZ/LANG are removed from the individual role env.j2 so each value stays single-source.

New lint tests/lint/ansible/test_env_base_include.py verifies every role env.j2 includes the base and never re-sets a base-owned key (keys derived from the base file).

mattermost and gitea meta/volumes.yml drop the /etc/timezone host-path bind: swarm hard-rejects a task whose bind source is missing on the node (Debian-trixie nodes have no /etc/timezone). TZ now comes from the base env. /etc/localtime bind also dropped; the container timezone follows TZ.

Non-directive comments were removed from the touched env.j2 (banners, restatements, reference URLs, narration) so the changed files pass the comment-lint; only tool directives remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07c6ad8e7c test(e2e): scale every nav click + timeout literal for onion targets
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.

Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).

Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 11:35:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 5c6f656413 style(meta): normalize inline-comment spacing via autoformat
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 10:56:12 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f76bfcf531 refactor(svc-bkp-volume-2-local): rename role from svc-bkp-container-2-local
Rename the role, its test tree, the primary entity key
(volume-2-local), the derived vars (BKP_VOLUME_2_LOCAL_*,
SYS_SERVICE_BACKUP_VOLUME_2_LOCAL), the run-once fact, and every
group_names consumer flag, include path, doc, and lint reference. The
container_backup service key and the on-disk backup-docker-to-local
tree keep their names: both are contracts with existing consumer
entries, snapshot trees, and the baudolo pull tooling.

Operators must rename the svc-bkp-container-2-local inventory group and
disable the previous versioned units on already-provisioned hosts; the
pre-deploy snapshot glob derives its base from the new role name and
will not find them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:52:02 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3875eb2cf9 feat(variants): pin container_backup per matrix variant
Variant 0 pins the backup consumer flag enabled so the matrix baseline exercises the in-group branch; every other variant pins it disabled (shared: false carries the static-override suppression since the base value stays literally true). The matrix and wordpress variant entries move their services mapping to the list-item head so the static-override line parser resolves the override paths; yaml semantics are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:04:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4473edafb2 feat(roles): declare the container_backup consumer flag on stack roles with volumes
Every invokable stack role that declares named volumes in meta/volumes.yml now carries the standard container_backup consumer entry: enabled resolves via 'svc-bkp-container-2-local' in group_names, shared stays literally true since there is no per-consumer backup instance. Playwright/dynamic-flag/bond-baseline suppressions ride on the block per the established idioms; stale explanatory comments in the touched files are dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 23:04:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 00829757a0 feat(checkmk): pin to swarm manager so its node-local OMD volume survives
The checkmk OMD site cannot live on NFS (fcntl locks return EBADF) and must stay node-local, but an unpinned single-replica service loses that data on a swarm reschedule. Pin checkmk to the manager so the task never leaves the node holding its data, and teach the nfs-false-data-loss lint to accept a manager-pinned role as justification (dropping checkmk's now-redundant per-volume opt-out).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 02:19:09 +02:00
kevinveenbirkenbachandClaude Opus 4.8 22a391ac56 test(lint): require data-loss justification for nfs: false volumes
A node-local volume does not follow a swarm reschedule, so its data is lost on drain. Add a meta/volumes.yml lint that fails when a 'nfs: false' opt-out lacks both '# nocheck: nfs-false-data-loss' and a '# Reason:' arguing why that loss is acceptable. Justify the 10 existing entries (checkmk OMD site; gitlab repositories, logs, runtime state).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 01:03:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 485574bcae fix(checkmk): keep the OMD site store node-local (nfs:false)
The OMD site config store uses fcntl.flock, which returns EBADF over NFS, so in swarm the checkmk site never serves and oauth2-proxy 502s. Opt checkmk_data out of the swarm NFS rewrite so it stays node-local.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 00:03:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4c91df5de7 test(e2e): forbid raw page.goto, migrate all specs to Tor-resilient gotoOnion
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 20:59:38 +02:00
Alejandro Roman 2f7d0cd461 fix port collision after merge 2026-07-08 11:13:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 35dee215c8 test(playwright): route all timeouts through resolveTimeout for onion-compat
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 10:38:06 +02:00
kevinveenbirkenbachandClaude Fable 5 15f95f0725 fix(curl): timeout-hardened curl SPOT filter, lint, and fleet-wide rollout
A curl against an accepted-but-silent peer hangs forever: task retries
never fire because the first attempt never returns, and the CI job dies
at the 6h runner cut (proven by the mailu admin-API hang that took the
suitecrm/peertube swarm legs down). The new `curl` filter renders the
hardened base command ("{{ 30 | curl }}" -> curl -s --connect-timeout 5
--max-time 30); scaling composes with lookup('timeout').

test_curl_via_filter enforces it: bare curl invocations in role tasks,
templates and shipped scripts fail the lint with a pointer to the
filter. Plain .sh files, which cannot render Jinja, pass by pinning
--max-time inline; Docker healthchecks are exempt (their timeout field
caps them); `# nocheck: curl-timeout` opts out (catalogued in
suppression.md).

Convert all 34 existing invocations: mailu (replacing the short-lived
MAILU_CURL var), elasticsearch, checkmk, yourls (300s for the
schema-creating installer POST), the telegram alert script, and the
plain scripts of jellyfin, svc-runner, apt_repo, moodle (600s tarball)
and the mattermost provisioner. Mailu's user/token tasks additionally
carry task-level timeouts so a wedged container exec cannot stall the
play.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:37:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3a713529c5 fix(checkmk): move manager-side tasks behind an include gate
Worker play-hosts fataled with 'CHECKMK_HOST_NODE is undefined': the resolve_host_cid include was IS_STACK_HOST-gated, but the consuming wait/configure tasks sat at the top level of 01_core.yml and their delegate_to was evaluated on every host despite the task-level when. Move the three manager-side tasks verbatim into 02_manager_ops.yml included behind when: IS_STACK_HOST (the gitea/espocrm pattern), so workers never instantiate them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 02:37:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c2fa2da88e style: apply make autoformat (whitespace only)
Indentation/whitespace normalization emitted by make autoformat across role meta/services.yml, meta/variants.yml and a task file. No semantic changes (verified: git diff --ignore-all-space is empty for every file). Kept separate from the Tor egress-router feature.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 08:22:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 283108f802 test(lint): require a single-replica nocheck marker on every replicas:1 pin
Adds tests/lint/ansible/roles/test_single_replica_justified.py: a bare replicas:1 in meta/services.yml is flagged so a single-replica pin is never applied silently; services that can scale should carry no replicas key and let compose_replicas pick the topology default. Marks the pins on genuinely single-instance services (audit-verified as unscalable): checkmk OMD/RRD single-writer, erpnext websocket/scheduler/configurator sidecars, jenkins controller, listmonk campaign-manager singleton, mastodon db:prepare one-shot, minio single-node (no erasure), opencloud decomposedfs+NATS single-writer, seaweedfs single-node, taiga admin sidecar, ejabberd standalone Mnesia.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 01:14:29 +02:00