Commit Graph
24 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 4ac07edaea refactor(meta): move the info-media exemption to the file it speaks about
The marker names the reason a role has no upstream homepage or demo video, and
it sat in meta/main.yml because a comment-only info.yml parses to an empty
document, which the schema lint rejected outright. So the exemption lived one
file away from the two fields it explains, and a reader looking for the video
found neither the field nor the reason it was missing.

The schema lint now accepts an empty document exactly when it carries the
marker, and that is what allows the move. The media lint reads the marker from
info.yml rather than main.yml, and its failure message names that same file, so
the fix it suggests lands where the absent field would have gone.

A hundred and two roles follow. Every occurrence was a standalone comment on
the second line of main.yml, directly under the document start, so the move is
a line delete and an insert at the same offset. Seventy-three roles already had
an info.yml and gain the line; twenty-nine get one whose whole content is the
document start and the marker. All hundred and two reasons carry over verbatim,
including the thirty-five bespoke ones.

Nothing downstream sees a change. Of the four readers of meta/info.yml, three
coerce a null parse through `or {}` behind an isinstance guard, and
utils/cache/applications.py skips a falsy parse before assignment, so
applications.<role>.info stays absent for the twenty-nine exactly as it was
while the file did not exist.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-09 22:26:38 +02:00
kevinveenbirkenbachandClaude Opus 5 e16eb5728d test(lint): drop the minimal variant's exemption from the onion
72fd9c870 required the onion in every variant but exempted variant 1, on the
argument that the minimal round is the one place the stack runs without it. That
argument does not hold: the onion is part of the deployment under test, not a
feature bolted onto the full shape. A round that skips it is not a smaller
deployment, it is a different one.

So the exemption is gone and 95 variant-1 entries pin the onion on. Two
exclusions remain, both read from data rather than a comment: a role with no tor
block has no onion to pin, and a role whose meta/services.yml pins tor.enabled to
a literal false has already declared it has none.

That put the rule in conflict with test_coverage, which requires both polarities
of every dynamic flag: with tor on everywhere, the false side has no variant left
to live in. 94 roles take the exemption that contract already provides for
exactly this - "flags whose group-derived value is a hard invariant that variants
must never override", the same one 4e6ef2542 used for the seaweedfs frontend.
web-app-nextcloud needs none: its variant 8 pairs the clearnet-only PeerTube and
keeps the false polarity.

Measured, because the declaration is not what CI runs. assign_tor gives the onion
to every second capable entry with the run number as the parity switch, so more
capable variants do not mean proportionally more onion jobs: across five roles
and 21 entries the count goes 8 -> 10 on an even run and 9 on an odd one. The
clearnet path also survives - checkmk reports v1=false on run 0 and v1=true on
run 1, so both states are still covered across two consecutive runs. What the
94 markers give up is a declarative guarantee in variants.yml, not the coverage
itself.

The real cost is weight, not job count. web-app-nextcloud variant 1 goes from
3.09GB reservation / 7.27GB limit to 18.26GB / 43.01GB once svc-net-tor and its
closure join, so the cheap round is no longer cheap. Every variant stays under
the 32/64GB budget, and the same effect applies to the other 94 unmeasured.

Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles/meta/
variants plus the role-meta lints 39 passed. The rule was checked in both
directions - it reported the variant-1 offenders before the flips and none after.

Not measured: what the heavier rounds do to wall-clock, against swarm jobs capped
at 355 minutes and one that has hit that ceiling before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:02 +02:00
kevinveenbirkenbachandClaude Opus 5 ab72cff929 test(lint): require the onion in every variant, not just SSO ones
7a70d27b7 made a variant that switches SSO on also switch the onion on, because
that pairing is where the platform breaks in ways clearnet never shows. The same
argument reaches further than SSO: a Secure cookie is dropped, an https identity
provider refuses to embed, and crypto.subtle is undefined for any feature on a
non-secure origin, not only for a login. A variant that never runs the onion is
not testing the deployment users get.

So the rule now covers every variant of a role that declares a tor block.
Variant 1 stays exempt - it is the minimal, partner-free round every role
declares and the one place the stack is exercised without the onion at all - and
a role whose meta/services.yml pins tor.enabled to a literal false is skipped
outright, because that declaration already says the role has no onion and needs
no comment repeating it.

That found 137 variants. 101 were variant 1. Of the rest, 28 are the LDAP-only
auth-isolation rounds, which gain real coverage from running LDAP over the onion
and are flipped on here. Four cannot: svc-net-tor is the provider itself, and
nextcloud's variant 8 pairs the clearnet-only PeerTube. Both carry a marker
naming the reason; opentalk and coturn need none, since their services.yml pins
the literal false the rule now reads.

Flipping them exposed a contract that contradicted itself. test_auth_isolation
and test_non_baseline_explicit_disables demanded `shared: false` beside
`enabled: false`, while variants-static-override flagged that same line as a
static override - which is why all 133 of them carried a nocheck marker. The
demand only makes sense where `shared` is dynamic: a literal base value cannot
leak through deep-merge, so restating it protects nothing. Both contracts now
ask for it only in that case, and the 133 lines are gone.

The result is a stricter rule with fewer lines: 189 insertions against 193
deletions.

Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles 107
passed / 610 subtests. The rule is checked in both directions - it reported 137
offenders before the resolution and none after.

Not measured: 28 additional onion rounds cost matrix wall-clock, against swarm
jobs already capped at 355 minutes and one that has hit that ceiling before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:02 +02:00
kevinveenbirkenbachandClaude Opus 5 b2a0a317fa refactor(tor): loosen the onion bond to 0.5
Every variants.yml header states the rule the bond feeds: variant 0 carries each
tightly-coupled (bond>=1, same-host) service, while loosely-coupled (bond<1,
separate-host) partners are excluded from the closure and distributed across
later variants to keep every host under budget. tor was declared bond: 1 in all
101 consumers, so it had to sit in every variant-0 closure.

It does not belong there. The onion is a separate-host provider reached over the
node's hidden service, the same shape as the bridged partners that already carry
0.5, and f1dff50fb established both the reasoning and the precedent when it
dropped xwiki and bigbluebutton from 1 to 0.5: bridged partners with their own
role and their own spec, whose bond declaration was the outlier rather than
their placement.

This also unblocks the variant-level tor contract. Requiring a non-baseline
variant to pin the onion on while tor still counted as tightly coupled would
have pulled variant 0's closure requirements along with it.

Verified: tests/lint 336 passed / 307 subtests and tests/integration/roles 107
passed / 610 subtests, including test_resource_budget over every variant's
deduplicated footprint and the bond-aware test_order closure.

Not measured, and the honest cost: f1dff50fb's own caveat applies unchanged
here. Whether redistributing a service out of the baseline closure buys back
wall-clock is not established; role-runtimes has the per-role seconds and no run
has been made with this shape.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:01 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0a5cfadcef style(meta): apply make autoformat to cdn/css variants
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-19 11:29:13 +02:00
kevinveenbirkenbachandClaude Fable 5 194671d90f feat(readme): uniform description schema and clearer role titles
Rewrite all 254 role descriptions to a uniform noun-phrase schema
(function-first, no leading verb, no trailing period, <=120 chars) and
enforce it via the extended description lint. Rename 12 role README H1
titles for clarity (e.g. Backup Container to Local -> Backup Docker
Volumes, CSS Service -> Corporate Design, NGINX Legal -> Imprint) and
regenerate their Cosmos and Quick Setup sections. Overview table
regenerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 01:11:48 +02:00
kevinveenbirkenbachandClaude Fable 5 da598a2d67 feat(readme): roles overview table with homepage/video backfill
Add cli.build.readme.overview (make readme-index / make docs) rendering a
roles overview in the root README: Name, Lifecycle, Homepage, Video,
Description, Integrated with, Install. New info-media lint requires
homepage + video in every invokable role's meta/info.yml (opt out via
'# nocheck: info-media' in meta/main.yml); backfilled 147 roles with
researched, adversarially-verified homepages and official videos.

Retire the 'unsupported' lifecycle value: fold it into 'eol' as a valid
live end-of-life status (shipped but untested/unmaintained) across the
stage lists, symbol glossary, allow-list lint and docs. Retag confluence,
jira, minio, phpldapadmin to eol and magento to pre-alpha.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-17 19:10:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 672e93246b feat(deploy): add host deploy mode + matrix symbol view
Host lane for invokable non-stack roles (drivers, backup, wireguard, swapfile, ...). modes.host.enabled declared on 56 such roles; new lint test_role_host_mode enforces it. The complexity matrix gains a host column (not stack and modes.host.enabled), --deploy-mode host, and host bundles (one per variant); apps.sh accepts INFINITO_DEPLOY_MODE=host and filters host == true. New test-deploy-host.yml workflow (INFINITO_MAX_JOBS=10) deploys discovered host roles via the mode-agnostic ci/all.sh path, wired into ci-orchestrator (modes default 'swarm compose host').

Also: complexity -s/--symbol compact view (emoji-only headers, true/false as check/cross, lifecycle stage symbols, grouped legend with per-entry explanations, display-width aligned); stack column moved after host. scripts/meta/env/load.sh made zsh-compatible by delegating to bash and importing INFINITO_/PYTHON/VENV/PATH back.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c2fa2da88e style: apply make autoformat (whitespace only)
Indentation/whitespace normalization emitted by make autoformat across role meta/services.yml, meta/variants.yml and a task file. No semantic changes (verified: git diff --ignore-all-space is empty for every file). Kept separate from the Tor egress-router feature.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 08:22:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 77be21f5fd feat(tor): mark tor as non-playwright-gateable in domain services.yml
Onion routing is transport-layer with no persona/UI surface, so tor cannot be gated from the per-persona Playwright specs. Mark the tor block with '# nocheck: playwright-service-flag' in the 60 domain roles that ship a playwright.env.j2, per test_env_services_match's documented exemption for services that legitimately cannot be Playwright-gated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1d9d355bc9 feat(tor): add onion on/off variant coverage to domain roles
Each role's meta/variants.yml now pins the tor onion gate both ways so the matrix exercises it: variant 0 pins services.tor.enabled: true, every non-baseline variant pins enabled: false + shared: false (# nocheck: variants-static-override, since tor.shared is a static literal). This gives real test_coverage / test_bond_baseline coverage without the removed per-key code exemptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0a7164af48 feat(tor): per-role onion service block in meta/services.yml
Domain-serving roles reference the svc-net-tor provider via a tor: block. Consumers declare enabled (dynamic group_names gate) and shared: true (static — onion is always the shared node model, inline nocheck: dynamic-flag). The provider svc-net-tor additionally declares exclusive/primary defaults that consumers inherit via resolve_service_config unless overridden.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c6a8924525 fix(roles/meta): canonicalise galaxy_info.author spelling
Correct 'Birchenbach' -> 'Birkenbach' and normalise the non-breaking
hyphen (U+2011) to an ASCII hyphen in galaxy_info.author across 34
roles, so the author SPOT matches the regenerated README Credits.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 16:47:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6cddce1db0 style(meta): normalize YAML indentation and wrap long scalar
Autoformat (mbake/yamllint) re-indented sequence items under their mapping key
across the regenerated meta files and wrapped one long URL scalar in the matomo
inject task. Whitespace/formatting only, no semantic change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 20:36:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 59f22e3bda feat(meta): static web-svc scrape/track opt-out
Static-file (autoindex) and 301-redirect/alias web-svc vhosts route no request through lua-resty-prometheus and carry no user-facing HTML, so they have no per-vhost app=<role> scrape target and no _paq tracker page. scrape:false (asset, cdn, css, html, file, collabora) drops them from PROMETHEUS_TARGET_ROLES_JSON; track:false (asset, cdn, file) drops them from MATOMO_TARGET_ROLES_JSON. suppression.md catalogs both markers. Real apps stay scrape/track targets so monitoring is default-on.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 02:16:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6ce6780575 refactor(meta): split server.yml into csp/domains/networks.yml
Extract the csp, domains, and networks blocks out of roles/*/meta/server.yml
into dedicated meta files, each mapping to its own top-level applications key
via the recursive meta-walk:

  csp:      -> meta/csp.yml      -> applications.<app>.csp
  domains:  -> meta/domains.yml  -> applications.<app>.domains
  networks: -> meta/networks.yml -> applications.<app>.networks

Residual nginx/proxy keys (status_codes, client_max_body_size, locations,
ip6_enabled, config_upstream_url) stay in server.yml; an emptied server.yml is
deleted. 98 server.yml -> 249 new files, 74 deleted.

- Consumers repointed applications.<app>.server.{csp,domains,networks} ->
  applications.<app>.{csp,domains,networks} across templates, vars, tasks,
  lookups and filters.
- Infra readers migrated: canonical_domains_map, application_domain_index,
  build_ldap_role_entries, service-registry overlay discovery,
  roles_with_service, and the file-based readers utils/domains/list.py +
  utils/meta/scan.py (read the now-deleted server.yml directly).
- Detection + mapping: _META_TOPICS gains csp/domains/networks;
  ROLE_FILE_META_{CSP,DOMAINS,NETWORKS} constants + ROLE_FILES entries.
- New self-expiring lint guard (test_server_meta_no_split_fields) fails if a
  server.yml still carries a split field; auto-fails from 2026-12-28 as a
  delete reminder.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3fa781004d feat(roles): add bond integration-importance factor to shared services
`bond` is a 0..1 factor for how strongly a service is tied to the overall
integration: bond: 1 = always required, bond: 0 = omittable. Groundwork for
tooling that later scores service importance and decides what can be dropped.
Named `bond` (not `bound`) to avoid clashing with the resource bounds/limits.

Add test_service_bond.py (lint): every enabled + shared service must declare a
valid `bond` (number in 0..1). Seed `bond: 1` as a placeholder on all 638
enabled+shared service entries across 88 roles, to be refined later.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 06:03:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 50fe3bf623 feat(roles): declare compose resource limits for self-run services
Extend test_compose_resource_limits.py to enforce resource limits on every
container a role runs itself (not provided by another role via the service
registry): the primary entity needs min_storage/cpus/mem_reservation/mem_limit/
pids_limit; uncovered sidecar containers need the four resource keys. A service
is exempt only when another role covers it (e.g. discourse's postgres ->
svc-db-postgres, whose entity declares the limits), replacing the previous
blanket shared:True skip.

Declare the previously-missing limits across 18 roles: mostly self-run redis
cache sidecars (cpus "0.5" / 128m / 512m / 512), plus the minio mc client,
the coturn gateway, and the web-svc-* css/html/asset/cdn/file provider entities.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 06:03:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 99646f4001 fix(web-svc-css): pin cdn/matomo statically, keep only prometheus in matrix
web-svc-css is a pure redirect vhost to web-svc-cdn (tasks/01_core.yml).
That makes cdn a hard dependency, not a matrix dimension, and leaves the
redirect with no own page views for matomo to track. Hardcode both,
shrink variants.yml to the one truly optional observability peer
(prometheus). Fixes the round-1 Playwright failure where cdn was purged
under the css variant despite css being unreachable without it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 04:23:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f24b2d903b style(playwright,csp): strip explanatory comments per minimal-comments rule
Removes prose block comments, JSDoc headers, and multi-line nocheck
rationale paragraphs added in the previous two commits — keeps only
the bare nocheck markers (which carry actual exception semantics
the lint test enforces) and the unchanged function/test bodies. No
behaviour change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 01:09:02 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d03db7f73e test(web-svc-css): own consumer CSS-indexing assertion + close variants/playwright gates
Adds the web-svc-css provider Playwright spec that mirrors the
dashboard/prometheus SPOT-provider pattern: emits CSS_TARGET_ROLES_JSON
via roles_with_service('css') and asserts every consumer's canonical
surface indexes a <link rel="stylesheet"> pointing at the shared CSS
asset host (CSS_BASE_URL / CDN_BASE_URL). The baseline test exercises
CDN reachability first so per-consumer failures unambiguously point
at the consumer's sys-front-inj-css wiring, not at the provider.

Suppresses the per-role playwright env-services-match guard above each
consumer's `css:` key with the canonical `# nocheck:
playwright-service-flag — verified by web-svc-css provider spec`
marker. web-svc-css/meta/services.yml gets the self-provider nocheck
on `css:` and `provider role; no consumer surface to gate` nochecks on
`cdn:` / `matomo:`.

Adds css.{enabled,shared} true/false pinning to every variants.yml
that declared the dynamic flag without coverage so
test_every_dynamic_flag_has_true_and_false_variant passes. Unifies
the css service declaration across espocrm / mediawiki / openproject
(was static enabled: false, shared: true) so the matrix matches the
rest of the consumer fleet.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 01:09:02 +02:00
kevinveenbirkenbachandClaude Opus 4.7 40cde27eef fix(web-app-odoo): collapse generic biber+admin persona tests via PERSONA_*_BLOCKED
The generic `runBiberFlow` / `runAdminFlow` helpers in
roles/test-e2e-playwright/files/personas/{biber,admin}.js single-click
the role's "Sign In" link and expect the next URL to contain
`openid-connect/auth`. Odoo's login page is a two-step OIDC flow
(landing → username/password form with OIDC button below → click OIDC
button → Keycloak), which the helper does not navigate. The
`biber/administrator: app → universal logout` tests fail with
`no in-app logout control reachable on the current authenticated
surface` because the test lands back on the public Odoo landing page.

The role already ships bespoke biber+admin coverage:

  - `odoo: admin sso login, verify ui, logout`
  - `odoo: biber sso login, verify ui, logout`

Both drive the full OIDC flow through Odoo's own login surface and
both have been passing in CI. The generic persona tests are
redundant for Odoo and add nothing on top.

Set `PERSONA_BIBER_BLOCKED=true` / `PERSONA_ADMINISTRATOR_BLOCKED=true`
in `templates/playwright.env.j2` so the generic helpers cleanly
`test.skip(...)` per the documented escape clause.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 12:50:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 96e6a3daa5 feat(web-svc-css): introduce canonical owner of the css service flag
Decouples the CSS service concept from its underlying transport
(currently `web-svc-cdn`) so the flag becomes opt-in via inventory
group membership instead of staying literal-`true` everywhere.

- `roles/web-svc-css/`: new role. Marker / wrapper role that owns
  the `css` entry in the service registry, declares `web-svc-cdn` as
  the upstream that serves the actual bytes, and 301-redirects its
  own canonical domain to the cdn domain so health probes return a
  valid response.
- `roles/web-svc-cdn/meta/services.yml`: drop the `css` block — the
  canonical owner is now web-svc-css; cdn keeps `javascript`.
- 51 consumer roles (web-app-* and web-svc-{coturn,file,html,
  libretranslate,logout,onlyoffice}) migrated their `css:` block from
  literal `{ enabled: true, shared: true }` to dynamic
  `{ enabled: "{{ 'web-svc-css' in group_names }}", shared: ... }`,
  with matching matrix coverage added to each role's
  `meta/variants.yml` (true-pinning + false-pinning entries) so the
  dynamic-flag variant lint stays green.
- 41 `# nocheck: playwright-service-flag` rationale strings updated
  from the now-incorrect "canonical:cdn alias of static-asset
  provider" to "css consumer, no Playwright surface (provider:
  web-svc-css)".

Bundled refactor: `roles/web-svc-asset/tasks/01_core.yml` now
includes `web-opt-rdr-domains` directly with a single-entry mapping
to redirect its canonical domain to web-svc-file, instead of having
that wiring live in the global `tasks/stages/01_constructor.yml`
constructor. The `add_redirect_if_group` filter
(`plugins/filter/redirect_filters.py`) and its unit test are dropped
as dead code — both call sites are now role-local.

Dashboard's consumption of the new pattern is intentionally not part
of this commit and follows separately.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 02:20:37 +02:00