Commit Graph
340 Commits
Author SHA1 Message Date
Alejandro Roman 7332bd66ae Merge upstream/main into feature/role-stalwart-mail-swarm 2026-08-18 01:42:23 +04:00
kevinveenbirkenbachandClaude Opus 5 c55f508e05 feat(readme): draw mandatory and disabled service relations
The cosmos generator linked a consumer to its provider only through an
'<role>' in group_names reference or an explicit provides: name. A hard
dependency has neither -- pinning the provider on unconditionally is
exactly what removes the Jinja reference the generator was reading -- so
web-svc-asset's bond to web-svc-file vanished from the diagram the moment
it became binding. Match a service key against the provider's own primary
key from the registry, in both directions.

Services the role turns off drew a solid 1:1 edge, claiming a mandatory
relationship for something that carries nothing at deploy time. They now
render as a red 0..0 edge; edge merging gained a real precedence so a
provider consumed through two keys stays live when only one is off.

Named colour, not hex: a '#' opens an entity code in the mermaid lexer and
turns the whole diagram into a parse error on GitHub.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:31 +02:00
kevinveenbirkenbachandClaude Opus 5 a46348b6cf refactor(handlers): bind notify to a listen topic, not to a name
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.

Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.

Freed from the binding, the names take the emoji prefix every other task line
already has.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 03:38:42 +02:00
kevinveenbirkenbach a7c7e8393d Merge remote-tracking branch 'origin/main' into feature/ai-agent-employees 2026-08-15 14:07:15 +02:00
kevinveenbirkenbach 4f673030ce Merge branch 'main' into feature/svc-net-tor 2026-08-15 06:09:46 +02:00
kevinveenbirkenbachandClaude Opus 5 e16eb5728d test(lint): drop the minimal variant's exemption from the onion
72fd9c870 required the onion in every variant but exempted variant 1, on the
argument that the minimal round is the one place the stack runs without it. That
argument does not hold: the onion is part of the deployment under test, not a
feature bolted onto the full shape. A round that skips it is not a smaller
deployment, it is a different one.

So the exemption is gone and 95 variant-1 entries pin the onion on. Two
exclusions remain, both read from data rather than a comment: a role with no tor
block has no onion to pin, and a role whose meta/services.yml pins tor.enabled to
a literal false has already declared it has none.

That put the rule in conflict with test_coverage, which requires both polarities
of every dynamic flag: with tor on everywhere, the false side has no variant left
to live in. 94 roles take the exemption that contract already provides for
exactly this - "flags whose group-derived value is a hard invariant that variants
must never override", the same one 4e6ef2542 used for the seaweedfs frontend.
web-app-nextcloud needs none: its variant 8 pairs the clearnet-only PeerTube and
keeps the false polarity.

Measured, because the declaration is not what CI runs. assign_tor gives the onion
to every second capable entry with the run number as the parity switch, so more
capable variants do not mean proportionally more onion jobs: across five roles
and 21 entries the count goes 8 -> 10 on an even run and 9 on an odd one. The
clearnet path also survives - checkmk reports v1=false on run 0 and v1=true on
run 1, so both states are still covered across two consecutive runs. What the
94 markers give up is a declarative guarantee in variants.yml, not the coverage
itself.

The real cost is weight, not job count. web-app-nextcloud variant 1 goes from
3.09GB reservation / 7.27GB limit to 18.26GB / 43.01GB once svc-net-tor and its
closure join, so the cheap round is no longer cheap. Every variant stays under
the 32/64GB budget, and the same effect applies to the other 94 unmeasured.

Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles/meta/
variants plus the role-meta lints 39 passed. The rule was checked in both
directions - it reported the variant-1 offenders before the flips and none after.

Not measured: what the heavier rounds do to wall-clock, against swarm jobs capped
at 355 minutes and one that has hit that ceiling before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:02 +02:00
kevinveenbirkenbachandClaude Opus 5 ab72cff929 test(lint): require the onion in every variant, not just SSO ones
7a70d27b7 made a variant that switches SSO on also switch the onion on, because
that pairing is where the platform breaks in ways clearnet never shows. The same
argument reaches further than SSO: a Secure cookie is dropped, an https identity
provider refuses to embed, and crypto.subtle is undefined for any feature on a
non-secure origin, not only for a login. A variant that never runs the onion is
not testing the deployment users get.

So the rule now covers every variant of a role that declares a tor block.
Variant 1 stays exempt - it is the minimal, partner-free round every role
declares and the one place the stack is exercised without the onion at all - and
a role whose meta/services.yml pins tor.enabled to a literal false is skipped
outright, because that declaration already says the role has no onion and needs
no comment repeating it.

That found 137 variants. 101 were variant 1. Of the rest, 28 are the LDAP-only
auth-isolation rounds, which gain real coverage from running LDAP over the onion
and are flipped on here. Four cannot: svc-net-tor is the provider itself, and
nextcloud's variant 8 pairs the clearnet-only PeerTube. Both carry a marker
naming the reason; opentalk and coturn need none, since their services.yml pins
the literal false the rule now reads.

Flipping them exposed a contract that contradicted itself. test_auth_isolation
and test_non_baseline_explicit_disables demanded `shared: false` beside
`enabled: false`, while variants-static-override flagged that same line as a
static override - which is why all 133 of them carried a nocheck marker. The
demand only makes sense where `shared` is dynamic: a literal base value cannot
leak through deep-merge, so restating it protects nothing. Both contracts now
ask for it only in that case, and the 133 lines are gone.

The result is a stricter rule with fewer lines: 189 insertions against 193
deletions.

Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles 107
passed / 610 subtests. The rule is checked in both directions - it reported 137
offenders before the resolution and none after.

Not measured: 28 additional onion rounds cost matrix wall-clock, against swarm
jobs already capped at 355 minutes and one that has hit that ceiling before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:02 +02:00
kevinveenbirkenbachandClaude Opus 5 b2a0a317fa refactor(tor): loosen the onion bond to 0.5
Every variants.yml header states the rule the bond feeds: variant 0 carries each
tightly-coupled (bond>=1, same-host) service, while loosely-coupled (bond<1,
separate-host) partners are excluded from the closure and distributed across
later variants to keep every host under budget. tor was declared bond: 1 in all
101 consumers, so it had to sit in every variant-0 closure.

It does not belong there. The onion is a separate-host provider reached over the
node's hidden service, the same shape as the bridged partners that already carry
0.5, and f1dff50fb established both the reasoning and the precedent when it
dropped xwiki and bigbluebutton from 1 to 0.5: bridged partners with their own
role and their own spec, whose bond declaration was the outlier rather than
their placement.

This also unblocks the variant-level tor contract. Requiring a non-baseline
variant to pin the onion on while tor still counted as tightly coupled would
have pulled variant 0's closure requirements along with it.

Verified: tests/lint 336 passed / 307 subtests and tests/integration/roles 107
passed / 610 subtests, including test_resource_budget over every variant's
deduplicated footprint and the bond-aware test_order closure.

Not measured, and the honest cost: f1dff50fb's own caveat applies unchanged
here. Whether redistributing a service out of the baseline closure buys back
wall-clock is not established; role-runtimes has the per-role seconds and no run
has been made with this shape.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:01 +02:00
infinito-nexus-updater[bot] cefe0ba48b update(images): update Docker image versions (web-app-gitea) 2026-08-15 02:07:53 +00:00
kevinveenbirkenbachandClaude Opus 5 642055a20a style(e2e): drop the timeout argument isVisible discards
Playwright declares the timeout option on locator.isVisible() and isHidden() as deprecated and ignored: both return immediately and never wait. Passing resolveTimeout(10_000) to one therefore changes nothing at runtime and only tells the reader a wait happened that never did. That reading is what let 7538b8a0b conclude two genuine call sites were fine, and what hid the false negative b97cf8888 had to fix.

Sixty call sites across fifty-eight role-local specs lose the argument and nothing else. Every added line in this commit is a bare isVisible(); a grep for added lines that are not one returns zero, so a behaviour change cannot have entered here. Files whose diff also alters waiting semantics are deliberately not part of this commit.

A repository lint added in b97cf8888 keeps the construct out. It matches isVisible and isHidden only, because isChecked, isDisabled, isEditable and isEnabled take a real timeout.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 13:49:18 +02:00
kevinveenbirkenbachandClaude Opus 5 e3caa074b2 fix(mcp): set the service account password on every run, not only at creation
Four roles created their MCP service account and never touched its password
again, while the same play authenticates as that account moments later. A
deployment whose declared password moved on therefore failed at the login,
not at the point the drift was introduced.

gitea now runs 'admin user change-password' before the token dance. The
--must-change-password=false flag is mandatory rather than cosmetic: at the
pinned v1.27.1 the flag defaults to true, so omitting it locks the account out
of the very API call that follows.

nextcloud resets through 'occ user:resetpassword --password-from-env' before
minting the app token that verifies the password.

snipe-it compared nothing at all - its docstring promised to pin the password
while the assignment sat inside the create branch. It now rehashes only when
Hash::check says the stored hash no longer matches, so the changed flag stays
honest.

home assistant swallowed a failed credential creation for accounts that already
existed and then logged in with them. It now calls admin_change_password and
falls back to creation only when the reply is credentials_not_found.

jellyfin and zammad already converged; wordpress, gitlab and moodle never
authenticate with this password and are left alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 21:34:30 +02:00
kevinveenbirkenbachandClaude Opus 5 f713ba8350 refactor(credentials): address them at secrets.credentials, like every other meta file
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.

The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.

--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.

Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.

tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.

Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 16:51:56 +02:00
kevinveenbirkenbachandClaude Opus 5 28475455db feat(mcp): give every provider its own service account from the registry
Nine roles reached their MCP surface as the deployment administrator or with
an account password kept beside the account in credentials. A Moodle site
admin bypasses every capability check, a Nextcloud app password is not
scopable, a Gitea token reads every private repository, and a GitLab token on
root reads every private project. Each role now declares its own principal in
meta/users.yml and provisions it, so the exposed surface is what the service
role permits. jellyfin, zammad and snipe-it kept their account but took its
password from a credential named token; they read the registry too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 15:44:12 +02:00
kevinveenbirkenbachandClaude Opus 5 e96c752549 Merge fix/playwright/env-template-gate into feature/svc-net-tor
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.

Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.

Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:

- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
  Same value, so nothing breaks today; editing the documented one would have
  silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
  the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
  rename never saw it. Its credential would not have resolved and the role
  would have deployed with an unusable administrator account.

The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.

make quality green on the merge result.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 14:20:19 +02:00
kevinveenbirkenbachandClaude Opus 5 ef5ea7cb55 refactor(roles): name the credential file meta/secrets.yml
meta/schema.yml held credential definitions and their runtime values, but the
name said 'schema' and read like structural metadata next to services.yml and
volumes.yml. Nothing in the file is a schema in the sense the other meta files
use the word, and a contributor had to open it to learn it holds secrets.

62 role files renamed. ROLE_FILE_META_SCHEMA becomes ROLE_FILE_META_SECRETS and
keeps its role as the SPOT the path is built from.

Two readers bypassed that constant and spelled the name themselves:
utils/cache/applications.py built meta_dir / 'schema.yml' twice, and
plugins/lookup/application_gid.py carries the filename in the marker list it
detects application roles by. Left behind, the rename would not have raised —
credentials would have resolved empty and the role would have stopped counting
as an application.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 12:08:06 +02:00
kevinveenbirkenbachandClaude Opus 5 43d8d36a0a fix(web-app-gitea): converge the database config through the environment
The image seeds app.ini from the flat DB_* variables only while the file is
absent, so a persisted install kept whatever credentials it was first given and
the role patched the current ones back in with sed. environment-to-ini runs
unconditionally on every start and overwrites the [database] keys in place, so
declaring them as GITEA__database__* converges the config on its own.

The restart that makes the new values take effect already comes from rendering
env.j2, whose handler the backend stack flushes before the manager tasks run,
so the patch task loses its last reason to exist.

Drops 02_database.yml and renumbers the following task files to keep the
sequence gapless.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:49:59 +02:00
kevinveenbirkenbach efaa9b1c49 Merge remote-tracking branch 'origin/main' into feature/ai-agent-employees
# Conflicts:
#	roles/web-app-flowise/meta/services.yml
2026-08-10 00:14:48 +02:00
kevinveenbirkenbach 13f2c406e8 fix(mcp): follow the renamed MCP task path in the manager-ops entry points
Three roles reach their MCP work from a manager-ops file rather than
from the role entry point, and the rename missed them, leaving them
including task files that no longer exist.
2026-08-10 00:06:32 +02:00
kevinveenbirkenbach b48346212e fix(networks): admit only the MCP clients a provider named
Being an MCP client was enough to join a provider's overlay, so a single
admission anywhere reached every provider that opened one. Attachment now
requires both that the role speaks as a client and that the provider
lists it in mcp.allowed_consumers.

consumer.kind accepts a list so one overlay can carry both its
service-flag consumers and its MCP clients, which is what the vector
store needs.
2026-08-09 22:55:43 +02:00
kevinveenbirkenbachandClaude Opus 5 a423bde6cc refactor(mcp): unify client entrypoints and read provider tokens through one lookup
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.

Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.

The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.

The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.

The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 21:06:31 +02:00
kevinveenbirkenbach 25f89cf8a7 Merge branch 'main' of github.com:infinito-nexus/core into fix/playwright/env-template-gate
# Conflicts:
#	package.json
2026-08-08 18:48:34 +02:00
kevinveenbirkenbach 2789d10287 Merge branch 'main' into feature/svc-net-tor 2026-08-08 11:16:05 +02:00
infinito-nexus-updater[bot] 6939b174ac update(images): update Docker image versions (web-app-gitea) 2026-08-08 06:04:03 +00:00
kevinveenbirkenbach 3e8a4c6cb3 feat(mcp): prove the contract and pin token ownership on the native servers
The eight roles whose MCP server is built into the application now end their MCP
step with the shared contract probe. It builds URL and Authorization header from
the role's own declaration, calls initialize and tools/list, invokes one
deterministic read tool per role, and compares the served surface against the
declared allowlist. It cannot be a Playwright test: the check needs the real
provider credential and a Playwright run would capture it in its trace.

Eleven sites wrote the MCP token into the store under the administrator's
identity while their metadata declared an owner of their own. The schema lint
forbids administrator as a credential owner, so declaration and storage
disagreed, and the probe would have looked for the bearer under the declared
owner and found nothing. They now read the declared owner.
2026-08-06 20:40:01 +02:00
kevinveenbirkenbachandClaude Opus 5 ed0f5af9db feat(playwright): make a blocked persona state why
Blocking a persona removes the only end-to-end proof that a role's auth chain
works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has
always required each flag to carry a one-line nocheck stating the role contract
that justifies it, plus a matching paragraph in the role's README or TODO.
Nothing enforced it, so 89 flags across 46 roles carried neither.

The lint mirrors test_has_spec's shape and uses is_suppressed_at with
mode="line-above", so the marker must sit directly above its flag -- a blank
line between them breaks the association, which is what the persona helper's
reader will look at. A literal =false blocks nothing and is exempt; the
sso-conditional ternary is not, because it blocks in every variant that pins
services.sso.enabled false.

Writing 89 reasons turned out to be the point. Five roles audited in one pass
blocked for five entirely different mechanisms, all of which read as the same
shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than
the Keycloak username; phpMyAdmin authenticates as the MariaDB root account;
GitLab's seeded password belongs to the built-in root, not the administrator;
Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds
with an LDAP DN and a different secret. Further out: MediaWiki removes the form
that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false)
and bounces anonymous requests back into the IdP, so the unauthenticated
landing the contract asserts can never be observed; Mattermost Team Edition has
no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled
link; MinIO's console returns redirectRules: null and renders no SSO button at
all; XWiki's administrator is an XWikiUsers object with no password property.

Two flags turned out to be unjustifiable and are gone rather than annotated.
web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but
runGuestFlow does one anonymous goto and asserts no signed-in marker -- it
never touches the dashboard. A leftover from a helper version that entered
through the dashboard account menu. And web-app-mig's rationale claimed the
role is an in-app module of web-app-dashboard; it has its own compose stack,
healthcheck and canonical domain, and dashboard is merely a tile consumer.

Nine of the reasons say plainly that the block is not a role property but a gap
in the shared persona helpers -- decidim's sign-out is a data-method link the
generic click cannot activate, fider's SSO entry is a two-step modal, taiga,
wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit,
jenkins blocks unconditionally although its OIDC variant looks runnable, and
opentalk's own spec proves both personas work. That list did not exist before;
it is the actual yield here, not the 89 comment lines.

45 pre-existing comments came into the comment lint's scope because this sweep
touched their files -- it only inspects files that differ from HEAD. 28 of them
were the banner "Service gate flags rendered for closure.", every one sitting
above a line that is not a service gate flag at all, orphaned by the
alphabetical re-sort these templates once got.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 20:36:39 +02:00
kevinveenbirkenbach 06bc856c32 refactor(mcp): make MCP a role topic instead of a service entry
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.

Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.

The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.

Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
2026-08-06 20:36:00 +02:00
kevinveenbirkenbachandClaude Opus 5 946b3769cb feat(svc-net-tor): let CI deploy the role in compose and swarm
modes.compose.enabled and modes.swarm.enabled were both false, and a
stack role is forced host=false, so all three test_<mode> columns were
false and the CI discovery filter (test_<mode> == true) dropped the role
from every matrix. It was resolved into the priority whitelist as a new
role and then silently filtered out again — svc-net-tor was only ever
exercised indirectly, as a dependency other roles pull in.

The flag is read solely by the discovery model and the README generator;
nothing in the deploy path consults it, so this has no runtime effect.

meta/categories.yml pins the whole net category to compose/swarm false,
which fits svc-net-wireguard-{core,plain,firewalled}: none of them ships
a compose template. svc-net-tor does, and is the only role in the
category that does, so it takes the category-modes exemption the lint
provides rather than bending the category default for three host roles.

Regenerated READMEs follow: svc-net-tor now carries its mode symbols in
every cosmos diagram that names it as a dependency.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 19:02:24 +02:00
kevinveenbirkenbach 342de43111 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	roles/svc-ai-ollama/tasks/01_manager_ops.yml
#	roles/web-app-decidim/templates/compose.yml.j2
#	roles/web-app-joomla/templates/compose.yml.j2
#	roles/web-app-matrix/files/flavor/ansible/runner/bootstrap.service
#	roles/web-app-matrix/templates/flavor/compose/services.yml.j2
#	roles/web-app-nextcloud/files/playwright/_shared.js
#	roles/web-svc-cdn/tasks/01_assets.yml
#	scripts/tests/deploy/swarm/utils/trigger_units.sh
#	scripts/tests/deploy/utils/filesystem/docker_dataroot.sh
2026-08-05 02:32:58 +02:00
kevinveenbirkenbachandClaude Opus 5 e355bbdb8b feat(mcp): gate Open WebUI tool servers on the serving role's mcp group
Eight server roles declare an `mcp` RBAC role, so the platform mints
roles/<app>/mcp and the administrator joins it. Open WebUI now honours those
groups instead of flattening them to user/admin, and each MCP tool server is
restricted to the group of the role that serves it.

The grant cannot be rendered: Open WebUI matches it against a group id that
insert_new_group mints as a fresh UUID, so tasks/01_mcp.yml resolves the group
after the container is up and writes the grant through the admin API, the way
every instance-minted credential in this repository is provisioned.

Entries render disabled. An empty access_grants is not 'nobody', Open WebUI
reads it as every administrator, and ENABLE_PERSISTENT_CONFIG=false makes the
env authoritative again on every start. Enabling only together with the grant
keeps the fallback at no access rather than too much.

The provisioning account is the role-internal openwebui-api-bot, never a
platform identity: OAUTH_MERGE_ACCOUNTS_BY_EMAIL ships disabled and an OIDC
login whose email a local account already holds is refused, so seeding a
person's address would lock them out.

Known limits are documented rather than claimed away: a bare restart leaves MCP
off until the next deploy, a user's roles list carries no application context so
roles: [mcp] grants every deployed mcp group at once, and Open WebUI drops a
stale membership only while the groups claim is non-empty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 22:06:47 +02:00
kevinveenbirkenbach 255ee13f60 Merge branch 'feature/swarm-nfs' into feature/ai-agent-employees 2026-08-03 04:21:37 +02:00
kevinveenbirkenbachandClaude Opus 5 e3e81e816f refactor(healthcheck): declare probes in services.yml, render them from one lookup
Seven include templates under sys-svc-container decided how a container was
probed, and they read their inputs from template-local `{% set %}` scope.
That scope leaks: a value set for one service silently reached every service
below it, which is how taiga's front and gateway ended up probing a port
declared for a different service, and how bluesky's frontend probed a PDS
endpoint.

Probes now live in utils/docker/healthcheck.py, one class per flavor behind a
PROBES registry, and each service declares its own:

    healthcheck:
      flavor: curl
      path: health/ready
      start_period: 20m

`port_key` picks another entry under ports.internal for the services that
expose their probe elsewhere (keycloak on management, nextcloud talk on
signaling). Without a flavor the entry must carry an explicit test argv, so a
missing declaration fails loudly instead of rendering a probe against nothing.

Rendering HEAD's templates against the new output confirms 49 of 56 call
sites are byte-identical; the remaining seven are covered by an Ansible run.
Three probes change on purpose: erpnext's backend gains a check it never had,
bluesky's frontend drops the leaked PDS path, and msmtp_curl moves from a
hardcoded localhost:80 to the service's own port on 127.0.0.1 like every
other flavor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 04:05:38 +02:00
kevinveenbirkenbachandClaude Opus 5 3036b7ba19 feat(gitea): serve MCP through the project-owned sidecar
Gitea ships no MCP server, so the role runs the project-owned gitea-mcp
package as a sidecar next to it and points it at the instance over the
container network.

Two things the image forces. Its command carries the binary path
explicitly because the image declares no ENTRYPOINT, so a bare flag list
would replace the binary instead of extending it. The port is stringified
because Ansible's native types make a bare lookup an int, and compose
rejects a non-string in a command array.

Keep the surface internal. The public vhost answers a bearerless probe
with an SSO redirect, so the guest spec pins maxRedirects to 0 and
asserts no MCP response is ever served; following the redirect would land
on a login page and pass for the wrong reason.

Calls run with administrator rights, so the tool policy is pinned
read-only and the sidecar starts with the read-only flag and a scope
list.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 22:10:15 +02:00
kevinveenbirkenbach e04c448cdd Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Three conflicts, none resolved by picking a side wholesale:

- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
  instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
  rc now counts as failure; the previous default(0) let an aborted Playwright
  run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
  which this branch's lint enforces, and adopted the incoming retry-inside-poll
  for the OIDC button plus its networkidle wait.
2026-07-30 15:38:15 +02:00
kevinveenbirkenbachandClaude Opus 5 028f4b85f4 chore(storage): set every image-backed min_storage to its image size plus 200MB
The declared values were guesses that ran up to 60x the size of the image they belong to: seaweedfs asked for 30GB against a 104MB image, nextcloud for 20GB against 353MB. Because min_storage feeds the storage guard, the runner packer and the sufficient-storage filter, those numbers made hosts look too small and pruned caches that deploys did not need to lose.

Every service that declares min_storage together with image and version now carries the measured compressed transfer size of that image plus 200MB of headroom. Sizes were read from the registries through utils.docker.registry.manifest_transfer_size, the same path the headroom test uses, so the declarations and the gate agree by construction. Values are written in MB throughout, matching the existing 500MB / 100MB / 50MB style.

119 declarations across 89 roles. Range: 206MB (memcached) to 3473MB (ollama); four exceed 1GB. The 84 min_storage entries that name no image are untouched, since there is nothing to measure them against.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:08:51 +02:00
kevinveenbirkenbach 5dbbaca4a1 Merge branch 'feature/swarm-nfs' of github.com:kevinveenbirkenbach/infinito-nexus-core into feature/svc-net-tor 2026-07-29 23:23:56 +02:00
kevinveenbirkenbachandClaude Opus 5 c6133c157d docs(gitea): point the authentication link at its current location
Gitea moved the authentication docs from usage/ to administration/; the old path returns 404 and failed the URL reachability test. Probed the candidates: usage/authentication and usage/authentication/ both 404, administration/authentication 308s to administration/authentication/ which answers 200 -- use the 200 form so the result does not depend on whether the checker follows redirects.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 23:23:55 +02:00
kevinveenbirkenbachandClaude Opus 5 4a5a9e2c24 docs(gitea): point the authentication link at its current location
Gitea moved the authentication docs from usage/ to administration/; the old path returns 404 and failed the URL reachability test. Probed the candidates: usage/authentication and usage/authentication/ both 404, administration/authentication 308s to administration/authentication/ which answers 200 -- use the 200 form so the result does not depend on whether the checker follows redirects.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 23:09:21 +02:00
Alejandro Roman 871b00670a Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-27 11:25:42 +02:00
kevinveenbirkenbach 0e64865297 Merge branch 'feature/swarm-nfs' into feature/svc-net-tor 2026-07-25 19:12:37 +02:00
kevinveenbirkenbachandClaude Opus 5 5d8253f004 fix(web-app-gitea): hoist delegate_to to apply:
Commit 99bc9a94d gave every top-level task in cleanup/ldap.yml and cleanup/oidc.yml an identical delegate_to, tripping the include-uniform-apply-keyword lint and failing CI run 30160077828.

apply: on include_tasks wraps the included tasks in a Block, and delegate_to is Block-valid, so hoisting it to the include site is semantically identical: register still resolves on the original host, the per-task when: gate is unaffected, and the include-site when: does not leak into the block. Leaf-only keywords (failed_when/changed_when) stay per-task per 64eef3440.

Matches the established pattern in keycloak, mediawiki, mastodon, matomo, espocrm, collabora and shopware. setup/ldap.yml and setup/oidc.yml deliberately keep the per-task form, since their undelegated set_fact must not run on the hosting node.

make quality green: test-external, test-integration, test-lint, test-unit all pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:12:37 +02:00
kevinveenbirkenbachandClaude Opus 5 386f27e901 fix(web-app-gitea): hoist delegate_to to apply:
Commit 99bc9a94d gave every top-level task in cleanup/ldap.yml and cleanup/oidc.yml an identical delegate_to, tripping the include-uniform-apply-keyword lint and failing CI run 30160077828.

apply: on include_tasks wraps the included tasks in a Block, and delegate_to is Block-valid, so hoisting it to the include site is semantically identical: register still resolves on the original host, the per-task when: gate is unaffected, and the include-site when: does not leak into the block. Leaf-only keywords (failed_when/changed_when) stay per-task per 64eef3440.

Matches the established pattern in keycloak, mediawiki, mastodon, matomo, espocrm, collabora and shopware. setup/ldap.yml and setup/oidc.yml deliberately keep the per-task form, since their undelegated set_fact must not run on the hosting node.

make quality green: test-external, test-integration, test-lint, test-unit all pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 16:17:31 +02:00
kevinveenbirkenbach 98b6b3db8e Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	plugins/lookup/application_gid.py
2026-07-25 15:44:55 +02:00
kevinveenbirkenbachandClaude Opus 4.8 99bc9a94d9 fix(web-app-gitea): delegate ldap/oidc auth exec to host node
In swarm the ldap/oidc setup+cleanup 'container exec' tasks used
GITEA_CONTAINER_ADDRESS (a swarm-wide-resolved container id) and ran on
the manager without delegation. When gitea is scheduled on a worker,
'docker exec' is node-local, so the manager's daemon can't see that
container -> 'No such container', failing the deploy.

Bring these tasks to parity with the DB/manager-ops tasks (commit
6a5cf7b66): use GITEA_LOCAL_CID + delegate_to GITEA_HOST_NODE (facts
already set by resolve_host_cid.yml in 01_manager_ops before setup/
cleanup run), with the container-exec-resolver nocheck marker. Applies
to every exec incl. the id-lookup steps, else idempotency/cleanup
misfire on worker placement.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 15:31:13 +02:00
kevinveenbirkenbachandClaude Opus 4.8 911152d3b6 fix(web-app-gitea): justify replicas:1 with single-replica nocheck marker
The single-replica lint requires every 'replicas: 1' pin to carry a '# nocheck: single-replica <reason>' marker, since pinning to one replica forgoes horizontal scaling. The --no-verify commit that added the pin skipped that hook; add the marker with the reason (single-writer state on swarm-shared NFS corrupts under concurrent multi-node writers). No runtime change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 12:44:04 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6a5cf7b663 fix(web-app-gitea): pin single writer and delegate DB patches to the hosting node
In swarm the gitea service defaulted to replicas = group size (3), so three replicas on manager + both workers bind-mounted the same NFS-shared /data and concurrently rewrote app.ini with no cross-node locking, leaving it NUL-filled and failing migrate. Gitea is a single-writer app (external MySQL, repos + app.ini on disk), so pin it to one replica; it stays on the shared NFS volume and remains node-mobile for failover.

The DB-patch sed tasks ran manager-local via GITEA_CONTAINER_ADDRESS, which cannot reach a replica scheduled on a worker. Resolve the hosting node + container id first (the same resolve_host_cid idiom 01_manager_ops.yml already uses) and delegate each patch to GITEA_HOST_NODE, so the patches follow the single replica wherever swarm places it. Compose is unaffected: replicas is swarm-only and the resolver short-circuits to localhost.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-25 12:44:03 +02:00
kevinveenbirkenbach 1a0af0ce09 Merge remote-tracking branch 'fork/feature/swarm-nfs' into feature/svc-net-tor
# Conflicts:
#	roles/sys-ctl-mtn-cert-deploy/tasks/main.yml
#	roles/sys-svc-compose-ca/tasks/main.yml
#	roles/web-app-dashboard/templates/javascript/oidc.js.j2
#	roles/web-app-mastodon/tasks/01_overrides.yml
#	roles/web-app-peertube/templates/compose.yml.j2
#	scripts/github/guide/compose_deploy.sh
#	scripts/github/guide/host_deploy.sh
#	scripts/make/alias.sh
#	tests/lint/ansible/roles/test_service_core_first_task_run_once.py
2026-07-24 04:30:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4039313266 refactor(tasks): enforce and apply a single task-file numbering scheme
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.

Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Fable 5 62a46e472f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 16:27:25 +02:00
kevinveenbirkenbachandClaude Fable 5 3125c5ac03 docs: regenerate role catalogs with the tor onion wiring
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 15:49:50 +02:00
Alejandro Roman 1915ec115b Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-22 12:16:40 +02:00