A variant that pins `services.tor.enabled: true` while enabling a role whose
own `meta/services.yml` pins `tor.enabled: false` declares a combination the
deployment cannot hold. The role rides no node onion, so on that round its
surface lives at a clearnet host no onion browser reaches, and the dashboard's
`frame-src 'self' http://*.<onion>` forbids embedding it outright. Run
36651124458 job 109692287333 shows the result: five specs fail at once,
dashboard tile, matomo tracker, css injection, logout and the coturn metric
label, all on hosts that are the only non-onion ones in the whole job.
web-app-nextcloud variant 11 was such a round: bigbluebutton on, tor on, while
bigbluebutton declares no onion. Variant 8 already answers the identical
question for PeerTube (65538c40da, "clearnet, because the OIDC plugin rejects a
plaintext onion issuer"), so variant 11 now does the same. With that,
bigbluebutton never meets a tor node and needs no consumer gates of its own.
Two roles keep a gate, for reasons that differ from each other:
web-app-seaweedfs is not clearnet-only. It deploys on tor as the object store
other roles reach over the internal network, and only its browser frontend is
off there, which `frontend.enabled` on main already says. Its logout, dashboard
and prometheus declarations claimed a surface that round does not deploy, so
they take the same gate the file's own frontend and sso blocks use.
web-svc-coturn belongs on a tor node and cannot serve one: STUN/TURN runs on
UDP plus a 20000-port relay range that no onion descriptor can publish, and its
only HTTP is a gateway answering a plaintext OK. Deploying it is right,
advertising it as a prometheus consumer is not, because that scrape goes
through the browser path. It is exempt from the rule itself, since it claims no
visitable surface, and its `dynamic-flag` markers now carry the reasons the
rule's own message asks for; one of the four was a duplicate of the line below
it and is gone.
The lint states the rule. It found these five pairings on its own, matching a
hand count made before it existed, and reports none after the variant fix.
`test_sso_variant_requires_tor` covers the complementary half, every variant of
a role that has an onion must pin it on, and exempts the clearnet-only role
itself; this rule covers the consumers that bind one. The suppression marker
belongs on the bound role when the reason is that role's own shape, because a
consumer-side marker would also silence every other clearnet role in the file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fourteen roles repeated the same two lines to say they use the gateway: a
`shared` flag gated on group_names, and an `enabled` flag conjoining that gate
with LITELLM_CHAT_MODEL_SERVED. Both are now single-sourced.
`shared` becomes literal true. The flag means the provider is a shared instance
rather than one deployed per consumer, and a central gateway is always that;
the dynamic form only said "shared if deployed", which `enabled` already
decides. The 50 shared pins under `litellm:` leave meta/variants.yml with it.
The tree had already noticed: 20 of those 50 lines carried
`# nocheck: variants-static-override`, two of them with copy-pasted text about
container_backup. This narrows e5de4e085a for one service key rather than
reversing it; every other dynamic flag keeps the group_names form.
`enabled` becomes `{{ LITELLM_USABLE }}`, defined in group_vars/all/16_ai.yml.
That shorthand is only possible because is_explicit_truth learns about it.
The resolver reads a dependency out of the flag TEXT — "in group_names" in
value — so a constant hides the role id and the provider silently leaves the
static closure of every role referencing it. Measured: the old expression
resolves True, `{{ LITELLM_USABLE }}` resolves False. GROUP_GATED_CONSTANTS
names the constants whose own definition carries the gate, and the docstring
states the obligation: a SPOT of this kind that is not listed there drops its
provider without a word.
The wider fix, accepting any Jinja as dynamic, was measured and rejected: 35
other flags carry a Jinja value without the marker (MCP sidecars, javascript
and SSO gates, recaptcha), and they would all enter closures they are not in
today.
A literal shared flag is no longer dynamic, so test_dynamic_flags wants the
`# nocheck: dynamic-flag` its own message documents for legitimate exceptions;
where a marker already existed the rule list is extended rather than a second
comment added.
Behaviour is unchanged where it counts. The closure keeps litellm with the same
app counts (wordpress 26, openwebui 23, hermes 23, mediawiki 22, flowise 23),
unrelated Jinja still resolves False, and the truth table of the new enabled
form matches the old one in all four served/deployed combinations. Three roles
resolve without litellm, and `git show HEAD:` proves that predates this change:
zammad pins enabled false in variant 0 at line 39, nextcloud at line 97.
Six unit tests cover the resolver, including one asserting that every
registered constant is actually defined in group_vars, so an entry cannot
become a dead promise.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nextcloud#2 of run 33923190220 probes the MCP surface three times and gets
"could not reach http://contextagentmcp:8080/mcp: <urlopen error [Errno 111]
Connection refused>" before failing on both workers. Connection refused, not
an unresolved name: the sidecar's address existed and nothing served it.
meta/services.yml derives both halves of that surface from the same switch,
context_agent and contextagentmcp each carrying
enabled: "{{ lookup('config', 'web-app-nextcloud', 'mcp.enabled') | bool }}".
mcp.enabled is itself derived, from peer presence rather than by hand:
utils/roles/applications/mcp.py renders "'peer' in group_names or ..." over
the consumers the role admits, and for nextcloud that is hermes, openclaw and
openwebui. Variant 2 is the openwebui variant, so the switch is true, the
sidecar comes up, and only the app behind it stayed down.
It stayed down because the variant pinned it off by hand. That pin is an
orphan. a2be4f081a introduced context_agent paired with an mcp pin in each
variant and kept the two in step; 06bc856c32 lifted mcp out of the services
map when it became a role topic; ffdef28da3 then dropped the hand-maintained
mcp values entirely, calling them "wrong in both directions", and replaced
them with the peer derivation. The mirrors it left behind have contradicted
that derivation ever since.
Removing them changes exactly one variant. Evaluated across all fourteen:
variants 0 and 1 keep their explicit enabled: true and are untouched, variant
2 gains the agent its sidecar needs, and variants 3 through 13 admit none of
the three peers, so the derivation yields false and they behave as before.
Eleven of the twelve removals were writing a value the derivation already
produced.
The budget objection does not hold either. collect_role_resources counts any
enabled that is not literally false, so a rendered template counts as on;
test_resource_budget therefore charges the agent's 512m to every variant that
no longer pins it off, and still passes.
merge/svc-net-tor still carries all fourteen declarations and will reintroduce
the orphans if it lands.
An MCP client spec signs in through the client's own OIDC button, which
redirects to Keycloak. Every provider put its MCP pairing in the lean variant,
and that variant pinned sso off - so the round that exists to prove the pairing
was the one round without an identity provider. Run 33574056480 shows what that
costs: openwebui's MCP specs reached _shared.js:131 and died on "openwebui OIDC
sign-in button must be visible on the auth page".
The rule now demands two things of the variant that carries the consumers: the
provider pins sso on, and each admitted consumer, in the variant that round
actually pulls in, does not pin it off. The second half matters because a round
deploys every dependency at its own index and falls back to variant 0 only when
the dependency has fewer, so the client shipping alongside is that one and not
the client's variant 0. A role that declares no sso, or pins it to a literal
false because upstream speaks no OIDC, is exempt from a flag it cannot have.
Twenty roles moved to satisfy it. Ten already had a second sso: false pin and
only needed the flip. Six could host the pairing in variant 0, which already
has sso on, and dropped back to two variants. Six kept a third lean variant to
carry the false polarity variants-coverage wants. moodle and mattermost took
swaps instead: moodle's variant 1 is the mandated LDAP-only branch, and
mattermost was already at the three-variant cap with sso on in its variant 2.
nextcloud also turns context_agent on in that round, the ExApp that serves the
endpoint - with it off the round proved nothing it was built to prove.
The rule moves to tests/lint/ansible/services/ because it reads meta YAML and
executes nothing, which is where integration.md says such a check belongs, and
its suppression key joins the catalog it was never listed in.
nextcloud_exapp_reregister becomes nextcloud_exapp_renew: mcp-secret-masking
matches `register:` unanchored, so the substring in the old name read as a task
that prints a secret. register.yml already masks the two tasks that carry one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mcp-consumer-variant guard demanded three things of a provider's MCP
round: every admitted consumer enabled, each of them pinned `services: null`
so it dragged no dependencies along, and no other service left enabled. Two
of those were the rule legislating a round's contents rather than checking
that the pairing gets proven at all.
Keep the first and drop the rest. What else the round carries is the role's
own call, so the tor/litellm exemption, the datastore exemption and the
`allowed` set they fed have no reader left and go with them.
Strip the 63 `services: null` pins the rule forced across 20 variants, all
written bare as `services:`. The topic-emptying capability behind them
stays: apply_topic still normalises None to {} for the inventory payload,
and the unit tests still cover it. Only the uses and the mandate are gone.
That leaves overridden_providers scanning a repository with nothing left to
find, so its three assertions passed over an empty dict and would have
stayed green however the function broke. Rebuild them on a synthetic role
tree, which also covers the case the repository scan never held: a variant
naming a plain flag is not an override.
Removing the pins changes what deploys. Measured across all 20 variants: 18
identical, prometheus#1 grows 23 to 31 roles, moodle#2 grows 31 to 32 and
crosses the memory budget until the following commit shrinks it back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A round deploys a provider once, so the role that pulls it in is the only place that can say how lean it has to be. A variant entry may now carry any application config topic under a service it depends on: a mapping deep-merges, so naming one key changes that key alone, and null empties the topic. Empty was the case that needed a marker of its own, because merging an empty mapping is a no-op and would leave exactly the services the round was trying to shed.
The topic vocabulary and the merge semantics live in utils.roles.applications.topics, which is also where resources.py now gets them rather than spelling the same null-or-merge out a second time. Collecting who demands what stays in the inventory tree, refuses two roles demanding different values for one provider in the same round, and skips a role's own entity entry so the existing services.<own>.users declarations are not read as overrides.
The CI set-cover learns the same fact. Its greedy cover retires a role's variant-0 row when an earlier row of another role embeds it, on the stated premise that embedding pulls the provider's variant 0. A variant that dictates that provider's config breaks the premise from the other side: whatever ran, it was not the provider as the provider declares it. Such a pair now covers nothing, so the retired row keeps its job, its artefact and its verdict.
Measured on the current tree: 20 override pairs, 41 covered rows before and after, so nothing is revoked today. The rule bites the moment an override lands in a row that covers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
83c1c729de gave every provider a group_names-gated services entry per MCP
client, including the clients it refuses with mcp_consumer: false. The gate
is what makes the inventory resolver plan provider and client into one round,
so 19 of the repo's 82 co-deploy pairs started a client for a pairing the
provider had already declined. flowise is a live MCP client of qdrant,
homeassistant and prometheus; the other 17 providers were bringing up its
node stack for nothing.
The entries now read enabled: false, shared: false, mcp_consumer: false, and
the variants that pinned them to true are gone. Setting those variants to
false instead is not an option: test_variants_no_static_override forbids any
variant override of a literal base value in either direction.
bond leaves with them, and a new lint holds that line: collect_role_resources
skips a literally disabled entry before it reads any resource key, so a bond
there describes a cost nobody pays. Thirteen entries unrelated to MCP carried
one and are cleared too.
test_mcp_consumer_services_entry demanded the group_names shape for every
client entry, refused ones included, and now demands the opposite there.
dynamic-flag keeps its rule and takes its own documented per-key exemption.
make docs redraws twenty graphs: the edges go, the nodes stay as
'openwebui ❌' and 'flowise ❌', so the refusal is visible rather than read as
an integration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
enabled and shared were hand-maintained in 24 roles and wrong in both
directions. Both now resolve from the mcp_consumer edges already declared
in meta/services.yml: a server takes the consumers it admits, a client
takes the servers that admit it, and the pair renders as "'peer' in
group_names or ...", or False when no peer is admitted at all. setdefault
keeps meta/mcp.yml and meta/variants.yml as the override channel, so
pinning either key still wins.
The resolved block replaces the mapping instead of being written into it.
_build_role_base_config parks the YAML cache's own object under mcp, so an
in-place setdefault injected the derived keys into every later raw read and
made test_mcp_schema pass alone but fail in a directory run.
The derivation keys on peer presence, never on a peer's own value: a
re-entrant read under _RENDER_GUARD returns the unrendered dict.
Four lints now ask the resolver instead of reading meta/mcp.yml, which they
could not do while the key was hand-written. That is what exposed the eleven
missing hostile-path specs the previous commit closed.
Carries one change that belongs to the WordPress probe but shares a file
with this one: the tool allowlist moves to the hyphenated names the adapter
actually serves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
allowed_consumers was twenty hand-kept lists of which fourteen were identical, so every new client meant an edit in every provider. A client now declares services.<own-entity>.mcp_consumer: true once and a provider refuses one with mcp_consumer: false, which is also what tells a deliberate exclusion apart from a forgotten entry.
The group_names-gated consumer entries give the inventory resolver the co-deploy edge meta/mcp.yml never had: one of sixty-nine matrix rounds carried an MCP client before, twenty-three role/round pairs do now. bond 0.5 keeps the clients out of round 0 so the host memory budget still holds.
BREAKING CHANGE: meta/mcp.yml no longer accepts allowed_consumers; declare services.<entity>.mcp_consumer on the client instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both partners in one variant overran the 20400s swarm drill budget by
~2%; the job then hit timeout-minutes and surfaced as cancelled.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
a34cb44ad split variant 3 into openproject, mattermost+matrix and the rest, and
closed with the open question: 'Whether dropping one partner moves 214 sync
under 185 is open.' Run 31864448027 answers it for the pair that was left.
Job 94965456921 (swarm, variant 10) died at rc=137, SIGKILL, 221s past the
20400s sweep deadline. Its sync pass alone took 11547s = 192.5 minutes - still
above the ~185 that commit derived as the survivable boundary. One partner was
not enough; the round needs one app.
matrix is the heavier half. Of PASS 2's 2811s of Playwright, its suite is
1090.55s - 39%, against 401.96s for nextcloud itself and 25-90s for ten of the
remaining twelve. PASS 1 carries the same shape at 1071.81s.
The new round is appended as variant 12 rather than inserted after 10, so every
existing index keeps its job name and its artifact names.
The alternative considered and rejected: restricting the async pass to the app
under test. It would have saved more seconds and cost the co-deployment coverage
of thirteen roles in the update path, to fix an overshoot one role caused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three rounds of run 31823845576 died at 356 minutes against the 355-minute cap
in call-test-deploy-swarm.yml: variants 0, 2 and 3, with 214.9, 214.0 and 202.5
minutes in the sync pass alone and no async pass reached. The role-runtimes
artifacts put the boundary between 183 sync (variant 4, 303 total, survived) and
202.5 (cancelled), so a round has to come in under roughly 185.
Each of the three carried more than it could finish. Variant 0 is the baseline
and had bigbluebutton on top of every tightly-coupled service; variant 2 paired
openwebui with flowise; variant 3 carried openproject, mattermost and matrix.
Splitting them leaves one partner each, and the freed partners fill three new
rounds.
bigbluebutton is the lever for variant 0 because f1dff50fb already prepared it:
that commit dropped its bond to 0.5 and noted "bigbluebutton stays in variant 0
for now, by explicit request; its bond change only makes the move possible
later." Everything else in variant 0 is bond >= 1 and would need redeclaring
first.
The xwiki split 6fcbffe08 merged away comes back here without costing a variant
of its own: xwiki takes flowise along, which variant 2 had just given up. sso and
tor stay on everywhere except variant 1, the minimal round, and variant 8, which
pairs the clearnet-only PeerTube.
Verified: tests/integration/roles/meta/variants plus the role-meta lints, 39
passed. Every variant resolves between 18.26GB and 27.77GB reservation against
the 32GB cap. The planner confirms the split is clean - round 2 pulls
web-app-openwebui without web-app-flowise, round 9 pulls flowise and xwiki
without openwebui, and neither role references the other in meta/services.yml.
Costs three more jobs in compose and in swarm, measured 9 -> 12 each. The slot
pool is fixed, so they displace three lower-priority rows per mode rather than
growing the run. host is unaffected: test_host is false for this role because it
ships a stack.
Not measured, and the reason to expect less than the arithmetic suggests: the
round closures sit at 21-24 roles before and after, because the shared base -
keycloak, mailu, matomo, dashboard, prometheus, the database engines, tor -
dominates the partner. Whether dropping one partner moves 214 sync under 185 is
open, and every measurement here predates 8f98e1648, which put the onion stack
into every round.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
72fd9c870 required the onion in every variant but exempted variant 1, on the
argument that the minimal round is the one place the stack runs without it. That
argument does not hold: the onion is part of the deployment under test, not a
feature bolted onto the full shape. A round that skips it is not a smaller
deployment, it is a different one.
So the exemption is gone and 95 variant-1 entries pin the onion on. Two
exclusions remain, both read from data rather than a comment: a role with no tor
block has no onion to pin, and a role whose meta/services.yml pins tor.enabled to
a literal false has already declared it has none.
That put the rule in conflict with test_coverage, which requires both polarities
of every dynamic flag: with tor on everywhere, the false side has no variant left
to live in. 94 roles take the exemption that contract already provides for
exactly this - "flags whose group-derived value is a hard invariant that variants
must never override", the same one 4e6ef2542 used for the seaweedfs frontend.
web-app-nextcloud needs none: its variant 8 pairs the clearnet-only PeerTube and
keeps the false polarity.
Measured, because the declaration is not what CI runs. assign_tor gives the onion
to every second capable entry with the run number as the parity switch, so more
capable variants do not mean proportionally more onion jobs: across five roles
and 21 entries the count goes 8 -> 10 on an even run and 9 on an odd one. The
clearnet path also survives - checkmk reports v1=false on run 0 and v1=true on
run 1, so both states are still covered across two consecutive runs. What the
94 markers give up is a declarative guarantee in variants.yml, not the coverage
itself.
The real cost is weight, not job count. web-app-nextcloud variant 1 goes from
3.09GB reservation / 7.27GB limit to 18.26GB / 43.01GB once svc-net-tor and its
closure join, so the cheap round is no longer cheap. Every variant stays under
the 32/64GB budget, and the same effect applies to the other 94 unmeasured.
Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles/meta/
variants plus the role-meta lints 39 passed. The rule was checked in both
directions - it reported the variant-1 offenders before the flips and none after.
Not measured: what the heavier rounds do to wall-clock, against swarm jobs capped
at 355 minutes and one that has hit that ceiling before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
7a70d27b7 made a variant that switches SSO on also switch the onion on, because
that pairing is where the platform breaks in ways clearnet never shows. The same
argument reaches further than SSO: a Secure cookie is dropped, an https identity
provider refuses to embed, and crypto.subtle is undefined for any feature on a
non-secure origin, not only for a login. A variant that never runs the onion is
not testing the deployment users get.
So the rule now covers every variant of a role that declares a tor block.
Variant 1 stays exempt - it is the minimal, partner-free round every role
declares and the one place the stack is exercised without the onion at all - and
a role whose meta/services.yml pins tor.enabled to a literal false is skipped
outright, because that declaration already says the role has no onion and needs
no comment repeating it.
That found 137 variants. 101 were variant 1. Of the rest, 28 are the LDAP-only
auth-isolation rounds, which gain real coverage from running LDAP over the onion
and are flipped on here. Four cannot: svc-net-tor is the provider itself, and
nextcloud's variant 8 pairs the clearnet-only PeerTube. Both carry a marker
naming the reason; opentalk and coturn need none, since their services.yml pins
the literal false the rule now reads.
Flipping them exposed a contract that contradicted itself. test_auth_isolation
and test_non_baseline_explicit_disables demanded `shared: false` beside
`enabled: false`, while variants-static-override flagged that same line as a
static override - which is why all 133 of them carried a nocheck marker. The
demand only makes sense where `shared` is dynamic: a literal base value cannot
leak through deep-merge, so restating it protects nothing. Both contracts now
ask for it only in that case, and the 133 lines are gone.
The result is a stricter rule with fewer lines: 189 insertions against 193
deletions.
Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles 107
passed / 610 subtests. The rule is checked in both directions - it reported 137
offenders before the resolution and none after.
Not measured: 28 additional onion rounds cost matrix wall-clock, against swarm
jobs already capped at 355 minutes and one that has hit that ceiling before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Variant 7 pinned sso, tor and peertube together, and since 8ffb1f567 those three
cannot coexist. test_sso_variant_requires_tor forbids sso: true without
tor: true, so tor cannot give way. With tor on, keycloak is published onion-only
- auth.infinito.example does not appear once in the job log - and
align_domain_to_consumer falls through to `return primary_domain` because there
is no clearnet sibling to align to. PeerTube therefore receives
http://auth.<onion>/realms/.../.well-known/openid-configuration, oauth4webapi
rejects it on scheme before any request leaves ("only requests to HTTPS are
allowed", OAUTH_HTTP_REQUEST_FORBIDDEN), the plugin registers without building a
client, and the gate at oidc/enable.yml:105 fails the deploy. Both deploy modes
reproduced it, with different onions, which rules out routing: the SSO
back-channel pin from container_extra_hosts is emitted for peertube and fixes
where a name points, not which scheme a library will accept.
peertube is the only one of the three that can move, so it moves to a variant of
its own with tor off. sso follows tor off, which costs nothing here: the addon
gate is services.peertube.enabled alone and the spec signs in with a direct
password, not through Keycloak. PeerTube keeps its OIDC coverage in its own
variant 0, which is sso-on and clearnet - the topology where the plugin works.
The slot is paid for by folding xwiki into variant 7 rather than adding a tenth
variant. This does not undo f1dff50fb: that commit moved xwiki out of variant 0
because job 94319317296 was cancelled at exactly the 355-minute ceiling, and
variant 0 is untouched here. Nor does it grow a round - variant 7 carried two
partners before (discourse + peertube) and carries two now (discourse + xwiki),
while variant 3 already carries three. Measured: the merged variant 7 resolves to
20.26GB reservation / 48.01GB limit against caps of 32/64, the lightest
non-baseline variant in the file, and the two-partner variant 7 of run
31823845576 spent 101.4 minutes against a 355-minute box.
mariadb and redis stay shared: true in the new variant because
variants-shared-false-once allows exactly one shared: false pin each and variant
1 already holds both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An addon's enabled is either dynamic -- a Jinja expression resolving per deploy
against a service flag, an API credential, the SSO plugin choice or group_names
-- or static, a literal boolean. A dynamic addon already follows its variant:
switch the service off and the expression yields false. A statically-on addon
does not. It installs in every round, so each non-baseline variant pays install
time for plugins it never exercises, and variant 0 stops being the only
maximum-footprint shape that meta/variants.yml exists to document.
web-app-nextcloud carried 22 of them across nine variants. They are now pinned
off once in variant 1 behind a YAML anchor, and variants 2-8 alias it, so the
set has one spot -- the same shape meta/volumes.yml and the
volumes-target-single-spot lint already prescribe.
test_addons_variant_pinning.py holds the rule for every role that ships
meta/addons/ and declares more than one variant:
1. an addon whose base enabled is literal true MUST be pinned enabled: false
in variant 1; a literal false is out of scope, the pin would override
nothing,
2. an addon whose base enabled is a Jinja expression MUST NOT be pinned in
any variant -- that replaces the expression with a constant and decouples
it from the service it follows,
3. every later variant MUST carry the same set, in practice by aliasing
variant 1. Variant-specific pins beyond that set stay allowed, so a
variant may still switch a statically-off addon on (web-app-matrix does
exactly that for its bridges in variant 2).
Measured on the real role through utils.cache.applications.get_variants: all
nine variants keep 51 addon keys, variant 0 keeps its 22 statics on, variants
1-8 hold none, and every dynamic entry keeps its Jinja string -- xwiki still
resolves true only where services.xwiki.enabled is. The 51 keys matter: the
override merges per addon instead of replacing the map, which is what an
earlier lookup-based attempt at this got wrong.
Each rule was broken on purpose and seen to fail: calendar pinned true, xwiki
pinned (reported for variants 1-8, which also proves the alias resolves), and
variant 5's alias removed (22 findings, exactly the required set).
web-app-discourse is exempted per addon with a reason. Its plugin set is
compiled into the image from services.discourse.plugins, and meta/addons/ is a
second declaration of the same thing; an addon-level disable would still build
the plugin and only flip the gate flag, saving no time while silencing the
spec. web-app-wordpress is not exempt: vars/main.yml:16 builds the plugin list
from the merged map and 07_manager_ops.yml:45 gates the install loop on
enabled, so its variant 1 genuinely skips a wp plugin install. Its README
claimed activitypub was "always enabled"; that is corrected rather than left to
contradict the file.
Not verified: no deploy has run. The saving is derived from the install loops
being gated, not observed, and the WordPress Multisite variant losing
activitypub rests on the same reading plus its skipUnlessAddonEnabled gate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Variant 0 of this role no longer fits the box CI gives it. Job 94319317296 of
run 31658270172 started at 01:49:01Z and logged
"##[error]The operation was canceled." at 07:44:14Z -- 355 minutes later, to the
minute, which is the ceiling .github/workflows/call-test-deploy-swarm.yml:161
sets. It died inside its own step, at
"TASK [test-e2e-playwright : ▶️ Run Playwright container]". The drill costs about
6h40m and the box is 5h55m.
The cost is dominated by the deploy itself, not by a repeatable sweep: PASS 1
alone is 13695.90s in role-runtimes, of which sys-svc-compose is 3094.19s and
test-e2e-playwright 2832.15s. It therefore scales with how many services variant
0 switches on, which is what this commit reduces.
The file's own header states the rule: variant 0 carries every tightly-coupled
(bond>=1, same-host) service, while loosely-coupled (bond<1) partners are
excluded from the closure and distributed across later variants to keep each
host under budget. xwiki and bigbluebutton were declared bond: 1 and therefore
had to sit in variant 0. Both are bridged partners with their own role and their
own spec -- the same shape as openproject, gitlab and mattermost, which already
carry bond: 0.5 -- so the declaration was the outlier, not the placement.
Both bonds drop to 0.5. xwiki moves out of variant 0 into a new variant 8 with
sso, ldap and tor, the same shape as variant 5 (zammad) and variant 6 (gitlab).
bigbluebutton stays in variant 0 for now, by explicit request; its bond change
only makes the move possible later. Each edge is one-directional -- neither
web-app-xwiki nor web-app-bigbluebutton declares a nextcloud service -- so there
is exactly one bond per partner to change.
coturn is deliberately absent from variant 8. BigBlueButton speaks WebRTC and
the reflex is to bring a TURN server, but the only coupling in the repo is
meta/addons/spreed.yml:8, which bridges coturn for Nextcloud Talk, and talk is
off in that variant. An extra service there would be exactly the overhead this
commit removes.
Measured: the file loads to 9 variants, variant 0 falls from 16 enabled services
to 15, and variant 8 resolves to ['ldap', 'sso', 'tor', 'xwiki']. make test-lint
stays at 328 passed -- variant-limit (capped at 3, opened by the file's existing
nocheck), variants-static-override and the bond lint (which only requires a
number in 0..1) all accept it.
Not measured, and the honest cost: a ninth variant is one more deploy round, and
the limit lint's own rationale says total deploy time grows linearly with
max(variant_count) across the primary apps -- Nextcloud sets that maximum, so
this is roughly +12.5% for the whole matrix, spent to relieve the single job that
dies on its own ceiling. Whether removing one service of sixteen closes a ~45
minute overrun is not established; role-runtimes has the per-role seconds and no
run has been made with this shape.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SSO over a plaintext onion is where this platform actually breaks, and none of
it shows on clearnet: a framework's Secure session-cookie default makes Chromium
drop the cookie, an https identity provider cannot be embedded in an http page,
and crypto.subtle is undefined because an onion is no secure context. Pixelfed,
Taiga, Joomla and WordPress each cost a red job over exactly that. The
combination is only exercised if a variant deploys both, so the lint requires
it: services.sso.enabled true implies services.tor.enabled true.
69 variants enable SSO; 59 already complied. Of the ten that did not, seven are
flipped on because the mechanism is proven there - matrix and nextcloud both run
SSO behind the onion in variant 0 already (20ddda8c1), and their later variants
were pinned false only by the blanket rule from 1d9d355bc. Nextcloud variant 1
stays false: SSO is off there, and it is what gives the coverage test its
false-pinning variant.
Three roles opt out, each for a structural reason named in the marker. jitsi and
bigbluebutton pin tor.enabled false in meta/services.yml because the conference
needs getUserMedia, which no plaintext onion can offer. seaweedfs gates its own
sso flag on 'svc-net-tor not in group_names', so on a tor node there is no login
flow to test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The block sat under services but never behaved like a service. A service has
exactly one providing role, declared with provides, so the registry can map the
key to that role and consumers bind their flag to its presence. No role declares
provides: mcp. There are seventeen providers and the pairing is done by
allowed_consumers inside the block itself.
Every rule that enforces the service model therefore needed an exemption for it:
a shared-provider suppression in fourteen roles, a playwright-service-flag
suppression, and two freshly added exemptions in the dynamic-flag and
variants-static-override checks. Four independent rules, four escape hatches,
one cause. They are all gone now, together with a bond key that only the
resource budgeter reads and only for entries under services, and a wordpress
addon bridge that pointed at a service block that no longer exists.
The block moves to meta/mcp.yml. The applications loader maps meta/<name>.yml to
applications.<app>.<name> generically, so the move needs no wiring; the file
takes the same shape as meta/rbac.yml. Every config path follows, the variant
overrides are lifted out of their services map, and the vocabulary module leaves
the services package because it no longer describes one. roles_with_service
gains an explicit topic parameter rather than an implicit fallback, so nothing
silently resolves against the wrong container.
Files that are otherwise unrelated to this move are included where the committed
tree would otherwise keep pointing at a path that no longer resolves: thirteen
vars files, eleven playwright env templates and one task file.
Nextcloud carries no MCP server of its own, so the role runs the Context
Agent as an ExApp: render its container while MCP is enabled, wait for
its heartbeat, register it with AppAPI, add the MCP route, enable it, and
only then mint the app password clients present.
Split the credentials by origin. The shared secret between the AppAPI
proxy and the ExApp is generated by this deployment and stays a vault
credential; the app password is minted by Nextcloud through occ and is
persisted through sys-token-store instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nextcloud reached its OpenAI-compatible backend through Flowise, which
only forwarded to the LiteLLM sidecar that Flowise no longer runs. The
consumer flag, the activation condition and the endpoint now bind to
svc-ai-litellm directly, one hop shorter.
The endpoint switches from Flowise' public TLS URL to the internal
gateway address plus /v1, so a container-to-container call no longer
travels through the reverse proxy.
The api_key stops crossing a role boundary: it was read from
web-app-flowise credentials.llm_master_key, handing Nextcloud
administrative access to the whole gateway, and is now this role's own
per-consumer virtual key.
The litellm variant pin sits on variant 7 rather than 3 because adding
the gateway to variant 3 pushed that round past the matrix memory budget.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Variant 0 pins the backup consumer flag enabled so the matrix baseline exercises the in-group branch; every other variant pins it disabled (shared: false carries the static-override suppression since the base value stays literally true). The matrix and wordpress variant entries move their services mapping to the list-item head so the static-override line parser resolves the override paths; yaml semantics are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Each role's meta/variants.yml now pins the tor onion gate both ways so the matrix exercises it: variant 0 pins services.tor.enabled: true, every non-baseline variant pins enabled: false + shared: false (# nocheck: variants-static-override, since tor.shared is a static literal). This gives real test_coverage / test_bond_baseline coverage without the removed per-key code exemptions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move discourse out of variant 4 (now onlyoffice + mastodon) into a new dedicated variant 7, mirroring the gitlab isolation, so its loosely-coupled (bond 0.3) bridge footprint gets its own budget-bin. Add a numbered overview comment (# Variant N) before each variant entry.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SSO-without-LDAP left the OIDC admin user without nextcloud admin rights -> the admin-settings Playwright addon tests hit 'Access forbidden' in every sso variant that lacked ldap (v2-v6), while v0 (sso+ldap) passed. Enable ldap (enabled+shared) in every sso variant and make mariadb shared everywhere except variant 1 for a consistent multi-app topology.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reclassify the nextcloud->onlyoffice bond from 1 to 0.5 and move onlyoffice out of the variant-0 host baseline into variant 4, so the document server is tested as a separate-host partner. Collabora stays the tightly-coupled (bond 1) default in the baseline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Disable talk_recording in variants where talk is off, and isolate gitlab into its own variant so it no longer shares a CI runner with discourse and mastodon.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
matomo: force-restart the swarm service after patching config.ini.php DB creds (the compose-up handler is a no-op in swarm), extracted into 10_resolve_host_cid. nextcloud: gate the redis PONG wait on the engine being local. prometheus: config 0444 (swarm config objects are root) and gate the host-monitor binds plus privileged cadvisor/node-exporter to compose only. dashboard: probe the public icon URL in swarm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mariadb.enabled is a static literal in meta/services.yml, so variants must not
pin it (test_variants_no_static_override). Regenerate variants.yml overriding
only the dynamic flag per service (mariadb keeps shared-only). With mariadb
correctly always-on and flowise+openwebui sharing ollama, the bond<1 partners
repack into 4 bins -> 5 variants (v0 coupled baseline + 4 partner bins), each
under the 73GB/37GB host budget.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_order now only pulls bond>=1 (tightly-coupled, same-host) services into a
role's closure; bond<1 partners deploy on their own host and are excluded, so a
later variant may enable them without variant 0 having to list them.
New test_bond_baseline enforces the placement: a bond>=1 dynamic flag must be
pinned true in variant 0 (the host baseline); a bond<1 flag must be pinned true
in at least one variant. Escape: '# nocheck: bond-baseline'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Schema-migrate every nextcloud addon to the unified per-file addon syntax (mechanism/source/required/bridges/config); add the non-proprietary integration addons (matrix, mattermost, moodle, suitecrm, zammad) and the local gitlab bridge; wire the bridged service blocks into meta/services.yml and pin every new (service,enabled)/(service,shared) pair in meta/variants.yml for variant coverage; relocate the per-addon user_ldap hook from tasks/plugin/hooks/ to tasks/addons/ for a uniform tasks/addons/ layout, dropping the obsolete vars/plugins README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Each Nextcloud app declared as a per-file addon; sociallogin bridges sso, user_ldap bridges ldap, bbb/onlyoffice/richdocuments/spreed/whiteboard/xwiki bridge their partner services. Per-plugin runtime config absorbed from vars/plugins/* (now deleted); install/configure tasks read the addon config via include_vars for lazy var rendering.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switch penpot, nextcloud, gitea, mattermost, openwebui, funkwhale, baserow, openproject, pixelfed, mastodon, bookwyrm, listmonk, peertube and opentalk to the engine-agnostic objstore lookup: the minio service entries are pinned off, a seaweedfs service (gated on the provider being in group_names, public for the media apps) replaces them, and the matrix variants pin the new flag. OpenTalk keeps object storage mandatory through an embedded fallback and provisions its bucket via weed shell.
Add objstore lookup SPOT keys consumed by these roles: endpoint (host:port), public_domain, public_bucket (public S3 host plus bucket) and public_url (scheme plus public_bucket, scheme derived from the provider TLS state). This removes the repeated lookup('config', lookup('objstore', ...), 'server.domains.canonical.api') / host:port concatenations from the consumer templates. Listmonk settings render their JSON values via the to_json filter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Switch penpot, nextcloud, gitea, mattermost, openwebui, funkwhale, baserow, openproject, pixelfed, mastodon, bookwyrm, listmonk, peertube and opentalk to the engine-agnostic objstore lookup: the minio service entries are pinned off, a seaweedfs service (gated on the provider being in group_names, public for the media apps) replaces them, and the matrix variants pin the new flag. OpenTalk keeps object storage mandatory through an embedded fallback and provisions its bucket via weed shell.
Add objstore lookup SPOT keys consumed by these roles: endpoint (host:port), public_domain, public_bucket (public S3 host plus bucket) and public_url (scheme plus public_bucket, scheme derived from the provider TLS state). This removes the repeated lookup('config', lookup('objstore', ...), 'server.domains.canonical.api') / host:port concatenations from the consumer templates. Listmonk settings render their JSON values via the to_json filter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`_build_application_defaults` returned variant 0 of every role with a
meta/variants.yml, which then deep-merged with the variant-N override
that build_dev_inventory bakes into the inventory. Any service flag
that variant N did not explicitly disable carried through from
variant 0 into the runtime applications map. The most visible casualty
was svc-bkp-container-2-local variant 1 (postgres + mariadb only),
where gitea + keycloak leaked in via variant 0, sys-service-loader
pulled in svc-db-openldap via keycloak.services.ldap, and the deploy
crashed in sys-svc-compose/04_files.yml on openldap's env.j2 with
"credentials is defined in schema at roles/svc-db-openldap/meta/schema.yml
but not set in application 'svc-db-openldap'".
Defaults now come from each role's variant-free base config
(meta/<topic>.yml plus schema-defaults, no variants overlay). Variants
stay accessible via get_variants for any caller that explicitly needs
a specific variant. Inventory-level applications.<app> overrides win
exactly as before.
Supporting changes that keep the variant matrix consistent under the
new defaults model:
- New lint guard tests/integration/roles/meta/variants/
test_non_baseline_explicit_disables.py: for every dynamic
service-key declared in a role's meta/services.yml that variant 0
pins to literal true, every non-baseline variant MUST explicitly
pin the same key to either literal true or to literal
enabled: false + shared: false. Statically-enabled keys are exempt
(same rule as test_auth_isolation.py).
- roles/web-app-{fusiondirectory,lam,nextcloud,phpldapadmin}/meta/
variants.yml: explicit `false` pins for dashboard / email / logout /
matomo / prometheus / talk in restrictive variants so deep-merge no
longer hands them the deleted variant-0 inheritance.
- roles/web-app-moodle/templates/playwright.env.j2: gate the LAM
block on `lookup('config', application_id, 'services.lam.enabled',
false)`. LAM is a transitive helper, not a moodle-owned service, so
the default-bearing lookup tolerates the missing path when the
variant does not deploy LAM.
- tests/unit/utils/cache/test_variants.py: rename the legacy
"always variant zero" coverage to TestMergedApplicationsUsesBaseConfig
and add test_get_application_defaults_omits_variant_zero_only_keys
as the regression guard for the leak this commit fixes.
- docs/contributing/artefact/files/role/variants.md +
docs/contributing/design/variants.md: variant 0 now manually
enumerates every dynamic services.yml key with
enabled: true + shared: true; `{}` is only permitted when the role
has no dynamic-enabled service-key.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Commit f63118f34 ("feat(lint): enforce auth-isolation in
variants.yml") added `enabled: false, shared: false` overrides for
mariadb / postgres to every auth-isolated LDAP-only variant. That
disables the role's hard-required DB at deploy time, so e.g. the
moodle baseline blows up with `Required env var MOODLE_DB_HOST is
not set` as soon as round 2 of the matrix hits variant 1.
The auth-isolation rule's premise — "auth-isolated rounds must trim
non-auth services from the include closure" — only buys closure
savings for services whose `meta/services.yml` base declares
`enabled` as a dynamic Jinja expression. DB-engine entries are
already a static `enabled: true`, so the resolver pulls them in
regardless of the variant pin; the variant `false` only breaks the
deploy without trimming anything.
Two changes:
- `test_auth_isolation`: exempt service-keys whose `services.yml`
base has a static literal `enabled: true` from the per-variant
pin obligation. Such keys should not be declared in auth-isolated
variants at all; the inherited base value is correct.
- 10 web-app roles in the active deploy scope (akaunting, baserow,
bookwyrm, espocrm, friendica, gitea, moodle, nextcloud, odoo,
openproject): drop the entire mariadb/postgres stanza from the
LDAP-only variant — both `enabled` and `shared` inherit from
`meta/services.yml`. Matches the pre-f63118f34 state.
- 6 more roles out of the current iteration scope (funkwhale,
opentalk, shopware, snipe-it, suitecrm, taiga) carry the same
broken `enabled: false` line: removed it, kept the pre-existing
`shared: false` to preserve the variant's own-DB-container
topology intent.
`make test` is green; `variants-auth-isolation` and
`variants-static-override` both pass with no per-variant nochecks
required.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Autoformat byproduct — normalize the double space before the trailing
`# nocheck: variants-static-override` marker across seven role variants.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Auth-isolated matrix variants (exactly one of LDAP / OIDC-OAuth2
enabled, and no non-auth `enabled: true` alongside) must explicitly
pin every non-auth service-key in the role's variant union to literal
`enabled: false, shared: false`. Without this, the variant's
auth-only test scope balloons into a full integration deploy via the
services.yml-default propagation, wasting CI minutes and obscuring
the regression signal.
Adds `tests/integration/roles/meta/variants/test_auth_isolation.py`
(rule key `variants-auth-isolation`) plus the matching catalog entry.
The trigger is narrow on purpose: a variant that enables auth
alongside other services is a deliberate integration test, not an
auth-isolated one, and the rule does not fire there.
Mass-fixes the 24 legacy auth-isolated variants the rule turned up:
each gets the missing `enabled: false, shared: false` declarations
appended to its `services:` block. Where the false-override clashes
with a static-literal base in meta/services.yml, the override line
also picks up a `# nocheck: variants-static-override` marker. The
svc-bkp-container-2-local variant 2 additionally carries
`# nocheck: variants-services-match` on the new keys because they
are resolver-only matrix hooks with no consumer contract in the base
meta.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two-line block
# variant pinning every dynamic services.yml flag to false so the
# matrix also exercises the out-of-group_names branch.
sat inside the variant-0 (true-pinning) block of 46 variants.yml files —
inverted relative to its content. Removed across the board along with
the now-orphaned blank line that used to separate it from the
preceding service entry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the web-svc-css provider Playwright spec that mirrors the
dashboard/prometheus SPOT-provider pattern: emits CSS_TARGET_ROLES_JSON
via roles_with_service('css') and asserts every consumer's canonical
surface indexes a <link rel="stylesheet"> pointing at the shared CSS
asset host (CSS_BASE_URL / CDN_BASE_URL). The baseline test exercises
CDN reachability first so per-consumer failures unambiguously point
at the consumer's sys-front-inj-css wiring, not at the provider.
Suppresses the per-role playwright env-services-match guard above each
consumer's `css:` key with the canonical `# nocheck:
playwright-service-flag — verified by web-svc-css provider spec`
marker. web-svc-css/meta/services.yml gets the self-provider nocheck
on `css:` and `provider role; no consumer surface to gate` nochecks on
`cdn:` / `matomo:`.
Adds css.{enabled,shared} true/false pinning to every variants.yml
that declared the dynamic flag without coverage so
test_every_dynamic_flag_has_true_and_false_variant passes. Unifies
the css service declaration across espocrm / mediawiki / openproject
(was static enabled: false, shared: true) so the matrix matches the
rest of the consumer fleet.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the Talk env vars, settings-text scraping helpers, spreed
test-button verifier, and the former `verifyNextcloudTalkAdminSettings`
flow out of `_shared.js` into `test-talk-admin-settings.js` so the gate
flag, env-presence guard, and assertion logic live in a single module.
Drop the redundant `verifyNextcloudTalkAdminSettings` calls from the
admin OIDC and native login specs — the dedicated Talk test now owns
that coverage. Add `talk: true` / `talk: false` variant pins for the
new dynamic `services.talk.enabled` flag so coverage stays complete.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
declare it at file start with default `true`; still env-overridable.
Reword the worker comment so the literal `# shellcheck` token no
longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
is always fatal. NAMESPACE is an internal constant. Add the same
PARALLEL env var (default `true`, env-overridable); falsy demotes the
xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
add the lint-galaxy parallelism note.
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two coupled deltas to ``meta/variants.yml`` across every role whose
``meta/services.yml`` declares ``oidc`` / ``oauth2`` / ``ldap`` via
the dynamic ``"… in group_names"`` shape:
1. **Pair-polarity backfill** in 14 roles whose existing all-true /
all-false reference variants only set ``shared`` (relying on
``services.yml``'s literal ``enabled: true # nocheck:
dynamic-flag``). The auth-coverage rule is stricter: it counts a
pair only when BOTH ``enabled`` AND ``shared`` are explicitly
pinned in the same variant entry. Added the missing ``enabled:
true`` (true-variant) / ``enabled: false`` (false-variant) for
``ldap`` (and ``oidc`` in friendica/gitea/bookwyrm). Touched
roles: web-app-{friendica, funkwhale, fusiondirectory, keycloak,
kix, lam, nextcloud, openproject, opentalk, phpldapadmin,
snipe-it, suitecrm, taiga} and web-svc-xmpp.
2. **LDAP-only third variant** in 25 roles where both an auth
service (``oidc`` / ``oauth2``) AND ``ldap`` are dynamic. The
variant pins ``ldap.enabled: true`` together with ``enabled:
false`` on every dynamic auth service in the same entry, so the
matrix exercises the LDAP-only branch in isolation from OIDC —
the explicit Playwright LDAP-coverage gap tracked by
docs/requirements/018-playwright-ldap-coverage.md. Touched roles:
web-app-{akaunting, baserow, bluesky, bookwyrm, discourse,
espocrm, flowise, friendica, funkwhale, fusiondirectory, gitea,
jenkins, kix, lam, minio, nextcloud, odoo, opencloud, openproject,
opentalk, openwebui, phpldapadmin, snipe-it, suitecrm, taiga}.
Mechanically applied via a one-off ruamel.yaml round-trip script
(comments and key order on existing variants survive); the new
ldap-only entries are a flat ``services:`` block at the end of each
file.
Verification: ``test_oidc_oauth2_ldap_variants_cover_auth_matrix``
passes; ``test_every_dynamic_flag_has_true_and_false_variant``
still passes; full integration suite is now ``172 tests, 0 failures,
1 skip`` — first time the suite is fully green on this branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new integration lints under tests/integration/roles/meta/:
* test_variants_coverage.py — every services.yml flag declared via
the dynamic ``"{{ '<role>' in group_names }}"`` form MUST be
exercised by meta/variants.yml: at least one variant entry pins
it to literal ``true`` AND at least one pins it to ``false``.
Pairs may share variant entries — the test only checks the union
of all variants covers both polarities for every dynamic flag.
* test_services_no_self_reference.py — a role's own primary entity
(matching ``get_entity_name(role)``) MUST NOT use the dynamic form
for ``enabled`` / ``shared``. Both MUST be literal ``true``: the
role's own services.yml is loaded only when the role itself is in
group_names, so a ``'<self>' in group_names`` Jinja is tautological.
The dynamic-flag test now skips the role's own entity to avoid a
contradiction with this new guard.
Self-reference fixes (Jinja → literal ``true: true``) on 19 roles:
svc-ai-ollama, svc-db-{mariadb, openldap, postgres}, web-app-{
dashboard, discourse, keycloak, mailu, matomo, prometheus},
web-svc-{asset, cdn, collabora, coturn, file, libretranslate, logout,
onlyoffice, simpleicons}.
Variants coverage rollout (62 new + 17 merged meta/variants.yml):
* Roles without a pre-existing variants.yml get a new file with two
entries — one pinning every dynamic flag true, one pinning every
dynamic flag false. No ``- {}`` baseline (matrix runs only the
two coverage variants).
* Roles with a pre-existing variants.yml (akaunting/baserow/bluesky/
bookwyrm/flowise/fusiondirectory/jenkins/jira/joomla/keycloak/lam/
minio/phpmyadmin/postmarks/wordpress/web-svc-xmpp) keep their
matrix variants (OIDC vs LDAP, Single-Site vs Multisite, dev creds
+ TOTP-disabled realm, autologin, etc.); the all-true coverage is
deep-merged into variant 0 and the all-false coverage into
variant 1, so each role still ships exactly two matrix rounds.
* svc-ai-ollama keeps its single CI image-shrink variant; after the
self-reference fix it has no remaining dynamic flags so no extra
coverage entry is required.
* svc-db-{mariadb, openldap, postgres} drop variants.yml entirely
for the same reason.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
