The private blocks were written out wherever something needed to say "this is us": NETWORK_TRUSTED_PROXY_CIDRS listed all three, Tor's SocksPolicy repeated them, unbound's access-control repeated them again. Four copies of a fact, and nothing tying them together, so narrowing one narrows nothing else.
Worse, the block role networks are carved from was never declared at all. The allocator behind 'infinito contributing network address suggest' derives its umbrella from the subnets already in use, so the tree defined the rule instead of the rule defining the tree. Anything consuming "our own address space" had to guess it, and a guess that is one role out of date silently excludes that role. Measured: 106 of 107 role subnets sit in 192.168.0.0/16 and one does not, so a firewall range derived from the pool by eye would have cut BigBlueButton off.
The blocks are now named once and everything else is derived, NETWORK_TRUSTED_PROXY_CIDRS included, which renders byte-identically to before for its four consumers.
BigBlueButton stays outside because its upstream pins both the block and its gateway, which meta/services.yml repeats as upstream_host 10.7.7.1. The exception list therefore carries the role id rather than the CIDR: the block is already declared in that role's meta/networks.yml, and repeating it here would be the same duplication one layer up. The role_subnets lookup resolves the ids and raises on a role that declares no subnet, because a silently dropped entry would narrow whatever firewall or trust list consumes the result.
A lint keeps the declaration honest in both directions: every role subnet must sit in the declared space, and an exception whose block the pool already admits is rejected so it cannot outlive its reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The list form could publish a host:container pair and nothing else, so thirteen blocks stayed hand-written: the ones needing a protocol suffix, a port range, the swarm long syntax with mode host, or a container-port-only publish. Each of those duplicated numbers the role already declares in meta/services.yml, which is the drift the lookup exists to remove.
The lookup keeps the list form unchanged for its 93 existing call sites and gains a dict form: transport emits one item per protocol, container 'same' serves the services that cannot remap (STUN/TURN, media relays), mode 'host' emits the long block an edge role needs in swarm so the client address survives the hop, publish 'ephemeral' emits the container port alone, and key names the block for Discourse's launcher file, whose expose: list is what becomes docker run -p there. A declared {start, end} mapping renders as start-end on both sides, so a relay range needs no term of its own.
svc-prx-openresty had no ports block at all and bound 80 and 443 as literals in its template, while torrc.j2 carried a third literal 80 for the hidden service. None of the three referenced the others. openresty now declares the ports, the compose template reads them, and the onion's port 80 comes from ports.onion.http through TOR_ONION_EXTRA_PORTS.
Three blocks stay raw, each with the reason it cannot route through one call: keycloak's sso_proxy maps a published port owned by the consuming role to a container port owned by web-app-keycloak, and matrix's ansible flavor takes element on 8080 and federation on 8048 from the upstream playbook rather than from this repo's ports.internal.
The enforcing lint scanned only files with compose in the name and only double-quoted list items, so nine of the thirteen blocks were invisible to it. It now reads every role template that can carry a ports block, catches unquoted items, protocol suffixes, ranges, flow style and the long syntax, and tracks the block by the indentation of its key: keying on the next line closed the block on anything shaped like a key, which target, published, protocol and mode all are. Under expose: only a real mapping counts, because there the single-segment form documents a container port and publishes nothing.
Verified by rendering all 110 container_ports call sites through the real templar, the real lookup and the real merged applications view.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A port under ports.local or ports.public binds an address on the stack host, which is what a HiddenServicePort can forward to. Whether it should be forwarded has a cost in both directions: silently off leaves a service unreachable that the operator believes is on the onion, and silently on publishes a database, an admin interface or a SOCKS proxy to everyone holding the onion address. Neither belongs in a default, so the flag is now mandatory and every false carries the reason it is false.
The first run produced 165 findings, of which 131 were one architectural fact repeated rather than 131 decisions. Four groups are therefore settled once in the lint instead: the proxy-fronted http, sso and websocket, because the node onion already forwards port 80 to the reverse proxy that routes every vhost, oauth2 hop and socket upgrade behind it; the implicit-TLS variants, because Tor is already the transport encryption and container_ports drops those listeners when TLS is off, which an onion deployment always is; the categories a hidden service cannot carry, listed once in plugins.lookup.tor_ports.UDP_ONLY_CATEGORIES, which the lint imports rather than restates; and any entity that already answers the question through exposed:.
That leaves 23 real per-port decisions, each with a reason naming its own consumer. svc-net-tor keeps its own socks, trans and dns off the onion: a published SOCKS port is an open proxy, and SocksPolicy cannot refuse it because the forward arrives from the node's own address.
Mailu turns smtp, submission, imap and pop3 on, which is what makes sending and receiving work over the onion, and leaves sieve off because ManageSieve manages filters rather than mail. Measured end to end through the merged applications view, across all 105 roles the deploy now forwards exactly 25, 110, 143 and 587.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The public-port sweep never forwarded anything. It read tor.enabled from meta/services.yml on disk, where 104 of 105 roles declare it as the Jinja string "{{ 'svc-net-tor' in group_names }}", which _as_bool answers False for. Running HEAD's collect_public_ports the way HEAD's run() calls it, against the real roles directory with every role deployed, returns []. Only svc-net-tor reads True on disk, and it declares no public ports. The node onion therefore carried port 80 and port 22 and nothing else.
Reviving the sweep by reading the merged view instead would have been worse than leaving it dead: it publishes 16 ports across 7 roles, and declared is not published. openldap's ldaps 636 sits next to network.public false, Mailu's implicit-TLS ports are dropped by container_ports whenever TLS is off (which an onion deployment always is), and svc-net-tor's own SOCKS port would become an open proxy. So the source and the selection rule change together: read the variant-merged applications view, and forward only the categories a service names under ports.onion, resolving the number from ports.local when declared there and ports.public otherwise, which is the precedence container_ports publishes by.
collect_public_ports had no consumer outside this module and its own test. UDP_ONLY_CATEGORIES becomes public so the accompanying lint imports the list rather than restating it, and gains stun_turn and stun_turn_tls: those speak TCP, but the allocation TURN hands back is a UDP relay address an onion client can never receive on.
No role declares ports.onion yet, so the rendered HiddenServicePort set is unchanged by this commit alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/addons/oidc_login.yml gates on lookup('sso_oidc_plugin'), which the applications render evaluates while it is still in flight. Under that re-entry guard the merged view hands out service flags as their own Jinja text, and bool("{{ ... }}") is True. The lookup read services.sso.enabled and services.ldap.enabled as enabled on a deployment with neither Keycloak nor OpenLDAP and announced oidc_login, so Nextcloud advertised a login hand-off to a provider that is not there.
lookup('config', ...) renders the value it returns, which is why the roughly ninety '| bool' sites in meta/ do not have this problem. 238bc285c7 established reaching the merged config through the lookup loader; this one had not been converted. utils.roles.applications.config.get is the raw accessor and does not render.
Measured against the real render: with only web-app-nextcloud deployed the addon set is now empty; adding Keycloak yields sociallogin; adding OpenLDAP as well yields oidc_login. That is the resolution order the lookup's own docstring documents. All 14 nextcloud variants pin both flags as literal booleans, so no CI-deployed variant changes.
The stub next to the plugin answered every lookup with the applications payload; it now mirrors lookup('config', app, path, default). A stub resolves a path but does not render it, so it cannot see this defect at all: the regression test renders for real.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
services.tor.enabled is Jinja in 99 of the 105 roles that declare it. A lookup('tls') or lookup('domains') inside a meta/*.yml value runs while the applications render is in flight, the re-entry guard hands it the unrendered tree, and _as_bool read the flag's own source text as False. The onion injection then did nothing for any app and the clearnet domain was returned.
fa3797540a already described this and quarantined it: the render state joined the domains cache key so the poisoned map answers no later read. The value stayed wrong, which is enough until a call site bakes it into applications. roles/web-svc-asset/meta/services.yml:15 is the only one in the repo that does, and in run 34922994823 it froze https://file.infinito.test/assets into the payload while every vhost of that round was published under the node onion. web-app-dashboard then waited 60 times on a host openresty answers with ssl_reject_handshake, and the deploy died on TLSV1_UNRECOGNIZED_NAME.
_resolved_flag renders enabled, exclusive and primary and raises when template text survives, the shape plugins/lookup/proxy_pass.py already uses. plugins/lookup/config.py and plugins/lookup/email.py render before coercing for the same reason; this module was the one that did not.
Measured across the whole map: 44 of 12819 leaves change, none added, none removed, 17 roles. 42 are clearnet to onion. The two others are repairs as well: keycloak's csp frame-ancestors moves from https to http, which is what an onion node serves, and the nextcloud sociallogin provider name stops being a hostname and becomes the bounded name that test_sociallogin_provider_name.py documents.
The new tests drive the same function with Jinja instead of the literal booleans the existing ones pass, which is why those never saw this. Three of the four fail without the fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
f8da0fd122 made the pre-pull download model.source instead of the bare model.name, but the Features list still said an entry 'carries the LM Studio name to download'. An author following that sentence writes the search term that caused the failure: lms get answers 'Multiple models found. Automatically selecting the first one due to --yes' and fetches a different model, after which the gateway addresses a name nothing serves.
Name all three fields with their roles. source is the repository that gets downloaded, name is the key LM Studio indexes it under and templates/config.yaml.j2 addresses as openai/<name>, alias is what the gateway publishes.
A second entry records that the pre-pull does not load. That is what makes the upstream 'No models loaded' body readable, and it is why svc-ai-litellm warms the gateway at deploy time rather than leaving the cold load to the first consumer.
Measured on the image CI runs (sha256:48abf3b7e061): the URL form resolves the pinned repository and indexes it as qwen2.5-0.5b-instruct, the search-term form resolves to the Coder variant and indexes it as qwen2.5-coder-0.5b-instruct, and a chat completion against an empty lms ps loads the model just in time and answers 200.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 34893363518 lost the fedora workspace job at certs.sh with exit 125. docker run resolved alpine:latest itself, the token request to auth.docker.io came back as 'read: connection reset by peer', and that single transport fault ended the job. Five of the six workspace jobs pulled the same image successfully in the same run, so the cause is transport, not a Docker Hub policy response, whose signature would be toomanyrequests after a successful token exchange.
Every other network operation on this path already retries: scripts/github/runner/pull_with_retry.sh serves two workflows, scripts/image/build.sh has its own ladder for the build frontend, and cli/administration/deploy/development/compose.py retries compose up six times. This call site had no attempts at all, so pre-pull through the existing helper rather than a new loop.
The attempt budget is 3x10s instead of the helper's 7x20s default because this runs as a prerequisite on the interactive local path, where two minutes of sleeping without a network would be felt. The call stays non-fatal so a locally cached image still carries an unreachable registry, the same semantics 0df75d83a6 established for the buildx bootstrap; set -euo pipefail is why it needs the if-guard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
15a6251d7 introduced the merge but tripped over three of the repository's own rules and over the lookup's existing contract test.
_templated called templar.template(value, fail_on_undefined=False). A templar whose signature is template(self, value) raises TypeError on that kwarg, the except swallowed it, and the value came back unrendered: the lookup returned ['{{ vault_cf }}'] where ['secret-cf'] was due. It now falls back to the positional call, the way utils/cache/base.py has always done it.
The SPOT read goes through load_yaml_any instead of yaml.safe_load, which tests/lint/repository/yaml/test_no_direct_calls.py requires so the file is parsed once per process. The module docstring drops the inventories/development/default.yml literal, which tests/unit/python/cli/administration/deploy/development/test_inventory_vars_file_spot.py whitelists by path.
test_missing_api_global_raises_unknown_key asserted exactly the behaviour that broke CI: a scope without API had to raise. It now asserts the fallback, and two new cases pin the edges. A provider the scope omits still resolves; a provider the declaration does not know still raises.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
e90cac4ee2 staged the lint but not what the gate rewrote after it. Whitespace only; the join, the fix hint and the assertion are unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
691911279f moved roles/svc-net-tor/meta/services.yml from alpha to beta but staged only the source. The README role table is generated from those declarations, so the two had drifted apart. No behaviour attaches to the table; cli/meta/ci/validate.py reads the meta file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
API is a plain mapping, so an inventory that configures three providers replaces the whole declaration from group_vars/all/18_api.yml instead of merging into it. inventories/development/default.yml declares openai, anthropic and openrouter, so github, google and jira did not exist in CI at all and lookup('api', 'github.client_id') raised.
A raising lookup inside a value the applications renderer is templating does not fail the render: _templar_render_best_effort returns the string untouched and, for a value carrying a non-env lookup(, refuses the fallback as well. The raw template text reached plugin_value.enabled | bool, which coerced it to False and emitted the deprecation warning that fails the deploy gate. Run 34893363518, job 104153194109 lost web-app-nextcloud to exactly that, for three integrations at once. The configvalue entries would have stored template text as the client id.
utils/api.py merges the scope's override over the declaration, so an override narrows values and never the provider set, and holds the enable rule: every credential a provider declares must carry a value. The api lookup reads through it; the new api_enabled lookup gates on it and returns a real boolean instead of a string a filter has to coerce.
The same class sat in group_vars/all/16_ai.yml and roles/svc-ai-litellm/meta/secrets.yml as bare API.<provider>.<key> dereferences; both now go through the lookups.
Two lints keep it shut. test_addon_enabled_renders_to_bool renders the whole applications map against a scope with API blanked, the most hostile override an inventory may legally write, and requires every addon gate to come out a boolean. test_api_accessed_via_lookup forbids a bare API dereference in Jinja under roles/, group_vars/ and inventories/.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
roles/test-e2e-cli and roles/test-e2e-playwright run only in CI, against a
throwaway cluster whose credentials are generated per deploy and discarded with
it. no_log there protects nothing and costs the one thing a red test job exists
for: a masked task reports that its output was hidden, and the whole matrix
round has to run again to learn what a single assertion saw.
The rule is absolute, so it ships without a nocheck escape hatch. A test task
whose output looks too sensitive to print calls for extracting the field the
assertion reads, not for blinding the task.
A second case asserts the glob still reaches the roles. A path rule that
matches nothing passes forever, which is how test_gateway_backend_registered
nearly went quiet earlier today.
Both roles are clean right now, so this starts green. Reproduced against an
injected no_log in run_one.yml to confirm it is not vacuous.
Note for whoever hits it: test_credential_tasks_are_masked scans roles/*/tasks
and roles/*/handlers without excluding these two, and requires no_log on a task
whose printed arguments name a password, secret, token, api_key or private_key.
Nothing in the test roles trips it today, but a future task that does would
deadlock the two rules. The exemption belongs in that lint's file selection.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Job 104155149593 failed its own CLI test with
[FAIL] svc-ai-ollama is not deployed yet the gateway publishes
['qwen2.5:0.5b']
In that round ollama is deployed and the variant enables it, so the gateway was
right and the probe was wrong. templates/test.env.j2 renders through
lookup('template') in run_one.yml, which evaluates in the current play host's
scope, while config.yaml.j2 renders through _render_one_template.yml with
delegate_to STACK_HOST and run_once. On a worker the backend flags read false
and the probe compared its own view against a gateway built from another one.
The three flag-derived values are now frozen next to the config render, with
the same delegate_to plus run_once. Which host run_once elects does not matter;
what matters is that it is the same host for both, and that follows from the
same construct in the same play.
CLI_LOCAL_CID stays per host. 9fb8f63286 introduced it precisely because the
container lives on whichever node the service scheduled it to, and the test
script task carries no run_once, so it runs once per play host against its own
resolved node. Freezing the render itself would have pinned one node's
container id into every host's env and broken the CLI test for every role that
execs into its own container, not just this one.
Two facts this rests on, both measured rather than assumed: a run_once set_fact
does reach the other hosts, as web-app-gitea's mcp provisioning shows by
asserting on gitea_mcp_token from a task that carries no run_once; and
03_destructor.yml includes test-e2e-cli as a stage of the same play the deploy
ran in, so the fact is still there when the template renders.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replaces the note that treated a rotated seal key as local stale state to purge
around. It is the platform's behaviour, the role now carries it, and what is
left to know is where the window ends: two generations, with MODE_RESET the
concrete way to fall out of it, since it removes DIR_COMPOSITIONS while the
named volumes survive.
Every one of these breaks into something that looks like ordinary drift rather
than a defect: a state read moved after the backend include leaves the node
sealed, a token helper fed the first login's stdout gets an empty string on
every rotated deploy and fails later at the first write, an ungated SecretID
swap answers 500 on every deploy that did not rotate, and a swap that never
destroys the old SecretID revokes nothing at all.
Closes the rotation: the instance is moved onto this run's credentials and the
result is written to .applied.json, which is what lets the next deploy log in
before it re-pins in turn. The file sits in the compose project directory, which
already holds the seal key in plaintext in .env/env, so this adds no new
exposure class; it is written 0600 root-owned and last, after both pins, so a
run that dies partway leaves the previous generation intact.
custom-secret-id is not idempotent -- writing a SecretID the role already holds
answers 500 'SecretID is already registered' rather than doing nothing -- so the
swap is gated on the value having changed. The previously registered SecretID is
destroyed afterwards, without which the rotation would add a credential and
revoke none.
A deploy is handed credentials the instance has not been told about yet, so the
inventory login is expected to fail on every rotation. Try it, fall back to what
the previous deploy recorded, and fail with a message naming the state file when
neither is accepted -- a bare 403 here reads like an unrelated defect.
Both logins are non-fatal so the guard is what reports the failure. Whichever
one succeeded also identifies the SecretID the instance actually holds, which is
the value the rotation has to destroy; taking it from the state file instead
would destroy the wrong one whenever the inventory credentials still worked.
The static seal supports n-1 to n rotation. Emitting the previous pair only
while the two keys differ lets the node unseal with the old key and rewrap to
the new one; a first deploy still renders a single-key stanza.
sys-stk-backend renders env.j2, which is what carries the previous seal key
into the container. Reading the applied state after that include leaves the
node with no way to decrypt its own store, so the read has to precede it.
The platform regenerates generated-algorithm credentials on every deploy, so
the seal key and AppRole this role is handed are not the ones the running
instance holds. Derive current_key_id from the key itself -- upstream requires
the identifier to match its key, and a derived id cannot drift out of step with
one. Carry the previously applied credentials and the two rotation predicates
alongside it, so the config render and the AppRole swap can both tell a
rotation from a steady state.
Job 104155149606 died at the warm request with
litellm.APIConnectionError: OllamaException - Cannot connect to host
ollama:11434 [Connect call failed ('192.168.244.10', 11434)]
192.168.244.10 is not ollama. It is the manager node on the lab bridge: the
name never resolved through the overlay, because the litellm task was attached
to ingress, litellm and postgres and to nothing else. The lmstudio round is the
control, where the same replica does carry the lmstudio network.
The attachment comes from _is_consumer, which reads svc-ai-litellm's own
services.<backend>.enabled and .shared. Variant 1 pins both false. The config
template meanwhile asked `'svc-ai-ollama' in group_names`, which answers a
different question: is ollama deployed anywhere in the round. It is, six edges
away, because litellm pulls postgres, postgres pulls tor, tor pulls the mirror,
the mirror pulls prometheus and prometheus declares flowise, which declares
ollama. So the gateway advertised a route it had no network for.
The template and LITELLM_CHAT_MODEL_SERVED now read the same two flags the
network attachment reads, through one constant each. Publication and routing
cannot diverge any more, and in a variant that declines a deployed backend the
warm task no longer runs at all.
LITELLM_CHAT_MODEL and LITELLM_BACKEND_ROLES are deliberately left alone.
5324eadd8b made them inventory-scoped because web-app-mattermost's assert, the
one task in its file carrying neither delegate_to nor run_once, re-templates in
each worker's own scope; web-app-discourse's addon tasks carry no delegate_to
at all. A host-scoped model name fails there again. Both templates here render
delegate_to STACK_HOST plus run_once, where group_names is authoritative.
meta/services.yml keeps its `in group_names` flags, so the five separate
substring detectors that decide whether a flag counts as dynamic keep seeing
svc-ai-litellm.
Measured over every (deployed, pinned) combination against the real
expressions and the real template: the model consumers are told to ask for is
published whenever the warm task runs, and nothing is published without its
flag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lifecycle is a selection axis rather than a label: cli/meta/ci/validate.py
hands --lifecycles to matrix.entries_of, so a run whose envelope admits beta
but not alpha picks svc-net-tor up from now on. Recent sweeps ran with an empty
envelope and are unaffected.
The maturity call is the operator's; this commit only records it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Job 104153193860 died at "Ensure the administrator account carries the current
password" with:
Validation failed: Password is the same as your current password.
Discourse's UserPasswordValidator adds :same_as_current when password_hash_
changed? and confirm_password?(value) are both true. Assigning the current
password again satisfies both, because the reassignment salts a fresh hash. The
script assigned unconditionally, so any run that did not follow a rotation was a
hard failure rather than a no-op, and the task was never idempotent.
The assignment is now guarded by confirm_password?, which makes the rule
unreachable: it requires confirm_password? to be true, and that is exactly when
nothing is assigned. A new user is covered without a new_record? check, since
User#confirm_password? returns false when no user_password row exists.
web-app-openproject/files/ruby/ensure_admin_user.rb already guards the same way
with check_password?; this brings Discourse in line with it.
The two saves collapse into one while we are here. The second existed only to
set `active`, and one save writes username, password and active together.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 34893750601 lost four jobs at svc-ai-litellm's warm request, every one of
them on the same upstream body:
No models loaded. Please load a model in the developer page or use the
`lms load` command.
The catalogue handed `lms get --yes --gguf` a bare name, which that command
treats as a hub search term. Its own help spells out the consequence: "If there
are multiple staff picks matching the search term, the first one will be used."
For `qwen2.5-0.5b-instruct` the first pick was Qwen2.5-Coder-0.5B-Instruct, so
the key LM Studio indexed never matched the `openai/{{ _model.name }}` the
gateway sends upstream, and just-in-time loading had nothing to load.
A `source:` now carries the full HuggingFace URL, the one spelling lms get
resolves deterministically; a bare owner/model pair addresses an LM Studio Hub
artifact instead and fails even for a model already on disk.
Measured against the shipped image rather than reasoned: the URL yields modelKey
qwen2.5-0.5b-instruct, `lms ps` is empty right before the request, and a plain
/v1/chat/completions for that name answers 200 and leaves the model resident. No
load step belongs here, because the server reports "Just-in-time model loading
active" and does it itself.
The embedding entry is renamed in the same breath. LM Studio indexes that model
as text-embedding-nomic-embed-text-v1.5, so the gateway was addressing a key
that never existed there either, and the first consumer to embed would have hit
the same 404.
The three fields stay separate and each has exactly one reader: Ollama pulls by
alias, LM Studio downloads by source, the gateway addresses name.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 34883491022 lost all 15 of its jobs and run 34883296966 four more, every
one of them on the same line and before any role deployed:
ERROR: the value supplied for 'credentials.openai_api_key' does not match
the pattern its schema declares: \S{20,}
786899700f gave the three provider keys a regex and wired validation onto the
value that sits beside a `default:`. That value is the default itself. The
credential leaf writes meta["default"] into the inventory verbatim, so the next
read hands `{{ API.openai.api_key }}` back as if an operator had typed it. The
literal is 24 characters and carries spaces, so \S{20,} cannot match it, and
svc-ai-litellm sits in every closure that holds a litellm consumer, which is
why roles as distant as Baserow and Memcached died with it.
A pattern describes the rendered secret. Template text is not that value, so
validate_supplied_value returns on it, next to the guard that already returns
on an empty one. One guard covers all three call sites. It can only skip a
check, never add one, and the three roles carrying a Jinja default for far
longer (listmonk, keycloak, espocrm) declare no regex and keep their behaviour.
The test class asserted that `<your-key-here>` is rejected but never that the
schema's own default survives, so the gap shipped green.
test_every_declared_default_survives_its_own_schema closes it for every default
the file declares, which is why _schema() widened into _credentials().
Reproduced at the call site that failed: the credentials subprocess taken from
the job log exits 0, and the value it emits decrypts back to the template.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make quality-high pass 75 rewrote it: a concatenation whose right operand is a
multi-line literal gets parenthesised, with the operator leading its own line.
The string itself is unchanged, so
test_narrowing_the_credentials_still_rotates_every_user keeps the declaration
that separates web-app-keycloak from web-app-a and still fails when the user
half of the rotation is narrowed along with the credentials.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The matrix gate rotates credentials between the two deploy passes so PASS 2
has to carry fresh values. It rotated every application block in the host_vars
file, and on a matrix inventory that is not the round. Provision applies its
--include as a strict intersection (filters.py:57-62), so it generates
credentials for at most derive_includes, while mirror_overrides.py adds a block
per mirror artefact carrying only image and version. Rotating those costs one
credential subprocess each for values provision never wrote.
matrix.py now passes --include derive_includes(app_id, variants=round_variants),
the same call 02_provision_inventory.sh:11 feeds provision, from the same
round_variants the provision step got one call earlier. The rotation set is
therefore a superset of what provision generated: every database_password still
rotates, because utils/manager/inventory.py:236-242 writes it under an id from
that same filtered list. Measured for web-app-wordpress: 35 ids.
The scope stops at the credentials. reset_credentials takes credential_ids
beside application_ids because the two halves shared one list, and narrowing it
narrowed _drop_user_passwords too: that resolves through
required_user_policies(roles_dir, application_ids), so 60 of 66 declared users
would have stopped rotating, root, backup and every mcp-* bot among them. The
user half keeps the full document list and is byte-identical to before.
The test fixture declared the same users in both roles, which made the
regression test pass against the regression. web-app-keycloak now declares a
user web-app-a does not, so the assertion has something to bite on.
The CLI aborts when --include matches no block instead of rotating nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two probes that describe a host were re-running per application. Measured in CI
run wp-swarm-tor2: 42 invocations of the pip probe and 44 of the NFS assert.
sys-pip-install guarded only its dependency include. Everything expensive sat
below that guard in main.yml, unguarded: resolving the interpreter, probing pip
for --break-system-packages, checking for `packaging` and bootstrapping it. All
four ran on every one of the 21 include sites. They move into 00_core.yml, where
the existing `when: run_once_sys_pip_install is not defined` covers them, and
main.yml keeps the guarded include and the pip task itself. The role recap reads
sys-pip-install ---- 87.35s.
The two facts this freezes per host are run constants: ansible_python_interpreter
is pinned once in group_vars/all/00_general.yml and no call site overrides it,
and all 21 sites pass global_install: true. main.yml now defends the probe fact
with `| default(false)` the way 00_core.yml already did, because the probe sits
behind `when: global_install | bool` and a first invocation with false would
otherwise leave it undefined for every later one.
The NFS assert emulated a per-host check with run_once plus delegate_to over
groups['svc-swarm-node'], and the include it lives in happens per application.
It becomes a plain per-host task in sys-svc-compose/00_core.yml, which main.yml
already gates once per host per play: 44 instances and 132 remote commands
become 4 and 4. Dropping run_once also removes a real fragility — at log line
35355 only wrk-01 and wrk-02 reach the include because mgr-01's flag is already
set, so run_once there designates a host that skipped.
It reaches the same hosts as the loop it replaces, measured rather than assumed:
sys-svc-compose/00_core fires only on mgr-01, wrk-01 and wrk-02, the backup play
never runs it, and nfs-server and bkp-01 never execute it. The two conditions
that do not travel are a per-application volume count, meaningless for a
per-host check, and an exemption for manager-placed roles — mgr-01 answered the
assert ok 14 times out of 14, so the exemption was hiding nothing.
The assert is appended to 00_core.yml rather than prepended: utils/once/flag.yml
must stay that file's first task for test_service_core_first_task_run_once.
Worth 128.6s: 84.6s from the pip probe and 44.0s from the assert.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An unset secret or variable resolves to the empty string, not to an error, so
an undocumented one is invisible twice: the administrator cannot know to set it,
and the run that needed it degrades quietly instead of failing. The guard reads
every ${{ secrets.X }} and ${{ vars.X }} out of .github/workflows and demands
the name in the page that owns it, secrets.md or configuration.md.
It compares the reference against the page rather than the page against itself,
so it also catches a doc naming something the workflows no longer read.
Three gaps were already there. DOCKERHUB_USERNAME and DOCKERHUB_TOKEN
authenticate the image mirror against Docker Hub and lift the anonymous
pull-rate limit; both are declared required: false and the login step is gated
on them being non-empty, so their absence does not fail a run, it walks the
mirror into the rate limit. Neither was documented anywhere.
PLAYWRIGHT_KEEP_ALL is the sharper one: configuration.md listed
INFINITO_PLAYWRIGHT_KEEP, which is the name the workflow forwards it under, not
the name an administrator sets. Following the documentation produced a variable
nothing reads.
Commented-out lines are skipped. cron-security-scorecard.yml carries
`# repo_token: ${{ secrets.SCORECARD_TOKEN }}` as an option nobody has taken,
and demanding a doc row for it would describe an input no run asks for.
GITHUB_TOKEN is exempt in _PROVIDED_SECRETS because GitHub mints it per run; it
is not administered and cannot be set. A second case asserts the scan finds
workflow files at all, so the guard cannot pass vacuously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gateway could reach exactly one remote provider, through a bare
OPENROUTER_API_KEY group_var with an empty-string default and an `{% if %}`
around a single hardcoded model entry. OpenAI and Anthropic had no path at all.
All three are now declared in roles/svc-ai-litellm/meta/secrets.yml, which
mapping.py already calls the application config schema, so a key lands at
applications.svc-ai-litellm.secrets.credentials.<provider>_api_key like every
other credential. Each carries the `type` and `regex` its value must satisfy and
defaults to the central API.<provider>.api_key block, which keeps one
operator-facing place to fill a provider in while the role owns the shape.
type and regex had to become real to be worth declaring. 192 `validation:`
blocks sit in the role schemas today and not one line of Python enforces them,
so a `regex` would have joined a pile of decoration. validate_supplied_value
now runs on every path a value arrives from outside — the --set override, a
value already in the inventory, and the value standing next to a `default:`.
A generated value stays exempt because the algorithm that produced it is named
by the same schema and generate_value takes no schema argument. min_length is
deliberately left unenforced: the generators ignore it, so holding them to it
would fail where nothing is wrong.
The pattern is written for the paste, not for the vendor. OpenAI moved issuance
from sk- to sk-proj- without notice and Anthropic issues sk-ant-api03-; a schema
pinning a prefix would reject a valid key and abort inventory creation. `\S{20,}`
accepts every form those three issue today and one nobody has announced, and
rejects a trailing newline, an embedded space, a truncated copy and a
<placeholder>. A key pasted into the wrong provider's slot is caught at deploy
instead, where the route it publishes answers nothing.
The gateway iterates one LITELLM_REMOTE_BACKENDS list instead of three `{% if %}`
blocks; config.yaml.j2 and env.j2 read the same list and selectattr('key')
decides. LITELLM_CHAT_MODEL falls back to the first configured remote alias
rather than to a hardcoded openrouter/auto.
For CI the keys travel as INFINITO_API_{OPENAI,ANTHROPIC,OPENROUTER}_KEY through
default.env, the passthrough allowlist, the infinito service in compose.yml, the
workflow env block and the development inventory. The compose leg is not
optional: the env lint rejected the change without it, because a key the
inventory reads but compose does not forward resolves to empty inside the
container with no error. The workflow reads `secrets.X || vars.X` so a variable
works as asked while a secret wins, which matters because a variable is not
masked and the development inventory sets MASK_CREDENTIALS_IN_LOGS: false.
An unset key is not an error anywhere: the provider's routes stay unpublished
and the CLI test asserts their absence rather than skipping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gateway publishing a model proves nothing about a backend answering it: LM
Studio shipped a published lmstudio/default route for as long as it had no
model to load. This CLI test closes that gap by asking the running gateway what
it serves and comparing it against what the deployment enabled.
templates/test.env.j2 is both the input and the entry point — discover_cli_roles
finds a role by that file and filters on group_names, so the test runs only
where the gateway is deployed. meta/tests.yml names the container service key so
run_one.yml resolves the node holding it in swarm, and allows 600s for a cold
model load.
probe.py runs inside the gateway container, where the API answers on loopback
and the backends resolve by name, and uses the standard library only because
the image ships the litellm app rather than a test toolchain. It asserts that
every declared model is served, that the LM-Studio-only aliases are present
exactly when that backend is deployed, that no Ollama-backed model is published
without Ollama, that LITELLM_CHAT_MODEL is routable, that an empty deployment
publishes nothing, and that a real completion comes back non-empty.
The fourth assertion is the one worth having: LITELLM_CHAT_MODEL is built from
lookup('deployment').inventory while the config is built from this host's
group_names. Two sources for one fact, and this is where they are made to
agree.
The alias list is rendered whether or not LM Studio is deployed. Gating it
would empty the set in exactly the case the negative direction needs, turning
"a route must not stand without its backend" into a vacuous check.
The verdict logic sits in a pure evaluate() so it is exercised without mocks:
four deployment shapes, each with its positive and negative cases, including a
both-backends case that asserts a shared alias proves nothing about LM Studio
and only the exclusive one counts.
files/test/.nocheck is required rather than convenient. role-files-layout sorts
files/ by language while test-e2e-cli stages files/test/ under that fixed path;
the other 29 CLI test directories carry the same marker.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
328113e7c added a guard that reads every group_vars/all file to confirm each
GROUP_GATED_CONSTANTS entry is really defined, and read them with
path.read_text(). test_no_uncached_file_reads rejects that under tests/: the
cached read_text shares one decoded read per file across the whole session, and
these files are read by several other tests already.
The nocheck escape does not apply. It covers reads outside the project tree,
and group_vars/all is inside it.
What the guard asserts is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Variant 0 pinned both local backends on, so LITELLM_CHAT_MODEL always resolved
to an Ollama model and every completion went there. The LM Studio route was
published and never travelled. A third variant turns Ollama off and LM Studio
on, which is the only shape that exercises it, and variant 0 pins LM Studio
literally off so a WordPress round stops co-deploying a backend it never
prompts: 27 apps become 26.
bond drops to 0.5 and that is what makes the variant legal. test_order's
_direct_deps filters services on bond >= 1.0 before building a closure, so at
0.5 LM Studio leaves that computation and the new variant no longer violates
"variant 0 is the maximal deploy footprint". The deploy planner does not filter
on bond — resolve_service_dependency_roles_from_config reads enabled and shared
only — so the variant still deploys it. test_bond_baseline takes its bond < 1
branch, which asks for any variant pinning the flag true rather than variant 0
specifically, and variant 2 supplies it. No suppression and no guard change.
bond < 1 states that the provider is a separate-host partner rather than a
same-host service. Reaching the gateway through LMSTUDIO_BASE_LOCAL_URL, a
container-network hostname, is consistent with that.
Measured against the planner: variant 0 resolves 21 apps without LM Studio,
variant 1 stays at 1, variant 2 resolves 8 apps with LM Studio and without
Ollama, and a WordPress round 0 drops from 27 to 26.
The variant is reachable only when svc-ai-litellm is swept as a primary row and
pinned to it: legacy_resolver clamps a non-primary role to the round index, and
a two-round WordPress job never reaches index 2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fourteen roles repeated the same two lines to say they use the gateway: a
`shared` flag gated on group_names, and an `enabled` flag conjoining that gate
with LITELLM_CHAT_MODEL_SERVED. Both are now single-sourced.
`shared` becomes literal true. The flag means the provider is a shared instance
rather than one deployed per consumer, and a central gateway is always that;
the dynamic form only said "shared if deployed", which `enabled` already
decides. The 50 shared pins under `litellm:` leave meta/variants.yml with it.
The tree had already noticed: 20 of those 50 lines carried
`# nocheck: variants-static-override`, two of them with copy-pasted text about
container_backup. This narrows e5de4e085a for one service key rather than
reversing it; every other dynamic flag keeps the group_names form.
`enabled` becomes `{{ LITELLM_USABLE }}`, defined in group_vars/all/16_ai.yml.
That shorthand is only possible because is_explicit_truth learns about it.
The resolver reads a dependency out of the flag TEXT — "in group_names" in
value — so a constant hides the role id and the provider silently leaves the
static closure of every role referencing it. Measured: the old expression
resolves True, `{{ LITELLM_USABLE }}` resolves False. GROUP_GATED_CONSTANTS
names the constants whose own definition carries the gate, and the docstring
states the obligation: a SPOT of this kind that is not listed there drops its
provider without a word.
The wider fix, accepting any Jinja as dynamic, was measured and rejected: 35
other flags carry a Jinja value without the marker (MCP sidecars, javascript
and SSO gates, recaptcha), and they would all enter closures they are not in
today.
A literal shared flag is no longer dynamic, so test_dynamic_flags wants the
`# nocheck: dynamic-flag` its own message documents for legitimate exceptions;
where a marker already existed the rule list is extended rather than a second
comment added.
Behaviour is unchanged where it counts. The closure keeps litellm with the same
app counts (wordpress 26, openwebui 23, hermes 23, mediawiki 22, flowise 23),
unrelated Jinja still resolves False, and the truth table of the new enabled
form matches the old one in all four served/deployed combinations. Three roles
resolve without litellm, and `git show HEAD:` proves that predates this change:
zammad pins enabled false in variant 0 at line 39, nextcloud at line 97.
Six unit tests cover the resolver, including one asserting that every
registered constant is actually defined in group_vars, so an entry cannot
become a dead promise.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-ai-lmstudio had no model-load step at all: tasks/00_core.yml set the
once-flag and the container network, nothing else. The gateway published
lmstudio/default anyway, so the route existed and nothing could answer it.
wp-compose-tor.log:48994 shows the route written into the config, :51770 shows
the warm step reaching qwen2.5:0.5b on Ollama instead, and the green ai-engine
spec ran against that. LM Studio was deployed, published and never prompted.
The pre-pull mirrors the one that works. tasks/01_manager_ops.yml resolves the
hosting node and container id, loops the declared models through
`lms get --yes --gguf`, overlaps them with async/poll and reaps before
playwright, exactly like svc-ai-ollama. That `lms` exists is measured, not
assumed: /app/daemon-run.sh:67 in the image starts ./.bundle/lms server start,
and `lms get --help` documents -y for scripting and --gguf for the format.
Synonymity needs both namespaces in one place. Ollama knows qwen2.5:0.5b, LM
Studio knows qwen2.5-0.5b-instruct, so AI_LOCAL_MODELS carries both per entry:
`alias` is what the gateway publishes and what Ollama pulls, `name` is what
`lms get` resolves. Both roles now reference that one list, and config.yaml.j2
publishes each backend under the alias, routing Ollama to ollama/<alias> and LM
Studio to openai/<name>. The lmstudio loop skips an alias Ollama already
publishes, so two backends never produce a duplicate model_name and Ollama
keeps precedence. lmstudio/default is gone; the model name is now the same
whichever backend answers it.
Ollama's preload_models changes shape from a list of strings to the {alias,
name} entries. Every consumer moves with it: the pull command, the template
loop, and both expressions in 16_ai.yml. An inventory that overrides
`applications.svc-ai-ollama.services.ollama.preload_models` with the old string
list will break at map(attribute='alias'); this is a deliberate shape change,
in the spirit of 7b15c61dfc.
The CI model list is not a property of an installation, so AI_LOCAL_MODELS_CI
lives in inventories/development/default.yml, whose own header scopes that file
to the cross-cutting environment knobs. test_variable_definitions only knew
group_vars, so it reported the constant as undefined; the shipped inventories
are a definition source too, because the deploy CLI bakes them into host_vars,
and the guard's own docstring already claimed "defined somewhere in the
repository".
A latent crash came out of this: `| list | first` raises on an empty sequence,
so a backend deployed without preload models blew up LITELLM_CHAT_MODEL. Ollama
with only embedding models could already hit it; the existing test dodged it by
rendering only _SERVED. `| default('')` now yields the empty string that
_SERVED already reports as false.
Tests: config.yaml.j2 is rendered rather than read, over ollama-only,
lmstudio-only, both and neither, asserting the published model_name set, that
both backends name the same alias, that a shared alias is published once and
routed to Ollama, and that the render is valid YAML. test_litellm_chat_model
gains the lmstudio branch, the no-preload case and the same-model assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
113113e553 parks the carrier after the token store, 165 lines below the merge
block, so each of the block's six tasks still renders the applications SPOT
cold in its own forked worker. Measured in CI run wp-swarm-tor2 they cost
323.51s in the PASS 1 main play, 180.39s in the backup play and 479.79s in
PASS 2.
The cost is the render, not the work. Two tasks in that block share the exact
construct `defaults_X | combine(X | default({}, true), recursive=True)`: design
takes 0.247s and service_provider 75.94s. The difference is that
defaults_service_provider is dense with lookup('config'/'users'/'domains'/'tls')
and every one of them lands in get_merged_applications.
The carrier cannot simply move up. get_merged_applications serves the carrier
fact whenever the key matches, and the carrier action goes through that same
path, so a second carrier task would be handed the first one's payload instead
of rendering. merged_applications_cache_key reads neither group_names nor the
token file, and with a real templar 71 of 265 applications differ by
group_names alone, so a carrier left standing across add_host or the token
store would be served where a fresh render belongs.
So the block opens its own carrier and drops it again at the end. Nothing
outside the block changes: every task from sys-version to the stage-wide
carrier renders exactly as before.
Placement follows measurement. A per-key perturbation test over the whole
265-application payload, with the credential regeneration noise floor
subtracted and every role in group_names, finds exactly two reads of the seven
facts the block sets:
.web-app-matomo.services.matomo.excluded_ips[0] <- networks
.web-app-wordpress.services.wordpress.title <- service_provider
networks is the block's first task and service_provider its last, which is why
the carrier opens after the one and is dropped before the other takes effect.
The block's tasks already render before service_provider is merged today, so
the frozen payload is what they would have rendered anyway.
The action gains a `clear` argument so the playbook never spells the private
fact name. Parking the carrier serves a payload in 0.000s against a 3.417s cold
render; a wrong key and a cleared fact both force the render again, which is
what makes the drop task work.
Worth roughly 806s of the 20199s job, against a floor of 673.6s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_file_spot failed the gate on the pathlib chain that spelled
meta/services.yml out by hand. Both paths now come from utils.roles.mapping as
ROLE_FILE_META_SERVICES and ROLE_DIR_META_ADDONS, following ea37027f49 and
8243aa571b.
All four chains move, not only the reported one. The lint reads the role-file
shape and lets a "meta" / "addons" directory chain through, so leaving those
behind would have split one fixture across two sources of truth, which is the
split the lint exists to prevent. The constants are slash-bearing strings that
Path accepts as a single segment, so the paths are unchanged.
Only the throwaway role built under tempfile in TestCollectEntriesCoversAddons
is affected; its assertions on entity_path and config_path.name are untouched.
autoformat then reflowed the two write_text concatenations, which fit on one
line once the names shortened, and the generator in test_oidc_setting_order.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
DISCOURSE_LDAP_AUTH_PLUGIN_REPOSITORY and DISCOURSE_LDAP_AUTH_PLUGIN_REF date
from fd22d34559, when every role mirrored its git URL into a per-role
REPOSITORY/REF pair. 84a4b04f0 pointed the clone loop in config.yml.j2 at
plugin_config.config.repository and plugin_config.config.ref instead, reading
the addon declaration directly, and left the mirror without a reader.
They are not merely noise. Both are unconditional top-level vars whose
lookup('config', ...) carries no fourth argument and is therefore strict, and
both address a path that exists only while this one addon keeps this one id.
Renaming or removing discourse-ldap-auth would have failed every Discourse task
at resolution time over a value no caller wanted.
The surviving LDAP path is untouched: DISCOURSE_LDAP_ENABLED reads
services.ldap.enabled, and stack_host/02_credentials.yml takes the site
settings through DISCOURSE_PLUGINS['discourse-ldap-auth'].config.site_settings.
A tree-wide grep for DISCOURSE_LDAP_AUTH_PLUGIN now returns nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard recognised meta/services.yml as the only place a git URL may be
declared. 84a4b04f0 made meta/addons/<id>.yml the one place a Discourse plugin
pins itself, so the addon files now carry the clone URL literally, and
_iter_role_files picks up every .yml under the role: each declaration reported
itself as an undeclared reference.
Two halves. The declaration set is now the union over meta/services.yml and
every meta/addons/*.yml, so an addon pin legitimises that URL for the whole
role, not just for the file holding it. Path.glob on a missing meta/addons
yields nothing, so roles without addons need no special case. The self-skip
that kept services.yml out of the scan is now a membership test against exactly
those declaration paths.
Scope stays per role: declared is rebuilt inside the role loop, so a
declaration in one role legitimises nothing in another. The failure message and
the docstring name both sites, otherwise they send the next reader to the wrong
file.
This is the test-side counterpart of e4141cb826, which taught collect_entries
to see the same addon refs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Only two files were touched by both sides. env.js merged automatically: the
incoming side owns recordVideoOptions and this branch had cherry-picked that
same function from it, so the result carries one definition. README.md is the
generated roles index and both sides append rows; resolved as a union, taking
the incoming side for the three rows it edits (OnlyOffice, Open WebUI, OpenClaw)
and keeping ours only for OpenBao, then re-sorted.
The incoming side adds lint rules and resources that this role then violated.
Five fixes, each the remedy the failing rule names:
* services.openbao.ports.local.http 8081 -> 8093; svc-ai-litellm now holds 8081.
* meta/networks.yml subnet 192.168.221.0/24 -> 192.168.222.0/24; svc-ai-mcp-adapter
now holds 221, and the suggester only sub-allocates inside occupied /24s, so
the next free block was taken directly.
* services.email bond 1 -> 0; the entry is disabled and the resource model skips
it before reading the weight.
* README Quick Setup regenerated from the template it had drifted from.
* Credits link the implementer, matching how they are credited elsewhere.
* docs/requirements/035-mcp-proxy-expansion.md disposes of web-app-openbao as
privileged_control_plane, which is what a secrets manager is, and the list
count follows.
Verified against a worktree of #454 alone so the baseline is measured rather
than assumed: lint 501 passed with only the branch-name rule failing, which
fails identically on that baseline because the worktree branch is named
merge/*; unit 5665 passed with the same three host-environment failures the
baseline has; integration 324 passed, 0 failed.
collect_entries walked roles/*/meta/services.yml only, so a repository and ref
declared in an addon file was invisible to it. That mattered once 84a4b04f0
made meta/addons the one place a Discourse plugin is pinned: the pins would
have left the updater's view entirely. The other updater cannot stand in,
utils/update/docker.py reaches version: through iter_role_images and skips any
service without an image:.
e317f08ac9 built the addon contract with its own catalog-driven SPOT in
utils/update/addons.py and left repository.py untouched without saying why, so
this widens the ref path rather than replacing the catalog one; an addon can
be maintained by either.
The per-file work moves into _entries_of_file so services.yml and every
meta/addons/*.yml share it. Addon entries carry an ("addons", <stem>) prefix
on their entity path, which keeps them distinguishable from a service entry
holding the same ref. The line index, the semver filter and the per-file
"# nocheck: repository-version" suppression apply unchanged.
Two tests cover it: an addon ref is collected beside a service ref with its
own path and config_path, and a suppressed addon ref yields nothing. Against
this repository the extension currently collects nothing, because every addon
ref is a branch and is_semver rejects those; the tests are what prove the walk
rather than the absence of output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pinning sat in two places and covered one plugin out of four.
discourse-ldap-auth carried repository and ref both in the services.yml
plugins map and in its addon file, the duplicate silenced by a
"# nocheck: repository-declared". The three official plugins carried neither,
so the template hardcoded https://github.com/discourse/<name>.git and cloned
whatever the default branch held, while only the LDAP one got a --branch
through a special-cased block beside the loop.
The addon layer was already the real source: DISCOURSE_PLUGINS reads
lookup('config', application_id, 'addons'), and the services.yml plugins block
survived only in nocheck comments pointing at it. It is dropped.
Every cloned addon now declares its own repository and ref plus the update
contract, so the template reads the declaration instead of inventing it and
one loop covers all four. Checked against the assembled addon map: the four
with source != bundled all carry repository and ref; discourse-ai carries
neither and is bundled, which the loop condition already excludes.
Pinning these while 4c2e480d5 pins the core is what upstream expects.
docker_manager, discourse-akismet and discourse-activity-pub each ship a
.discourse-compatibility map from core versions to plugin commits, and the
bootstrap falls back to the matching commit for an older core. Their newest
threshold is 2026.4.0-latest, below the pinned core, so the declared branch is
the intended pairing rather than a drift.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
params.version sat commented out in app.yml, so Discourse silently inherited
tests-passed and every bootstrap could pull a different application revision
with nothing recording which.
Declaring it as services.discourse.version would have been invisible to both
updaters: utils/update/docker.py reaches version: only through
iter_role_images, which skips any service without an image: (discovery.py,
"if not image or not version: continue"), and Discourse is built by the
launcher and has none. A nested application entry carrying repository: and
ref: lands in utils/update/repository.py instead, whose walk_repo_ref_pairs
descends the whole tree; the collector now reports
discourse/application | discourse.git | v2026.8.0
so git ls-remote --tags keeps the pin current rather than freezing it, and the
external URL test sees the repository through iter_non_ignored_files. This is
the shape 7b15c61dfc established when it migrated version/branch keys to ref.
v2026.8.0 is the newest stable tag: v2026.9.0 answers 404, only
v2026.9.0-latest exists. The auto-bump will not drift onto those pointers,
version_flavor("v2026.8.0") is empty and latest_semver with that flavor picks
the plain tag back.
Pinning the core while the plugins clone from their default branch is safe
because upstream solves it: docker_manager, discourse-akismet and
discourse-activity-pub each ship a .discourse-compatibility map, and the
bootstrap checks out the plugin commit matching an older core. The newest
threshold in those maps is 2026.4.0-latest, below this pin, so the default
branch is the intended pairing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All 22 run: hooks in app.yml were rails and rake calls against the database,
yet the launcher executes them only while building the image. Two of them
carried rotating values, the OIDC client secret and the administrator
password, so every rotation forced a full rebuild with bundle install and
asset precompile: 961.82s in one pass and 13m16s in the next, on a job that
then missed its wall-clock budget by 201s.
They now run against the running instance through container exec, following
the pattern web-app-openproject already uses: real .rb files under files/ruby,
values through ENV.fetch, a readback that prints booleans rather than values,
and an assert that fails the deploy when a rotated credential did not arrive.
ensure_admin_user.rb mirrors the Y/Y path of admin:create using upstream's own
grant_admin!, change_trust_level! and activate; setting the flags by hand had
skipped auto_approve_user, the staff welcome message, the notification levels,
trust level 1 and the email confirmation.
What stays in run: is the CA trust call, which writes into the image and would
not survive a recreate. With the rest gone, the bootstrap digest f47ea0e5bb
records covers templates, expose, params, volumes, after_code and docker_args,
and nothing in that set rotates.
00_core.yml carried the same IS_STACK_HOST condition three times; it now
includes 04_stack_host.yml once, which holds the three with their own
conditions, and their implementations moved under tasks/stack_host/.
The OIDC values are read only when OIDC is enabled. Ansible resolves vars
lazily, so referencing DISCOURSE_OIDC_LOGOUT_REDIRECT_URL unconditionally
would have run its lookup against web-svc-logout on a deployment that has no
SSO, failing where the template did not.
test_oidc_setting_order.py follows the settings into apply_settings.rb. The
invariant 88218fc740 introduced is unchanged: openid_connect_enabled comes
after the discovery document, the client id and the client secret.
Four explanatory "# Exception:" comments are removed, two of them from
f47ea0e5bb. They described mechanisms rather than naming a trip-wire, which
belongs in a commit body.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three notifiers pointed at one handler, and that handler always ran
./launcher rebuild. A rotated database password therefore cost a full image
build with bundle install and asset precompile: 961.82s in pass 1 and 13m16s
in pass 2 of job 103878400427, on a run that then missed its wall-clock
budget by 201s.
The launcher draws the line itself. set_template_info reads app.yml on every
call and turns env: into -e flags that run_start hands to docker run, while
templates:, params: and hooks: are consumed during bootstrap only. An env:
change needs a new container, not a new image. run_start takes the docker run
branch only when no container exists, which is why the start path still stops
and removes it first.
The digest task is the discriminator: it records a sha256 over every top-level
key except env:, so a credential-only change leaves it untouched and the
bootstrap handler is never notified. A hash rather than the content because
that section carries the OIDC client secret and the administrator password.
The guard against running both paths lives in a fact cleared per role run. A
register from inside the included file would have outlived the play and, in a
later one, silently skipped the recreate while the config change never reached
the container.
from_yaml adds no failure mode: the launcher parses the same file with
YAML.load, so anything it rejects was already a broken deployment. start
without an image is structurally impossible, since a running container
implies its image and a missing one notifies the bootstrap.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
host_header made the contract test send the public vhost as Host, and
web-app-wordpress was the only role in the repo carrying the flag. Under an
onion deployment home_url() is http, so the mu-plugin's https short circuit
does not fire and the filter falls through to comparing HTTP_HOST against
WORDPRESS_INTERNAL_HOST. That comparison sees the onion vhost and the service
name, never matches, application passwords stay unavailable, core never
validates the credential, and the transport guard answers 401. Clearnet hid
the defect because the first branch returned before the comparison was ever
reached.
Reproduced in three jobs across two repositories, both deploy modes and three
distributions, each with the same line: REJECTED authenticated initialize
answered 401 infinito_mcp_unauthorized. The split measured in the job logs is
https://blog.infinito.test against http://blog.<onion>.
The route without the spoof is already proven in the same failing job. hermes
reports [OK] web-app-wordpress and openclaw reads 3 tools, resources, prompts,
both against "url": "http://wordpress:80/wp-json/mcp/mcp-adapter-default-server".
WordPress serves the REST MCP route under Host: wordpress, and the filter
branch b260cc5604 added is what lets them through.
This completes 5e810c4a5a, which dropped both spoofed headers for Moodle and
moved the accommodation to the provider. b260cc5604 did the provider side for
WordPress and removed X-Forwarded-Proto but left the Host spoof standing, so
the test stayed green on a route no client uses while every client was
refused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The OIDC step runs `container exec <db> mariadb -e "UPDATE ..."`, and the
address it used named the shared service unconditionally, so it always
resolved to the literal `mariadb`. meta/variants.yml splits exactly here:
variant 0 sets mariadb.shared true, variant 1 sets it false and gets a
dedicated container. On the dedicated one the task died with "Error response
from daemon: No such container: mariadb" after an otherwise clean deploy,
ok=801 changed=147 failed=1 in job 103734390485 of run 34760282443.
The database lookup already carries that branch, container = dbtype when
central and <entity>-database otherwise, and its address key adds a second
branch the old value also got wrong: under swarm it resolves through
resolve-container-id instead of handing back a bare name.
This drops the one-off in favour of the house pattern. web-app-listmonk and
web-app-matrix already build their container exec target the same way;
web-app-moodle was the only role reaching for container_address against a
foreign role id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
