The systemd drop-in is written inside a block gated on TOR_DNSMASQ_OWNS_LISTENER, which turns false as soon as a foreign file sits in /etc/dnsmasq.d. The start task read that block's register unconditionally, so the expression's correctness depended on ansible defining a register for a task it skipped. Short-circuiting on the same condition means the register is never reached when the block did not run, and the question stops mattering.
The test asserted the literal declaration string, which exercises nothing. It now renders the expression read from the role with ansible's own bool filter and a changed test, in both states: owning the listener with a rewritten drop-in yields True, not owning it yields False without touching the register. The previous form raises UndefinedError under the second case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dnsmasq binds the docker bridge address with the clearnet upstreams configured and the node's own .onion records loaded, and no rule covered port 53. Any source that could route a packet to that address got an open resolver plus the split-DNS records. c89ca42c46 claimed local-service closed this; abb6f37297 showed the keyword never took effect.
The guard now carries two named sets instead of one plus a special case. 'containers' is the docker pool list, for tor's own transparent and DNS ports. 'internal' is NETWORK_INTERNAL_CIDRS, for the resolver and the SOCKS port. Every guarded entry names its set, which folds the hand-written SOCKS rule and its manual append into the same list the reverse-path sets are built from.
The resolver takes the wider set deliberately: 'internal' is the list torrc.j2 already spends on SocksPolicy accept. The narrow set carries only 10.208.0.0/12 and 10.7.7.0/24 out of 10.0.0.0/8, which leaves out swarm's ingress network, every overlay docker allocates without an ipam block, and a node whose own address sits elsewhere in 10/8. A query bound to such an address was measured timing out against the narrow set. Guarding the resolver more strictly than the proxy beside it would refuse a name lookup to a source that is handed a full SOCKS proxy six lines earlier.
NETWORK_DNS_PORT lives next to the other resolver constants rather than in meta/services.yml: ports.local describes host-bound container ports, where 53 collides with svc-dns-unbound and falls outside every PORT_BANDS band.
Measured on the rendered artifact: the table loads and reloads with a byte-identical listing, every compose source of port 53 resolves inside 'internal' and every source of 9040/9053 inside 'containers', and a mistyped set name, an empty CIDR list or a missing admitted key each fail loudly at validate time. Port 53 newly passes a strict reverse-path check, so an asymmetrically routed client loses name resolution.
test_fragments built its guard list from a fixture of three entries while the role shipped six, so it validated the template against a port list nobody deploys. It now takes the shape from the declaration and fills only the numbers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SOFTWARE_NAME, SOFTWARE_DOMAIN, SOFTWARE_URL and ORGANIZATION are declared once in group_vars/all/00_general.yml and derive from each other. Eleven templates wrote the value instead, so a rename would reach the four declarations and every consumer that reads them, and silently miss the literal.
The lint reads the name from the declaration rather than carrying it, and searches case-insensitively: the domain is the name lowered, the URL is https:// plus the domain and the organization is the name again, so one literal finds the value of all four. A hit is reported with the variable it should have been, derived from the scheme in front of it and its case. The escape is nocheck: software-identity, same line or the one above.
Every migrated template renders the same bytes as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The line that mocks the role_subnets dependency exceeded the line length; the project formatter broke it across three. No behaviour changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ansible warns that the name is already in use and carries on. Carrying on is the problem: inside the loop the original value is gone, so every lookup, template and nested include reading that name sees the loop item instead, and the task still reports success. The damage lands in a log nobody greps.
The lint collects every key declared in group_vars and in each role's vars or defaults, then reports any loop_var matching one. A denylist of known-bad names would have found application_id and missed domain, which is the same class in four more places.
All seven existing sites are marked, each with the reason read at the source rather than inferred. The three application_id loops pass the role id as the item, and the include_vars that follows loads that role's own application_id with the same value, so both meanings coincide by construction. The four domain loops hand the vhost to sys-stk-front-proxy, whose tasks read front_proxy_domain from exactly that name, so the shadowing is the interface.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A fragment declares its own nftables table. An accept there terminates that table's base chain and nothing else: the packet still traverses every other base chain at the same hook, and a drop in any of them wins. Docker sets the filter FORWARD policy to drop by default, so a fragment that accepts is not permitting anything, it is declining to decide.
Measured rather than argued. Converting a role's appended iptables FORWARD accept into a fragment rule at priority -10 delivered 0 of 3 forwarded packets under that policy where the rule it replaced delivered 3 of 3, and the fragment's own counter showed the accept had matched all three. Both versions deployed green, which is why this belongs in a lint and not in a paragraph.
The message names both ways out: express the intent as a drop of what must not pass, or put the rule in the chain that carries the policy it has to overrule.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard drops every source outside the list tor_egress_clients returns, and containers reach the TransPort and the DNSPort through exactly those rules. Nothing held that relationship: narrowing a pool, renaming a key or dropping an entry from the lookup would have taken the resolver and the transparent proxy away from every container at once, in production.
The test runs the real lookup over the repository's real network declarations rather than a fixture, so it fails where the mistake is made. Its templar resolves the cross references those declarations carry, because without it the lookup would compare unrendered template text and the check would pass on nothing.
It also pins the decision that the admitted list is deliberately narrower than SocksPolicy: 10.0.0.0/8 stays out, because a transparent proxy and a resolver with no access control of their own do not belong to the whole LAN. Left unwritten, that reads like an oversight and gets repaired on the next pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An audit of what the host actually filters found a single input chain in the whole ruleset, covering three tor ports. ss showed two more binds with nothing in front of them: the SOCKS port on 0.0.0.0, and dnsmasq on the bridge address.
Each gets the layer it belongs in. SOCKS gets a packet filter rule against the same CIDRs SocksPolicy already accepts, so no client loses access while the filter closes if tor's own config ever stops enforcing. Reusing the narrower guard set was deliberately avoided: it excludes the LAN blocks SocksPolicy admits, so it would have changed the policy rather than hardened it.
dnsmasq gets local-service, which answers only directly connected networks. A packet filter rule there would have sat on the container DNS path, which daemon.json points at that same bridge address, in exchange for closing an exposure that needs an extra route onto the bridge.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A templated script is opaque to every tool that reads shell. shellcheck parses Jinja as syntax errors, the repository's own .sh guards skip the file because it is not a .sh, and nothing can execute it to test it: what runs on the host never exists in the repository, so a defect in the rendered half is found by deploying it.
The lint reports any *.sh.j2 under roles and points at the static slot sys-service already resolves, files/shell/script.sh, with the values passed as ExecStart arguments or Environment entries the way this repository's python scripts already take them.
None of the seven templates present meets the bar the marker is for, which is a script whose shape is generated rather than its values. Each is grandfathered with a marker naming what it interpolates, because converting a deployed script to argv changes what runs on the host. The guard exists to stop the next one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
sys-service resolves the template, writes it into the unit directory, reloads the daemon, enables and starts the unit, and carries the timer, async and flush behaviour. A role that renders its own service.j2 with a bare template task gets the file and none of the rest, and every one of those omissions is silent: a unit that exists but was never enabled looks installed until the host reboots.
The lint reports any src pointing at a service.j2 inside a role's tasks. desk-ssh was the one finding left after sys-svc-firewall moved over, and it migrates here: its ssh-agent unit now comes from sys-service in the user scope, with the type, ordering, environment and Install section supplied as template fields. RuntimeMaxSec is pinned empty on purpose, because the default applies it to anything that is not oneshot and would kill a long-running agent after a day.
The unversioned unit is retired first. sys-service names units by version, so the old file would otherwise stay behind enabled and a second agent would bind the same socket.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role advertised holding a forwarding or masquerade rule for a tunnel. A fragment cannot do that: it lives in its own table, and an accept there terminates only its own base chain, so a DROP policy another component set still executes. svc-net-wireguard-firewalled was measured losing every forwarded packet that way.
The use case is replaced by one the design does serve, and the rule that decides it is written down where the next author will look rather than left in a commit body.
The fragment test no longer seeds the two wireguard interface names; the fragment they described is gone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-net-tor rendered its own guard script, its own unit and its own handler, and hooked the rules into DOCKER-USER with an insert, because a rule appended to a chain docker owns lands behind the RETURN docker puts there. The script flushed its chain and refilled it rule by rule, so the boundary was gone for the duration of the refill.
A role that needs packet filter rules now writes templates/nftables.conf.j2 declaring its own table and includes svc-net-firewall, which renders it to /etc/nftables.d/<role>.conf, rejects it with nft -c, loads it, reads the table back and refuses one that carries no chain. Deployed on its own the role installs nftables and the ruleset that loads every fragment at boot. That ruleset is a distribution decision, so its path follows os_family: RedHat reads /etc/sysconfig/nftables.conf, the others /etc/nftables.conf. The unit is only enabled, never started, because on Debian its ExecStop flushes the whole ruleset, docker's tables included.
One table per role removes the shared path: a fragment replaces its own table and touches no other, and docker never touches either. nft -f on an existing table merges, so each fragment creates, deletes and redeclares its table in one transaction. The tests load every fragment twice into a throwaway network namespace and require the listing to match, because a merging fragment loads twice without error and shows up only as a duplicated rule.
The tor guard now applies in both deploy modes. It was pinned to swarm while torrc binds TransPort and DNSPort on the bridge address in compose too, so on a single-host install those ports answered every on-link neighbour. The guard drops rather than admits, so nothing short-circuits another table, and ip saddr carries an implicit ipv4 dependency, which leaves the ipv6 exposure exactly where the iptables rules left it. svc-net-tor reads its own chain back and refuses a guard that drops nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
lint-python runs ruff, which the pytest lint suite does not cover, so these
slipped through locally: an unnecessary lambda around bool, and a Jinja
Environment left on the implicit autoescape default. The suppression has to sit
on the autoescape argument itself, which is where ruff anchors the violation.
Sending to a .onion needs two things of the node: a resolver that answers the suffix, and a route from the container to Tor's transparent port. Neither existed. Compose pointed Mailu at unbound, which knows nothing about .onion, and in swarm Tor's DNSPort and TransPort listen inside the container where the host's dnsmasq and nat rules cannot reach them, so the forwarding rules that were already there pointed at a closed port on every swarm node.
Compose resolves through unbound, which now forwards the onion zone to Tor's DNSPort on the bridge it already owns. The forward alone would have answered SERVFAIL: the signed root proves that onion does not exist, so with auto-trust-anchor-file set every answer below it validates as bogus, and Tor's DNSPort can supply no DS. The zone is therefore declared insecure. Swarm resolves through the node dnsmasq instead, which every container already uses because container_dns puts the bridge resolver into daemon.json wherever svc-net-tor is deployed, and which now reaches Tor because its ports are published host-mode next to the SOCKS port that was published there already.
The transport rule changed target rather than mechanism. REDIRECT rewrites the destination to the primary address of the interface the packet arrived on, which is a different project bridge for every role, so Tor would have had to listen on all of them. A DNAT naming the bridge keeps one listener, and in swarm the same address reaches the published port through the proxy docker runs there.
That leaves the exposure. SocksPort has SocksPolicy; DNSPort and TransPort have no equivalent, and a swarm-published port carries no host address to bind to, so in swarm the packet filter is their only boundary. Compose needs none, because there Tor binds loopback and the bridge and nothing else. The swarm guard admits this deployment's own address space and drops the rest, and it is installed as a unit ordered after docker rather than as ansible-applied rules: nothing here persists netfilter state, so docker recreating its chains at boot would have left Tor back on the wildcard with the guard gone. It rebuilds its chain instead of appending to it, because the appending version could only ever be loosened. It hooks INPUT and DOCKER-USER both, since host-local traffic reaches a published port through docker's proxy and external traffic through FORWARD.
The deploy now proves all of it back and fails when a rule is missing, the router no longer runs on swarm nodes that carry no Tor, and the CLI test probes the forwarded ports over SOCKS instead of assuming a HiddenServicePort line means a reachable service.
Measured so far: Tor accepts the non-loopback binds and reports circuit-established, and the chain, its order and the DOCKER-USER hooks read back correctly from a deployed host. The swarm path is not yet exercised.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
b036f6003 gave svc-prx-openresty the ports it had been binding as literals, including ports.public.http and the onion flag that turns the node onion's port 80 into a declaration instead of a hardcoded torrc line. The matching lint change stayed in the working tree, so that commit does not pass its own gate: the onion-flag lint excludes http wholesale, never finds the category among the declared ones, and reports the flag as out of scope.
The exclusion was right for the 128 roles that declare http, sso or websocket on the loopback and are reached through the proxy on the onion's port 80. It was wrong for the single role that publishes http on the host, because that role is the proxy. The lint now splits on scope rather than on category: exempt under local, decision required under public.
The companion lint gets the same treatment it needed for the dict-form call sites: the ports block is tracked by the indentation of its key rather than by the next line that looks like one, since target, published, protocol and mode are all valid keys and closed the block while it was still being read. Under expose: only a real host:container mapping counts, because there the single-segment form documents a container port and publishes nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make lint-shellcheck walks files on disk, so a .sh.j2 is invisible to it. The thing that actually runs on the host is the render, and nothing looked at the render until it failed on a node. That is an unpleasant place to discover a quoting mistake in a script which runs as root and rewrites the host packet filter.
Every roles/*/templates/*.sh.j2 is now rendered and handed to bash -n, plus shellcheck where it is installed. A value the check does not know renders as a placeholder token, an unknown collection renders as empty, an unknown filter passes its value through, and lookup returns a token, so a template written for ansible still renders to whole bash and the check judges the shape of the script rather than the values one deployment happens to supply.
SC2034 is excluded with its reason: emptying a conditional branch orphans the variables that branch used, so a partial render cannot tell a genuinely unused variable from one whose only reader was not rendered. Everything else stays on.
First run, first finding: the backup ssh wrapper substituted with backticks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The private blocks were written out wherever something needed to say "this is us": NETWORK_TRUSTED_PROXY_CIDRS listed all three, Tor's SocksPolicy repeated them, unbound's access-control repeated them again. Four copies of a fact, and nothing tying them together, so narrowing one narrows nothing else.
Worse, the block role networks are carved from was never declared at all. The allocator behind 'infinito contributing network address suggest' derives its umbrella from the subnets already in use, so the tree defined the rule instead of the rule defining the tree. Anything consuming "our own address space" had to guess it, and a guess that is one role out of date silently excludes that role. Measured: 106 of 107 role subnets sit in 192.168.0.0/16 and one does not, so a firewall range derived from the pool by eye would have cut BigBlueButton off.
The blocks are now named once and everything else is derived, NETWORK_TRUSTED_PROXY_CIDRS included, which renders byte-identically to before for its four consumers.
BigBlueButton stays outside because its upstream pins both the block and its gateway, which meta/services.yml repeats as upstream_host 10.7.7.1. The exception list therefore carries the role id rather than the CIDR: the block is already declared in that role's meta/networks.yml, and repeating it here would be the same duplication one layer up. The role_subnets lookup resolves the ids and raises on a role that declares no subnet, because a silently dropped entry would narrow whatever firewall or trust list consumes the result.
A lint keeps the declaration honest in both directions: every role subnet must sit in the declared space, and an exception whose block the pool already admits is rejected so it cannot outlive its reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The list form could publish a host:container pair and nothing else, so thirteen blocks stayed hand-written: the ones needing a protocol suffix, a port range, the swarm long syntax with mode host, or a container-port-only publish. Each of those duplicated numbers the role already declares in meta/services.yml, which is the drift the lookup exists to remove.
The lookup keeps the list form unchanged for its 93 existing call sites and gains a dict form: transport emits one item per protocol, container 'same' serves the services that cannot remap (STUN/TURN, media relays), mode 'host' emits the long block an edge role needs in swarm so the client address survives the hop, publish 'ephemeral' emits the container port alone, and key names the block for Discourse's launcher file, whose expose: list is what becomes docker run -p there. A declared {start, end} mapping renders as start-end on both sides, so a relay range needs no term of its own.
svc-prx-openresty had no ports block at all and bound 80 and 443 as literals in its template, while torrc.j2 carried a third literal 80 for the hidden service. None of the three referenced the others. openresty now declares the ports, the compose template reads them, and the onion's port 80 comes from ports.onion.http through TOR_ONION_EXTRA_PORTS.
Three blocks stay raw, each with the reason it cannot route through one call: keycloak's sso_proxy maps a published port owned by the consuming role to a container port owned by web-app-keycloak, and matrix's ansible flavor takes element on 8080 and federation on 8048 from the upstream playbook rather than from this repo's ports.internal.
The enforcing lint scanned only files with compose in the name and only double-quoted list items, so nine of the thirteen blocks were invisible to it. It now reads every role template that can carry a ports block, catches unquoted items, protocol suffixes, ranges, flow style and the long syntax, and tracks the block by the indentation of its key: keying on the next line closed the block on anything shaped like a key, which target, published, protocol and mode all are. Under expose: only a real mapping counts, because there the single-segment form documents a container port and publishes nothing.
Verified by rendering all 110 container_ports call sites through the real templar, the real lookup and the real merged applications view.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A port under ports.local or ports.public binds an address on the stack host, which is what a HiddenServicePort can forward to. Whether it should be forwarded has a cost in both directions: silently off leaves a service unreachable that the operator believes is on the onion, and silently on publishes a database, an admin interface or a SOCKS proxy to everyone holding the onion address. Neither belongs in a default, so the flag is now mandatory and every false carries the reason it is false.
The first run produced 165 findings, of which 131 were one architectural fact repeated rather than 131 decisions. Four groups are therefore settled once in the lint instead: the proxy-fronted http, sso and websocket, because the node onion already forwards port 80 to the reverse proxy that routes every vhost, oauth2 hop and socket upgrade behind it; the implicit-TLS variants, because Tor is already the transport encryption and container_ports drops those listeners when TLS is off, which an onion deployment always is; the categories a hidden service cannot carry, listed once in plugins.lookup.tor_ports.UDP_ONLY_CATEGORIES, which the lint imports rather than restates; and any entity that already answers the question through exposed:.
That leaves 23 real per-port decisions, each with a reason naming its own consumer. svc-net-tor keeps its own socks, trans and dns off the onion: a published SOCKS port is an open proxy, and SocksPolicy cannot refuse it because the forward arrives from the node's own address.
Mailu turns smtp, submission, imap and pop3 on, which is what makes sending and receiving work over the onion, and leaves sieve off because ManageSieve manages filters rather than mail. Measured end to end through the merged applications view, across all 105 roles the deploy now forwards exactly 25, 110, 143 and 587.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The public-port sweep never forwarded anything. It read tor.enabled from meta/services.yml on disk, where 104 of 105 roles declare it as the Jinja string "{{ 'svc-net-tor' in group_names }}", which _as_bool answers False for. Running HEAD's collect_public_ports the way HEAD's run() calls it, against the real roles directory with every role deployed, returns []. Only svc-net-tor reads True on disk, and it declares no public ports. The node onion therefore carried port 80 and port 22 and nothing else.
Reviving the sweep by reading the merged view instead would have been worse than leaving it dead: it publishes 16 ports across 7 roles, and declared is not published. openldap's ldaps 636 sits next to network.public false, Mailu's implicit-TLS ports are dropped by container_ports whenever TLS is off (which an onion deployment always is), and svc-net-tor's own SOCKS port would become an open proxy. So the source and the selection rule change together: read the variant-merged applications view, and forward only the categories a service names under ports.onion, resolving the number from ports.local when declared there and ports.public otherwise, which is the precedence container_ports publishes by.
collect_public_ports had no consumer outside this module and its own test. UDP_ONLY_CATEGORIES becomes public so the accompanying lint imports the list rather than restating it, and gains stun_turn and stun_turn_tls: those speak TCP, but the allocation TURN hands back is a UDP relay address an onion client can never receive on.
No role declares ports.onion yet, so the rendered HiddenServicePort set is unchanged by this commit alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/addons/oidc_login.yml gates on lookup('sso_oidc_plugin'), which the applications render evaluates while it is still in flight. Under that re-entry guard the merged view hands out service flags as their own Jinja text, and bool("{{ ... }}") is True. The lookup read services.sso.enabled and services.ldap.enabled as enabled on a deployment with neither Keycloak nor OpenLDAP and announced oidc_login, so Nextcloud advertised a login hand-off to a provider that is not there.
lookup('config', ...) renders the value it returns, which is why the roughly ninety '| bool' sites in meta/ do not have this problem. 238bc285c7 established reaching the merged config through the lookup loader; this one had not been converted. utils.roles.applications.config.get is the raw accessor and does not render.
Measured against the real render: with only web-app-nextcloud deployed the addon set is now empty; adding Keycloak yields sociallogin; adding OpenLDAP as well yields oidc_login. That is the resolution order the lookup's own docstring documents. All 14 nextcloud variants pin both flags as literal booleans, so no CI-deployed variant changes.
The stub next to the plugin answered every lookup with the applications payload; it now mirrors lookup('config', app, path, default). A stub resolves a path but does not render it, so it cannot see this defect at all: the regression test renders for real.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
services.tor.enabled is Jinja in 99 of the 105 roles that declare it. A lookup('tls') or lookup('domains') inside a meta/*.yml value runs while the applications render is in flight, the re-entry guard hands it the unrendered tree, and _as_bool read the flag's own source text as False. The onion injection then did nothing for any app and the clearnet domain was returned.
fa3797540a already described this and quarantined it: the render state joined the domains cache key so the poisoned map answers no later read. The value stayed wrong, which is enough until a call site bakes it into applications. roles/web-svc-asset/meta/services.yml:15 is the only one in the repo that does, and in run 34922994823 it froze https://file.infinito.test/assets into the payload while every vhost of that round was published under the node onion. web-app-dashboard then waited 60 times on a host openresty answers with ssl_reject_handshake, and the deploy died on TLSV1_UNRECOGNIZED_NAME.
_resolved_flag renders enabled, exclusive and primary and raises when template text survives, the shape plugins/lookup/proxy_pass.py already uses. plugins/lookup/config.py and plugins/lookup/email.py render before coercing for the same reason; this module was the one that did not.
Measured across the whole map: 44 of 12819 leaves change, none added, none removed, 17 roles. 42 are clearnet to onion. The two others are repairs as well: keycloak's csp frame-ancestors moves from https to http, which is what an onion node serves, and the nextcloud sociallogin provider name stops being a hostname and becomes the bounded name that test_sociallogin_provider_name.py documents.
The new tests drive the same function with Jinja instead of the literal booleans the existing ones pass, which is why those never saw this. Three of the four fail without the fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
15a6251d7 introduced the merge but tripped over three of the repository's own rules and over the lookup's existing contract test.
_templated called templar.template(value, fail_on_undefined=False). A templar whose signature is template(self, value) raises TypeError on that kwarg, the except swallowed it, and the value came back unrendered: the lookup returned ['{{ vault_cf }}'] where ['secret-cf'] was due. It now falls back to the positional call, the way utils/cache/base.py has always done it.
The SPOT read goes through load_yaml_any instead of yaml.safe_load, which tests/lint/repository/yaml/test_no_direct_calls.py requires so the file is parsed once per process. The module docstring drops the inventories/development/default.yml literal, which tests/unit/python/cli/administration/deploy/development/test_inventory_vars_file_spot.py whitelists by path.
test_missing_api_global_raises_unknown_key asserted exactly the behaviour that broke CI: a scope without API had to raise. It now asserts the fallback, and two new cases pin the edges. A provider the scope omits still resolves; a provider the declaration does not know still raises.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
e90cac4ee2 staged the lint but not what the gate rewrote after it. Whitespace only; the join, the fix hint and the assertion are unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
API is a plain mapping, so an inventory that configures three providers replaces the whole declaration from group_vars/all/18_api.yml instead of merging into it. inventories/development/default.yml declares openai, anthropic and openrouter, so github, google and jira did not exist in CI at all and lookup('api', 'github.client_id') raised.
A raising lookup inside a value the applications renderer is templating does not fail the render: _templar_render_best_effort returns the string untouched and, for a value carrying a non-env lookup(, refuses the fallback as well. The raw template text reached plugin_value.enabled | bool, which coerced it to False and emitted the deprecation warning that fails the deploy gate. Run 34893363518, job 104153194109 lost web-app-nextcloud to exactly that, for three integrations at once. The configvalue entries would have stored template text as the client id.
utils/api.py merges the scope's override over the declaration, so an override narrows values and never the provider set, and holds the enable rule: every credential a provider declares must carry a value. The api lookup reads through it; the new api_enabled lookup gates on it and returns a real boolean instead of a string a filter has to coerce.
The same class sat in group_vars/all/16_ai.yml and roles/svc-ai-litellm/meta/secrets.yml as bare API.<provider>.<key> dereferences; both now go through the lookups.
Two lints keep it shut. test_addon_enabled_renders_to_bool renders the whole applications map against a scope with API blanked, the most hostile override an inventory may legally write, and requires every addon gate to come out a boolean. test_api_accessed_via_lookup forbids a bare API dereference in Jinja under roles/, group_vars/ and inventories/.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
roles/test-e2e-cli and roles/test-e2e-playwright run only in CI, against a
throwaway cluster whose credentials are generated per deploy and discarded with
it. no_log there protects nothing and costs the one thing a red test job exists
for: a masked task reports that its output was hidden, and the whole matrix
round has to run again to learn what a single assertion saw.
The rule is absolute, so it ships without a nocheck escape hatch. A test task
whose output looks too sensitive to print calls for extracting the field the
assertion reads, not for blinding the task.
A second case asserts the glob still reaches the roles. A path rule that
matches nothing passes forever, which is how test_gateway_backend_registered
nearly went quiet earlier today.
Both roles are clean right now, so this starts green. Reproduced against an
injected no_log in run_one.yml to confirm it is not vacuous.
Note for whoever hits it: test_credential_tasks_are_masked scans roles/*/tasks
and roles/*/handlers without excluding these two, and requires no_log on a task
whose printed arguments name a password, secret, token, api_key or private_key.
Nothing in the test roles trips it today, but a future task that does would
deadlock the two rules. The exemption belongs in that lint's file selection.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every one of these breaks into something that looks like ordinary drift rather
than a defect: a state read moved after the backend include leaves the node
sealed, a token helper fed the first login's stdout gets an empty string on
every rotated deploy and fails later at the first write, an ungated SecretID
swap answers 500 on every deploy that did not rotate, and a swap that never
destroys the old SecretID revokes nothing at all.
Job 104155149606 died at the warm request with
litellm.APIConnectionError: OllamaException - Cannot connect to host
ollama:11434 [Connect call failed ('192.168.244.10', 11434)]
192.168.244.10 is not ollama. It is the manager node on the lab bridge: the
name never resolved through the overlay, because the litellm task was attached
to ingress, litellm and postgres and to nothing else. The lmstudio round is the
control, where the same replica does carry the lmstudio network.
The attachment comes from _is_consumer, which reads svc-ai-litellm's own
services.<backend>.enabled and .shared. Variant 1 pins both false. The config
template meanwhile asked `'svc-ai-ollama' in group_names`, which answers a
different question: is ollama deployed anywhere in the round. It is, six edges
away, because litellm pulls postgres, postgres pulls tor, tor pulls the mirror,
the mirror pulls prometheus and prometheus declares flowise, which declares
ollama. So the gateway advertised a route it had no network for.
The template and LITELLM_CHAT_MODEL_SERVED now read the same two flags the
network attachment reads, through one constant each. Publication and routing
cannot diverge any more, and in a variant that declines a deployed backend the
warm task no longer runs at all.
LITELLM_CHAT_MODEL and LITELLM_BACKEND_ROLES are deliberately left alone.
5324eadd8b made them inventory-scoped because web-app-mattermost's assert, the
one task in its file carrying neither delegate_to nor run_once, re-templates in
each worker's own scope; web-app-discourse's addon tasks carry no delegate_to
at all. A host-scoped model name fails there again. Both templates here render
delegate_to STACK_HOST plus run_once, where group_names is authoritative.
meta/services.yml keeps its `in group_names` flags, so the five separate
substring detectors that decide whether a flag counts as dynamic keep seeing
svc-ai-litellm.
Measured over every (deployed, pinned) combination against the real
expressions and the real template: the model consumers are told to ask for is
published whenever the warm task runs, and nothing is published without its
flag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 34883491022 lost all 15 of its jobs and run 34883296966 four more, every
one of them on the same line and before any role deployed:
ERROR: the value supplied for 'credentials.openai_api_key' does not match
the pattern its schema declares: \S{20,}
786899700f gave the three provider keys a regex and wired validation onto the
value that sits beside a `default:`. That value is the default itself. The
credential leaf writes meta["default"] into the inventory verbatim, so the next
read hands `{{ API.openai.api_key }}` back as if an operator had typed it. The
literal is 24 characters and carries spaces, so \S{20,} cannot match it, and
svc-ai-litellm sits in every closure that holds a litellm consumer, which is
why roles as distant as Baserow and Memcached died with it.
A pattern describes the rendered secret. Template text is not that value, so
validate_supplied_value returns on it, next to the guard that already returns
on an empty one. One guard covers all three call sites. It can only skip a
check, never add one, and the three roles carrying a Jinja default for far
longer (listmonk, keycloak, espocrm) declare no regex and keep their behaviour.
The test class asserted that `<your-key-here>` is rejected but never that the
schema's own default survives, so the gap shipped green.
test_every_declared_default_survives_its_own_schema closes it for every default
the file declares, which is why _schema() widened into _credentials().
Reproduced at the call site that failed: the credentials subprocess taken from
the job log exits 0, and the value it emits decrypts back to the template.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make quality-high pass 75 rewrote it: a concatenation whose right operand is a
multi-line literal gets parenthesised, with the operator leading its own line.
The string itself is unchanged, so
test_narrowing_the_credentials_still_rotates_every_user keeps the declaration
that separates web-app-keycloak from web-app-a and still fails when the user
half of the rotation is narrowed along with the credentials.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The matrix gate rotates credentials between the two deploy passes so PASS 2
has to carry fresh values. It rotated every application block in the host_vars
file, and on a matrix inventory that is not the round. Provision applies its
--include as a strict intersection (filters.py:57-62), so it generates
credentials for at most derive_includes, while mirror_overrides.py adds a block
per mirror artefact carrying only image and version. Rotating those costs one
credential subprocess each for values provision never wrote.
matrix.py now passes --include derive_includes(app_id, variants=round_variants),
the same call 02_provision_inventory.sh:11 feeds provision, from the same
round_variants the provision step got one call earlier. The rotation set is
therefore a superset of what provision generated: every database_password still
rotates, because utils/manager/inventory.py:236-242 writes it under an id from
that same filtered list. Measured for web-app-wordpress: 35 ids.
The scope stops at the credentials. reset_credentials takes credential_ids
beside application_ids because the two halves shared one list, and narrowing it
narrowed _drop_user_passwords too: that resolves through
required_user_policies(roles_dir, application_ids), so 60 of 66 declared users
would have stopped rotating, root, backup and every mcp-* bot among them. The
user half keeps the full document list and is byte-identical to before.
The test fixture declared the same users in both roles, which made the
regression test pass against the regression. web-app-keycloak now declares a
user web-app-a does not, so the assertion has something to bite on.
The CLI aborts when --include matches no block instead of rotating nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An unset secret or variable resolves to the empty string, not to an error, so
an undocumented one is invisible twice: the administrator cannot know to set it,
and the run that needed it degrades quietly instead of failing. The guard reads
every ${{ secrets.X }} and ${{ vars.X }} out of .github/workflows and demands
the name in the page that owns it, secrets.md or configuration.md.
It compares the reference against the page rather than the page against itself,
so it also catches a doc naming something the workflows no longer read.
Three gaps were already there. DOCKERHUB_USERNAME and DOCKERHUB_TOKEN
authenticate the image mirror against Docker Hub and lift the anonymous
pull-rate limit; both are declared required: false and the login step is gated
on them being non-empty, so their absence does not fail a run, it walks the
mirror into the rate limit. Neither was documented anywhere.
PLAYWRIGHT_KEEP_ALL is the sharper one: configuration.md listed
INFINITO_PLAYWRIGHT_KEEP, which is the name the workflow forwards it under, not
the name an administrator sets. Following the documentation produced a variable
nothing reads.
Commented-out lines are skipped. cron-security-scorecard.yml carries
`# repo_token: ${{ secrets.SCORECARD_TOKEN }}` as an option nobody has taken,
and demanding a doc row for it would describe an input no run asks for.
GITHUB_TOKEN is exempt in _PROVIDED_SECRETS because GitHub mints it per run; it
is not administered and cannot be set. A second case asserts the scan finds
workflow files at all, so the guard cannot pass vacuously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gateway could reach exactly one remote provider, through a bare
OPENROUTER_API_KEY group_var with an empty-string default and an `{% if %}`
around a single hardcoded model entry. OpenAI and Anthropic had no path at all.
All three are now declared in roles/svc-ai-litellm/meta/secrets.yml, which
mapping.py already calls the application config schema, so a key lands at
applications.svc-ai-litellm.secrets.credentials.<provider>_api_key like every
other credential. Each carries the `type` and `regex` its value must satisfy and
defaults to the central API.<provider>.api_key block, which keeps one
operator-facing place to fill a provider in while the role owns the shape.
type and regex had to become real to be worth declaring. 192 `validation:`
blocks sit in the role schemas today and not one line of Python enforces them,
so a `regex` would have joined a pile of decoration. validate_supplied_value
now runs on every path a value arrives from outside — the --set override, a
value already in the inventory, and the value standing next to a `default:`.
A generated value stays exempt because the algorithm that produced it is named
by the same schema and generate_value takes no schema argument. min_length is
deliberately left unenforced: the generators ignore it, so holding them to it
would fail where nothing is wrong.
The pattern is written for the paste, not for the vendor. OpenAI moved issuance
from sk- to sk-proj- without notice and Anthropic issues sk-ant-api03-; a schema
pinning a prefix would reject a valid key and abort inventory creation. `\S{20,}`
accepts every form those three issue today and one nobody has announced, and
rejects a trailing newline, an embedded space, a truncated copy and a
<placeholder>. A key pasted into the wrong provider's slot is caught at deploy
instead, where the route it publishes answers nothing.
The gateway iterates one LITELLM_REMOTE_BACKENDS list instead of three `{% if %}`
blocks; config.yaml.j2 and env.j2 read the same list and selectattr('key')
decides. LITELLM_CHAT_MODEL falls back to the first configured remote alias
rather than to a hardcoded openrouter/auto.
For CI the keys travel as INFINITO_API_{OPENAI,ANTHROPIC,OPENROUTER}_KEY through
default.env, the passthrough allowlist, the infinito service in compose.yml, the
workflow env block and the development inventory. The compose leg is not
optional: the env lint rejected the change without it, because a key the
inventory reads but compose does not forward resolves to empty inside the
container with no error. The workflow reads `secrets.X || vars.X` so a variable
works as asked while a secret wins, which matters because a variable is not
masked and the development inventory sets MASK_CREDENTIALS_IN_LOGS: false.
An unset key is not an error anywhere: the provider's routes stay unpublished
and the CLI test asserts their absence rather than skipping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gateway publishing a model proves nothing about a backend answering it: LM
Studio shipped a published lmstudio/default route for as long as it had no
model to load. This CLI test closes that gap by asking the running gateway what
it serves and comparing it against what the deployment enabled.
templates/test.env.j2 is both the input and the entry point — discover_cli_roles
finds a role by that file and filters on group_names, so the test runs only
where the gateway is deployed. meta/tests.yml names the container service key so
run_one.yml resolves the node holding it in swarm, and allows 600s for a cold
model load.
probe.py runs inside the gateway container, where the API answers on loopback
and the backends resolve by name, and uses the standard library only because
the image ships the litellm app rather than a test toolchain. It asserts that
every declared model is served, that the LM-Studio-only aliases are present
exactly when that backend is deployed, that no Ollama-backed model is published
without Ollama, that LITELLM_CHAT_MODEL is routable, that an empty deployment
publishes nothing, and that a real completion comes back non-empty.
The fourth assertion is the one worth having: LITELLM_CHAT_MODEL is built from
lookup('deployment').inventory while the config is built from this host's
group_names. Two sources for one fact, and this is where they are made to
agree.
The alias list is rendered whether or not LM Studio is deployed. Gating it
would empty the set in exactly the case the negative direction needs, turning
"a route must not stand without its backend" into a vacuous check.
The verdict logic sits in a pure evaluate() so it is exercised without mocks:
four deployment shapes, each with its positive and negative cases, including a
both-backends case that asserts a shared alias proves nothing about LM Studio
and only the exclusive one counts.
files/test/.nocheck is required rather than convenient. role-files-layout sorts
files/ by language while test-e2e-cli stages files/test/ under that fixed path;
the other 29 CLI test directories carry the same marker.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
328113e7c added a guard that reads every group_vars/all file to confirm each
GROUP_GATED_CONSTANTS entry is really defined, and read them with
path.read_text(). test_no_uncached_file_reads rejects that under tests/: the
cached read_text shares one decoded read per file across the whole session, and
these files are read by several other tests already.
The nocheck escape does not apply. It covers reads outside the project tree,
and group_vars/all is inside it.
What the guard asserts is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fourteen roles repeated the same two lines to say they use the gateway: a
`shared` flag gated on group_names, and an `enabled` flag conjoining that gate
with LITELLM_CHAT_MODEL_SERVED. Both are now single-sourced.
`shared` becomes literal true. The flag means the provider is a shared instance
rather than one deployed per consumer, and a central gateway is always that;
the dynamic form only said "shared if deployed", which `enabled` already
decides. The 50 shared pins under `litellm:` leave meta/variants.yml with it.
The tree had already noticed: 20 of those 50 lines carried
`# nocheck: variants-static-override`, two of them with copy-pasted text about
container_backup. This narrows e5de4e085a for one service key rather than
reversing it; every other dynamic flag keeps the group_names form.
`enabled` becomes `{{ LITELLM_USABLE }}`, defined in group_vars/all/16_ai.yml.
That shorthand is only possible because is_explicit_truth learns about it.
The resolver reads a dependency out of the flag TEXT — "in group_names" in
value — so a constant hides the role id and the provider silently leaves the
static closure of every role referencing it. Measured: the old expression
resolves True, `{{ LITELLM_USABLE }}` resolves False. GROUP_GATED_CONSTANTS
names the constants whose own definition carries the gate, and the docstring
states the obligation: a SPOT of this kind that is not listed there drops its
provider without a word.
The wider fix, accepting any Jinja as dynamic, was measured and rejected: 35
other flags carry a Jinja value without the marker (MCP sidecars, javascript
and SSO gates, recaptcha), and they would all enter closures they are not in
today.
A literal shared flag is no longer dynamic, so test_dynamic_flags wants the
`# nocheck: dynamic-flag` its own message documents for legitimate exceptions;
where a marker already existed the rule list is extended rather than a second
comment added.
Behaviour is unchanged where it counts. The closure keeps litellm with the same
app counts (wordpress 26, openwebui 23, hermes 23, mediawiki 22, flowise 23),
unrelated Jinja still resolves False, and the truth table of the new enabled
form matches the old one in all four served/deployed combinations. Three roles
resolve without litellm, and `git show HEAD:` proves that predates this change:
zammad pins enabled false in variant 0 at line 39, nextcloud at line 97.
Six unit tests cover the resolver, including one asserting that every
registered constant is actually defined in group_vars, so an entry cannot
become a dead promise.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-ai-lmstudio had no model-load step at all: tasks/00_core.yml set the
once-flag and the container network, nothing else. The gateway published
lmstudio/default anyway, so the route existed and nothing could answer it.
wp-compose-tor.log:48994 shows the route written into the config, :51770 shows
the warm step reaching qwen2.5:0.5b on Ollama instead, and the green ai-engine
spec ran against that. LM Studio was deployed, published and never prompted.
The pre-pull mirrors the one that works. tasks/01_manager_ops.yml resolves the
hosting node and container id, loops the declared models through
`lms get --yes --gguf`, overlaps them with async/poll and reaps before
playwright, exactly like svc-ai-ollama. That `lms` exists is measured, not
assumed: /app/daemon-run.sh:67 in the image starts ./.bundle/lms server start,
and `lms get --help` documents -y for scripting and --gguf for the format.
Synonymity needs both namespaces in one place. Ollama knows qwen2.5:0.5b, LM
Studio knows qwen2.5-0.5b-instruct, so AI_LOCAL_MODELS carries both per entry:
`alias` is what the gateway publishes and what Ollama pulls, `name` is what
`lms get` resolves. Both roles now reference that one list, and config.yaml.j2
publishes each backend under the alias, routing Ollama to ollama/<alias> and LM
Studio to openai/<name>. The lmstudio loop skips an alias Ollama already
publishes, so two backends never produce a duplicate model_name and Ollama
keeps precedence. lmstudio/default is gone; the model name is now the same
whichever backend answers it.
Ollama's preload_models changes shape from a list of strings to the {alias,
name} entries. Every consumer moves with it: the pull command, the template
loop, and both expressions in 16_ai.yml. An inventory that overrides
`applications.svc-ai-ollama.services.ollama.preload_models` with the old string
list will break at map(attribute='alias'); this is a deliberate shape change,
in the spirit of 7b15c61dfc.
The CI model list is not a property of an installation, so AI_LOCAL_MODELS_CI
lives in inventories/development/default.yml, whose own header scopes that file
to the cross-cutting environment knobs. test_variable_definitions only knew
group_vars, so it reported the constant as undefined; the shipped inventories
are a definition source too, because the deploy CLI bakes them into host_vars,
and the guard's own docstring already claimed "defined somewhere in the
repository".
A latent crash came out of this: `| list | first` raises on an empty sequence,
so a backend deployed without preload models blew up LITELLM_CHAT_MODEL. Ollama
with only embedding models could already hit it; the existing test dodged it by
rendering only _SERVED. `| default('')` now yields the empty string that
_SERVED already reports as false.
Tests: config.yaml.j2 is rendered rather than read, over ollama-only,
lmstudio-only, both and neither, asserting the published model_name set, that
both backends name the same alias, that a shared alias is published once and
routed to Ollama, and that the render is valid YAML. test_litellm_chat_model
gains the lmstudio branch, the no-preload case and the same-model assertion.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
113113e553 parks the carrier after the token store, 165 lines below the merge
block, so each of the block's six tasks still renders the applications SPOT
cold in its own forked worker. Measured in CI run wp-swarm-tor2 they cost
323.51s in the PASS 1 main play, 180.39s in the backup play and 479.79s in
PASS 2.
The cost is the render, not the work. Two tasks in that block share the exact
construct `defaults_X | combine(X | default({}, true), recursive=True)`: design
takes 0.247s and service_provider 75.94s. The difference is that
defaults_service_provider is dense with lookup('config'/'users'/'domains'/'tls')
and every one of them lands in get_merged_applications.
The carrier cannot simply move up. get_merged_applications serves the carrier
fact whenever the key matches, and the carrier action goes through that same
path, so a second carrier task would be handed the first one's payload instead
of rendering. merged_applications_cache_key reads neither group_names nor the
token file, and with a real templar 71 of 265 applications differ by
group_names alone, so a carrier left standing across add_host or the token
store would be served where a fresh render belongs.
So the block opens its own carrier and drops it again at the end. Nothing
outside the block changes: every task from sys-version to the stage-wide
carrier renders exactly as before.
Placement follows measurement. A per-key perturbation test over the whole
265-application payload, with the credential regeneration noise floor
subtracted and every role in group_names, finds exactly two reads of the seven
facts the block sets:
.web-app-matomo.services.matomo.excluded_ips[0] <- networks
.web-app-wordpress.services.wordpress.title <- service_provider
networks is the block's first task and service_provider its last, which is why
the carrier opens after the one and is dropped before the other takes effect.
The block's tasks already render before service_provider is merged today, so
the frozen payload is what they would have rendered anyway.
The action gains a `clear` argument so the playbook never spells the private
fact name. Parking the carrier serves a payload in 0.000s against a 3.417s cold
render; a wrong key and a cleared fact both force the render again, which is
what makes the drop task work.
Worth roughly 806s of the 20199s job, against a floor of 673.6s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_file_spot failed the gate on the pathlib chain that spelled
meta/services.yml out by hand. Both paths now come from utils.roles.mapping as
ROLE_FILE_META_SERVICES and ROLE_DIR_META_ADDONS, following ea37027f49 and
8243aa571b.
All four chains move, not only the reported one. The lint reads the role-file
shape and lets a "meta" / "addons" directory chain through, so leaving those
behind would have split one fixture across two sources of truth, which is the
split the lint exists to prevent. The constants are slash-bearing strings that
Path accepts as a single segment, so the paths are unchanged.
Only the throwaway role built under tempfile in TestCollectEntriesCoversAddons
is affected; its assertions on entity_path and config_path.name are untouched.
autoformat then reflowed the two write_text concatenations, which fit on one
line once the names shortened, and the generator in test_oidc_setting_order.py.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard recognised meta/services.yml as the only place a git URL may be
declared. 84a4b04f0 made meta/addons/<id>.yml the one place a Discourse plugin
pins itself, so the addon files now carry the clone URL literally, and
_iter_role_files picks up every .yml under the role: each declaration reported
itself as an undeclared reference.
Two halves. The declaration set is now the union over meta/services.yml and
every meta/addons/*.yml, so an addon pin legitimises that URL for the whole
role, not just for the file holding it. Path.glob on a missing meta/addons
yields nothing, so roles without addons need no special case. The self-skip
that kept services.yml out of the scan is now a membership test against exactly
those declaration paths.
Scope stays per role: declared is rebuilt inside the role loop, so a
declaration in one role legitimises nothing in another. The failure message and
the docstring name both sites, otherwise they send the next reader to the wrong
file.
This is the test-side counterpart of e4141cb826, which taught collect_entries
to see the same addon refs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Only two files were touched by both sides. env.js merged automatically: the
incoming side owns recordVideoOptions and this branch had cherry-picked that
same function from it, so the result carries one definition. README.md is the
generated roles index and both sides append rows; resolved as a union, taking
the incoming side for the three rows it edits (OnlyOffice, Open WebUI, OpenClaw)
and keeping ours only for OpenBao, then re-sorted.
The incoming side adds lint rules and resources that this role then violated.
Five fixes, each the remedy the failing rule names:
* services.openbao.ports.local.http 8081 -> 8093; svc-ai-litellm now holds 8081.
* meta/networks.yml subnet 192.168.221.0/24 -> 192.168.222.0/24; svc-ai-mcp-adapter
now holds 221, and the suggester only sub-allocates inside occupied /24s, so
the next free block was taken directly.
* services.email bond 1 -> 0; the entry is disabled and the resource model skips
it before reading the weight.
* README Quick Setup regenerated from the template it had drifted from.
* Credits link the implementer, matching how they are credited elsewhere.
* docs/requirements/035-mcp-proxy-expansion.md disposes of web-app-openbao as
privileged_control_plane, which is what a secrets manager is, and the list
count follows.
Verified against a worktree of #454 alone so the baseline is measured rather
than assumed: lint 501 passed with only the branch-name rule failing, which
fails identically on that baseline because the worktree branch is named
merge/*; unit 5665 passed with the same three host-environment failures the
baseline has; integration 324 passed, 0 failed.
collect_entries walked roles/*/meta/services.yml only, so a repository and ref
declared in an addon file was invisible to it. That mattered once 84a4b04f0
made meta/addons the one place a Discourse plugin is pinned: the pins would
have left the updater's view entirely. The other updater cannot stand in,
utils/update/docker.py reaches version: through iter_role_images and skips any
service without an image:.
e317f08ac9 built the addon contract with its own catalog-driven SPOT in
utils/update/addons.py and left repository.py untouched without saying why, so
this widens the ref path rather than replacing the catalog one; an addon can
be maintained by either.
The per-file work moves into _entries_of_file so services.yml and every
meta/addons/*.yml share it. Addon entries carry an ("addons", <stem>) prefix
on their entity path, which keeps them distinguishable from a service entry
holding the same ref. The line index, the semver filter and the per-file
"# nocheck: repository-version" suppression apply unchanged.
Two tests cover it: an addon ref is collected beside a service ref with its
own path and config_path, and a suppressed addon ref yields nothing. Against
this repository the extension currently collects nothing, because every addon
ref is a branch and is_semver rejects those; the tests are what prove the walk
rather than the absence of output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
All 22 run: hooks in app.yml were rails and rake calls against the database,
yet the launcher executes them only while building the image. Two of them
carried rotating values, the OIDC client secret and the administrator
password, so every rotation forced a full rebuild with bundle install and
asset precompile: 961.82s in one pass and 13m16s in the next, on a job that
then missed its wall-clock budget by 201s.
They now run against the running instance through container exec, following
the pattern web-app-openproject already uses: real .rb files under files/ruby,
values through ENV.fetch, a readback that prints booleans rather than values,
and an assert that fails the deploy when a rotated credential did not arrive.
ensure_admin_user.rb mirrors the Y/Y path of admin:create using upstream's own
grant_admin!, change_trust_level! and activate; setting the flags by hand had
skipped auto_approve_user, the staff welcome message, the notification levels,
trust level 1 and the email confirmation.
What stays in run: is the CA trust call, which writes into the image and would
not survive a recreate. With the rest gone, the bootstrap digest f47ea0e5bb
records covers templates, expose, params, volumes, after_code and docker_args,
and nothing in that set rotates.
00_core.yml carried the same IS_STACK_HOST condition three times; it now
includes 04_stack_host.yml once, which holds the three with their own
conditions, and their implementations moved under tasks/stack_host/.
The OIDC values are read only when OIDC is enabled. Ansible resolves vars
lazily, so referencing DISCOURSE_OIDC_LOGOUT_REDIRECT_URL unconditionally
would have run its lookup against web-svc-logout on a deployment that has no
SSO, failing where the template did not.
test_oidc_setting_order.py follows the settings into apply_settings.rb. The
invariant 88218fc740 introduced is unchanged: openid_connect_enabled comes
after the discovery document, the client id and the client secret.
Four explanatory "# Exception:" comments are removed, two of them from
f47ea0e5bb. They described mechanisms rather than naming a trip-wire, which
belongs in a commit body.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both template tests listed the filters they hand to jinja by name, which made
the FilterModule a second source: adding a filter to it left these two behind,
and the templates that used it failed to compile with "No filter named …" in
test-unit and test-integration at once. The list is derived from the module now,
so a filter reaches the tests the moment the role can use it.
The two entries that are not filters at all, to_json and bool, stay spelled out
because jinja needs them and the module does not ship them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`process.env.X || <value>` is a second source for something the deployment
already renders, and the two drift without a sound. playwright.config.js read
`process.env.PLAYWRIGHT_ACTION_TIMEOUT || 30_000` while nothing ever set that
variable, so every run used the unscaled 30 s and no failure said so.
Normalising an absent variable stays allowed, because it substitutes nothing:
`|| ""`, `|| '[]'`, `|| false`, `|| 0` and falling through to another
`process.env` entry only let the caller test absence itself.
The six sites the rule found:
n8n and flowise built a Postgres port from `|| '5432'` while env.j2 renders it
from `lookup('database', application_id, 'port')`. Node throws
ERR_SOCKET_BAD_PORT on the resulting NaN, so an unfed port now names itself
instead of connecting somewhere plausible.
The bluesky broker did the same with BROKER_PORT, which compose.yml.j2 renders
unconditionally. Its INSECURE_TLS default was `"false"`, which is what `""`
already evaluates to.
matomo defaulted MATOMO_TRACKING_SCOPE to "sub"; the only comparison is against
"root", so `""` takes the identical branch.
moodle and simpleicons defaulted a name their playwright.env.j2 always renders.
simpleicons already asserts the value is truthy; moodle now fails on the scope
it cannot find rather than on a name nobody configured.
Four values the code itself owns keep a default under `// nocheck: env-default`:
PDS_INTERNAL_URL (which falls through to requireEnv, not to a literal),
HANDOFF_COOKIE_NAME, HANDOFF_COOKIE_MAX_AGE and PASSWORD_SALT_HASH_ROUNDS. No
role renders any of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The config held literals for values vars/main.yml already owned, and the two
drifted without a sound. actionTimeout carried `|| 30_000` for its whole life
while nothing ever set PLAYWRIGHT_ACTION_TIMEOUT, so every run used the
unscaled 30 s. expect had no entry at all, leaving Playwright's built-in 5 s to
govern every assertion over Tor while its neighbours were scaled by the onion
multiplier. That is what killed the second Nextcloud login attempt: an
expect(usernameField).toBeVisible() that gave up after 5 s inside a 150 s
budget.
vars/main.yml now owns the values in TEST_E2E_PLAYWRIGHT_CONFIG_TIMEOUTS, the
harness renders that map into the staged .env both the deploy and
scripts/tests/e2e/rerun-spec.sh already pass, and the config demands each name
through requiredTimeout() instead of inventing a fallback. The two `-e` flags
that fed the old pair are gone with it.
scripts/lint/playwright.sh stages the same config and runs `playwright test
--list`, which loads it, but built its stub env only from each role's
playwright.env.j2 and no such template carries a timeout. Without this it would
have aborted all 87 specs on a missing variable. It now takes the names from
the same map; the value is a placeholder because --list consumes none of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ansible renders the values of a structure loaded through include_vars and leaves its keys as written, so the mapping key a564385068 wrote as a Jinja expression reached Nextcloud as its own source text. occ config:app:get sociallogin custom_providers on a live variant-1 instance returns "groupMapping": {"/{{ lookup('rbac_group_path', application_id=application_id, role='administrator') }}": "admin"} while every neighbouring value of the same provider is rendered, scope among them.
Keycloak emits /roles/web-app-nextcloud/administrator, so the mapping matched nothing and the account sociallogin provisions on first login stayed out of admin. occ group:list on that instance shows admin holding only the local administrator while keycloak-ac165525-e9e8-4230-9381-13b1acd83fc0 exists and belongs to no group, and every Playwright spec that needs an admin settings section answered 'Access forbidden'.
An independent reproduction outside this repo showed the same split on ansible-core 2.21.4 and 2.20.6: in a payload of the same shape the sibling value rendered and the key did not, so it is not a version artifact. Nextcloud 34.0.3 settles the other half: Manager.php hands an admin the unfiltered settings of a section and ai is a core section of the always-enabled settings app, so an admin cannot receive 403 there whatever is installed.
Writing the whole mapping as one expression makes jinja2_native, which this deployment has on, return a real mapping with the key computed. The lint takes the class: no meta payload may carry Jinja in a mapping key. It found exactly this one case in the whole meta tree, and it ignores a line that itself opens with the expression so the corrected form does not trip it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Hugo image build in run 34653542922 died on a Debian mirror mid-sync: 'Err:13 libruby ... File has unexpected size (6300780 != 5440). Mirror sync in progress?', and apt closed with its own advice, 'maybe run apt-get update or try with --fix-missing?'. The index the build had fetched disagreed with the pool it then read.
Acquire::Retries=3 does not cover that. It re-fetches the same URL against the same stale index and reads the same wrong size, which is why three retries produced three identical failures.
install_package_candidates now runs its candidate loop through a helper and, when every candidate failed and the manager is apt-get, fetches a fresh index and runs the loop once more. The helper returns two facts rather than one because the two existing error paths distinguish them: whether the caller's command became callable, and whether anything installed at all.
The retry is bounded so it cannot mask a real absence: apt-get only, once, only when nothing installed, and only behind a new index. A package that does not exist fails the second pass the same way and raises the same RuntimeError. The tests pin both directions, the exact update, install, update, install sequence for apt-get and two calls only for a manager without an index, so the retry cannot spread to pacman or dnf.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
