The shared Playwright config passes --unsafely-treat-insecure-origin-as-secure
for every .onion origin in the environment, but Playwright's default
chrome-headless-shell ignores that switch. On the tor axis the odoo origin
therefore stayed an insecure context: navigator.clipboard was undefined and
Odoo 20's onboarding tour crashed with "Cannot read properties of undefined
(reading 'writeText')" and "Cannot destructure property 'TourInteractive'",
opening error dialogs that the addon specs read as a missing module (odoo#2,
swarm tor, main run 38038541631).
When ODOO_BASE_URL is an onion URL the odoo specs now use channel "chromium",
the full Chromium build in the Playwright image, and block service workers;
clearnet runs keep every option unchanged. The worker is blocked because a
secure context lets Odoo register /web/service-worker.js, whose fetch handler
answers a failed navigation with a cached page or /odoo/offline: gotoOnion
would get a 200 instead of the transport error it retries on. The switch stays
in the role: a use.channel in the shared config would also reach the Firefox
launch of onion-test.js, which rejects it with "Unsupported firefox channel",
and would move every other role's onion run to a browser none of them was
tested on.
Verified with Playwright 1.63 in the runner's staging layout and the real
odoo specs: an onion ODOO_BASE_URL resolves channel "chromium" and
serviceWorkers "block", also for the contexts _shared.js creates itself, and
launches chromium-<rev>/chrome-linux64/chrome; a clearnet one, and HEAD's
specs on onion, keep the headless shell and "allow". All spec files stay on
one worker, which the suite's one-worker test requires. Against Odoo 20's
service worker with a dropped connection, full Chromium returned the cached
/odoo page with status 200, while with the worker blocked page.goto threw
net::ERR_CONNECTION_FAILED as the headless shell does. In the v1.64.0 image
full Chromium made no :443 attempt to an onion host and the onion origin
reported isSecureContext with crypto.subtle, which the headless shell did not.
A local deploy of web-app-odoo variant 2 over Tor ran the odoo suite on
/ms-playwright/chromium-1248/chrome-linux64/chrome and passed all seven addon
specs, the one-worker test and the guest flow without a retry; in the variant
0 round the suites of the other roles kept chrome-headless-shell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Memcached, Joomla, Moodle, Odoo, Shopware, Snipe-IT and SuiteCRM set
their cache size, PHP memory_limit, OPcache, worker limits and search
heap as literals next to a mem_limit they never read. They now ask
lookup('service_memory_mb') for a share of that limit. The shares are
chosen so every rendered value stays what it was.
Where the parameter lives in a file that cannot hold a template, the
value is handed in: Shopware's init script reads INIT_PHP_MEMORY_LIMIT
and refuses to start without it, SuiteCRM's image takes it as a build
argument. Shopware's worker gets --memory-limit as a command appended
to its entrypoint, and Moodle's OPcache size moves from the static ini
into the rendered one.
The performance.php.memory_limit keys of Joomla and Moodle are gone
from services.yml; nothing else read them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Nobody could sign in to Odoo through Keycloak. Odoo sent an http redirect
URI, and Keycloak refused it as "Invalid parameter: redirect_uri".
The addon auth_oauth_https builds that URI from web.base.url. Two things
kept it from doing so after the image moved to Odoo 20.
Its manifest declared version 19.0.1.8.0, and Odoo marks a module whose
version names another series as not installable. It only warns about that,
so the install task stayed green and the addon was missing. The manifest
now declares 1.8.0, which Odoo prefixes with its own series.
Once installed, the addon read the parameter with get_param, which Odoo 20
replaced by typed getters. The call raised AttributeError, a broad except
logged it and fell back to the URL of the request, which is http behind
the proxy. The addon reads the parameter with get_str now and no longer
catches the error.
A deploy now fails when the addon is not installed afterwards. The check
reads ir_module_module, because the output of "odoo -i" does not tell an
installed module from a skipped one.
The spec logged out by navigating to /web/session/logout. Odoo 20 accepts
POST only on that route and answered 405, so both SSO tests failed after
a successful login. The spec logs out through the user menu now.
A host that runs the suite with several workers still failed it. Odoo
computes the session token of an account from its OAuth access token, so
every new sign-in ends the other sessions of that account, and the spec
files share the administrator. The role caps its suite at one worker
with PLAYWRIGHT_MAX_WORKERS, which the central config now honours, and
the spec asserts the cap.
Closes#822
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
MAIL_PROVIDER moved to web-app-stalwart without a make docs run, so every generated README still drew Mailu as the email dependency. This is the generator output for the current tree: the root role table and each role cosmos now name Stalwart, Mailu loses its dependents, and Stalwart gains its swarm marker, the Tor dependencies and the 0..0 edges.
The hand-written cosmos of svc-net-tor-smtp gives way to the generated one, which no longer shows the SOCKS5 hop through svc-net-tor because the role does not declare that dependency.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
At -vvv Ansible prints each module argument and result, so a password in
a command line, a request body, a header or a fact lands in the deploy
log. Nothing enforced masking outside the MCP task files, and no check
looked at parents at all.
A new lint, test_credential_tasks_are_masked, finds tasks whose printed
parts (module arguments, environment, loop) reference an identifier ending
in password, secret, token, api_key or private_key, or
secrets.credentials. A task passes when it or an enclosing block sets
no_log, when every file that pulls it in does so through include_tasks
with apply.no_log or import_tasks with no_log (transitively), when the
secret only reaches a parameter the module masks itself, or when the
module masks its values. no_log on an include_tasks task itself does not
count, because it does not reach the included tasks. The rule can be
suppressed per task with `# nocheck: credential-masking`, now listed in
the suppression catalog, and the lint tests its parent and suppression
logic on synthetic cases.
It found 39 unmasked tasks in 19 roles, all real: join tokens, client
secrets, API tokens in Cloudflare headers, database passwords in
command lines, the ChatGPT bot login body and more. Each now sets
no_log from MASK_CREDENTIALS_IN_LOGS. The sys-dns-hetzner-rdns alias
hetzner_no_log, which only restated that switch, is gone.
Not covered yet: template and copy tasks whose rendered file carries a
secret print it as a diff under --diff.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
start_interval is the cadence while start_period runs, and only while it runs.
Once the window closes the probe falls back to interval, so a service that
becomes ready just after it pays a whole tick anyway and the cadence never
reached it. Six of the seven flavors were shaped that way, Curl worst: a thirty
second window against a one minute interval.
Measured across 151 converge waits in four job logs, 63 sit exactly on a tick
boundary, 36 in the thirty second step and 27 in the sixty second one.
Attributed by application, web-app-seaweedfs is the largest at 1501s over
sixteen waits with a median of 57.7s: it probes with curl and fell straight into
the gap. web-app-zammad probes with connect, whose window is fifteen minutes,
and its 154s median is real boot time with nothing to recover.
So the window moves to ten minutes and the steady state interval to two, and the
flavor overrides that sat below them go: Curl, Wget and Connect lose interval
1m, Http its twenty second window, Nc its ten. Connect keeps fifteen minutes.
Neither number costs anything while a container works. start_period delays
nothing, it only stops failures counting, and interval no longer touches
convergence at all since that hangs on the window alone; it is steady state
load, which drops from 120 probes an hour per service to 30.
Forty six role files follow. Twenty seven declarations go that repeat the new
default or undercut it, an undercut window being the defect itself. Eight
windows longer than ten minutes keep their value and now say why. The literal
blocks and the three Dockerfiles this repository builds spell every timing out,
because outside the lookup an omission takes docker's value and not the
platform's.
A fourth lint rule keeps the shape: start_period must outlast one interval,
checked against the effective values, so raising interval without the window
cannot quietly bring the cliff back.
Two faults in the lint surfaced while building it and are fixed here. It
compared timings as strings, so 600s and 10m read as different values, and it
matched the healthcheck key exactly, so a line carrying a trailing nocheck was
skipped whole, which is how the two swarm-init partials went unexamined.
One regression, bounded. A container that starts and never turns healthy is
neither rejected nor errored, so stack_ready.sh only calls it fatal after three
killed tasks, and each cycle now spans the longer window. That reaches twenty
minutes rather than six, capped there by swarm_stack_converge_retries at 600
with a two second delay, and it lands only on runs that were failing anyway.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The root-README overview renders a Video link per role and a reader following it expects a demo, but 26 of the 62 entries addressed a channel, a playlist or a product page: youtube.com/@zammadhq, /channel/UC..., framatube.org/c/mobilizon/videos, pgadmin.org/videos. None of those can be embedded and none shows the product without a second click.
Each replacement was resolved through YouTube's oEmbed endpoint, which returns the real title and channel, so a candidate could be checked rather than assumed. That caught a supposed LibreOffice intro published by an unrelated channel, a Discourse overview belonging to a hosting vendor, and a Zoom video answering 403 with embedding disabled. Fourteen of the new entries come from the vendor itself and two are PeerTube.
The reachability half needed the same endpoint for a different reason: YouTube answers a removed video with HTTP 200 and a Video-unavailable page, so a plain fetch proves only that YouTube is up, while oEmbed answers 400. Measured both ways before relying on it.
The check is external because reachability needs live HTTP, and like its siblings it reports an unanswered probe as unverified rather than failing, so a firewalled runner cannot redden a good URL. Only the shape is asserted, because that is a repository fact.
README.md follows rather than leads: overview.py renders the Video column from this field.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Production block never set one. inventory provision writes only
ansible_connection for a local host, so DOMAIN_PRIMARY stayed at the
group_vars default of infinito.localhost and the deployment served a zone its
own resolver knows nothing about. CoreDNS templates answers for
INFINITO_DOMAIN_RE and nothing else, so every lookup of the deployment's own
name came back NXDOMAIN.
Mastodon is where that surfaced. tootctl accounts create validates the address
through EmailMxValidator, which resolves MX and then, for the domain itself,
A and AAAA. All six queries returned NXDOMAIN in run 33574056480, the
validator recorded email :unreachable, and the admin account was never created.
Only the guide rows were affected: the matrix rows write
DOMAIN_PRIMARY: "{{ lookup('env', 'INFINITO_DOMAIN') }}" into their host_vars
and resolve fine.
The template now carries DOMAIN=<your-domain> in both the host and the
container form and threads it into --vars, with the container form passing it
through the docker run boundary as well. The two guide runners substitute it
the way they already substitute the SSH key, so the replay deploys under
INFINITO_DOMAIN like every other CI deploy. compose_deploy.sh moves its env
load ahead of the substitutions to have the value; host_deploy.sh had no env
load at all.
A production reader is asked for a domain now instead of silently inheriting
infinito.localhost, which no real deployment wants either. The shipped default
keeps its value: sys-svc-nss documents that *.localhost resolves through the
myhostname NSS module without any zone, and host-side tasks lean on it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A role-files-layout exemption frees a tree from files/<lang>/ because
its import paths are fixed from outside. The mirrored-test lint derived
its scope from that same directory, so every layout exemption silently
became a test exemption nobody ever argued for.
The language now comes from the file suffix. Test code is excluded by
scope rather than exempted: a test-* role and a files/playwright/ or
files/test/ tree are the tests, so demanding a test of them is circular.
The eleven files this newly reaches are framework-instantiated and carry
a stated reason each.
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.
As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).
The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.
Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).
Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.
Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
unsafe-inline re-admits the inline execution an XSS needs and unsafe-eval re-admits string-to-code, so a CSP relaxation must be verified against the running app rather than copied from another role. The new lint requires a nocheck marker plus a Reason on the flag's own line, so a marker cannot drift to the neighbouring directive.
The 197 pre-existing grants across 70 roles are checked out with a TODO: they were never verified and still have to be.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The cosmos generator linked a consumer to its provider only through an
'<role>' in group_names reference or an explicit provides: name. A hard
dependency has neither -- pinning the provider on unconditionally is
exactly what removes the Jinja reference the generator was reading -- so
web-svc-asset's bond to web-svc-file vanished from the diagram the moment
it became binding. Match a service key against the provider's own primary
key from the registry, in both directions.
Services the role turns off drew a solid 1:1 edge, claiming a mandatory
relationship for something that carries nothing at deploy time. They now
render as a red 0..0 edge; edge merging gained a real precedence so a
provider consumed through two keys stays live when only one is off.
Named colour, not hex: a '#' opens an entity code in the mermaid lexer and
turns the whole diagram into a parse error on GitHub.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.
Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.
Freed from the binding, the names take the emoji prefix every other task line
already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
72fd9c870 required the onion in every variant but exempted variant 1, on the
argument that the minimal round is the one place the stack runs without it. That
argument does not hold: the onion is part of the deployment under test, not a
feature bolted onto the full shape. A round that skips it is not a smaller
deployment, it is a different one.
So the exemption is gone and 95 variant-1 entries pin the onion on. Two
exclusions remain, both read from data rather than a comment: a role with no tor
block has no onion to pin, and a role whose meta/services.yml pins tor.enabled to
a literal false has already declared it has none.
That put the rule in conflict with test_coverage, which requires both polarities
of every dynamic flag: with tor on everywhere, the false side has no variant left
to live in. 94 roles take the exemption that contract already provides for
exactly this - "flags whose group-derived value is a hard invariant that variants
must never override", the same one 4e6ef2542 used for the seaweedfs frontend.
web-app-nextcloud needs none: its variant 8 pairs the clearnet-only PeerTube and
keeps the false polarity.
Measured, because the declaration is not what CI runs. assign_tor gives the onion
to every second capable entry with the run number as the parity switch, so more
capable variants do not mean proportionally more onion jobs: across five roles
and 21 entries the count goes 8 -> 10 on an even run and 9 on an odd one. The
clearnet path also survives - checkmk reports v1=false on run 0 and v1=true on
run 1, so both states are still covered across two consecutive runs. What the
94 markers give up is a declarative guarantee in variants.yml, not the coverage
itself.
The real cost is weight, not job count. web-app-nextcloud variant 1 goes from
3.09GB reservation / 7.27GB limit to 18.26GB / 43.01GB once svc-net-tor and its
closure join, so the cheap round is no longer cheap. Every variant stays under
the 32/64GB budget, and the same effect applies to the other 94 unmeasured.
Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles/meta/
variants plus the role-meta lints 39 passed. The rule was checked in both
directions - it reported the variant-1 offenders before the flips and none after.
Not measured: what the heavier rounds do to wall-clock, against swarm jobs capped
at 355 minutes and one that has hit that ceiling before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
7a70d27b7 made a variant that switches SSO on also switch the onion on, because
that pairing is where the platform breaks in ways clearnet never shows. The same
argument reaches further than SSO: a Secure cookie is dropped, an https identity
provider refuses to embed, and crypto.subtle is undefined for any feature on a
non-secure origin, not only for a login. A variant that never runs the onion is
not testing the deployment users get.
So the rule now covers every variant of a role that declares a tor block.
Variant 1 stays exempt - it is the minimal, partner-free round every role
declares and the one place the stack is exercised without the onion at all - and
a role whose meta/services.yml pins tor.enabled to a literal false is skipped
outright, because that declaration already says the role has no onion and needs
no comment repeating it.
That found 137 variants. 101 were variant 1. Of the rest, 28 are the LDAP-only
auth-isolation rounds, which gain real coverage from running LDAP over the onion
and are flipped on here. Four cannot: svc-net-tor is the provider itself, and
nextcloud's variant 8 pairs the clearnet-only PeerTube. Both carry a marker
naming the reason; opentalk and coturn need none, since their services.yml pins
the literal false the rule now reads.
Flipping them exposed a contract that contradicted itself. test_auth_isolation
and test_non_baseline_explicit_disables demanded `shared: false` beside
`enabled: false`, while variants-static-override flagged that same line as a
static override - which is why all 133 of them carried a nocheck marker. The
demand only makes sense where `shared` is dynamic: a literal base value cannot
leak through deep-merge, so restating it protects nothing. Both contracts now
ask for it only in that case, and the 133 lines are gone.
The result is a stricter rule with fewer lines: 189 insertions against 193
deletions.
Verified: tests/lint 336 passed / 307 subtests, tests/integration/roles 107
passed / 610 subtests. The rule is checked in both directions - it reported 137
offenders before the resolution and none after.
Not measured: 28 additional onion rounds cost matrix wall-clock, against swarm
jobs already capped at 355 minutes and one that has hit that ceiling before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every variants.yml header states the rule the bond feeds: variant 0 carries each
tightly-coupled (bond>=1, same-host) service, while loosely-coupled (bond<1,
separate-host) partners are excluded from the closure and distributed across
later variants to keep every host under budget. tor was declared bond: 1 in all
101 consumers, so it had to sit in every variant-0 closure.
It does not belong there. The onion is a separate-host provider reached over the
node's hidden service, the same shape as the bridged partners that already carry
0.5, and f1dff50fb established both the reasoning and the precedent when it
dropped xwiki and bigbluebutton from 1 to 0.5: bridged partners with their own
role and their own spec, whose bond declaration was the outlier rather than
their placement.
This also unblocks the variant-level tor contract. Requiring a non-baseline
variant to pin the onion on while tor still counted as tightly coupled would
have pulled variant 0's closure requirements along with it.
Verified: tests/lint 336 passed / 307 subtests and tests/integration/roles 107
passed / 610 subtests, including test_resource_budget over every variant's
deduplicated footprint and the bond-aware test_order closure.
Not measured, and the honest cost: f1dff50fb's own caveat applies unchanged
here. Whether redistributing a service out of the baseline closure buys back
wall-clock is not established; role-runtimes has the per-role seconds and no run
has been made with this shape.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Same class as 642055a20, missed by the filter that selected it: files were grouped by whether they CONTAIN a waitFor state call rather than by whether the diff ADDS one, so four whose change is a pure deletion were sorted into the waiting-semantics bundle instead. Their diffs alter nothing but the discarded argument, and every added line here is a bare isVisible().
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/volumes.yml says where a volume lands inside its container. Wherever a
second file spelled the same path out, the two drifted independently: the mount
moves, the consumer keeps pointing at the old place, and nothing fails until the
process looks for content that is not there. e031b7f61 fixed exactly this for
web-app-moodle; this generalises it and makes the rule hold.
The lookup gains what a caller needs. `targets` maps each mounting service to
its container path, and `target` is the one path an entry mounts at -- present
only when every mount agrees. An entry that lands at different paths per service
omits `target` rather than picking one, and the caller names the service:
"{{ lookup('volume', application_id, 'prometheus_config').target }}"
"{{ lookup('volume', 'web-app-prometheus', 'host_sys').targets['node-exporter'] }}"
66 literals across 48 role files now derive. Replacement only happened where the
path stood alone -- /srv/app inside /srv/app/data is a different path -- and
every rewritten file was re-parsed, Jinja and YAML alike. Three sites the tool
refused rather than guess were done by hand: zammad's path sat in a Jinja list
literal, seaweedfs' inside a quoted string feeding a filter, and openresty's
/var/run/docker.sock is not a mount target at all but the socket on the host,
which only happens to spell the same string. That one carries the single
nocheck.
tests/lint/repository/spot/test_volumes_path_single_spot.py keeps it that way.
roles/*/files/ is out of scope by construction: it is copied verbatim into an
image and executed there, so a Dockerfile or entrypoint has no Ansible to
evaluate a lookup with.
One defect this found, and it was in the tooling rather than the roles: deciding
ambiguity AFTER filtering short paths made web-app-prometheus' host_sys look
single-valued -- it mounts /sys for cadvisor and /host/sys for node-exporter,
and /sys is below the length filter. The template would have rendered an
undefined `.target`. Both the converter and the rule now decide ambiguity over
every target and filter afterwards, and that call reads targets['node-exporter'].
Verified by resolving rather than by reading: all 70 accesses in the diff were
run against the real registry and every one yields a non-empty path equal to the
literal it replaced. make test-lint: 328 passed.
Not verified: no Ansible render, so a call site whose surrounding context cannot
build the applications registry would still fail at deploy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.
The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.
--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.
Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.
tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.
Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.
Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.
Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:
- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
Same value, so nothing breaks today; editing the documented one would have
silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
rename never saw it. Its credential would not have resolved and the role
would have deployed with an unusable administrator account.
The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.
make quality green on the merge result.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml held credential definitions and their runtime values, but the
name said 'schema' and read like structural metadata next to services.yml and
volumes.yml. Nothing in the file is a schema in the sense the other meta files
use the word, and a contributor had to open it to learn it holds secrets.
62 role files renamed. ROLE_FILE_META_SCHEMA becomes ROLE_FILE_META_SECRETS and
keeps its role as the SPOT the path is built from.
Two readers bypassed that constant and spelled the name themselves:
utils/cache/applications.py built meta_dir / 'schema.yml' twice, and
plugins/lookup/application_gid.py carries the filename in the marker list it
detects application roles by. Left behind, the rename would not have raised —
credentials would have resolved empty and the role would have stopped counting
as an application.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each of the seven addon specs opened a fresh context and ran the full SSO
round-trip. Over Tor that is roughly twelve minutes apiece, and odoo 0 of run
31340965955 died on 'Test timeout of 600000ms exceeded' inside
gotoOnion -> loginToOdoo -> contacts.spec.js:13.
authenticatedContext caches the storage state after the first login and restores
it for later callers. A restored state that no longer renders the web client is
discarded and the full login runs, so an expired session degrades to today's
behaviour rather than failing the spec.
Coverage is not reduced: the SSO flow keeps its own dedicated tests in
playwright.spec.js:98 and :160 (admin and biber, log in / verify ui / log out),
which this change does not touch. The seven addon logins were repetition of an
already-covered path, and no addon spec asserts anything about the login or
needs an anonymous session. What does change is isolation -- the specs now share
one restored session instead of each starting clean.
The TEST_E2E_PLAYWRIGHT_ONION_TEST_TIMEOUT bump belongs in this commit: four
specs lost their own test.setTimeout(resolveTimeout(180_000)), which resolved to
900s on an onion. Without the bump those four would regress from 900s to 600s.
Cause B is untouched: the global ONION_RUN_TIMEOUT of 5400s left eight tests
unstarted. The cache should relieve it, but that is arithmetic on measured
single-login times, not an observed run, and raising a timeout that spans every
onion role needs its own measurement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Blocking a persona removes the only end-to-end proof that a role's auth chain
works. docs/contributing/artefact/files/role/playwright.specs.js.md:146 has
always required each flag to carry a one-line nocheck stating the role contract
that justifies it, plus a matching paragraph in the role's README or TODO.
Nothing enforced it, so 89 flags across 46 roles carried neither.
The lint mirrors test_has_spec's shape and uses is_suppressed_at with
mode="line-above", so the marker must sit directly above its flag -- a blank
line between them breaks the association, which is what the persona helper's
reader will look at. A literal =false blocks nothing and is exempt; the
sso-conditional ternary is not, because it blocks in every variant that pins
services.sso.enabled false.
Writing 89 reasons turned out to be the point. Five roles audited in one pass
blocked for five entirely different mechanisms, all of which read as the same
shrug behind a bare =true: pgAdmin's login id is the account e-mail rather than
the Keycloak username; phpMyAdmin authenticates as the MariaDB root account;
GitLab's seeded password belongs to the built-in root, not the administrator;
Mastodon's tootctl accounts create never sets a password; phpLDAPadmin binds
with an LDAP DN and a different secret. Further out: MediaWiki removes the form
that would accept the Keycloak secret ($wgPluggableAuth_EnableLocalLogin=false)
and bounces anonymous requests back into the IdP, so the unauthenticated
landing the contract asserts can never be observed; Mattermost Team Edition has
no native OIDC, so Keycloak rides the GitLab OAuth slot behind a relabelled
link; MinIO's console returns redirectRules: null and renders no SSO button at
all; XWiki's administrator is an XWikiUsers object with no password property.
Two flags turned out to be unjustifiable and are gone rather than annotated.
web-app-mailu gated PERSONA_GUEST_BLOCKED on services.dashboard.enabled, but
runGuestFlow does one anonymous goto and asserts no signed-in marker -- it
never touches the dashboard. A leftover from a helper version that entered
through the dashboard account menu. And web-app-mig's rationale claimed the
role is an in-app module of web-app-dashboard; it has its own compose stack,
healthcheck and canonical domain, and dashboard is merely a tile consumer.
Nine of the reasons say plainly that the block is not a role property but a gap
in the shared persona helpers -- decidim's sign-out is a data-method link the
generic click cannot activate, fider's SSO entry is a two-step modal, taiga,
wordpress, xwiki, odoo and penpot bind OIDC on a path the personas never visit,
jenkins blocks unconditionally although its OIDC variant looks runnable, and
opentalk's own spec proves both personas work. That list did not exist before;
it is the actual yield here, not the 89 comment lines.
45 pre-existing comments came into the comment lint's scope because this sweep
touched their files -- it only inspects files that differ from HEAD. 28 of them
were the banner "Service gate flags rendered for closure.", every one sitting
above a line that is not a service gate flag at all, orphaned by the
alphabetical re-sort these templates once got.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
modes.compose.enabled and modes.swarm.enabled were both false, and a
stack role is forced host=false, so all three test_<mode> columns were
false and the CI discovery filter (test_<mode> == true) dropped the role
from every matrix. It was resolved into the priority whitelist as a new
role and then silently filtered out again — svc-net-tor was only ever
exercised indirectly, as a dependency other roles pull in.
The flag is read solely by the discovery model and the README generator;
nothing in the deploy path consults it, so this has no runtime effect.
meta/categories.yml pins the whole net category to compose/swarm false,
which fits svc-net-wireguard-{core,plain,firewalled}: none of them ships
a compose template. svc-net-tor does, and is the only role in the
category that does, so it takes the category-modes exemption the lint
provides rather than bending the category default for three host roles.
Regenerated READMEs follow: svc-net-tor now carries its mode symbols in
every cosmos diagram that names it as a dependency.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Seven include templates under sys-svc-container decided how a container was
probed, and they read their inputs from template-local `{% set %}` scope.
That scope leaks: a value set for one service silently reached every service
below it, which is how taiga's front and gateway ended up probing a port
declared for a different service, and how bluesky's frontend probed a PDS
endpoint.
Probes now live in utils/docker/healthcheck.py, one class per flavor behind a
PROBES registry, and each service declares its own:
healthcheck:
flavor: curl
path: health/ready
start_period: 20m
`port_key` picks another entry under ports.internal for the services that
expose their probe elsewhere (keycloak on management, nextcloud talk on
signaling). Without a flavor the entry must carry an explicit test argv, so a
missing declaration fails loudly instead of rendering a probe against nothing.
Rendering HEAD's templates against the new output confirms 49 of 56 call
sites are byte-identical; the remaining seven are covered by an Ansible run.
Three probes change on purpose: erpnext's backend gains a check it never had,
bluesky's frontend drops the leaked PDS path, and msmtp_curl moves from a
hardcoded localhost:80 to the service's own port on 127.0.0.1 like every
other flavor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, each resolved by combining both sides rather than picking one.
compose.yml passes both env vars through to the container: swarm-nfs adds
INFINITO_SWARM_MGR_IP, this branch adds INFINITO_TOR_SOCKS_PORT, and neither
substitutes for the other.
daemon.json.j2 takes the data-root block from swarm-nfs and keeps HEAD's dns
key. The incoming side writes networks.internet.dns directly; lookup(
'container_dns') already returns that value as its clearnet entry and prepends
the docker0 bridge on a Tor node in compose mode, so the incoming form is a
strict subset and dropping it loses nothing.
The WordPress CA-trust addon spec gains skipUnlessServiceEnabled("sso") from
swarm-nfs alongside HEAD's gotoOnion and resolveTimeout, and asserts on the
Plugin Name the mu-plugin header declares - the mustuse screen renders that,
not the filename - with resolveTimeout applied to the literal the incoming
side left bare.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An untyped inspect searches every docker object kind, and a role's overlay network carries the same entity name as its main container, so the call can resolve the network and report a missing State key instead of naming the container. That is how the openresty assert died on every swarm job of run 30709070050.
Extend container-exec-resolver to cover inspect, stripping raw markers and quoted spans first so the flag walker survives Go template format strings. Add container-inspect-type, scoped to roles/ because the collision follows the entity naming convention and the outer act namespace carries no role networks. Sweep 40 call sites; suppress the delegate_to-resolved CIDs and the compose co-location probes with a stated reason.
Move the five container rules into tests/lint/ansible/shell/container/ and drop the container_ prefix from their filenames. Register both markers in the suppression docs, where the resolver marker was missing entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, none resolved by picking a side wholesale:
- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
rc now counts as failure; the previous default(0) let an aborted Playwright
run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
which this branch's lint enforces, and adopted the incoming retry-inside-poll
for the OIDC button plus its networkidle wait.
The declared values were guesses that ran up to 60x the size of the image they belong to: seaweedfs asked for 30GB against a 104MB image, nextcloud for 20GB against 353MB. Because min_storage feeds the storage guard, the runner packer and the sufficient-storage filter, those numbers made hosts look too small and pruned caches that deploys did not need to lose.
Every service that declares min_storage together with image and version now carries the measured compressed transfer size of that image plus 200MB of headroom. Sizes were read from the registries through utils.docker.registry.manifest_transfer_size, the same path the headroom test uses, so the declarations and the gate agree by construction. Values are written in MB throughout, matching the existing 500MB / 100MB / 50MB style.
119 declarations across 89 roles. Range: 206MB (memcached) to 3473MB (ollama); four exceed 1GB. The 84 min_storage entries that name no image are untouched, since there is nothing to measure them against.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A forced rolling update keeps N-1 replicas' ORM sessions alive while the
replacement runs -u all, deadlocking on AccessExclusiveLock. Scale the
service to 0, wait until no running task remains, then scale back to the
topology replica count so the entrypoint's advisory-lock winner is the
sole database actor during the update window.
Renumber the task files to the dispatcher convention: 02_manager_ops
keeps its repo-wide name, its children follow as 03-07 in execution
order, and sub-steps move into a subdirectory named after their parent
task instead of letter suffixes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add lookup('asset', app, pkg, path) and lookup('asset_host'): both resolve
the asset origin through one shared resolve_host, returning the internal
CDN domain when web-svc-cdn is deployed with flavor internal and
cdn.jsdelivr.net otherwise, so template URLs and CSP origins can never
diverge. Versions come from the role's committed files/package-lock.json.
Roles declare frontend dependencies in files/package.json; web-svc-cdn
aggregates every role lockfile, runs one npm ci per role in a node
container and mirrors the packages under /npm/<pkg>@<version>/ following
jsdelivr's path scheme, so internal and external URLs differ only by host.
Convert web-app-dashboard (keycloak-js 24.0.5 UMD) and web-app-littlejs
(bootstrap, fontawesome) to the asset lookup, drop the vendored keycloak
blob, and add a repository lint forbidding hardcoded external CDN hosts
in templates and CSP metas; pre-existing external CDN entries are checked
out with nocheck markers and a migration TODO.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Standardize the private inventory dir on the fully spelled "production" (matching inventories/development). Update the role README template (SPOT) and regenerate 153 role READMEs plus the admin deploy guide; add inventories/production/ to .gitignore, propagated to .dockerignore via make fix-dockerignore.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A transient Debian/Ubuntu mirror fetch aborts a docker build layer with exit 100 (it killed the web-app-nextcloud swarm openldap build while the same image built fine elsewhere). New lint under tests/lint/docker/dockerfile/apt/ flags any RUN with apt-get update/install that omits Acquire::Retries; retrofit all 15 apt Dockerfiles with -o Acquire::Retries=3. Both apt lints move into the apt/ subpackage. dnf retries by default and apk has no equivalent flag, so both stay out of scope.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
