f713ba835 moved credentials to applications.<app>.secrets.credentials and
updated every role that reads them, but four lookup plugins kept addressing
the bare credentials.<key> root. They call get() with strict=False and
default="", so the miss was silent rather than loud: MariaDB received
MYSQL_ROOT_PASSWORD= and refused to initialise, which took friendica-database
down as an unhealthy dependency and failed compose up. Object-store consumers
and shared redis auth resolved to empty secrets for the same reason.
The five call sites now build the path from OVERRIDE_SECTION, the single point
of truth that commit introduced, instead of spelling it beside it.
The two lookup unit tests pinned the pre-rename fixture shape, which is why
they stayed green straight through the regression; their fixtures move to
secrets.credentials so they fail against the old code.
Evidence: run 31649386343, job 94302347256, artifact
rescue-diagnostics-instructions-web-app-friendica,
containers/infinito_nexus_debian/nested/.../friendica-database.log:29
"Database is uninitialized and password option is not specified".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The body_filter lua render and the redirect-domain config both notified a full restart. Every accumulation window in the odoo swarm job carried one of them, so converting only the vhost render in 641925331 removed no restart at all.
Both write under directories.configuration.base, the same conf.d DIRECTORY bind the vhost uses, so SIGHUP is enough. Measured on job 93953781117: 27 of 30 restart windows become reloads, 27 x 39.9s = 18 min per swarm job. The remaining three are openresty's own base/default config and the prometheus http-block, which stay restarts.
Verified in openresty/openresty:alpine that a reload picks up a rewritten body_filter_by_lua_file across an inode swap -- lua_code_cache defaults to on, and SIGHUP forks workers with a fresh LuaJIT VM.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A per-host `when` on the looped include_tasks made ansible insert the included blocks in host order instead of loop order: whichever host reached an item first decided when that item ran. With the nfs-server host last in the inventory, svc-storage-nfs-client mounted the export before svc-storage-nfs-server had deployed it, and the play died on 'mount.nfs4: Connection refused' (run 31645153977, job 94280126636).
Gating on the include_role instead keeps every host on the same loop, so one block per item is inserted in loop order. This is what cli/build/include emitted before 351b64e2c: the when on the include_role, the flush unconditional behind it.
Reproduced with a four-host probe -- server host last inverts the order, gating one level down restores it. In run 31543714113 'setup svc-storage-nfs-server' ran 3 times; in 31645153977 it never ran.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_distros_spot globs **/*.md and friends across the whole project root and
reads every hit, so a named pipe, socket or device node with a matching suffix
crashes the check with PermissionError instead of reporting an offender. It now
skips anything that is not a regular file.
test_no_uncached_file_reads flagged three reads in the inventory reset tests.
Its suggested fix - the process-wide cached reader - would be wrong there: each
of those tests writes the file and reads it back in the same test, so a cached
read would serve the pre-write content and the assertion would stop testing
anything. They carry a nocheck marker instead, on the call line and short enough
that the formatter cannot wrap the expression away from it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds download.fedoraproject.org, ftp-stud.hs-esslingen.de, ftp.uni-stuttgart.de, ftp2.osuosl.org, mirror.dogado.de, mirror.netzwerge.de and mirrors.xtom.de to the WebFetch allow list, each in sorted position.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The adapter was the only image in the tree pinned by a hand-maintained digest,
and the pin was never recomputed by anything. It drifted twice: run 31374959721
lost seven of eight jobs to mirror/docker.io/python@sha256:6771159c... not
found, and the renewal to the newest upstream digest earlier today would have
reproduced it, because builds resolve FROM against the GHCR mirror while the
mirror is copied by tag and lags upstream by design.
The base is now FROM <image>:<version>, taken from the sidecar service in
meta/services.yml - the only file iter_role_images scans, so the tag the build
asks for is by construction the tag the mirror was told to copy. The version
moves from the moving 3.13-slim to 3.13.15-slim, which upstream stops rewriting
once published, and which the existing version drift job renews like every other
image.
With the build reading services.yml, the adapter block's digest, image and
version had no consumer left. They are removed together with the schema check
for the digest form, the warn-only drift test, and the manifest_digest helper
that existed only for it. What remains in the block describes the adapter rather
than the image it is built from.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/volumes.yml is read before the role's vars exist, so referencing OPENPROJECT_AWAIT_MIGRATIONS_ABS there broke test_no_role_vars_in_meta. Inline the expression vars/main.yml:14 already holds, with application_id spelled out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`cli build include` generated one tasks/groups/<group>-roles.yml per invokable category during setup, and the stage plays included those files. They were written by the in-container UID, which is why a dedicated clean-container-owned helper had to delete them from inside the container.
The stage plays now loop over lookup('group_roles', <group>) and include tasks/utils/setup/{group,role}.yml directly. ordered_roles() in utils/roles/order.py holds the run_after topological sort the generator used to perform at build time.
utils/roles/stage.py:_group_names() no longer globs the generated files; it reads get_invokable_paths(), the same categories.yml source those files were derived from.
Measured: handlers still flush after every role (ansible-core 2.20.2, three loop items, three separate handler runs), and resolving all 18 groups costs 0.60s cold against 1.4s for the generator.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PASS 2 read the exact values PASS 1 wrote, so no deploy ever had to propagate a secret that moved. Every role that only writes a credential at creation time therefore stayed green without the update path being exercised once.
Both orchestrators now call `inventory credentials reset` after the recover drill and before the async pass: compose through the container in the new .rotate module, swarm through the matrix orchestrator with --mirror, because its inventory keeps one host_vars copy per node. --backup keeps the pre-rotation file for the post-mortem, since after a rotation nothing records what PASS 1 deployed with. `administrator` stays exempt in both: its password is ansible_become_password, and rotating it would lock the deploy out of its own host.
Expect roles that never re-apply a changed secret to fail PASS 2 from here on; that is the gate working. Both database engines already converge their root password on every run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Provisioning fills a missing credential and leaves an existing one alone, so every deploy pass reads the value the first pass wrote and the update path is never exercised. Rotating the generated values makes a redeploy carry a secret that moved.
`infinito administration inventory credentials reset` drops every vault-encrypted leaf under applications.*.credentials.* (--schema) and the pinned password of every declared user (--users), then lets the existing generators refill the gaps, so no schema logic is duplicated. Plain credentials stay unencrypted in host_vars and are therefore left alone, as are users that exist only in the inventory and would get nothing back. --except exempts application ids and user keys, --backup keeps the pre-rotation file as <name>.YYYYMMDDTHHMMSSZ.backup, and --mirror realigns the other hosts of a multi-host inventory.
Adds the command only; no deploy path calls it yet. Measured against web-svc-coturn and web-app-flowise: eleven generated secrets move between two runs while ansible_become_password and the administrator password stay pinned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The path lived twice: as the mount target in meta/volumes.yml and again as a
literal in vars/main.yml. The variable now derives from the declaration through
lookup('volume', ...), the way web-svc-cdn and svc-opt-ssd-hdd already read
theirs, so the mount is the single place that defines where the cache lives.
The preceding commit's message already described this indirection; the edit
itself had not been staged with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous commit packed the personas into one archive but still transferred them per role. orchestrate.yml runs 01_setup.yml once and then loops 02_run_one.yml over every app plus a retry pass, which is where the 22 transfers came from. Moving the staging into the setup phase leaves exactly one.
Each container now mounts the shared tree read-only at /e2e/tests/personas, the same pattern node_modules already uses. Both import directions were checked: specs require ./personas from /e2e/tests and land in the mount, while personas/utils/env.js requires ../../timeouts and reaches two levels up into the writable per-app staging dir, where the shared helpers are still placed. The nested mount therefore does not cut the relative imports.
Measured saving is 4.1 of the 4.2 minutes the copy cost in odoo 0, since only one transfer remains.
Not verified: the nested bind mount was not executed. Docker orders mounts by destination depth so /e2e lands before /e2e/tests/personas, but if that ordering did not hold the spec would fail loudly on require, not silently.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copying a directory with ansible.builtin.copy walks it file by file, one round trip each. Measured in the green onion job odoo 0 from the log timestamps: 22 invocations at 10 to 13 seconds, 4.2 minutes total, for 15 files totalling 108 kB - about 0.7 s per file, all of it connection overhead rather than payload.
The tree is now packed once on the controller and transferred as a single archive. Uncompressed on purpose: at this size gzip saves 28 kB and would add a second binary that has to exist on every distro image. tar itself was verified present in the CI fedora image and in the arch and centos node images.
Equivalence checked rather than assumed. The old copy used a trailing slash and therefore copied the contents; `tar -C personas .` does the same. Building the archive produces 17 entries - 15 files plus ./ and ./utils/ - so the subdirectory layout survives.
This continues 6e14d12b5, which collapsed the per-file tasks into one task. That left the per-file transfer inside that task, which is what this removes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every wp-cli call in this role runs as www-data, whose home is root-owned, so
wp-cli cannot create its default cache directory. Without a cache each plugin
install re-downloads, and a single dropped connection ends the deploy instead of
being served locally - observed as 'Failed to create directory
/var/www/.wp-cli/cache/: mkdir(): Permission denied' followed by 'cURL error 52:
Empty reply from server' and 'Error: No plugins installed'.
An environment variable rather than the alternatives: chowning into the upstream
image would be permission surgery on every deploy, and a named volume would be
persistent storage for something defined as disposable. The cache now lives
exactly as long as the container, which is what a cache should do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Moodle kept everything under dataroot, which is the shared NFS export in swarm
mode - including localcachedir. Upstream is explicit that this is the one
directory that must not be shared: config-dist.php marks tempdir and cachedir
as 'MUST BE SHARED by all cluster nodes' and localcachedir as 'Intended for
local node caching'.
The path is declared once, as the mount target in meta/volumes.yml, and every
other place derives from it: vars/main.yml reads it back through
lookup('volume', ...), compose passes it as a build argument, the Dockerfile
exports it, the entrypoint requires it with :? and creates it with the runtime
owner, env.j2 hands it to config.php as MOODLE_LOCALCACHEDIR. The two mounts of
the same volume share one YAML anchor rather than repeating the literal.
The cron service mounts it too, because it runs the same code and would
otherwise write its cache somewhere else than the web service reads it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adding a domain redeployed the whole OpenResty task. Measured in the green onion job odoo 0: 26 redeploys at 41 to 43 seconds each, 18.1 minutes of a 4h44m job. A reload costs one to two seconds.
It is safe for a vhost because of where the file lands. nginx.py builds files.domain as servers_dir + protocol + domain.conf, and servers_dir is /etc/nginx/conf.d/servers/ - inside the conf.d DIRECTORY bind. A directory keeps its inode across the atomic write-and-rename that stack_host_template performs, so the running master already sees the new file. Only nginx.conf is bound as a FILE, whose inode the task pins at create time; that path keeps the redeploy, as do the default server config and the site-is-down case.
History matters here: a9ca2ae68 once made the handler reload for everything, including nginx.conf, and f9dea42c6 replaced it with the force-recreate. That replacement rode along in a commit about apt mirrors and explains itself nowhere, so the only recorded reason is the file header - the file bind and its pinned inode. That reason does not apply to a vhost in the directory bind, which is why this reload is confined to that case.
The reload cannot prove it took, so it does not have to. After the flush the domain is probed once, and anything other than 200, 301 or 302 notifies the full redeploy. The worst case is therefore one HTTP probe on top of the behaviour this replaces, never a domain left unserved behind a green deploy.
OpenResty runs replicated 1/1 (services.txt of a real deployment), so the exec reaches the only instance and no replica can keep stale config.
Not verified: the reload itself was never executed against a running stack.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A newly created swarm service starts every replica at the same instant -
update_config.Parallelism governs updates, not the initial scale-up - and each
process unpacks the prepackaged plugin into /mattermost/client/plugins, which
was an NFS export shared by all of them. The loser of that race aborts with
'unable to copy webapp bundle directory: chmod .../main.js: no such file or
directory', never activates the plugin, and answers 404 on its routes for the
rest of its life while the VIP keeps sending it a share of the traffic.
Measured on the live cluster: task IPs .30 and .32 answer 401 on the plugin
route, .31 answers 404; the shared directory holds the hashed bundle and the
LICENSE file but no main.js; byte-identical nanosecond mtimes on two nodes
prove one filesystem.
client/plugins is regenerated from the plugin at every activation, so it is
per-node state. plugins/ stays shared because uploaded plugin archives live
there, and sharing the generated bundle could never substitute for activation
anyway - a replica without the plugin in its process answers 404 regardless of
what sits in the directory.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four roles created their MCP service account and never touched its password
again, while the same play authenticates as that account moments later. A
deployment whose declared password moved on therefore failed at the login,
not at the point the drift was introduced.
gitea now runs 'admin user change-password' before the token dance. The
--must-change-password=false flag is mandatory rather than cosmetic: at the
pinned v1.27.1 the flag defaults to true, so omitting it locks the account out
of the very API call that follows.
nextcloud resets through 'occ user:resetpassword --password-from-env' before
minting the app token that verifies the password.
snipe-it compared nothing at all - its docstring promised to pin the password
while the assignment sat inside the create branch. It now rehashes only when
Hash::check says the stored hash no longer matches, so the changed flag stays
honest.
home assistant swallowed a failed credential creation for accounts that already
existed and then logged in with them. It now calls admin_change_password and
falls back to creation only when the reply is credentials_not_found.
jellyfin and zammad already converged; wordpress, gitlab and moodle never
authenticate with this password and are left alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
msmtp has defaulted tls_trust_file to the special value 'system' since 1.8.2,
and 'system' is exactly what the platform already curates: with-ca-trust.sh
installs the root anchor through update-ca-certificates, update-ca-trust or
trust, whichever the distribution has, and never needs a path. The deleted
apparatus reimplemented that by hand and measured the wrong machine - it
stat'ed candidate paths on the Ansible target while the rendered file is read
inside Debian containers, so a RedHat host would have named a path those images
do not have.
Measured on a live Debian node with no config file at all, i.e. the state this
commit produces: msmtp reports tls_trust_file = system with tls_certcheck = on
and verifies the relay leaf CN=INFINITO.EXAMPLE Global TLS against
CN=infinito.example Root CA; pointing it at an unrelated CA fails as expected.
Inside the deployed wordpress container the rendered msmtprc now carries no
trust line and msmtp 1.8.28 (GnuTLS) resolves it to system.
On p11-kit distributions this widens the anchor set from the serverAuth
extraction to the unfiltered store rather than being neutral; no anchor is lost
on any supported distribution.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Swarm ignores depends_on, so web and worker start while the seeder is still migrating and die on ActiveRecord::PendingMigrationError. The artifact records 12 failed attempts across the cluster before any replica survived, and all three survivors landed on the node with the fewest failures - one against five and seven - which is how the drain drill lost the worker task it asserts on.
The app services now start through files/shell/await_migrations.sh, which polls a read-only gate and then execs the upstream entrypoint. The gate never migrates, so replicas cannot race each other the way an `until db:migrate` loop does. The seeder keeps its job: the artifact shows it applying 465 migration lines and then seeding, so the wait is aimed at work that really happens during the deploy.
The wait is bounded because the gate is ambiguous. Measured against openproject/openproject:17: `bundle exec rails db:abort_if_pending_migrations` takes 11 s and exits 1 both for pending migrations and for an unreachable database (ActiveRecord::DatabaseConnectionError). An unbounded loop would spin forever whenever the database is down. Past the deadline the entrypoint starts regardless, so the worst case stays the crash loop that already exists, never a service that refuses to come up.
The container path is declared once, in meta/volumes.yml; the second mount takes it by YAML anchor and compose reads it back through lookup(volume). Both were verified through the projects own loader, not only yaml.safe_load.
Verified by running the script in that image with no database and no workdir override: it polls, reports the deadline and starts the entrypoint. Not verified: the success branch, since no migrated database was available, and no live deploy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
web, worker and cron depended on the seeder with condition service_started, which means "has begun". The seeder migrates first and seeds afterwards, so all three started mid-migration and died on ActiveRecord::PendingMigrationError. The artifact shows the consequence: 12 failed web and worker attempts across the cluster before any replica survived.
service_completed_successfully is what upstream uses and what funkwhale, mastodon, gitlab, zammad, erpnext and bookwyrm already use here. It is safe because the seeder genuinely terminates: openproject_seeder.ps.txt records CURRENT STATE "Complete", it carries docker_restart_policy on-failure, and meta/services.yml pins it to one replica with a note that it migrates and seeds once then exits. A seeder that kept running would deadlock every compose deploy on this condition, so that was worth checking rather than assuming. The service is declared unconditionally, so no variant can leave the condition pointing at an absent service.
This does not green the failing job. That one is a swarm job, and swarm ignores depends_on - container_depends_on emits only the short form there. The race exists on the compose path too and is what this fixes; the swarm side needs the migrations moved out of a service entirely and should not be patched blind.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Reverts the twelve pins to 9662417a, the digest the GHCR mirror currently
carries for python:3.13-slim. The previous commit renewed them to the newest
Docker Hub digest, which is the wrong registry: builds resolve FROM against the
mirror whenever a mirror prefix is configured, and the mirror is copied by tag,
so it holds whatever the tag pointed at when the sync last ran. A pin ahead of
the mirror resolves to nothing there while a local build keeps working - run
31374959721 lost seven of its eight jobs to exactly that, as
mirror/docker.io/python@sha256:6771159c...: not found.
The external check measured Docker Hub and would therefore have kept advising
the change that breaks the build. It now resolves the source the build uses via
mirror_image, asks whether the pin exists there at all, and only falls back to
upstream when no mirror is configured.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every one of the eleven declared tools carried the placeholder schema
{"properties": {}, "type": "object"}. The adapter answers tools/list purely
from this contract, so each client saw parameterless tools: nothing could know
that read_post needs a post_id, and the call failed upstream instead of being
formed correctly. The schemas here were measured against the running upstream;
list_agents keeps an empty property set because upstream declares none.
Both pins of the file follow, as the adapter rehashes the contract at startup.
Add a lint binding the declared surface to the shipped one. Only the allowlist
is an equality: writer_allowlist is a superset by construction, because
passthrough.py rejects a whole contract as soon as one tool declares
mutating: true while mutations are off, and upstream_serves is an observation
that may legitimately exceed what the platform allows.
The suppression lookup both MCP lints need moves into suppress.py rather than
being written twice.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
f713ba835 rewrote the BIND_CREDENTIAL lookup onto secrets.credentials and left a doubled blank line above it. Autoformat collected it after that commit had already landed. No key, value or indentation is touched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
BookWyrm 0 of run 31543714113 never converged, so Playwright never ran. The container log names the reason verbatim: "django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: bookwyrm. You may need to add bookwyrm to ALLOWED_HOSTS." The Host header is the bare entity name because container_hostname clamps to the 63 bytes sethostname(2) allows, and a v3 onion domain always exceeds that, so the lookup falls back from the primary domain to the entity name. The curl probe then reuses that kernel hostname as an HTTP Host, Django refuses it, the healthcheck fails and the container is killed every cycle.
Pinning the probe to localhost decouples the Host header from the kernel hostname. It follows e9817f825, which fixed pretix the same way and recorded why this stays per-role: reusing a kernel hostname as an HTTP Host is the underlying design bug, but defaulting the curl flavor would change six roles at once.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Friendica 0 and WordPress 0 of run 31543714113 never delivered their startup mail. An onion relay has no TLS certificate, so the resolver forces tls=false - and both port and auth hung off that flag: port collapsed to 25 and auth to off. Port 25 is the inbound MX and offers no SMTP AUTH at all, so the message reached Mailu unauthenticated with a local envelope sender and rspamd rejected it 554 5.7.1. Port and auth now hang off `external` instead: 465 with TLS, 587 without, and auth wherever a relay is external.
Moving the port alone would only relocate the failure. msmtp refuses to pick PLAIN or LOGIN itself on a cleartext link and aborts with EX_UNAVAILABLE, so the mechanism has to be named. The new `auth_mechanism` key resolves to off, on, or plain and sits in RESOLUTION_ORDER right after auth, whose result plus tls it reads.
It is one derived value rather than two template expressions because the two failing jobs run different msmtp instances: WordPress renders sys-svc-mail-msmtp/templates/msmtprc.conf.j2 through WORDPRESS_MSMTP_SRC and mounts it at /etc/msmtprc, while Friendica generates its own from SMTP_AUTH via the image script, whose line 32 writes `auth $smtp_auth` verbatim. Both now read the same key.
Verified against the shipped Mailu 2024.06 front rather than assumed. EHLO on 587 under TLS_FLAVOR=notls answers `250-AUTH PLAIN LOGIN`, so PLAIN is offered on the cleartext link; dovecot renders disable_plaintext_auth=no in that flavor. PORT_587 is set because socrate lists 587 outside PORTS_REQUIRING_TLS, unlike 465, which notls skips - so 587 is the only submission port available there. smtp and submission bind identically to MAILU_IP4_PUBLIC, so the path to 587 is the one that already reached 25.
This supersedes the premise of 563da9e5b, which disabled AUTH because the resolver had. That commit quoted a different msmtp error - "the server does not support authentication" - which was port 25 having none, not the relay refusing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Penpot 0 of run 31543714113 hung every OIDC login for 900 s. The Penpot backend is a JVM app and JDK 26 offers `Upgrade: h2c` on a plaintext POST. The generic HTML location relayed that upgrade verbatim and hardcoded `Connection: upgrade`, so Keycloak answered 101 Switching Protocols and continued in HTTP/2 framing. nginx proxies HTTP/1.1 and cannot consume those frames: it emitted chunked response headers with status 400 and zero body bytes, then held the connection until proxy_read_timeout. The client died on "chunked transfer encoding, state: READING_LENGTH" at oidc.clj:459, and Playwright timed out at 150 s long before that.
Measured, not inferred. Against Keycloak 26.7.0 the same token POST, differing only in the three upgrade headers, returns 401 with a 93-byte body without the offer and 101 Switching Protocols followed by HTTP/2 frames with it. Through nginx carrying the pre-fix directives the h2c request reproduces the CI signature exactly - status 400, Transfer-Encoding chunked, zero body bytes, elapsed equal to proxy_read_timeout - while the same request through the patched config returns the complete 401. The non-h2c path is byte-identical across both configs at 379 header bytes and 93 body bytes, and real WebSockets take the default branch unchanged.
Two maps in http{} do the work: $proxy_upgrade blanks the Upgrade header for h2c so nginx omits it, and $connection_upgrade answers close. html.conf.j2 also stops hardcoding `Connection: upgrade` on every HTML request, which misstated the hop state on requests that carried no Upgrade header at all. ws.conf.j2 already paired `Upgrade $http_upgrade` with $connection_upgrade and moves to $proxy_upgrade so the pair stays coherent under the new h2c branch.
Refusing the upgrade also removes a proxy/backend desync surface rather than adding one.
The comment deletions are the diff-scoped comment lint un-grandfathering the touched files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three fields pinned artefacts and nothing recomputed any of them.
The adapter base digest was stale for all twelve provider roles: the tag
python:3.13-slim moved to ffb752e1 while every role still pinned 9662417a.
The GHCR mirror copies by tag, so the pinned manifest is one it was never
asked to copy - a mirrored build resolves FROM to nothing while a direct
build keeps working.
adapter.specification_sha256 matched no revision of the file it pins in ten
of twelve roles; three of the old values were byte-identical to the same
role's tools.schema_sha256, so they had been copied rather than computed.
The field is enforced nowhere at runtime, which is exactly why it needs a
check.
Add a hard lint for both offline pins, computing the tool digest with the
adapter's own policy.schema_digest so it cannot drift from the code that
enforces it in production, and a warn-only external check that reports a
base digest the upstream tag no longer resolves to.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Alphabetical make target order, and the integration test walked the project
tree itself instead of the cached helpers, which share one walk and one read
per file across the whole session.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The DR drill starts the backup units itself, but the deploy arms their calendar
timers first, and a job that runs long enough crosses every backup window in
group_vars/all/07_schedule.yml. web-app-decidim ran 23:05-02:24 and hit exactly
that: the drill's own volume backup succeeded at 01:00:49, then the timer fired
a second, unattended run at 01:03:18 while step [8b/9] was lazily unmounting
/var/lib/infinito underneath it.
The failure is silent by construction. baudolo writes the dump in two
processes - 'pg_dump > x.tmp', then 'mv x.tmp x'. The redirect shell exits 0
because a shell never reports a failed writeback on a soft NFS mount, but the
unmount discarded that mount instance together with the unflushed file, so the
mv resolved through the fresh instance and got ENOENT. The unit died at
01:04:04, the same second the drill finished remounting, and 80 minutes later
the CLI test read 'failed' and aborted the job before running its own backup.
Disarming the timers makes the drill the only thing driving those units.
Stopping a timer does not stop its service, and both the drill and
roles/svc-bkp-volume-2-local/files/test/test.sh start their units explicitly,
so nothing under test loses coverage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.
The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.
--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.
Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.
tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.
Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The connect control replaces the location with the partner's authorize
endpoint, and GitLab bounces the unauthenticated browser on to its sign-in
page, so the spec asserted against the wrong URL and the redirect_to escape
hatch existed to paper over it. Capturing the request is what the Zammad
spec beside it already does. The README claimed the app password still
belongs to the administrator, which stopped being true.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine roles name their MCP principal by a literal key, and a typo there does
not fail loudly: the lookup yields an empty password and the application
answers with a rejected login, on the one variant that enables the surface.
The rule stays scoped to the mcp- principals because the registry is open by
design, so a repository-wide version would report the inventory's own
Playwright persona as missing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twenty-three PHP and twenty Ruby files reached the applications unchecked,
which is where the provisioning scripts live. php -l and ruby -c cover what
is reachable here; PHPStan and RuboCop would need the target project's
classes, since these scripts run inside Moodle, Snipe-IT or GitLab.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nine roles reached their MCP surface as the deployment administrator or with
an account password kept beside the account in credentials. A Moodle site
admin bypasses every capability check, a Nextcloud app password is not
scopable, a Gitea token reads every private repository, and a GitLab token on
root reads every private project. Each role now declares its own principal in
meta/users.yml and provisions it, so the exposed surface is what the service
role permits. jellyfin, zammad and snipe-it kept their account but took its
password from a credential named token; they read the registry too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Loading the backend first left flowise's swarm deploy notification pending,
and the front proxy's own handler flush consumed it, so the stack was never
rolled out and openresty answered 502 for thirty attempts. sys-stk-full
loads both in order and derives the domain and port the role passed by hand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A user password is pinned once by inventory creation from a shell-safe
alphabet, because it travels through Ansible, a shell and container exec.
An application whose registration demands a special character then had no
way to say so, and roles worked around it with a second credential beside
the account they already owned. The account now carries algorithm and
validation itself, two roles disagreeing about one account abort instead of
one silently winning, and an unsatisfiable pair fails naming both halves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Covered by PR #380 rather than merged on its own. This branch already carries
the Tor onion-service provider across the whole role set, and #495 touches the
same nextcloud Playwright specs; landing them separately would mean resolving
the same conflicts twice, once here and once on main. Reviewing #495 as part of
#380 keeps the resolution in one place.
One conflict, in the external-sites addon spec. Nextcloud 34 renamed the admin
section from #external to #external-vue and replaced the add-site control with
a 'New site' button, so the incoming selectors win — retargeting them is what
the PR is for. This branch's resolveTimeout wrappers stay on both assertions.
make quality green on the merge result.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Keeping the admin bypass off makes check_model_access run for every user,
and a model discovered through a connection has no row in the model table,
so the lookup raised and the chat answered 400 Model not found while the
same model was still listed. The flag restores the upstream default and is
read nowhere near the MCP tool-server gating, which stays on the admin
bypass alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The integration addons run container exec against the partner during
nextcloud's own deploy, and the loader orders the web-app bucket by role
name, so openproject and zammad had not been deployed yet and the exec
answered "No such container". Edges to roles outside the play are dropped
by the topological sort, so the entries are inert where the partner is
absent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The plugin caches mcp.enablePluginServer in process and refreshes it only
on activation or on a plugin cluster event, which Team Edition drops
silently. Toggling the plugin over mmctl reached one container, so the
other replicas kept serving 404 on the MCP route behind the round-robin
VIP. Patching the config row now notifies container recreate, which the
repository already defines for in-database state a process must reload.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
docker's name filter matches substrings, so every stack whose sidecar is
named after its app answered with two containers and two services. The
drill took the first, which after a reschedule was the MCP sidecar: it
declares no healthcheck and does not listen on the app port, so the step
waited sixty seconds on the wrong process. Selection now goes through the
swarm service label and an exact name match, both behind one helper.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.
Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.
Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:
- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
Same value, so nothing breaks today; editing the documented one would have
silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
rename never saw it. Its credential would not have resolved and the role
would have deployed with an unusable administrator account.
The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.
make quality green on the merge result.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The unauthenticated probe read a PHP fatal rendered as HTTP 200 as proof
that the endpoint serves anyone, wrote the REJECTED marker and thereby
settled probe.yml's retry condition on the first attempt. A body that is
not JSON-RPC now counts as not-yet-serving, which is what the existing
thirty retries are for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The drill probed for a fixed 60s. A container without a shell can only be
observed through its docker health status, and mattermost declares
start_period 30s with interval 1m, so that status cannot report healthy
before 90s. The step was unreachable for such an app however fast it
started. The budget now comes from the container's own healthcheck and
stays at 60s for every container that declares none.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An adapter-fronted provider holds two unrelated credentials, and handing
the upstream token to the contract probe produces a 401 the log blames on
the adapter. The rule binds the probe include, the vars definition reading
credentials.mcp_bearer and the ADAPTER_BEARER the sidecar starts with to
one variable, so the three cannot drift apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Playwright env file renders every value through dotenv_quote, so the
raw read carried the literal quotes into the URL and the parser took the
host up to the first slash. The guest spec asked mattermost.infinito.example"
and failed DNS on every attempt.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both tests reached for the filesystem themselves: one built the project root
from Path(__file__).parents[4], which silently points at the wrong directory the
moment the file moves, and both called Path.read_text directly, bypassing the
cache every other reader shares. The role-defaults path was spelled out by hand
where ROLE_FILE_DEFAULTS_MAIN already names it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SSO over a plaintext onion is where this platform actually breaks, and none of
it shows on clearnet: a framework's Secure session-cookie default makes Chromium
drop the cookie, an https identity provider cannot be embedded in an http page,
and crypto.subtle is undefined because an onion is no secure context. Pixelfed,
Taiga, Joomla and WordPress each cost a red job over exactly that. The
combination is only exercised if a variant deploys both, so the lint requires
it: services.sso.enabled true implies services.tor.enabled true.
69 variants enable SSO; 59 already complied. Of the ten that did not, seven are
flipped on because the mechanism is proven there - matrix and nextcloud both run
SSO behind the onion in variant 0 already (20ddda8c1), and their later variants
were pinned false only by the blanket rule from 1d9d355bc. Nextcloud variant 1
stays false: SSO is off there, and it is what gives the coverage test its
false-pinning variant.
Three roles opt out, each for a structural reason named in the marker. jitsi and
bigbluebutton pin tor.enabled false in meta/services.yml because the conference
needs getUserMedia, which no plaintext onion can offer. seaweedfs gates its own
sso flag on 'svc-net-tor not in group_names', so on a tor node there is no login
flow to test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A retrigger reproduces its source run's configuration, and the tor axis was
missing from it: rerunning a run dispatched with 'exclusive' silently fell back
to 'auto' and tested a different matrix than the one whose failures it was meant
to chase.
The mode is read from the job log's Inputs block, not from the run title. The
title renders it as a glyph, so recovering it there would mean translating a
symbol back into a word on every retrigger; the log already holds what the
operator picked, and inputs_from_jobs reads that same block for the priority
list today. LOG_INPUTS names the inputs that take this route.
That also removes a trap the glyph created. The tor segment renders through
format('{0} ', ...) with no opening literal, and run_name locates a value by
searching for its head - str.find('') matches at position 0, so the parser would
have handed back the title's own prefix as the tor value, plausibly and without
raising. segments() now skips a headless segment; its glyphs stay openings, so
they still terminate the value before them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
