Sphinx's i18n transform parses every msgstr into a temporary document
whose settings carry the whole BuildEnvironment (sphinx/transforms/
i18n.py, Sphinx 9.2). MyST nodes patched into the page keep pointing at
that document, and write_doctree only detaches the environment from the
page's own settings, so every translated Markdown page was pickled with
a full environment copy: a German doctree of drv-lid-switch grew from
21 KB to 5.5 MB, the whole doctree tree from 95 MB to 2.5 GB. The write
phase loads them all back and the forked -j worker copies the parent,
so the `deployed:de` build reached 5+ GB in the parent and was
OOM-killed inside the docs container in CI run 40625679270 (Jenkins#1,
Nextcloud#11), drove shared 16 GB swarm runners into swap and the
watchdog's memory and I/O-wedge aborts (GitLab#2, Nextcloud#7/#9/#10),
and left the daemon answering 500 to the queue task for the failed
build.
AdoptTranslatedNodes runs right after the Locale transform and points
every node whose document is not the page back at the page; conf.py
loads it for every build the image runs. No size, limit or budget
changes, and docs stays deployed wherever a variant declares it.
Measured locally on the full German build at -j 2: parent peak 685 MB
(was 5.3 GB and climbing when stopped at about a quarter of the write
phase), total 2.4 GB (was 8.35 GB), doctrees 92 MB (was 2.5 GB). The
English HTML of all 5,373 pages is byte-identical before and after. A
real German MyST build in the new test finds no foreign nodes and no
BuildEnvironment in the saved page; without the transform it fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2bf329cd3b put rescue_window.sh next to rescue_index.sh, and
test_prefix_grouped_files rejects two siblings sharing the `rescue`
prefix (quality-high pass 67). Both move to utils/rescue/ as index.sh and
window.sh, their tests to the mirrored tests/.../utils/rescue/, and every
caller follows: dedicated.sh, the swarm teardown in 00_one.sh, the
workspace workflow, and the swarm and diagnostics READMEs.
The budget-kill drill gave the stand-in a 1 s budget; under the full
parallel unit run the SIGTERM arrived before its EXIT trap was installed,
so nothing was captured and the test failed. The two kill drills now use
KILL_BUDGET = 3 s; their assertions are unchanged. A full `make test-unit`
passes them under load.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
distros.sh stops a distro at its budget with `timeout -k 60`: SIGTERM at
the deadline, SIGKILL 60 s later. dedicated.sh's cleanup trap then ran
the in-container capture under `timeout 1500` (266 s in Baserow#0 of run
40625679270) and the host capture under `timeout 900`, so the SIGKILL
always landed mid-capture, the snapshot was never copied out and the
rescue-diagnostics artifact was lost. The inner `timeout` also sat in its
own process group and outlived the kill, holding the step open.
- distros.sh keeps the grace in one place, KILL_GRACE_SECONDS, and
exports INFINITO_CI_DISTRO_KILL_EPOCH next to the deadline epoch.
- scripts/tests/deploy/utils/rescue_window.sh decides per trap step:
past the deadline, the step gets half of the time left until the
SIGKILL (at least 1 s) and an end epoch; before it, the nominal
timeout and no end epoch. Inside the trap `$?` is 0 on a budget kill,
so the deadline epoch is the only reliable signal.
- dedicated.sh runs both captures through `timeout --foreground` with
that window, keeping them in the trap's process group, and passes the
end epoch as RESCUE_DEADLINE.
- utils/diagnostics/container.py caps every command at RESCUE_DEADLINE,
writes `[skipped: past RESCUE_DEADLINE]` for what no longer fits and
hands the deadline to nested captures. Without the variable nothing
changes, so the swarm and workspace callers behave as before.
-k 60, the distro budget and every timeout-minutes are unchanged. Drill
tests replay the trap with the real helper and container.py: a budget
kill ends with rc 124 and the snapshot copied out, the old shape ends
with rc 137 and nothing copied, an early failure keeps the full capture.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_backup_nfs_excludes.py (4fcf097e5b) computed the repository root
with Path(__file__).resolve().parents[6], which
test_project_root_import rejects outside a package __init__.py
(quality-high pass 66). The package now exports PROJECT_ROOT exactly as
its sibling files/__init__.py does, and the test imports it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
svc-bkp-nfs-2-local snapshots the export base (/srv/nfs), so a
generation's files/ root already holds infinito-state/. Without an
explicit target, `recover nfs` and `recover full` restored it onto
/srv/nfs/infinito-state: the tree came out as
infinito-state/infinito-state/<volume>, and `rsync --delete` wiped every
live volume directory and the backup root under the old target.
- paths.NFS_EXPORT_BASE = get_export_base(), the function behind
lookup('nfs_export_base') that also feeds the backup unit's source,
replaces NFS_EXPORT_STATE. NfsRecoverer defaults to it; an explicit
`snapshot:/abs` target still wins, a remote target is still prefixed
with its host. `recover full` inherits the default.
- recover_drill.sh seeds its marker at files/infinito-state/<volume>/
and asserts it under the export base, so the drill can no longer pass
on a nested layout.
- The help examples, schema.yml and the CLI README show the export base;
the documented subtree example would have nested too and now names the
matching snapshot subtree.
Reproduced on a temp tree with the role's real script.sh and recover.py
and the CLI's argv: the old default loses every live volume and the
backup root, the new one restores in place. The new tests fail 6 of 6
against the old default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NFS backup rsynced the whole export and honoured only one exclude, the
shared backup root. Volumes declared `backup: false` were copied anyway and
then pulled a second time by remote-2-local; in the swarm DR drill of
Magento#0 (run 40625679270) docs_sites alone (a full repo clone plus its
prepared and work trees) took about 15.5 GiB per copy and filled the
runner. The volume backup already skipped these through the backup_volume
lookup; only the NFS path ignored the declaration.
- utils/storage/nfs.py: volume_dir() names the state directory a volume
binds to (pinned name or key), shared with compose_volumes;
backup_disabled_dirs() returns every NFS-backed `backup: false` volume
dir that no backed-up volume also binds to.
- The role-local lookup backup_nfs_excludes feeds them into
BKP_NFS_2_LOCAL_EXCLUDES next to the backup root; script.sh takes any
number of excludes. Compose installs still pass only the backup root.
- recover.py protects every live state directory the snapshot does not
hold, so `rsync --delete` keeps the excluded volumes (model weights,
the docs tree, caches) and volumes created after the snapshot instead
of wiping them.
- The swarm DR drill restores BACKUP_NFS_VOLUME, the first NFS volume
not declared `backup: false`; PRIMARY_NFS_VOLUME stays for the drain
tests. BookWyrm now drills bookwyrm_media; decidim, fusiondirectory,
meta, coturn, svc-registry-cache and svc-registry-docker have nothing
backed up on NFS and skip the drill.
Measured: compose_volumes renders identically before and after for 291
roles across compose/swarm x local/nfs (1940 of 1940); the 15 excluded
directories are exactly the backup-disabled device dirs; the systemd unit
renders shell-safe on ansible-core 2.20 and 2.21 and stays byte-identical
in compose and swarm-local. A new unit test runs the real script.sh over
two generations and the real recover.py back into the export, and fails
without the restore protection.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Job and run titles read `🐳🧅…𓏴𓏛·Nextcloud#2`; the `·` between the
hieroglyph block and the title only costs width. RoleDisplayName.encode
now joins the two directly (`𓏴𓏛Nextcloud#2`). Spaces inside a README
heading still become `·`, which keeps a display name one token in the
space-separated whitelist and priority inputs.
decode and decode_list still map the dotted form of older runs back to
the role id, so report_failures, the run tooling and selection tokens
pasted from an earlier run keep working. Old and new names cannot
collide: a glyph block holds only hieroglyphs and a title starts with a
letter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
320929e17c read the tempdir compose.yml back with Path.read_text, which
test_no_uncached_file_reads rejects (quality-high pass 64).
utils.cache.yaml.load_yaml keys its cache on path, mtime_ns and size, so
the file sync() rewrites in place is read fresh; the label assertions
would fail on a stale read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The autoformat step of `make quality-high` reflowed three lines of
320929e17c that exceeded the line length; tokens are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The chunk jobs read `🧱 Chunk N` and every priority row carried its own
trailing ⭐, so a priority sweep repeated the star down the whole job
list while the chunk that held those rows looked like any other.
- The four chunk jobs are named `0️⃣ Batch` to `3️⃣ Batch`, with ` ⭐`
when the batch holds priority rows, and their discover job reads
`🔍 Batch N ⭐` through the existing `marker` input.
- call-validation counts those batches once per run
(scripts/github/resolve/priority_batches.sh, which runs
`cli.meta.ci.matrix --priority-chunks`) and exposes the count as the
`starred` output. chunks.plan cuts the priority chunks from the
priority rows alone, so offset and the regular line cannot move them;
the orchestrator hands validation the same sweep and chunk size it
hands the chunk jobs.
- The row label drops its own star. report_failures and the job-name
parsers still accept a starred title from an older run.
- The glossary names the unit `batch` (`#️⃣`) instead of `chunk` (`📦`),
`ci plan` prints the batch column as the keycap the job name leads
with, and the name lint accepts a leading keycap emoji.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_backup_stop_decision_declared looked a volume mount's `service` up
as a meta/services.yml key. A role whose compose service is renamed via
`proxy.service_key` slipped through: web-svc-xmpp mounts `ejabberd`, its
entry is `xmpp`, and the lint never asked it to decide.
- compose_service_names() in utils/roles/applications/mounts.py is the
shared mapping from a meta key to the compose services it renders as
(the key plus a literal proxy.service_key); the lint resolves each
backed-up mount to its owning keys through it, and
test_image_volumes_mounted uses it instead of its own copy.
- web-svc-xmpp stops before the copy (`/opt/ejabberd/database` is a
Mnesia store spread over several files), which is what the missing
marker already did. Its upload volume is no longer backed up: the
ejabberd config enables no mod_http_upload, so nothing writes there.
- web-app-pretix deploys pretix_custom:3.13, the reference
lookup('backup_image', 'stop_required', false) resolves, instead of
overriding the tag with the source version. The override made the
deployed image miss --images-no-stop-required, so baudolo stopped
Pretix despite `stop_required: false`. The source version moves into
the build args, which keeps a bump rebuilding through the build
digest, and into an OCI version label on the image.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Swarm deploys with `docker stack deploy --resolve-image never`, so a task
only rolls when its spec changes. Since fdf1a33c5f BookWyrm, Decidim and
the Bluesky broker build under a constant tag (bookwyrm_custom:3.11-slim
and friends): a ref bump rebuilds the image through the build-args digest
and pushes it, but the image string in compose.yml stays the same and the
running tasks keep the old image.
registry_sync now stamps every service that runs a locally built image,
siblings sharing the tag included, with the container label
`infinito.image-id=<local image ID>`. A rebuild changes the ID, the label
changes the ContainerSpec, and swarm rolls the tasks; an unchanged image
leaves the label and the deploy untouched. List and mapping label forms
are both kept intact.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
baudolo stops every container on a volume before the authoritative copy
unless its image is listed via --images-no-stop-required. Leaving the
marker out silently picked "stop": the Stalwart webmail start that the
backup cycle triggered got SIGKILLed mid `composer update` (CI run
40625679270), and 71 services that mount a named volume had never
decided either way.
- The marker is now `backup.stop_required` (true = stop first, false =
copy live), replacing the double-negated `no_stop_required`; the 50
existing declarations are inverted unchanged. backup_image takes the
marker and the wanted value explicitly:
lookup('backup_image', 'stop_required', false) still feeds
--images-no-stop-required, lookup('backup_image', 'disabled', true)
the skip list.
- The 71 undecided services carry researched values (repo files plus
upstream backup guidance: GitLab, Jenkins, Elastic, RabbitMQ, Typesense,
Qdrant, OpenLDAP, Prometheus, Moodle, ClamAV, Alertmanager, ...): engine
files, git repositories, SQLite, mnesia and multi-file state stop; image
code, write-once blob stores with metadata in the database and
atomically rewritten files copy live. Stalwart's main service stays
stopped until /var/lib/stalwart is verified to hold only logs.
- Re-obtainable data is no longer backed up at all (`backup: false` on
the volume): model weights, ClamAV signatures, SearXNG config/cache,
transient RabbitMQ queues, Funkwhale's Typesense index, the meta clone,
the agent-broker socket dir, coturn's unused data dir, and volumes no
image reads (Decidim's /app paths outside /code, BookWyrm's /app/data
which v0.9.3 never reads, FusionDirectory's measured-empty dir).
- The backup lookup only matches what actually runs once the role
declares it: Bluesky's broker (`custom: bluesky_broker`) and Decidim
(`custom: true`) declare their custom build, BookWyrm's worker/migrate,
EspoCRM's daemon/websocket, Moodle's cron and OpenProject's proxy use
their owner's image, and the templates drop their render-time
overrides. Decidim deploys decidim_custom:0.31.6 and BookWyrm
bookwyrm_custom:3.11-slim from now on; a BookWyrm ref bump still
rebuilds through the build-args digest.
- test_backup_stop_decision_declared requires the boolean on every
service with its own image that mounts a backed-up named volume;
test_stop_required replaces the old integrity test.
Measured against HEAD: 43 -> 78 services copied live, none moved the
other way; 14 -> 30 volumes skipped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every role's base services carry `docs.enabled: "{{ 'web-app-docs' in
group_names }}"`, and the static closure counts any "in group_names" flag
as possibly true (is_explicit_truth). A round therefore pulled web-app-docs
in through any member whose active variant did not pin docs off, and a
round hands every role its index or clamps it to variant 0, where 84 roles
declare docs on. The README replay and everything it drags in (dashboard,
matomo, keycloak, prometheus with its MCP partners) landed in 365 of 469
rounds instead of one per role, which ran CI run 40625679270 out of
resources: Listmonk#0 memory, OpenProject#0 I/O in the DR drill,
Nextcloud#0/#2 budget (4310-4539 s in the replay each, 60 s over),
BookWyrm#0 drill disk, Matrix#1/#3 pids in the replay; none of those
primaries declares docs in the failing variant.
The guide host now stays on only where the primary app's own variant
declares it (declares_guide, the predicate instructions.py already used);
every other role has it switched off (guide_off), in the planner's round
overrides and in the swarm include set. The host_vars bake drops a docs
edge whenever the round does not deploy web-app-docs. Measured for
listmonk, openproject, translate and mariadb: web-app-docs is deployed in
exactly each primary's declared guide round, swarm and planner agree in
all 12 rounds, and rounds shed up to ten apps (mariadb#1 19 -> 9).
test_guide_rides_the_primary_only pins that on real role data (it fails
with the rule disabled); test_variant_pin_required leaves the guide host
out of its clone comparison and no longer counts the docs key as a
pairing edge, because the replay no longer follows a partner's index.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
translate#1 (CI run 40625679270) hit the disk floor and the watchdog
returned rc 75, but its proc.terminate() reached only the step's python
wrapper, which dies on SIGTERM without passing it on. ansible-playbook was
re-parented to PID 1 (rescue processes.txt) and kept deploying and testing
for another eleven minutes, ate the rescue headroom, and was SIGKILLed by
the teardown, which surfaced as a misleading rc 137.
The step now runs in its own process group; an abort signals the group
(SIGTERM, 30 s, SIGKILL). SIGINT and SIGTERM sent to the watchdog itself
are forwarded to that group while the step runs, so the outer `timeout`
in 00_one.sh (no --foreground) still stops the step.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
roles/web-app-nextcloud/tasks/01_fullstack.yml: both sides added a
replicated-template task at the same spot; store-i18n's Talk signaling
backend list and the updater's Janus config now stay as two tasks with
their own conditions.
roles/web-app-nextcloud/templates/compose.yml.j2: the updater rendered
the talk service's container_volumes a second time inside the direct-media
branch, next to store-i18n's unconditional call, which doubles the
volumes/configs keys once direct media is on. The janus_conf mount carries
its own direct_media `when` in meta/volumes.yml, so the single call
renders it; the duplicate is gone.
DEPLOYMENT_MODE == 'compose' became IS_COMPOSE_MODE | bool in
web-app-nextcloud/meta/services.yml, web-svc-coturn/vars/main.yml and
tasks/stages/03_destructor.yml, as store-i18n's deployment-mode lint
requires; the updater's coturn, Talk and decommission tests now pass
IS_COMPOSE_MODE alongside DEPLOYMENT_MODE. test_talk_direct_media.py
expects store-i18n's talk command line and mount position, and the
"host networking ignores direct media" check moved to the config level,
where direct_media already requires network_mode bridge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`infinito meta ci priority [run URL|ID]` prints what
`infinito administration deploy ci trigger --failed` would dispatch for a
run: every selection that was not green plus the priority entries the
run never deployed, space-separated on a single line so it can be pasted
into the manual CI trigger's priority field. Without an argument it reads
the latest deploy run of the current branch; --strict and --roles-only
narrow it like the trigger does.
Both commands now share gh.fetch_run_ref() and
selections.retrigger_entries(), so the trigger drops its own _fetch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Values an app renders once (env files, branding config, web manifests)
cannot follow the network a page is served on, so onion pages loaded
logos and icons from the clearnet CDN host (Mobilizon, Wazuh, Mini-QR in
run 40243434327). Like the object-store route from 40b178d9a4, every
front-proxy vhost now aliases the CDN directory under /_infinito/cdn/,
with nosniff and a sandboxing CSP so a mirrored HTML or SVG file cannot
script the app's origin. lookup('cdn') and lookup('design') add
relative_urls for consumers that want host-relative URLs; the absolute
urls stay for server-side users. The include lives outside the location
snippets, so test_vhost_cross_cutting_includes requires it of every
override, the compose synapse vhost included.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeQL flagged five notes on PR #1215: four test modules imported unittest both as a module and through from-import, and sys-front-inj-matomo's __init__ exported PROJECT_ROOT without declaring it, so the import from test_site_cache.py read as unused. The modules now bind mock through import unittest.mock as mock, and the package declares __all__, the same fixes main carries from 3610e7b9ef and 6e776a5780. The same pattern is fixed in the new Talk direct-media test and the decommission integration package, and the coturn test package drops an __init__ export nothing imports.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
janus_conf is mounted only when direct media is on, which requires compose mode, and compose ignores a config mode outside swarm and warns about it on every deploy. The file keeps the 0644 it is rendered with on the host.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
web-opt-rdr-domains renders a `return 301` vhost for each network sibling
of a redirect source (network_redirects), but the health expectations only
forced the clearnet source to [301]. A handover app's onion sibling kept
its [302] handover expectation, so Bluesky's disabled view domain failed
"view.bluesky.<onion>: ERROR: Expected [302]. Got 301." and
hlth-webserver aborted the deploy. Every expected domain whose clearnet
sibling is a redirect source now expects [301].
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A variant that pairs a clearnet-only partner with the onion (Matrix
variant 2 with web-svc-coturn) could draw the tor-only network mode, and
the deploy's domain map then aborted with "web-svc-coturn: allows
['clearnet'] but the node runs in 'tor' mode". disabled_services() now
also emits every deployed partner with domains and a Tor bond whose
networks the node mode does not serve, mirroring utils/cache/domains.py,
so the inventory provision drops it the same way it drops Tor on
clearnet rows. Measured across every role: 8 tor-only rows change, all of
which failed at that point before; clearnet and multi rows are untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removing a role from the inventory left its compose stack running under /opt/compose, holding ports, memory and a stale instance directory, until someone stopped it by hand.
On a full compose deploy with MODE_CLEANUP (no --id), the destructor now takes every directory below DIR_COMPOSITIONS that is named after a known role, carries the full instance layout and is owned by no active role (inventory-wide groups, group_names, the dependency closure and the instances this play prepared), stops it with compose down --remove-orphans without -v, verifies through the project and working_dir labels that no container is left, and only then deletes the directory; named volumes stay. Hand-made, partial or backup directories and instances a non-compose container is named after are left in place and listed. The step is the last block of the destructor, with the timer unlock in its always section, so a failure cannot leave timers locked.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The aio-talk image runs Janus with its stock janus.jcfg: no rtp_port_range and no nat_1_1_mapping, so Janus offered only its bridge address and every stream of a call went through coturn as relay. Calls above about six participants stuttered and failed.
When talk runs bridged next to standalone coturn in compose mode on a host that owns networks.internet.ip4, a rendered janus.jcfg is mounted over the stock one with rtp_port_range 61000-61999, nat_1_1_mapping = ip4 and keep_private_host, and that UDP range is published on ip4. TURN stays the fallback. Before rendering, the deploy reads the image's launcher and stock janus.jcfg and stops when Janus no longer reads the mounted path or the stock settings drifted from the template. Variant 0 publishes 64 ports so dev and CI exercise the path; the call capacity spec asserts a non-relay selected pair on ip4 in that range. With the flag off every rendered file is byte-identical to before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With host networking and no --relay-ip, coturn put every non-loopback IPv4 of the host into its relay list, Docker bridge gateways included, and handed IPv6 and bridge clients relays on those addresses while advertising the public ip4, so their media went nowhere. It also listened on every bridge address.
In compose mode with host networking, coturn now listens on networks.internet.ip4, 127.0.0.1 for the healthcheck and networks.internet.ip6 when set, and relays on ip4. It pins only when the host owns ip4, and owns ip6 too or leaves it unset; behind 1:1 NAT, without the ip binary or with a foreign ip6 it renders the previous command unchanged, because coturn exits when it cannot bind a listening address. A unit test renders the real template and vars for every cell.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The WriteAcp grant lands in roles/web-svc-seaweedfs/templates/s3.json.j2,
where store-i18n keeps the server templates; git's rename detection moved
the hunk there and roles/web-app-seaweedfs/templates/s3.json.j2 stays
absent. roles/web-svc-seaweedfs/README.md lists WriteAcp like the
web-app-seaweedfs README the hotfix updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SeaweedFS 4.49 (seaweedfs/seaweedfs#11592) authorizes a PutObject that
carries x-amz-acl as s3:PutObjectAcl as well. The consumer identities
only held Read/Write/List/Tagging, so every app that sends an ACL on
upload (Nextcloud, Fider and Pixelfed with private, Mastodon, PeerTube,
BookWyrm and Baserow with public-read) got AccessDenied after the 4.48 -> 4.51
bump in #1226. Main run 40401200083 failed Fider#0, BookWyrm#0,
Baserow#0, Pixelfed#0 and Nextcloud#0/#4/#8 on it.
Add WriteAcp to the s3.json identities and to the sidecar s3.configure
grant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
16 conflicts, resolved per hunk:
- sys-stk-front-proxy: tasks/main.yml keeps the network-sibling loop;
the single stack_host_contents write of http.conf plus vhost and the
front_proxy_vhost_changed gate move into tasks/vhost.yml, with the
http.conf decided per sibling domain so an onion vhost gets no HTTPS
redirect.
- sys-front-tls: 01_http.yml is gone as the updater intends; the
cert-renew include in main.yml gates on the vhost domain's TLS.
- sys-svc-compose: the updater's resolve_host_cid without the remote
echo and the prepared pending flag in up.yml, written with
IS_SWARM_MODE and the repeated-when block grouping.
- web-app-nextcloud: the updater's post-upgrade gate and pending-only
plugin configuration, which already carry the pinned-plugin handling.
- web-app-xwiki: one XWIKI_CORE_VERSION for the readiness probe and
style.css.j2; OIDC/LDAP follow xwiki_extensions_ready at bootstrap,
the S3 blob store stays off there as TestBlobstoreRenderOrder demands.
- roles_with_service: onion swap and dashboard card key combined, the
canonical domain is the canonical URL's host in both branches.
- sys-ctl-hlth-csp: --clearnet-timeout and --concurrency together;
run_checker forwards its options to build_docker_cmd, keeping the
script under the 500-line limit.
- web-app-matrix: Element branding and web_client_location side by side.
- web-app-stalwart/README.md: the generated Cosmos section is dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
csp-checker 3.2.0 checks every URL in its own browser context and takes --concurrency for a worker pool. The role pins 3.2.0, sets HEALTH_CSP_CONCURRENCY to 4, and the systemd unit hands it through script.py to the container command ahead of the -- URL separator, for the clearnet and the onion batch alike. TimeoutStartSec keeps its sequential sizing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_include_uniform_apply_keyword flagged legacy_checkout.yml after 11c2d8dcd3 moved its guard onto the include: both tasks repeated become: true. The include in 03_launcher.yml now applies become to the file and keeps the guard; the order test reads include_tasks in its mapping form too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every database consumer called baudolo-seed through the seed task, 37 times and always 'changed' in a full production run (54.4 s), although databases.csv rarely changes. The first seed of a play now reads the file once (05_read-seeded-databases.yml) into svc_bkp_volume_2_local_seeded; baudolo_seed_current compares the consumer's row with it and baudolo_seed_apply advances the cached state the way baudolo-seed writes the file, so the command runs only for a missing or differing row. A file baudolo did not write byte for byte (hand-edited, CRLF, short rows) falls back to one call per consumer; 02_reset forgets the cache with the file.
Replay harness old vs new: databases.csv byte-identical in fresh, empty, converged, drift and reset-mid-play scenarios; converged seeds 37 -> 0, drift 38 -> 2; the negative control without reset invalidation diverges. About 44 s per unchanged redeploy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each plugin ran its own occ config:app:set per configuration value right after its install or enable, and a redeploy rewrote every value. Install and enable now queue the plugin; the configure stage runs once for the queue: it loads every queued addon spec, reads occ config:list once, keeps only the values Nextcloud does not hold exactly (nextcloud_plugin_config_pending; typed, masked and structured values are always kept), and runs the hooks in queue order afterwards.
Combined with store-i18n's 7f72e53a58: the pending occ config:app:set calls go on the stdin of one container exec -i ... sh -s, so no value (Talk, OnlyOffice, whiteboard, BBB secrets) reaches a host argv; a failing call names its appid/configkey.
Replay harness old vs new: identical final state, updated keys and hook order in first-deploy, converged, drift and failure scenarios (failure stops at the same key); converged config:app:set calls 19 -> 3. Merge note: resolve 03_configure.yml with this side against 7f72e53a58.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every app's compose pass ran three remote tasks before compose up: create the directories, compose ps for running containers, and stat the pending marker (53 apps x ~2.7 s in a production run). compose_instance_prepare does all three in one call from 02_stack_host_prep.yml, sharing the directory creation with directory_batch through utils/directory_batch.py; compose ps runs only in compose mode. up.yml evaluates the registered result, named compose_instance_before_files because it predates 04_files: the running check is skipped whenever the compose, env or override file changed in this run (those changes notify compose-up themselves).
Replay harness old vs new: rendered files and the compose-up, marker and swarm handlers identical in every scenario; the only difference is the skipped running check when only the override changed, where compose-up still runs. About 85 s per full production run.
Merge note: store-i18n's a4c7fda974 hoists the removed marker stat into a block; resolve up.yml with this side.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DISCOURSE_REPOSITORY_DIR appended the repository URL to the services directory, so the launcher lived under services/https:/github.com/discourse/discourse_docker.git/. It now lives under services/discourse_docker. Before the clone, launcher/legacy_checkout.yml removes the URL-named tree (computed with relpath against the services directory) and prunes its emptied parents; a guard skips the cleanup when that path would escape the services directory or equal the new checkout. The cleanup must precede the clone: git otherwise resolves the old relative path as a URL and aborts with 'protocol ... is not supported'. Discourse data stays in the launcher's named volume and /var/discourse/shared; the first deploy rebuilds once because the bootstrap digest lived in the old containers/ directory.
relpath joins the builtin filters test_defined knows.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dashboard card of every app opened its primary domain, so Matrix led to the Synapse status page and Stalwart to its admin UI. A role can now declare services.dashboard.card: <key of meta/domains.yml canonical>; docker_cards builds the card URL with lookup('canonical_url', app, key), the Apps menu reuses it, and roles_with_service('dashboard') hands the same domain to the tile e2e spec. Matrix sets card: element, Stalwart card: webmail. Without card nothing changes, and the prometheus/matomo target lists are untouched. A lint requires the key to resolve in the role's own domains.canonical.
The Matrix ansible flavor also sets Synapse's web_client_location to Element through matrix_synapse_configuration_extension_yaml, as the compose flavor already does, so matrix.<domain>/ redirects there. Merge note: feature/iam-profile-rbac defines the same extension variable inside the OIDC block; the next merge must combine both into one block (agreed with that session).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A host that defines CAPTCHA with one provider replaces the group_vars mapping (no hash_behaviour merge), so galaxyserver, which configures only hCaptcha, left CAPTCHA.RECAPTCHA undefined. RECAPTCHA_ENABLED then failed to render, the applications render fallback returned the expression as text, and Keycloak's '| bool' coerced it to False with a deprecation warning that ansible-core 2.23 turns into an error. Both flags now check that their provider mapping is defined first; the effective values stay the same for every existing inventory.
test_captcha_enabled renders every role's services.recaptcha/hcaptcha.enabled through the real templar for defaults, both providers, only the other provider and only this provider, and fails on the raw text (negative control with the old expression: 4 subtests).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every redeploy ran occ maintenance:repair --include-expensive (43.5 s, maintenance mode on) and listed the appstore updates through Tor (49.7 s) although the version had not changed. 03_upgrade.yml now reads occ status and files/mimetype_version after occ upgrade; nextcloud_needs_post_upgrade decides whether the pass is due (version changed in this deploy, or the expensive repair has not run for the running version). A due deploy includes 03_upgrade/01_post_upgrade.yml with today's app update and repair; an unchanged one includes 03_upgrade/02_cached_updates.yml, which updates only the unpinned enabled apps whose release stored by the updatenotification background job is newer (nextcloud_cached_app_updates, PHP version_compare semantics). NEXTCLOUD_PLUGINS_PINNED is the single source of the pinned check.
Verified against echoserver's live occ output (status, marker, updatenotification cache, app:list, with debug log lines): unchanged redeploy is not due and finds no update. Scenario harness 24/24 incl. negative controls on the old file. About 86 s per unchanged redeploy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Conflict in roles/sys-stk-front-proxy/tasks/main.yml: store-i18n moved
the per-vhost tasks into the network-sibling loop of tasks/vhost.yml, so
main.yml keeps HEAD and the deferred probe lands in vhost.yml instead:
every sibling domain is queued in front_proxy_deferred_probes, and the
inline probe of an unchanged vhost is gone in favour of the batched one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each vhost used to run its own sequential reachability probe. The role now queues the vhost and deferred_probe.yml checks the whole queue with one uri_probe_batch call after the shared-service preload and after the server roles. A vhost answering like a stale config restarts OpenResty once; one still unserved afterwards fails the deploy by name. uri_probe_batch takes follow_redirects explicitly so the probe sees the redirect status itself.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Conflicts resolved:
- sys-front-inj-matomo: inject.yml keeps the store-i18n split; the cached
failed site list load (5b7284c4a4) is ported into resolve_site.yml next
to its token guard, and test_site_cache.py reads that file.
- web-app-bluesky: platform_administrator from store-i18n with the
prefix-free handle and its i8- reserved variant from feature/updater;
the broker CONFIG keeps siteTitle, storePath and chooseHandlePath.
- web-app-stalwart README: the managed Quick Setup section is dropped,
the docs role renders it.
- README.md: the role overview is regenerated from the merged tree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
5b7284c4a4 named its reset task with 81 characters, one over the limit of
test_name_emoji_and_length, and 754d35c4de built the path to
roles/web-app-keycloak/vars/main.yml with a pathlib chain that test_file_spot
requires to come from ROLE_FILE_VARS_MAIN. Both commits went in without the
lint suite. The task keeps its behaviour under a 70-character name, the test
reads the same file through the constant. make test-lint: 577 passed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
keycloak_kcadm_update built `cat <<'JSON' | kcadm update|create ... -f -`
with the JSON payload inside the command it hands to sh -c, and the
reCAPTCHA task did the same in its shell script. Client secrets, the SMTP
password, the LDAP bindCredential and the reCAPTCHA secret therefore sat
in a host process argument (#1213).
The library now sends the payload as subprocess input, also on its lock
and dead-container retries, and the reCAPTCHA task uses the shell
module's stdin. Every caller's kcadm_exec already runs `container exec -i`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While the private CA is missing from the prometheus stack, a self-signed
target has to use http_ready_private_ca. The blackbox job chose that
module once per application from lookup('tls', app, 'mode'), which aligns
to one sibling domain. For web-app-docs that sibling was the onion domain
(TLS off), so its clearnet https target kept the verifying http_ready and
failed with "x509: certificate signed by unknown authority" (CI run
40131272595: n8n#1, and hidden behind the first failure in API#1 and
Mattermost#1).
Every served domain now gets its own target group whose module follows
lookup('tls', probe_domain, 'mode'). Series labels are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SAML clients were created only by the realm import, and the import runs only
for a missing realm. echoserver keeps its legacy realm cymais.cloud and ran
SuiteCRM behind oauth2-proxy before v16, so the SuiteCRM SAML client
https://<suitecrm>/saml/metadata never existed there: the SAML hop showed
Keycloak's Invalid Request page and Keycloak logged LOGIN_ERROR
error=client_not_found reason=Cannot_match_source_hash.
The existing-realm reconcile now also loops over the clientId of every SAML
client in the rendered realm dictionary, the same source the import uses,
and creates or updates it through kcadm like the application-declared OIDC
clients. The same path already converged the stalwart-webmail and
stalwart-webui clients in echoserver full-10.
Same change as 2b4db90576 on feature/store-i18n, so a later merge of the two
lines resolves without a conflict, plus a unit test that pins the SAML client
selection and its place in the reconcile loop.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On a host whose Matomo was not deployed yet, every vhost injected before
web-app-matomo asked SitesManager.getSitesWithAtLeastViewAccess again and
waited out uri_retry (30 retries x 2 s): 6 vhosts, 84 to 98 s each, 8.7
minutes in galaxyserver full-2. matomo_site_cache only kept a loaded list.
inject.yml now caches every executed load: the list on success, none on a
failed request or an error mapping. A cache of none skips the cached-id step
and getSitesIdFromSiteUrl, so the existing skip task turns the injection off
for the vhost as before, without the retry wait. web-app-matomo resets the
cache of every play host (delegate_facts, swarm workers included) after
07_config and before 08_reinject_consumers, so the re-injection and every
later vhost load the list from the now running Matomo.
Evidence: the real inject.yml with the real uri_retry against a closed port
and a stub Matomo, 11 vhosts in 4 scenarios: the first probe took 68.6 s, the
next two 0.4 and 0.3 s with matomo off; after the reset a known site came
from the cache, an unknown one went getSitesIdFromSiteUrl -> addSite and was
appended, a later vhost of that domain hit the cache. HEAD inject.yml took
68 s per vhost with the same outcome. The swarm reset reached a second host.
ansible-lint passes; tests/unit/python/roles/sys-front-inj-matomo pins the
gates and the reset order.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
write_catalog writes the rendered catalogue to .<name>.po.tmp and moves it
over the target. When the write fails, the staging file stayed behind: the
translation run that hit a full disk left eleven empty .docs.po.tmp files
untracked in locale/. Any failure between the write and the move now
deletes the staging file and re-raises, so the target keeps its old bytes
and nothing is left in the tree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
resolve_term follows a redirect source to its target so an alias inherits the TLS settings of the application it points at. A redirect to a domain another host serves ends that chain on a domain this host does not know, and the tls and cert lookups failed: galaxyserver full-1 stopped in web-opt-rdr-domains on 'tls: domain project.infinito.nexus not found' for its project.veen.world -> project.infinito.nexus redirect.
When the chain ends outside the host's domains and web-opt-rdr-domains is deployed, the source now resolves to web-opt-rdr-domains with the source itself as primary domain, which is the role that serves its vhost and whose certificate bundle names it. Only the outermost call falls back, so the source asked for stays the primary domain across a longer chain; without the redirect role, or for a target this host serves, nothing changes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
