make i18n-translate addressed the engine directly, so a run could never reach more than the 47 languages Argos ships models for, while meta/languages.yml names 183. service_url now resolves web-svc-translate, which routes a pair the engine does not serve to a chat model, and the deploy brings both roles up.
The gateway advertised only what its backends declared, and the chat backend declared nothing, so /languages reported 51 targets of 183 and the readiness wait spun to its hour-long timeout without translating a line. The chat backend now declares every language the table names.
Routing offered libretranslate for every pair, because an undeclared catalogue read as any pair, so a language it holds no model for was answered with HTTP 400 instead of falling through to the next backend. serves() now reads the backend's own /languages, which catalogue() already exposed for the advertisement. Declaring the pairs in the role was the first attempt and carried a second copy that was already stale: the engine reports zh-Hans where the role's list says zh.
A small model copies the prompt markers back, echoes the source line and labels its answer with the target language. All three reached the catalog, and an entry carrying its own source scores below the oracle's middle level. clean() strips them, and only from an answer that carried scaffolding, because on its own a short word and a colon is a translation, not a label.
server() had lost its redeploy parameter while cli/build/i18n kept passing it, so every run raised TypeError before translating a line and the only symptom was an empty log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4aba897ca2 gave the pull and build subprocess a 3600s ceiling so a hang
could not stall a deploy. A pull of a large image set legitimately outlives
any fixed ceiling while it is still making progress, and the ceiling then
kills a healthy run.
run_streamed measures the gap between two output lines instead. Two reader
threads keep stdout and stderr separated the way run_cmd does, because
compose writes its warnings to stderr and merging them poisons every parser
downstream. Only the two calls that legitimately run long use it; the short
calls whose output is parsed keep run_cmd.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merge left the role with two ways to the same end. The wired shell path called setup.py update-user, so it could neither create a missing account nor clear a lockout, and it decided its own idempotency from a sha256 marker in the persistent volume, which lies after a restore: marker matching, database not, reported unchanged and did nothing. The python converger from 0709de6fa1 queries the application instead, creates what is missing, verifies the password, unlocks, resets the failed-login counter and ensures the Administrator role, and it carried the only tests either path has; it was wired to nothing after the merge resolved towards manager_ops.yml. That engine now serves the surviving wiring: renamed into the family spelling the sibling roles use, reading the env names the task already passes and emitting the markers the task already parses. tasks/administrator.yml had no caller anywhere in the tree and goes with it.
Trade-off worth knowing: the deleted path used the supported pgAdmin CLI while this one uses internal APIs, so a pgAdmin upgrade can move it. Neither path has ever executed in CI, the pgadmin job died at the health wait before reaching the realign, so the choice rests on capability and tests rather than on field evidence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard patched the silence budget down to 0.3 s, but the watchdog starts its clock at Popen while python -c still has to boot before it can print, so under the parallel load of a full gate run the startup alone exhausted the window and the process was killed before its first line. The same file passed in four seconds on its own. The budget is now 1.0 s against a process that prints every 0.2 s for three seconds, which keeps all three conditions true at once: the total run outlives the budget so the old total-runtime timer still kills it, the gap between lines stays under it, and so does interpreter startup. Verified against the old shape in isolation: it dies with signal 9 after five of fifteen lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The watchdog armed a single timer at process start, so it killed the German build with SIGKILL while that build was still emitting progress, and Library.failed() then answered 500 for the rest of the run. The container was not out of memory: OOMKilled is false, the restart count is zero and neither the kernel ring buffer nor the journal records a kill, while the sphinx worker BrokenPipeError that follows is the signature of killing the parent alone. The watchdog now stamps the time of every line read and kills only after a full silence window, which the existing test name already described. A regression guard runs a process that keeps printing past the deadline and requires its last line in the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`deadline` is whole seconds from `date +%s`. A run that starts just
before a second boundary reaches the deadline after the first probe and
one `sleep 1`, so the gate correctly gives up after one probe while the
test demanded two. With a stubbed clock (1000, 1000, 1002) the gate
probes once and prints `deadline of 2s reached`. The deadline message
already proves the pending migration held the entrypoint back; require
at least one probe.
Reported by CodeRabbit on #979.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Swarm Nextcloud#3 on main 748189e890 (job 117182318941) wedged the
OpenProject schema: the convergence check gave up after 74 attempts
while the seeder kept exiting 1, first with
`relation "ar_internal_metadata" already exists` from its structure
load, then with `relation "announcements" already exists` on every
`db:migrate`.
The seeder (`docker/prod/seeder`) loads `db/structure.sql` through
psql, statement by statement, when `\dt` finds no relation. In swarm,
`depends_on` is ignored, so web and worker start alongside it and poll
`await_migrations.sh`. That gate called
`rails db:abort_if_pending_migrations`, which runs
`migration_context.open`, i.e. `Migrator.new`, whose constructor
executes `schema_migration.create_table` and
`internal_metadata.create_table` (Rails 8.0.4,
activerecord/lib/active_record/migration.rb:1430-1431). The seeder
saw an empty database at 05:33:20; by 05:33:38 the gate had created
`ar_internal_metadata`, the structure load aborted at line 255 with
everything alphabetically before it already committed, and every
later attempt took the migrate path into tables that already existed.
Probe with `rails db:migrate:status` instead. It aborts while
`schema_migrations` is missing and otherwise only reads the recorded
versions (database_tasks.rb:303, migration.rb:1318); the gate treats
any `down` row as pending.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two conflicts, both resolved by combining rather than by picking a side.
`roles/web-app-socialhome/tasks/02_design.yml`: designer restructured the pass,
moving the token request and the theme write into the new
`02_design/converge.yml` and adding a stat of the name-state file. HEAD still
carried the converge block inline. Kept designer's stat, dropped HEAD's inline
block: the merged body below the conflict already includes `converge.yml` and
reads `socialhome_name_marker`, which only designer defines, so keeping HEAD's
side would have duplicated the token request and left that variable undefined.
HEAD's `retries: 0` on the token request is deliberately not carried over; the
default of 30 retries covers a service that is not yet accepting connections,
which matters more here than failing fast on bad credentials.
`roles/web-app-fider/files/playwright/test-design.js`: the import line takes
both sides, since the merged body uses designer's `apiGetOnion` and HEAD's
`requireDotenvValue`. The env block keeps HEAD's `requireDotenvValue` form,
because designer's `|| ""` defaults are what
`tests/lint/ansible/.../test_unguarded_env_default.py` exists to reject, and
drops HEAD's module-scope `title` and `logoEnabled`: the merged body defines
`title` locally and reads `DESIGN_LOGO_ENABLED` directly, so both were dead and
would fail `no-unused-vars`.
Checked after resolution: no conflict markers (`git diff --check`), no `nocheck`
line dropped by the merge, and no duplicated YAML key in the seven merged files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Without `--all`, `journalctl --output=json` encodes a field over 4096
bytes as `null`, so a long host error matched nothing and the check
passed. A repeated `MESSAGE` field arrives as a JSON array of its
values, and `bytes()` on that array raised `TypeError`, which aborted
the whole check.
Pass `--all`, and tell a byte array (all integers) from an array of
field values, decoding each value on its own.
Reproduced against the host journal: a 5000-byte error line read back
as `"MESSAGE":null` and produced no match; an entry with two `MESSAGE`
fields, sent over the native journal socket, read back as
`["error first","error second"]` and crashed `host_errors` with
`TypeError`. With the fix both are reported.
Also clear the two test-lint findings of this PR:
`test_comments_valid` rejects the narrative comment above the
`journalctl` Popen, so it goes; the stderr-to-file choice stays, because
an unread stderr pipe that fills up would block `journalctl` while
stdout is still being read. `test_no_uncached_file_reads` flags the
read of the stub's argument file, which lives in the test's tempdir and
is written per run, so the path-keyed `utils.cache.files.read_text`
could serve a stale body; the read opts out with
`# nocheck: cache-read tempdir fixture`, as the other stub-driven
script tests do.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
SH_ADMIN_PASSWORD seeds the administrator on first boot only: upstream provision_admin early-returns once platform_users holds a row. The rotation gate regenerates users.platform_administrator.password, the container keeps the old hash, and the household theme token request fails. The 31 retries then drained the 5-per-15-min budget upstream enforces on /api/auth/token, so every attempt after the fifth answered 429 and hid the underlying 401.
realign_admin_password.py runs inside the container, resolves the database through the config helper the application itself exposes, and rewrites platform_users.password_hash with the scrypt helper the application itself exposes when the declared password no longer verifies. It writes that one column, because set_password would also write is_admin and demote the administrator.
The token request drops its retries. 31 attempts into a five-per-window budget also poisons the bucket the Playwright design spec shares, and 0 is the only value multiply-timeouts.sh cannot inflate.
Proven by a two-pass compose deploy of variant 0, the row that failed in CI and the only one with design enabled: pass 1 [UNCHANGED], pass 2 [CHANGED] after the rotation, and the token task that failed in CI with 31 attempts and status 429 answering on attempt 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Problem:
Every compose service logs to journald through
sys-svc-container/templates/base.yml.j2. The daily journal health
check grepped the whole journal for "error", so routine application
errors from containers triggered the host alert (email/Telegram) and
buried real host problems.
Change:
- Read the last day of the journal as JSON (MESSAGE, CONTAINER_ID,
SYSLOG_IDENTIFIER, _SYSTEMD_UNIT) and skip every entry that carries
a CONTAINER_ID field. Docker daemon and system errors are still
reported.
- Report each match as one readable "source: message" line instead
of raw journal output.
- Stream the journalctl output line by line instead of buffering a
whole day of logs in memory.
- Exit 2 with journalctl's stderr if journalctl itself fails, instead
of silently reporting "no errors".
- Rewrite the check in Python (files/python/script.py, deployed by
sys-service like the other sys-ctl-hlth roles) and decode non-UTF-8
messages.
- Fix the success message, which claimed "All docker containers are
healthy".
Impact:
Container log errors no longer raise this host alert on any
installation. Container health stays covered by sys-ctl-hlth-container;
application log analysis belongs in a log platform such as Wazuh.
system_service_suppress_flush is unchanged until a deploy confirms
that no host errors remain.
Tests:
Unit tests run the script against a fake journalctl (8 cases):
container error ignored, daemon error reported, identifier fallback,
non-UTF-8 message, journalctl failure, many container lines streamed
past, clean journal passes. The core cases fail against the previous
script.
Under SSO the platform administrator signed in and landed on #profile as a
Customer, so the design spec never reached #manage/users. Three defects
stacked.
The group-to-role map was written with Jinja in its mapping keys. Ansible
renders a mapping's values and leaves its keys alone, so the rendered Ruby
keys were literal {{ ... }} and could never match a group path. The map is now
one expression that computes its keys through rbac_group_path, the shape
meta/addons already uses elsewhere.
The initializer patched User.create_from_hash and User.update_from_hash behind
`next unless User.respond_to?`. Zammad 7.1.2 defines only create_from_hash!,
so both guards fell through in silence and nothing was installed: the
initializer's name does not appear once in the failing run's Rails log. Patch
Authorization.find_from_hash and Authorization.create_from_hash instead, which
sessions_controller calls before current_user_set, so first and subsequent
logins are both mirrored.
administrator mapped to Admin alone. Zammad's Admin carries no ticket
permission, and the Dashboard navbar entry is gated on ticket.agent, so an
Admin-only account has neither Dashboard nor Overviews. Grant Admin and Agent,
matching what the password path converges.
The write is defended rather than assumed. Zammad's has_roles concern runs
validate_roles, validate_agent_limit_by_role and last_admin_check_by_role on
collection assignment and they raise, which inside the omniauth callback would
answer the sign-in with 500 - a lockout, worse than the demotion being fixed.
The assignment is wrapped, resets the association and logs on refusal, and the
sign-in completes on the roles the account already has. Role lookup filters on
active and reports a missing role instead of silently granting the subset. A
map that rendered as a String is refused at boot, because map[path] would
otherwise be String#[] returning a substring and the feature would be a silent
no-op. The patch targets are checked by signature, not by respond_to?, since
respond_to? was exactly what hid defect two.
An absent groups claim leaves the roles untouched and logs, rather than
demoting every account to Customer on a misconfiguration. A present but empty
claim still means "in no group" and maps to Customer.
biber is unaffected: inventories/development/default.yml declares roles: [] for
it, so it is in no RBAC group, maps to Customer as before, and keeps
#customer_ticket_new.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
assertReadable takes a selector list and measures contrast on whatever it
finds. An entry marked optional is skipped when it matches nothing, so a list
whose entries are all optional can measure nothing at all - and then the call
asserts nothing while reporting green. Reject such a list before the browser
is touched. A plain string normalises to a required entry, so only a
deliberately all-optional list is refused, and an empty list with it.
web-app-matomo found the defect the hard way in CI run 38818071194. Its KPI
card check listed four optional selectors, all of them trend modifiers that
the dashboard only renders at a non-zero change; every card read 0%, so
KPICard emitted kpiTrendNeutral and SitesTableSite left its evolution cell
unclassed, nothing matched, and the test failed on the minimum-measured guard
rather than on contrast. The sibling evolution-badge check has the same
all-optional shape and was green only by luck.
Give every all-optional call site an anchor that is already proven visible in
its own spec: .kpiCardEvolutionTrend and .evolutionBadge for matomo,
header.header for web-app-dashboard (waitForDashboardReady asserts it
unconditionally) and #login .panel-body for web-app-espocrm. The espocrm case
needs care: #btn-login and #sign-in each exist in only one branch of the sso
conditional, so neither is variant-safe, while assertLightAndDark measures
.panel-body on the line above.
The matomo theme rule is part of the fix, not a cosmetic extra. Without it the
new anchor keeps Matomo's hardcoded #76838f, which measures 3.60 against
--design-surface-1 and 3.82 against --design-surface-2 in light mode and would
fail the 4.5 threshold the anchor now enforces. The role already replaces
Matomo's hardcoded trend colours; this closes the same gap for the neutral
caption. Equal-specificity rules resolve by order and the role's block is
concatenated about 204 KB after Matomo's in the single served bundle, measured
by byte offset in the failing run's own trace.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
00_core.yml pins xwiki_blobstore_s3_enabled_switch false for the bootstrap
render on every deploy. On a wiki whose attachment content already lives in
the SeaweedFS bucket that strips the whole store.* block - the blob store type
and the object-store credentials with it - and restarts the container. In CI
run 38818071194 the credentials were rotated between the two passes at 04:49,
the SeaweedFS identities were re-rendered at 05:04, and the bootstrap render at
05:39 then removed the only configuration that would have carried the new keys
in. 06_design/logo.yml:7 is the first task that reads an attachment, so that is
where the play died, after 31 retries, with "could not be found in the
filesystem attachment store".
Move the bucket render behind 04_extensions in 02_manager_ops.yml, which
installs and asserts xwiki-commons-store-blob-s3. The precondition 26d588fa41
names - nothing answers to 's3' before the extension is installed - is then
established by execution instead of inferred, and 00_core.yml stays byte
identical to what it was.
An earlier attempt probed the deployed xwiki.properties for store.blobStoreType
and selected s3 when it was already there. That is not safe: the purge
primitives in scripts/container/purge/apps.sh:86-94 each run under `|| true`,
so a failed `rm -rf` leaves the config dir behind with the volume gone, and a
plain `compose down -v` in a debugging session does the same while the central
Postgres keeps the installed-extension rows. The probe would then have written
s3 into a wiki that cannot start. Removing that exposure beat pricing it.
Nothing before the new task can behave differently from today, so the only new
behaviour is one render into oidc off, ldap off, s3 on - two configurations
that each booted in this run, in a flat properties file where the blocks do not
interact. In the bootstrap window the new shape is strictly better: today's
window is a superset that extends through 06_design.
The test asserts both halves of the invariant, that no render in 00_core.yml
selects the bucket and that the bucket render precedes the 06_design include.
Against the previous tree it reports renders [8] design [7] ordered_ok False.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
weblate, beat and worker mount the same weblate_data and weblate_cache.
Docker fills a fresh named volume from the image once per
POST /containers/create, and that copy does an Lstat plus os.Mkdir per
directory without tolerating EEXIST. Compose created all three at once, two
populations of the image's /app/data raced, and the deploy died with
"failed to mkdir /var/lib/docker/volumes/weblate_data/_data/python: file
exists" in CI run 38818071194. The daemon log names the route as
containers/create, so this is the population path, not a mount of a
pre-existing file.
Make weblate and worker depend on beat. The lever is not startup order but
the Create phase, which compose runs in dependency order: beat populates both
volumes alone, and by the time the other two are created no copy runs. The
failing run proves that phase is ordered - COMPOSE_PARALLEL_LIMIT was 4, yet
only the two existing depends_on targets reached Created before the other
three began.
This is compose-only. In swarm, container_depends_on drops to a bare list
(plugins/lookup/container_depends_on.py:103) and docker stack deploy ignores
it, so swarm behaviour is unchanged and the same race stays reachable there. A
mode-independent fix needs nocopy on the non-owning mounts, which
plugins/lookup/container_volumes.py cannot express today.
The test reads meta/volumes.yml, collects every service sharing a volume and
requires each non-beat sharer to name beat. Against the previous template it
yields weblate False, beat True, worker False, so it fails without this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The web-app-litellm stylesheet swapped the navbar image with content: url(), so LiteLLM's own logo setting stayed empty and its UI theme page showed nothing. svc-ai-litellm now renders litellm_settings.ui_theme_config.logo_url from LITELLM_UI_LOGO_URL, the design lockup of web-app-litellm, only when that role is deployed with design and a logo; the stylesheet override is removed. The design spec asserts the served setting, the navbar src, the absence of a CSS swap, the loaded wide lockup and the value on the UI theme page; the config render test covers both the logo and the no-logo case.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`make quality-high` on main (fea6f31d9f) rewrote two files in its
autoformat step:
- roles/sys-service/handlers/refresh_systemctl_service.yml: the
handler's `when` was wrapped by hand; the formatter wraps it at line
width. Both forms parse to the same YAML.
- tests/unit/python/roles/web-app-nextcloud/test_onlyoffice_check_deadline.py:
two over-long assertIn calls that ruff format splits.
A second autoformat run leaves the tree unchanged, so main is stable
under the formatter again.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
identity.py built the services.yml path by hand, which test_file_spot.py names: utils/roles/mapping.py:175 holds ROLE_FILE_META_SERVICES, and bed4b3c84f already uses it in this role. The module meant to make services.yml the single source had duplicated a path of its own.
test_mirrored_unit_tests.py also required the matching test. It proves three things against a synthetic tree rather than the real values, so it cannot pass tautologically: the configured values arrive under the environment names conf.py reads, an undefined group variable is refused instead of rendering empty, and the repository's own configuration resolves with no {{ left in any value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
services.docs.sphinx is the single place the project, copyright and author are configured. conf.py takes them from the environment, which env.j2 fills inside the container, but two callers run outside it: the host side of make i18n-extract and the unit tests. Both had a copy of the values, which is the duplication this removes.
infinito_docs/identity.py renders services.docs.sphinx with the group variables it references, through the same Jinja environment utils/templating/jinja.py:62 uses: StrictUndefined, so an undefined variable raises instead of rendering empty, and select_autoescape for html and xml, which leaves a from_string template unescaped and satisfies ruff S701.
utils/i18n/extract.py loads that module from the tooling path it already computes for the subprocess PYTHONPATH, without touching sys.path. The two tests import it through the _TOOLING entry they already insert. conf.py is unchanged and still requires the environment: reading the configuration there would turn it back into a default.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The third post-merge quality-high pass wrapped the two context managers of 00393bb1b1 into the parenthesised form. Both revisions parse to an identical AST.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
conf.py reads DOCS_SPHINX_PROJECT, _COPYRIGHT and _AUTHOR with os.environ[...] since 9dc7f0cbc2, because a missing value must stop the build rather than render an empty project name into every title and the footer. The test loads the module at import time with only sys.argv patched, so it raised KeyError.
Supply the three through _REQUIRED_ENV in the same patch block that loads the module, and again in the reload test, whose own patch.dict opens after the import patch has closed. The test now states the requirement instead of a default hiding it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The second post-merge quality-high pass reported '1 file reformatted' and wrapped two long assertIn calls. Parsing both revisions yields an identical AST, so only the line breaks moved.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six conflicts, all from git pairing store-i18n's renamed roles with their designer originals.
web-app-meta/meta/services.yml and web-app-sphinx/vars/main.yml: store-i18n renamed web-app-sphinx to web-app-meta and dropped its custom build, so the role keeps the prebuilt ghcr.io/infinito-nexus/meta image and the old sphinx role stays deleted.
svc-ai-tika/templates/compose.yml.j2: a bogus rename pairing dropped sphinx build lines into the tika service. Rejected; the template is byte-identical to its pre-merge state again.
web-app-docs/templates/Dockerfile.j2: the designer file renders sphinx under the design title and reads SPHINX_IMAGE, SPHINX_OUTPUT_DIR and SPHINX_EXEC_DIR, none of which exist in web-app-docs, which builds from files/Dockerfile. Rejected.
web-app-docs/templates/playwright.env.j2 and files/playwright/test-design.js: both sides add independently, so both are kept. Of the designer's three new env reads only designTitle survives; iconUrl and faviconUrl are shadowed inside the test by stricter requireDotenvValue reads and were dead at module scope.
plugins/lookup/email.py merges clean toward designer, which is intended rather than a loss: 08ef66d8de (2026-10-08) replaces the submission_via_relay port-25 rule of 233f15f3e0 (2026-07-01) with per-bot app passwords under SSO, and carries the matching 04_manage_user/app_password tasks.
Checked after resolving: no duplicate YAML keys across the 17 merged YAML files, and no nocheck line dropped above a renamed key.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
allows() is called once per /v1/chat/completions and nowhere else, so the 60s cache saved two Keycloak admin GETs on a request that then runs an LLM inference. On an authorization decision that is not a trade worth making.
It produced the CI failure directly. The spec walks hermes then openclaw: it grants biber only hermes, the hermes completion re-reads and stores {hermes}, then it swaps the grant to openclaw. Open WebUI sees the new group on the fresh sign-in, which is why the models assertion passes, but about eight seconds later the openclaw completion hits a fresh cache entry, reads {hermes} and refuses. A hit returned the value without re-stamping the timestamp, so all three attempts - 13.9s, 15.2s, 15.2s - stayed inside one 60s window and failed identically.
The same cache was a security defect in the other direction: a revoked caller stayed entitled for up to a minute, contradicting requirement 037's own sentence that the broker refuses the request and stops the agent once a caller has lost the group. Step 3 of that document now says the groups are resolved on every completion, which is what the code does. The admin-token cache is untouched - that is a credential lifetime bounded by expires_in, not an entitlement answer.
Verified with a negative control: the CI scenario run against the real access.py through a transport fake, alongside a transcription of the pre-fix groups_of over the same transport. Pre-fix, the swapped-group check returns False, which is the CI failure; post-fix it returns True, and a revocation between two checks is seen by the second.
The repository had already worked around this twice: files/test/test.sh slept 65 seconds after granting a membership and membership.py constructed the client with the cache disabled. Both are removed.
The group-to-agent mapping was not at fault - broker and spec derive the path from the identical rbac_group_path expression, and the sibling spec that holds both groups passed.
Each completion now costs two Keycloak admin round trips, unmeasured on an onion deployment where KEYCLOAK_URL is a .onion against a 15s per-call timeout. The unit test that asserted the cached answer is replaced by two that encode the CI failure. Requirement 037 still has 47 translated catalogs carrying the removed access_cache_seconds row; the i18n tests pass, so it is obsolete rather than broken.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
01_manager_ops.yml waits for the moodle container to report healthy and then runs through to the ownership walks, but compose.yml.j2 gives a healthcheck to moodle alone. cron runs the same entrypoint against the same shared code volume and is not covered by that wait, so a bootstrap could still be filling and tearing down .bootstrap.stage while the walks traversed it. That is the 5089 'cannot access' lines and the closing 'Stale file handle' of job 116229890655.
5c3749c437 and 7bc5822421 serialized bootstraps against each other, append-only lock and a sentinel recheck when flock returns EIO across NFS clients. Neither serializes the deploy against a bootstrap, and no condition expressed 'the tree is final' for the deploy to wait on. The walks now wait for the state a finished bootstrap leaves behind: the stage gone and the sentinel holding the expected release. That observes the outcome rather than the lock, so it holds whether or not flock serializes on the mount.
The sentinel path moves to vars/main.yml and reaches the image as an ARG and ENV, like the stage, so the deploy and the entrypoint read one definition instead of two.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Job 116229890655 died in 03_ownership.yml after 69 seconds with 5089 chmod errors, every one of them under .bootstrap.stage, and the walk ended on 'find: failed to read file names from file system at or below /var/www/html: Stale file handle'. A second moodle-entrypoint.sh bootstrap completed inside that window and tore the staging tree away under the running find. The doubled phpspreadsheet path segment in the output is incidental.
0c748f74ed added the staging directory so a failed copy leaves the old tree in place, and guarded two instances on different releases through the version check. Two instances on the SAME release still race, because the lock lives on the code volume and does not serialize across NFS clients. moodle_bootstrap_code_dir now re-reads the sentinel after its copy and stands down when another container has finished the same bootstrap.
The three code-dir walks prune the staging directory, so a teardown mid-walk no longer fails them, and they batch through -exec + instead of one process per entry. MOODLE_BOOTSTRAP_STAGE moves into vars/main.yml and reaches the image as an ARG and ENV, so every service built from it, cron included, resolves the same path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`retries: 30` with `delay: 10` only bounded the wait while each
`occ onlyoffice:documentserver --check` failed fast. onlyoffice v10.2.1
gives the health check and the version command 60 s each and the
conversion 120 s, so a document server that accepts connections but
does not answer held the deploy for up to two hours before it failed.
The retries now also stop once a deadline of
`lookup('timeout', 300)` seconds, set right before the check, has
passed. A per-attempt `timeout:` would not bound the total: ansible-core
2.21 leaves the `until` loop on the first timed-out attempt, so only a
limit below onlyoffice's 60 s would end a hanging server's loop.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruff 0.17.0 adds `datetime` -> `dt` to the default aliases of
flake8-import-conventions, and the project selects ICN, so lint-python
failed with ICN001 in seven files.
The files now import `datetime as dt`. ruff's unsafe fix also renamed
two local names in utils/github/playwright_summary.py into the alias:
the `_to_iso_z` parameter and the parsed value in
`_parse_iso_timestamp`. Both are now called `moment`; with the bare fix,
nine tests of test_playwright_summary.py failed with UnboundLocalError
and AttributeError.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The role writes onlyoffice's settings with one occ call per key,
DocumentServerUrl first and jwt_secret fourth. Nextcloud loads app
config once per process, so a cron.php that starts inside that batch
runs onlyoffice's EditorsCheck job, registered when the app was
enabled, with the document server URL but without the secret. The
document server answers "Invalid token", EditorsCheck stores it in
settings_error, and onlyoffice stops loading its Files integration;
EditorsCheck itself skips every later run while that error is set.
Seen in run 38455782184 (Nextcloud#4, swarm): cron.php started at
06:50:00 inside the batch that ran from 06:49:59 to 06:50:04, logged
"Invalid token" at 06:50:22, and both Playwright passes found .docx
files offered for download only.
DocumentServerUrl is now written last, so a process that cached the
config before the batch ended sees no URL and EditorsCheck returns
early. After the batch, `occ onlyoffice:documentserver --check` sets
settings_error from a fresh check: a redeploy clears an error latched
earlier, and a document server that stays unreachable fails the
deploy. A unit test keeps the URL last.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The deploy decides what ADAPTER_PORT or OPENWEBUI_ADMIN_NAME is and
writes it into the environment. A default in the reader is a second place
that value lives, and it is the one nobody greps for. os.environ["X"]
fails loudly on a missing key, which is the behaviour a misconfiguration
deserves.
Empty values stay free, because they add no second source and only say
absent: "", None, False, 0, the empty containers, and the "[]"/"{}" a
json.loads reader hands an absent variable. A default that is itself an
environ.get is free too, since it reads the one source again under
another key; a literal ending such a chain is still reported. Detection
runs on the parsed tree, so a call split across lines reads the same.
Five sites were real duplication and lost their default: the adapter
port, openwebui's admin name (a role that runs as a bot user), odoo's
OIDC uid field, seaweedfs' public-primary flag, and the swarm hosts
entry IP. Nineteen carry a nocheck naming why the value is not the
deploy's: contributor tooling whose default is the only source, test
scaffolding that writes the file it names, and two readers of
CA_TRUST_CERT_EXTRA_HOST, which nothing in the repo sets.
utils/diagnostics/container.py resolves without a marker: RESCUE_DEPTH is
a recursion base case, and the integer 0 says what the string "0" said.
The file sits exactly at the 500-line cap, so a comment line was not
available.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The readiness module http_ready rendered insecure_skip_verify from
TLS_MODE and lookup('ca_injected', application_id) (#851). The lookup is
true only when Prometheus itself has a domain, TLS and the self-signed
mode, so in a self-signed deployment where Prometheus runs in another
mode the exporter had no CA file and skipped verification. All targets
share the module, so a target with a publicly trusted certificate was
probed without verification too, and anyone on the probe's path could
answer for a failed application.
http_ready now always verifies. A second module, http_ready_private_ca,
skips verification, and only the targets that need it select it: those
in the self-signed mode while the CA is not mounted into Prometheus,
through the __param_module label, which overrides the job's module for
that target. The default self-signed deployment mounts the CA and keeps
every target on the verifying module.
Verified on a local self-signed Tor round, where the CA is mounted:
the rendered blackbox.yml holds http_ready with insecure_skip_verify:
false and http_ready_private_ca with true, and no target of the scrape
configuration carries the module label. All targets of that round are
http onion names, so the https path itself ran only in the unit tests:
tests/unit/python/roles/web-app-prometheus pass, among them a publicly
trusted target that keeps the verifying module while the CA is missing,
a self-signed target that switches to the private-CA module, and the
verifying module rendering insecure_skip_verify: false without the CA.
Against the templates of main the three new cases fail.
Closes#851
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
