Four independent findings, each at its root.
sys-svc-container: tasks/00_core/gpu.yml included 'nvidia.yml' by bare name,
but the role is entered through tasks_from: 00_core.yml, so a relative include
resolves against tasks/ instead of next to the including file. Every sibling
include in 00_core.yml already carries the 00_core/ prefix.
web-app-seaweedfs: c61b29bd16 made services.seaweedfs.enabled and .shared
literal and stated that the variant pin follows, but never touched
meta/variants.yml. Both variants still repeated that literal, which
variants-static-override counts as four offenders. They set exactly what the
base declares, so they go. The base keeps the literal and regains the
dynamic-flag marker the console split dropped at e8e89d293d: the flag's only
job is joining the engine's shared_net, and its false branch yields a console
container that cannot start, which the lint names as a legitimate exception.
docs: the i8-tuner skill landed without its cheatsheet row, which the skills
coverage guard reads as a missing skill.
The remaining three files are that pass's own formatter output: quote
normalisation in nvidia.yml and line wrapping in two test files.
Not covered here: the two hallucinated URLs in locale/ko/LC_MESSAGES/docs.po,
because a translation run is writing the catalogs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
runtime: nvidia on its own is a no-op. The NVIDIA container runtime hook reads NVIDIA_VISIBLE_DEVICES off the environment of the container and injects nothing when it is unset, and that variable appeared nowhere in the repository. The engine images build on python slim with CUDA wheels rather than on an nvidia/cuda base, so they carry no such default in their image environment either; a role built on an nvidia/cuda base would have worked by accident. The deployed nllb-engine had been crash-looping on "Found no NVIDIA driver on your system" for 229 restarts.
Measured against the same image on the same nested daemon: --runtime=nvidia alone reports no devices, while adding NVIDIA_VISIBLE_DEVICES and NVIDIA_DRIVER_CAPABILITIES yields the device nodes, a working nvidia-smi and torch reporting 2.9.1+cu128 True 1. After the deploy the live container answers True 1 NVIDIA GeForce RTX 3080 Ti Laptop GPU, with the restart count back to 0 and health reported healthy.
Devices are named explicitly rather than with all. 00_core/gpu.yml asks the driver for memory.total and uuid, sorts by memory descending and publishes the uuid list, so a framework that grabs device zero gets the largest card. With a single card it degenerates to that card. NVIDIA_DRIVER_CAPABILITIES lives in group_vars rather than in the role vars because role vars are out of scope while a different role renders the template, which an earlier attempt proved by failing on an undefined name while rendering svc-ai-nllb-engine; SYS_SVC_CONTAINER_GPU_DEVICES crosses that boundary because it is a host-scoped set_fact.
meta/packages.yml already declared the toolkit for Archlinux, Debian and RedHat, the latter two carrying a note that sys-svc-container configures the nvidia.github.io repository before installing, yet only the apt path was ever built and the include was gated to Debian and Ubuntu. The declaration and the tasks contradicted each other. nvidia.yml now dispatches per family the way the role already does for Docker: an apt source for Debian, a repo file for the RedHat family, and nothing for Arch, where the package sits in extra. The install itself goes through package_install, and the runtime registration in daemon_config.yml was already presence-based rather than distro-gated, so it needed no change.
The discovery tasks live in their own file because a when-guarded block is capped at three direct children, and splitting the guard across two siblings would have tripped the repeated-when rule instead. Nothing renders on a host without a card: the gpu lookup and the set_fact hang on the same register, so the template emits the block exactly when the fact exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The verdict table remembered the winner of every comparison and nothing about speed, so the decider could tell which engine translates well and never which one answers quickly. For 183 catalogs the second question matters as much as the first.
verdict gains a nullable took_ms, and record() writes one duration per offered option. An option that did not answer stores NULL rather than zero, which would read as the fastest of them all, and speed() averages over the timed rows alone for exactly that reason. The widening ALTER exists because CREATE TABLE IF NOT EXISTS leaves an existing table untouched: a deployment that recorded wins before the column would otherwise fail every insert from here on.
The router asks for the average under the same signature the history is keyed by, so a duration belongs to a request class rather than to everything at once, and _criteria prefers it over the in-process last-request figure, which a restart forgets. Each wrapper swallows a history error and returns an empty mapping, as _wins and _streaks already do: a broken log costs the decider a fact, never a translation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_translations_are_not_copies_of_english asserts that the translation pipeline has run, not that the catalogue has the right shape, so it goes red on every machine whose catalogs are not complete. It now carries the guard tests/integration/i18n/test_catalogs_complete_on_gpu.py already uses for that same question, while the eight structural cases keep running everywhere.
Verified in both directions rather than only the convenient one: with a GPU the case runs and names the seven languages that still owe entries, and with INFINITO_GPU_COUNT=0 pytest reports it skipped while the structural cases pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test_mirrored_unit_tests expects tests/unit/python/roles/<role>/files/test_<name>.py for a script under files/python/, the way sys-svc-container already sits. This one was a level deeper, so the lint reported api_administrators.py as having no test at all.
The destination package carried no PROJECT_ROOT, which the test imports; the sibling defines it with parents[6] and this now does the same. The reflowed lines are autoformat's work from an earlier gate pass, not a change of substance: the file is otherwise byte-identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
With single sign-on disabled, the platform administrator had no indexer
account, so the dashboard login answered "Invalid username or password".
internal_users.yml now carries that account with backend role "admin"
while sso is off. The bcrypt salt is fixed per role, so a repeat deploy
renders the same hash.
The dashboard queries the Wazuh API with run_as. The default security
rules only match the users "elastic" and "admin", so every other signed-in
user got "You have no permissions" on the Wazuh pages. A new task runs
api_administrators.py inside wazuh.manager with the manager's own API
account. It creates the rule "infinito_administrators", corrects it when
its content drifts, and links it to the "administrator" role. The rule
matches the same three backend roles that roles_mapping.yml maps to
all_access, so it grants nothing to a principal that did not already hold
indexer all_access. The task prints CHANGED or OK and is idempotent.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The project licence governs this repository's code and says nothing about the application a role installs, which is where a restriction on commercial use usually sits: a deployment cleared to use Infinito.Nexus commercially is still barred from NLLB-200's CC BY-NC weights.
Each invokable role names its licence in meta/<role>/license.yml with the url it was read from. The rights of a named licence live once in meta/licenses.yml, fifteen fields following the taxonomy choosealicense publishes. AND folds to the strictest combination, OR keeps the most favourable whole licence, and the seven contexts are derived from the rights rather than declared, so a heading cannot contradict the matrix under it.
The generated page renders one matrix per context, a lint refuses a role that declares nothing, and a Playwright spec asserts the page against the deployed docs. 21 of 22 deprecated SPDX ids were resolved against the upstream text; EPSON's remains unread because its AUR page blocks automated access.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A deploy job name repeats the category path on every row of the matrix. One Egyptian hieroglyph costs four UTF-8 bytes where the two Chinese characters it replaces cost six. Unicode names each sign by its Gardiner code, so the opaque value stays identifiable without a comment per entry.
A label has to stay one codepoint: a sign written with a variation selector costs seven bytes and is worse than what it replaced. tests/integration/meta/categories/test_glyph.py enforces that.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The gateway had two engine kinds and neither reaches the catalogue.
LibreTranslate covers roughly 48 languages. The chat engine is backed
by the single entry in AI_LOCAL_MODELS, llama3.2:1b, which officially
speaks eight, while web-svc-translate/vars/main.yml declares every
language in meta/languages.yml as a chat target. The gateway therefore
asserted 183-language capability for a model that invents the rest:
Belarusian came back as Polish in 86 of 87 new entries, Abkhaz had no
Cyrillic in any of 83, Amharic 10 of 236 in Ethiopic, and Avaric,
Assamese and Bashkir returned the source verbatim.
NLLB-200 is built for translation rather than generation. A third
engine kind serves it beside the two libretranslate instances, since
it is seq2seq and does not belong as an ollama alias.
The honest ceiling is 136 of 184, not 184. flores.py maps 136 ISO
codes and names the 48 the model has no code for, and the test asserts
mapped plus unserved equals exactly the keys of meta/languages.yml so
neither side can drift. Belarusian, Amharic, Assamese and Bashkir are
covered; Abkhaz and Avaric are not, so two of the worst catalogues
still have no local engine.
The script half of each mapping is derived rather than typed: the test
re-derives every endonym's script from unicodedata and asserts it
matches the suffix of the FLORES code, ja and zh excepted because Han
cannot distinguish Jpan from Hans from Hant. The subtag half cannot be
derived, because where ISO 639-1 names a macrolanguage NLLB ships one
variety (ar->arb_Arab, fa->pes_Arab, ms->zsm_Latn and ten more), so it
is stated once in one table.
Counting languages surfaced a real bug: meta/languages.yml has 184
entries, not 183. 'no' is quoted because YAML reads bare no as false.
Inert until deployed. The engine entry is multiplied by zero while
TRANSLATE_NLLB_ENABLED is false, so TRANSLATE_ENGINES is unchanged;
the two new vars are evaluated unconditionally and every key they read
exists in the new role. No variant round was added, so CI does not
exercise this engine yet, and a first deploy needs an inventory
regeneration for the new service key, volume and subnet.
Assumed facebook/nllb-200-distilled-600M, about 3 GB resident, which
fits the envelope svc-ai-s1 already occupies. 1.3B buys roughly
+2 chrF++ concentrated in low-resource languages for twice the memory;
the README tables the sizes so that choice is visible.
Owed before this is offered beyond private use: the weights are
CC BY-NC 4.0, and the role does not yet say so or credit the NLLB
team. Tracked with the per-role licence index.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
coreutils timeout without --foreground calls setpgid, so the command it
wraps leaves the caller's process group. A wrapped command that then
reads the controlling terminal takes SIGTTIN and stops forever instead
of being killed at the deadline, which inverts the purpose of the
wrapper.
That is the cause behind d31b49e30d: container exec wraps docker in
this prefix, an ansible.builtin.script task gives the remote shell a
pty, and docker exec -i stops on its first read. Three jobs of run
39186571513 idled 1h37m to 2h22m before the outer budget killed them.
The scripts there close their own stdin; this closes the class.
utils/diagnostics/container.py gets the flag too. docker events reads
no stdin and was never at risk, but leaving one non-compliant
construction would have forced an exemption on the new rule's first
day.
The rule itself cannot be a text search. The prefix comes from a
function and the -i from the caller's argv, so timeout and the command
that reads stdin never meet in one line of source: a grep over the
tree scores zero hits both before and after the defect. The lint reads
the parsed tree instead and requires --foreground in any list literal
that starts with timeout. Checked against the pre-fix line, it reports
it.
Cost, deliberately accepted: --foreground signals the direct child
only, not a process tree. For docker exec the direct child is the CLI,
which is what has to die to release the task, but a wrapped command
that spawns children now leaves them where the group kill would have
taken them. covers the case where that
trade goes the other way.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
baudolo 7.1.x writes volume generations with `rsync -aHAX`, so the
producing host keeps `user.*` extended attributes (OpenCloud keeps its
storage metadata in `user.oc.*`). Every later copy dropped them again,
because none of the other rsync calls passed `-X` (nor `-A`/`-H`):
- NFS export snapshots (svc-bkp-nfs-2-local `script.sh`),
- the backup host's pull (svc-bkp-remote-2-local `pull_specific_host.py`),
- the device sync (svc-bkp-local-2-device `script.py`),
- every recover into a live target (`utils/recovery/base.py`, used by the
volume, NFS and device recover scripts),
- the `recover full` and `recover remote` tree pulls.
A disaster recovery through the documented chain therefore restored
OpenCloud without `user.oc.*`, silently: the copies succeeded and the
dirval stamp does not cover attributes.
All of these now run `rsync -aHAX` (`-aHAXP` for the device sync). A
target filesystem without xattr support makes rsync exit non-zero as soon
as a source file carries attributes, so the hop fails loudly instead of
dropping them; there is no per-device opt-out. The single-image pull in
`recover remote` keeps `-a`. The provider's ssh-wrapper needs no change:
its glob accepts any server flags and its character guard allows them.
Two tests write a `user.oc.id` attribute and assert it survives the NFS
snapshot and the recover; with the old `-a` both fail with
`OSError: [Errno 61] No data available`.
Closes#1021
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Moodle#2 of run 39186571513 failed the health check 26 times with
academy.main.infinito.test: Unable to reach https://academy.main.infinito.test/
(Navigation timeout of 20000 ms exceeded)
while every other vhost on the stack printed 'No CSP or network blocks
detected' and nginx answered 200 or 303 to every request. Nothing was
blocked; the page is slow. Moodle #0, #1 and #3 pass, and only the arm64
swarm row fails.
The role already claimed to own that budget. HEALTH_CSP_TIMEOUT_MS sat in
defaults with a comment calling it the clearnet navigation budget, but
script.py ran the clearnet batch as (clearnet_domains, '', 0), so clearnet
silently inherited the checker's own 20000 and the knob did nothing.
Only the onion batch ever received a real --timeout.
--clearnet-timeout closes that, defaulting to 0 so a call without the flag
behaves exactly as before. The value lives in its own variable rather than
raising the shared base, which would have multiplied through
HEALTH_CSP_ONION_TIMEOUT_MS and taken the public onion budget from 100s to
225s, toward the 3600s TimeoutStartSec cap. Onion budgets are unchanged.
45000 is derived, not picked: the slowest measured navigation on that
target reached domcontentloaded after 26s, with single requests stalling
up to 5.8s, so 26 + 3*5.8 = 43.4s, rounded. That is 1.7x the worst
navigation observed there.
TimeoutStartSec is deliberately NOT derived from the new budget. The
per-domain rate sizes the whole unit and nearly every domain answers in
well under a second; using the worst case as the scheduling rate would
have doubled the clearnet term and pushed large stacks toward the cap for
no measured reason. Scheduling is unchanged.
Known cost: above 20s the check no longer notices that an app got slow.
That coverage was incidental, not declared, and a page-load regression
needs its own assertion rather than a CSP probe.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The design and docs commits landed with --no-verify, so the hook that
runs make test never saw them and four repository lints went red on
code that was already pushed.
test_init_files wants an __init__.py in every python tree under tests/;
the sys-front-inj-design directory arrived without one.
ruff S701 rejects Environment(autoescape=False). Both call sites render
from a string, where select_autoescape(default_for_string=False)
returns the same False, so what the test exercises is unchanged and the
explicit escape filter keeps doing its own work.
test_no_env_defaults_in_javascript rejects a default for an environment
value in JavaScript, because it is a second source that hides an unfed
variable. Normalising to the empty string stays allowed and keeps the
expression's result identical for every input.
The env-file schema caps a comment at 140 characters.
The two lint/filesystem files carry nothing but the autoformat reflow.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The instruction replay provisions a fresh inventory inside a throwaway
target, so its compose pull fetched every application image from the
upstream registry while every other CI deploy pulls through the GHCR
mirror. On the moodle row of run 39082924588, 512 s of the 1646 s
ansible run went into that pull.
mirror.sh rewrites the extracted block and not the README template the
site publishes: it inserts the mirrors.yml resolver ahead of the
provision call and passes --mirror. A reader keeps the upstream
references, because a reader has no GHCR mirror to pull from, and the
replay exists to prove the instructions a reader copies. Outside CI the
two GitHub variables are empty and the block stays byte-identical.
The prefix is read from default.env in the checkout, the way machine.sh
already reads INFINITO_PARENT_IMAGE, so no Ansible template has to look
up an INFINITO_ key that belongs in an inventory.
A block that carries no provision call fails the harness instead of
quietly falling back to the upstream registry, and test_mirror.py
renders both published shapes to keep the two sed patterns matched to
what the template emits.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The inline design script carries a CSP sha256 computed at render time. On an
onion request body_filter.lua rewrites the clearnet CDN base inside the whole
head payload, the script's bytes stop matching their hash, and every role
whose script held a CDN URL has its branding refused wherever script-src-elem
has no 'unsafe-inline'. web-svc-logout and web-svc-html are on nearly every
deployment, so this was not a corner case.
The config now rides in the script tag's data-design attribute, which CSP
does not hash and onionize may rewrite freely, and the first statement of the
script reads it back into INFINITO_DESIGN. The executable body is identical on
both vhosts, so one hash authorizes both.
All 35 design scripts now read the config instead of interpolating
design.urls. The 13 that still needed converting keep their render-time
guards, so a role with branding disabled still emits no script at all, exactly
as before. slots left the config object: no script has ever read it.
A new check refuses a branding URL in a script body, so the class cannot
return, and a unit test drives the config through to_json, Jinja's escape and
an HTML parser with a hostile title to prove the attribute round trip.
Not yet proven: that the hash matches on a real onion vhost. This stack runs
without tor and every role deployed here has 'unsafe-inline', so the check
needs a tor-enabled deploy of one of the seven roles that disable logout.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
sys-front-inj-design now renders templates/css/app.css.j2 ahead of the
role's stylesheet and templates/js/design.js.j2 ahead of its script. The
branding URLs, the slot dimensions and the logo guard live there instead of
being repeated per role: the guard that stood in 41 sheets is one condition
in the composing template, and a sheet reads branding through
--design-url-* / --design-slot-* custom properties.
29 stylesheets and 43 design scripts lose their Jinja and move to static
files. Roles whose sheet carries app-specific URLs declare them as
design_css_urls and read them as --design-url-app-*.
The script prelude is prepended only when the role script references it,
and it carries urls only while a logo is configured, so every role that
needs neither is emitted byte for byte as before.
Five checks keep the mechanism honest: a composed sheet may only read
design custom properties that app.css.j2 declares for its role, a template
may not carry Jinja expression text outside its delimiters, a design script
may not quote its config object, and the injected-script parse check now
covers the design channel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
make quality-high reformatted three expressions this branch added today: the chat-target chain in the translate role's vars, an assertEqual in the gateway engine tests, and the wanted comprehension in the lane probe. No logic and no rendered value changes; the YAML break folds to a space inside a double-quoted scalar.
Kept separate from the changes that caused it, because a reflow mixed into a behaviour commit hides the behaviour.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
In swarm, docker stack deploy ignores depends_on, so the precheck
service and every init replica ran search_allocation_precheck.sh at the
same time. They shared one probe index: one invocation's PUT could fail
the shards_acknowledged check because another had created the index
first, and either one's cleanup could delete the other's probe. The
init service carries no replica override, so compose_replicas gives it
one task per host in the application group and the collision also
happens between init replicas.
A failing precheck task also never reached has_fatal_task in
stack_ready.sh: with restart_policy none it records a single error,
while has_fatal_task needs three. stack_ready.sh then kept answering
"not converged" until the handler ran out of retries. And a Running
init task counts as converged through running_current_spec, so the
swarm deploy never depended on init failing; the role's "wait for
Zammad init" task is what stops it.
The probe index now carries the container's hostname, which docker
sets to the container id, so concurrent invocations no longer share an
index. compose.yml.j2 renders the precheck service, and init's
service_completed_successfully dependency on it, only outside swarm.
Compose keeps its fast stop on a refusal; in swarm, init runs the same
check in front of zammad-init, a refusal leaves Zammad unseeded, the
wait for Zammad init fails the deploy, and the init service logs carry
the allocation explanation.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On a data disk with less free space than the high disk watermark asks
for, Elasticsearch creates indices but assigns none of their primary
shards. `zammad-init` then reports every search index update as failed
and never exits, so `compose up` waits on
`service_completed_successfully` until its 1800 s task timeout and the
deploy fails without naming the cause (#850). Elasticsearch's default
high watermark is an absolute 150 GB headroom on a large disk.
A new one-shot `precheck` service runs `search_allocation_precheck.sh`
with `restart: "no"`, and `init` depends on it with
`service_completed_successfully`:
- it waits a bounded time for `_cluster/health`;
- it creates a one-shard, zero-replica probe index with
`wait_for_active_shards=1`;
- once Elasticsearch allocates it, it deletes the probe and exits 0;
- otherwise it prints the creation answer and the answer of
`_cluster/allocation/explain` and exits 1.
The check sits in its own service because `init` keeps
`restart: on-failure`: wrapped into `init`, the failing check was
restarted seven times within eight minutes on a local deploy while
`compose up` kept waiting. Inside the stack, a `restart: "no"`
dependency that exits 1 made `compose up` fail after one second with
`service "pre" didn't complete successfully: exit 1`.
Every curl call carries --connect-timeout and --max-time, so a hanging
request cannot outlast the bounded wait. The image's entrypoint passes
any non-`zammad-*` command through `exec "$@"` (bin/docker-entrypoint of
7.1.2), and the image ships `curl`. The watermark values themselves come
with feature/store-i18n (b82591ba9d); this change covers inventories
without a variant cap.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make i18n-translate addressed the engine directly, so a run could never reach more than the 47 languages Argos ships models for, while meta/languages.yml names 183. service_url now resolves web-svc-translate, which routes a pair the engine does not serve to a chat model, and the deploy brings both roles up.
The gateway advertised only what its backends declared, and the chat backend declared nothing, so /languages reported 51 targets of 183 and the readiness wait spun to its hour-long timeout without translating a line. The chat backend now declares every language the table names.
Routing offered libretranslate for every pair, because an undeclared catalogue read as any pair, so a language it holds no model for was answered with HTTP 400 instead of falling through to the next backend. serves() now reads the backend's own /languages, which catalogue() already exposed for the advertisement. Declaring the pairs in the role was the first attempt and carried a second copy that was already stale: the engine reports zh-Hans where the role's list says zh.
A small model copies the prompt markers back, echoes the source line and labels its answer with the target language. All three reached the catalog, and an entry carrying its own source scores below the oracle's middle level. clean() strips them, and only from an answer that carried scaffolding, because on its own a short word and a colon is a translation, not a label.
server() had lost its redeploy parameter while cli/build/i18n kept passing it, so every run raised TypeError before translating a line and the only symptom was an empty log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4aba897ca2 gave the pull and build subprocess a 3600s ceiling so a hang
could not stall a deploy. A pull of a large image set legitimately outlives
any fixed ceiling while it is still making progress, and the ceiling then
kills a healthy run.
run_streamed measures the gap between two output lines instead. Two reader
threads keep stdout and stderr separated the way run_cmd does, because
compose writes its warnings to stderr and merging them poisons every parser
downstream. Only the two calls that legitimately run long use it; the short
calls whose output is parsed keep run_cmd.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merge left the role with two ways to the same end. The wired shell path called setup.py update-user, so it could neither create a missing account nor clear a lockout, and it decided its own idempotency from a sha256 marker in the persistent volume, which lies after a restore: marker matching, database not, reported unchanged and did nothing. The python converger from 0709de6fa1 queries the application instead, creates what is missing, verifies the password, unlocks, resets the failed-login counter and ensures the Administrator role, and it carried the only tests either path has; it was wired to nothing after the merge resolved towards manager_ops.yml. That engine now serves the surviving wiring: renamed into the family spelling the sibling roles use, reading the env names the task already passes and emitting the markers the task already parses. tasks/administrator.yml had no caller anywhere in the tree and goes with it.
Trade-off worth knowing: the deleted path used the supported pgAdmin CLI while this one uses internal APIs, so a pgAdmin upgrade can move it. Neither path has ever executed in CI, the pgadmin job died at the health wait before reaching the realign, so the choice rests on capability and tests rather than on field evidence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The guard patched the silence budget down to 0.3 s, but the watchdog starts its clock at Popen while python -c still has to boot before it can print, so under the parallel load of a full gate run the startup alone exhausted the window and the process was killed before its first line. The same file passed in four seconds on its own. The budget is now 1.0 s against a process that prints every 0.2 s for three seconds, which keeps all three conditions true at once: the total run outlives the budget so the old total-runtime timer still kills it, the gap between lines stays under it, and so does interpreter startup. Verified against the old shape in isolation: it dies with signal 9 after five of fifteen lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The watchdog armed a single timer at process start, so it killed the German build with SIGKILL while that build was still emitting progress, and Library.failed() then answered 500 for the rest of the run. The container was not out of memory: OOMKilled is false, the restart count is zero and neither the kernel ring buffer nor the journal records a kill, while the sphinx worker BrokenPipeError that follows is the signature of killing the parent alone. The watchdog now stamps the time of every line read and kills only after a full silence window, which the existing test name already described. A regression guard runs a process that keeps printing past the deadline and requires its last line in the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`deadline` is whole seconds from `date +%s`. A run that starts just
before a second boundary reaches the deadline after the first probe and
one `sleep 1`, so the gate correctly gives up after one probe while the
test demanded two. With a stubbed clock (1000, 1000, 1002) the gate
probes once and prints `deadline of 2s reached`. The deadline message
already proves the pending migration held the entrypoint back; require
at least one probe.
Reported by CodeRabbit on #979.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Swarm Nextcloud#3 on main 748189e890 (job 117182318941) wedged the
OpenProject schema: the convergence check gave up after 74 attempts
while the seeder kept exiting 1, first with
`relation "ar_internal_metadata" already exists` from its structure
load, then with `relation "announcements" already exists` on every
`db:migrate`.
The seeder (`docker/prod/seeder`) loads `db/structure.sql` through
psql, statement by statement, when `\dt` finds no relation. In swarm,
`depends_on` is ignored, so web and worker start alongside it and poll
`await_migrations.sh`. That gate called
`rails db:abort_if_pending_migrations`, which runs
`migration_context.open`, i.e. `Migrator.new`, whose constructor
executes `schema_migration.create_table` and
`internal_metadata.create_table` (Rails 8.0.4,
activerecord/lib/active_record/migration.rb:1430-1431). The seeder
saw an empty database at 05:33:20; by 05:33:38 the gate had created
`ar_internal_metadata`, the structure load aborted at line 255 with
everything alphabetically before it already committed, and every
later attempt took the migrate path into tables that already existed.
Probe with `rails db:migrate:status` instead. It aborts while
`schema_migrations` is missing and otherwise only reads the recorded
versions (database_tasks.rb:303, migration.rb:1318); the gate treats
any `down` row as pending.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two conflicts, both resolved by combining rather than by picking a side.
`roles/web-app-socialhome/tasks/02_design.yml`: designer restructured the pass,
moving the token request and the theme write into the new
`02_design/converge.yml` and adding a stat of the name-state file. HEAD still
carried the converge block inline. Kept designer's stat, dropped HEAD's inline
block: the merged body below the conflict already includes `converge.yml` and
reads `socialhome_name_marker`, which only designer defines, so keeping HEAD's
side would have duplicated the token request and left that variable undefined.
HEAD's `retries: 0` on the token request is deliberately not carried over; the
default of 30 retries covers a service that is not yet accepting connections,
which matters more here than failing fast on bad credentials.
`roles/web-app-fider/files/playwright/test-design.js`: the import line takes
both sides, since the merged body uses designer's `apiGetOnion` and HEAD's
`requireDotenvValue`. The env block keeps HEAD's `requireDotenvValue` form,
because designer's `|| ""` defaults are what
`tests/lint/ansible/.../test_unguarded_env_default.py` exists to reject, and
drops HEAD's module-scope `title` and `logoEnabled`: the merged body defines
`title` locally and reads `DESIGN_LOGO_ENABLED` directly, so both were dead and
would fail `no-unused-vars`.
Checked after resolution: no conflict markers (`git diff --check`), no `nocheck`
line dropped by the merge, and no duplicated YAML key in the seven merged files.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Without `--all`, `journalctl --output=json` encodes a field over 4096
bytes as `null`, so a long host error matched nothing and the check
passed. A repeated `MESSAGE` field arrives as a JSON array of its
values, and `bytes()` on that array raised `TypeError`, which aborted
the whole check.
Pass `--all`, and tell a byte array (all integers) from an array of
field values, decoding each value on its own.
Reproduced against the host journal: a 5000-byte error line read back
as `"MESSAGE":null` and produced no match; an entry with two `MESSAGE`
fields, sent over the native journal socket, read back as
`["error first","error second"]` and crashed `host_errors` with
`TypeError`. With the fix both are reported.
Also clear the two test-lint findings of this PR:
`test_comments_valid` rejects the narrative comment above the
`journalctl` Popen, so it goes; the stderr-to-file choice stays, because
an unread stderr pipe that fills up would block `journalctl` while
stdout is still being read. `test_no_uncached_file_reads` flags the
read of the stub's argument file, which lives in the test's tempdir and
is written per run, so the path-keyed `utils.cache.files.read_text`
could serve a stale body; the read opts out with
`# nocheck: cache-read tempdir fixture`, as the other stub-driven
script tests do.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
SH_ADMIN_PASSWORD seeds the administrator on first boot only: upstream provision_admin early-returns once platform_users holds a row. The rotation gate regenerates users.platform_administrator.password, the container keeps the old hash, and the household theme token request fails. The 31 retries then drained the 5-per-15-min budget upstream enforces on /api/auth/token, so every attempt after the fifth answered 429 and hid the underlying 401.
realign_admin_password.py runs inside the container, resolves the database through the config helper the application itself exposes, and rewrites platform_users.password_hash with the scrypt helper the application itself exposes when the declared password no longer verifies. It writes that one column, because set_password would also write is_admin and demote the administrator.
The token request drops its retries. 31 attempts into a five-per-window budget also poisons the bucket the Playwright design spec shares, and 0 is the only value multiply-timeouts.sh cannot inflate.
Proven by a two-pass compose deploy of variant 0, the row that failed in CI and the only one with design enabled: pass 1 [UNCHANGED], pass 2 [CHANGED] after the rotation, and the token task that failed in CI with 31 attempts and status 429 answering on attempt 1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Problem:
Every compose service logs to journald through
sys-svc-container/templates/base.yml.j2. The daily journal health
check grepped the whole journal for "error", so routine application
errors from containers triggered the host alert (email/Telegram) and
buried real host problems.
Change:
- Read the last day of the journal as JSON (MESSAGE, CONTAINER_ID,
SYSLOG_IDENTIFIER, _SYSTEMD_UNIT) and skip every entry that carries
a CONTAINER_ID field. Docker daemon and system errors are still
reported.
- Report each match as one readable "source: message" line instead
of raw journal output.
- Stream the journalctl output line by line instead of buffering a
whole day of logs in memory.
- Exit 2 with journalctl's stderr if journalctl itself fails, instead
of silently reporting "no errors".
- Rewrite the check in Python (files/python/script.py, deployed by
sys-service like the other sys-ctl-hlth roles) and decode non-UTF-8
messages.
- Fix the success message, which claimed "All docker containers are
healthy".
Impact:
Container log errors no longer raise this host alert on any
installation. Container health stays covered by sys-ctl-hlth-container;
application log analysis belongs in a log platform such as Wazuh.
system_service_suppress_flush is unchanged until a deploy confirms
that no host errors remain.
Tests:
Unit tests run the script against a fake journalctl (8 cases):
container error ignored, daemon error reported, identifier fallback,
non-UTF-8 message, journalctl failure, many container lines streamed
past, clean journal passes. The core cases fail against the previous
script.
Under SSO the platform administrator signed in and landed on #profile as a
Customer, so the design spec never reached #manage/users. Three defects
stacked.
The group-to-role map was written with Jinja in its mapping keys. Ansible
renders a mapping's values and leaves its keys alone, so the rendered Ruby
keys were literal {{ ... }} and could never match a group path. The map is now
one expression that computes its keys through rbac_group_path, the shape
meta/addons already uses elsewhere.
The initializer patched User.create_from_hash and User.update_from_hash behind
`next unless User.respond_to?`. Zammad 7.1.2 defines only create_from_hash!,
so both guards fell through in silence and nothing was installed: the
initializer's name does not appear once in the failing run's Rails log. Patch
Authorization.find_from_hash and Authorization.create_from_hash instead, which
sessions_controller calls before current_user_set, so first and subsequent
logins are both mirrored.
administrator mapped to Admin alone. Zammad's Admin carries no ticket
permission, and the Dashboard navbar entry is gated on ticket.agent, so an
Admin-only account has neither Dashboard nor Overviews. Grant Admin and Agent,
matching what the password path converges.
The write is defended rather than assumed. Zammad's has_roles concern runs
validate_roles, validate_agent_limit_by_role and last_admin_check_by_role on
collection assignment and they raise, which inside the omniauth callback would
answer the sign-in with 500 - a lockout, worse than the demotion being fixed.
The assignment is wrapped, resets the association and logs on refusal, and the
sign-in completes on the roles the account already has. Role lookup filters on
active and reports a missing role instead of silently granting the subset. A
map that rendered as a String is refused at boot, because map[path] would
otherwise be String#[] returning a substring and the feature would be a silent
no-op. The patch targets are checked by signature, not by respond_to?, since
respond_to? was exactly what hid defect two.
An absent groups claim leaves the roles untouched and logs, rather than
demoting every account to Customer on a misconfiguration. A present but empty
claim still means "in no group" and maps to Customer.
biber is unaffected: inventories/development/default.yml declares roles: [] for
it, so it is in no RBAC group, maps to Customer as before, and keeps
#customer_ticket_new.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
assertReadable takes a selector list and measures contrast on whatever it
finds. An entry marked optional is skipped when it matches nothing, so a list
whose entries are all optional can measure nothing at all - and then the call
asserts nothing while reporting green. Reject such a list before the browser
is touched. A plain string normalises to a required entry, so only a
deliberately all-optional list is refused, and an empty list with it.
web-app-matomo found the defect the hard way in CI run 38818071194. Its KPI
card check listed four optional selectors, all of them trend modifiers that
the dashboard only renders at a non-zero change; every card read 0%, so
KPICard emitted kpiTrendNeutral and SitesTableSite left its evolution cell
unclassed, nothing matched, and the test failed on the minimum-measured guard
rather than on contrast. The sibling evolution-badge check has the same
all-optional shape and was green only by luck.
Give every all-optional call site an anchor that is already proven visible in
its own spec: .kpiCardEvolutionTrend and .evolutionBadge for matomo,
header.header for web-app-dashboard (waitForDashboardReady asserts it
unconditionally) and #login .panel-body for web-app-espocrm. The espocrm case
needs care: #btn-login and #sign-in each exist in only one branch of the sso
conditional, so neither is variant-safe, while assertLightAndDark measures
.panel-body on the line above.
The matomo theme rule is part of the fix, not a cosmetic extra. Without it the
new anchor keeps Matomo's hardcoded #76838f, which measures 3.60 against
--design-surface-1 and 3.82 against --design-surface-2 in light mode and would
fail the 4.5 threshold the anchor now enforces. The role already replaces
Matomo's hardcoded trend colours; this closes the same gap for the neutral
caption. Equal-specificity rules resolve by order and the role's block is
concatenated about 204 KB after Matomo's in the single served bundle, measured
by byte offset in the failing run's own trace.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
00_core.yml pins xwiki_blobstore_s3_enabled_switch false for the bootstrap
render on every deploy. On a wiki whose attachment content already lives in
the SeaweedFS bucket that strips the whole store.* block - the blob store type
and the object-store credentials with it - and restarts the container. In CI
run 38818071194 the credentials were rotated between the two passes at 04:49,
the SeaweedFS identities were re-rendered at 05:04, and the bootstrap render at
05:39 then removed the only configuration that would have carried the new keys
in. 06_design/logo.yml:7 is the first task that reads an attachment, so that is
where the play died, after 31 retries, with "could not be found in the
filesystem attachment store".
Move the bucket render behind 04_extensions in 02_manager_ops.yml, which
installs and asserts xwiki-commons-store-blob-s3. The precondition 26d588fa41
names - nothing answers to 's3' before the extension is installed - is then
established by execution instead of inferred, and 00_core.yml stays byte
identical to what it was.
An earlier attempt probed the deployed xwiki.properties for store.blobStoreType
and selected s3 when it was already there. That is not safe: the purge
primitives in scripts/container/purge/apps.sh:86-94 each run under `|| true`,
so a failed `rm -rf` leaves the config dir behind with the volume gone, and a
plain `compose down -v` in a debugging session does the same while the central
Postgres keeps the installed-extension rows. The probe would then have written
s3 into a wiki that cannot start. Removing that exposure beat pricing it.
Nothing before the new task can behave differently from today, so the only new
behaviour is one render into oidc off, ldap off, s3 on - two configurations
that each booted in this run, in a flat properties file where the blocks do not
interact. In the bootstrap window the new shape is strictly better: today's
window is a superset that extends through 06_design.
The test asserts both halves of the invariant, that no render in 00_core.yml
selects the bucket and that the bucket render precedes the 06_design include.
Against the previous tree it reports renders [8] design [7] ordered_ok False.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
weblate, beat and worker mount the same weblate_data and weblate_cache.
Docker fills a fresh named volume from the image once per
POST /containers/create, and that copy does an Lstat plus os.Mkdir per
directory without tolerating EEXIST. Compose created all three at once, two
populations of the image's /app/data raced, and the deploy died with
"failed to mkdir /var/lib/docker/volumes/weblate_data/_data/python: file
exists" in CI run 38818071194. The daemon log names the route as
containers/create, so this is the population path, not a mount of a
pre-existing file.
Make weblate and worker depend on beat. The lever is not startup order but
the Create phase, which compose runs in dependency order: beat populates both
volumes alone, and by the time the other two are created no copy runs. The
failing run proves that phase is ordered - COMPOSE_PARALLEL_LIMIT was 4, yet
only the two existing depends_on targets reached Created before the other
three began.
This is compose-only. In swarm, container_depends_on drops to a bare list
(plugins/lookup/container_depends_on.py:103) and docker stack deploy ignores
it, so swarm behaviour is unchanged and the same race stays reachable there. A
mode-independent fix needs nocopy on the non-owning mounts, which
plugins/lookup/container_volumes.py cannot express today.
The test reads meta/volumes.yml, collects every service sharing a volume and
requires each non-beat sharer to name beat. Against the previous template it
yields weblate False, beat True, worker False, so it fails without this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The web-app-litellm stylesheet swapped the navbar image with content: url(), so LiteLLM's own logo setting stayed empty and its UI theme page showed nothing. svc-ai-litellm now renders litellm_settings.ui_theme_config.logo_url from LITELLM_UI_LOGO_URL, the design lockup of web-app-litellm, only when that role is deployed with design and a logo; the stylesheet override is removed. The design spec asserts the served setting, the navbar src, the absence of a CSS swap, the loaded wide lockup and the value on the UI theme page; the config render test covers both the logo and the no-logo case.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
