install-lint provisioned chrome-headless-shell on every run, and `make lint`
runs install-lint once per lint target in parallel whenever its stamp is not
fresh. On CI the stamp never is (eslint lives in node_modules, not on PATH),
so one lint job provisioned 16 times (run 38455552981). As root, puppeteer's
--install-deps runs `apt-get satisfy` even for an installed browser, and two
of those at once fail on the dpkg lock; a failed provisioning then deleted the
shared browser cache while lint-mermaid could be rendering with it.
The browser is now provisioned only when it is not installed, and --force
drops its cache so `make install-lint-force` still reinstalls it with its
libraries.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_no_uncached_file_reads rejects Path.read_text in tests unless the file is
written in the test itself. The mermaid script test reads the call log its
stubs append to during the run, so that read carries `# nocheck: cache-read`.
The mermaid cli test is formatted with ruff.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A local deploy stopped four times in two days in "Ensure Docker apt key
and sources are consistent" (#849). apt_repo.sh downloads the Docker GPG
key on every run, and when download.docker.com refused the connection,
curl's four attempts were over within about six seconds and the script
returned 1, although the key from an earlier run was installed and is
all the following apt operations need.
install_key now fails only when no key is installed. With a key in
place, a refused download prints a warning and keeps it. A downloaded
key that differs still replaces the installed one, and the 0644 mode is
enforced in every case. The keyring path became an argument:
ensure-key-and-sanitize passes the keyring it already hands to
sanitize_sources, so the role's call still resolves to
/etc/apt/keyrings/docker.asc, and a test can point the script at a
temporary keyring.
Verified: tests/unit/python/roles/sys-svc-container/files/test_apt_repo.py
stubs curl and sudo. A refused download keeps the installed key and ends
with changed=0, a refused download without a key exits 1, and a new key
replaces the installed one. A copy of the script whose keep branch always
errors, the old behaviour, fails the first test. Two local deploys ran
the new script with rc 0 and an empty stderr and left the installed key
untouched. shellcheck is clean.
Not verified live: the refused download itself, which needs
download.docker.com to refuse a connection during a deploy. A host
without a key still needs the download and still fails when it is
refused.
Closes#849
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
make lint runs install-lint once and then dispatches its fifteen targets
in parallel. scripts/lint/mermaid.sh provisioned chrome-headless-shell
inside that parallel phase with `npx puppeteer browsers install
--install-deps`, whose `apt-get satisfy` carries no lock timeout. In a
fresh stack container another target held the apt lock, the shared
libraries stayed missing, the script only warned, and all 289 Markdown
files then failed on libnspr4.so (#795).
utils/install/lint/mermaid_cli.py now provisions the browser right after
mmdc, inside the serial install-lint step, as root with --install-deps
and as a user without it, because puppeteer refuses to install libraries
without root. A failed install deletes the chrome-headless-shell cache
folder and retries once under a 900 second timeout per attempt; a cut-off
download otherwise leaves a folder without its executable, and puppeteer
refuses every later install of that build. The install-lint stamp counts
as fresh only while the browser executable exists, so a recreated
container provisions it again.
mermaid.sh no longer calls apt-get or npx. It renders one minimal diagram
first and, when that fails, exits with mmdc's output instead of reporting
every Markdown file.
Verified: tests/unit/python/scripts/lint/test_mermaid.py stubs mmdc,
npx and apt-get. With a browser that cannot load libnspr4.so the script
exits 1 with that line, before any Markdown file renders, and mmdc is the
only tool it calls. Against main's mermaid.sh the same test fails: it
calls apt-get and npx and reports the files one by one. The installer and
stamp cases pass, shellcheck is clean.
Not verified: a full make lint in a recreated stack container. The race
reproduced in one of two runs in the issue.
feature/store-i18n changed the provisioning in mermaid.sh with 2f5f52bd1a
and dc67bce5f0 (timeout, deleting a cut-off browser). Both behaviours
moved into mermaid_cli.py here, so its merge takes this version of
mermaid.sh.
Closes#795
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
report_failures asked `gh api --paginate` for the run's jobs with the jq
filter `[.jobs[] | {name, conclusion}]`. gh applies that filter to each page
and prints one array per page, so a run with more jobs than one page holds
produced two JSON documents, and json.loads stopped with "Extra data: line 2
column 1". Main run 38218402254 (113 jobs) failed its "Report main role
failures" job that way, and no issue was opened for its failed roles.
The listing now emits one job object per line and parses the lines one by
one, the same way logs/download/github.py lists jobs.
Verified: the old parse of the real output of run 38218402254 raises the
same "Extra data: line 2 column 1 (char 8491)" as CI; list_jobs on that run
returns all 113 jobs, 16 of them failed, grouped into 12 failed roles.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
roles/web-app-n8n/meta/services.yml conflicted on the n8n pin: main already
carries 2.43.4 from #900, newer than this branch's 2.43.2, so main's side is
kept. The provisioning fixes of this branch are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The repository is bind-mounted into the infinito container, and every deploy
runs `make setup` there, which depends on `dotenv`. The generator therefore
rewrote the host's .env with the values the container sees: the host lost
INFINITO_RUNNING_ON_GITHUB=true that way, and the guide replay pulled
alpine:latest past the GHCR mirror.
compose.yml now marks the infinito service with INFINITO_DOTENV_HOST_OWNED,
and `python -m cli.meta.env` keeps an existing .env while that marker is
"true". A missing .env is still generated, and `make dotenv-force` (env -i)
still rewrites it. The key stays out of the env registry: only compose.yml
sets it, it is no configuration value, and a .env entry could freeze the
host .env.
test_distros now also strips INFINITO_ENV_LOADED. load.sh returns early
while that guard is set and the mtime of .env is unchanged, so the test only
reloaded the budget keys it strips because the container kept rewriting .env.
Verified: unit tests cover a kept .env, a missing one being generated, and a
rewrite without the marker. Local compose deploys print "Kept .env: the host
checkout owns it" where they printed "Wrote .env" before, the host .env
checksum stayed the same across a deploy, and the four keys holding host
paths resolve inside the container under the same paths.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The network probe used community.docker.docker_network_info, whose
get_network() scans GET /networks and returns the first entry whose Name
equals the requested name OR whose Id starts with it, in a list order that
changes between calls. In main run 38038541631 (Zammad#1, swarm) the
central redis overlay got the ID n8nwzb1q39m3u4k7awwwmt6yg, the probe for
the not yet created provider network "n8n" returned it as existing, the
create step was skipped, docker stack deploy attached hermes to the redis
overlay as "n8n", and the oauth2-proxy then reached the password-protected
central redis by its bare host name: "NOAUTH Authentication required".
The same scan can also return the ID-prefix twin of a network that does
exist, so comparing the returned name alone would turn that network into
"missing" and fail the create with "already exists".
Both probes now run `container network ls --filter name=<entity> --format
'{{json .}}'`, whose name filter never matches an ID. The new lookup
network_driver picks the entry named exactly after the entity and returns
its driver, '' when there is none; the drift checks, their messages and the
create steps all read that one entry. No docker module is left in the file,
so its two docker-module-uses-container-cli suppressions go too.
Verified: on the host daemon GET /networks/<id prefix> returns a network of
another name and docker network ls --filter name=<id prefix> returns
nothing. In ansible-playbook against real docker network ls output the
lookup returned the exact network's driver in 20 of 20 probes beside an
ID-prefix twin, an empty driver (create path) for a missing network and for
a name only contained in a sibling, and the drift path for a foreign
driver. Unit tests cover the same cases; with a substring match instead of
the exact name the two sibling cases fail.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.
iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.
Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Both provisioners follow nextCursor for workflows and step skip for API
keys until a short page arrives. A cursor n8n repeats, or a key listing
that ignores skip, would keep that loop running forever and hang the
deploy instead of failing it.
workflows_named now remembers every cursor it followed and exits with
"FAILED listing workflows: nextCursor repeats" when one comes back.
api_keys_labelled exits when a non-empty page brings no key id it has not
already seen. Normal paging is unchanged.
Four regression tests drive both scripts against a fake API that repeats
its cursor or ignores skip; without the guards exactly those four fail.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`make quality-high` rewrote these files on every branch: the SuiteCRM
README and the root role table lacked the Redis service #881 added, the
session path test was not formatted, and an OpenTalk secret description
from #873 was not wrapped.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
n8n 2 returns ten API keys and a hundred workflows per page by default. A
stale managed key behind the first page survived, and a managed workflow
behind it was created a second time. Both scripts now follow every page,
narrowed by label or name, and keep the exact match in the script.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
n8n 2 pages GET /rest/api-keys as {data: {items, counts, totals, owners}},
ten keys at a time, and matches `label` as a substring, so walking the body
raised "'str' object has no attribute 'get'". It also enforces API key
scopes, so activating the MCP server workflow needs `workflow:activate`.
Both scripts now read every list and create response through one
normalisation that unwraps the page, look the managed key up by label, and
name an unexpected shape by its keys only. The normalisation is taken over
from a392e2dd49 on feature/store-i18n.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With three swarm replicas the PHP session files on the shared NFS volume
were read as empty under concurrent requests of one browser, so SuiteCRM
answered 403 on /api/graphql and dropped the session. Sessions now live in
Redis with session locking that waits like flock did; the phpredis version
is pinned in services.yml with its own update source, and the engine lookup
renders the save path.
Refs #879
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ruff format rewraps the read-back line in test_repository.py, and ruff
check reports the nested with in test_fetch.py as SIM117. Both fail
lint-python.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An http_regex source read one document, so a tag whose parts upstream
keeps in two files could only carry the second part as a literal. Every
pin also fetched its document on its own: a failed request left one pin
of a release behind while the others moved.
url now takes a list whose bodies are searched as one text, and every
document is fetched once per run, so all pins that read it resolve against
the same snapshot. The HTTP reads move into utils/update/fetch.py.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The repository updater assigns ref: lines to entities by position: every
entity that pins a value takes the next line carrying that value. A ref
that declares its own source was dropped before it took its line, so a
second entity pinning the same value was given the sourced entity's line,
and an update of it would have rewritten the wrong ref.
A sourced ref now takes its line first and is skipped afterwards, the
same order the nocheck marker already follows.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A pin that names its own upstream in an update: block was still collected
by the image updater or, for a ref, by the repository updater. Those raise
it to the newest tag of its registry or repository, which for a component
of a release is not the release's value. A tag built from several upstream
values, such as <freeswitch>-<bigbluebutton>, could not be resolved at
all: the resolver takes one capture and keeps a pin on its suffix.
The image updater now skips a version and the repository updater a ref
that declares a source for itself. An http_regex source accepts a template
over the pattern's named groups, and a templated pin moves whenever the
assembled value changes, its suffix included.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A pin that names its own upstream in an update: block was still collected
by the image updater, which raises it to the newest tag of its registry.
For a component of a release bundle that tag is not the bundle's, so the
two updaters would propose different versions for the same line and such
a pin had to carry nocheck: docker-version.
The image updater now skips a version that declares a source for itself.
The source resolver alone bumps it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`native_metrics_target` was a Jinja filter, so it could not read
DEPLOYMENT_MODE: filters see no Ansible variables. It returned
`<services.<key>.name>:<port>` unconditionally, which is `matrix-synapse:9000`
for matrix. Compose serves that name; swarm does not, because `docker stack
deploy` ignores `container_name` and the alias never exists. This repository
says as much itself, through `lookup('compose_only', 'container_name', ...)`.
The target was therefore scraped and refused, and `playwright.spec.js:274`
reported `web-app-matrix: 1 matching target(s) reporting up != 1`, meaning one
matching target whose `up` value is not 1.
It becomes a lookup that delegates host resolution to `scrape_target`, which
was already mode-aware, so exactly one plugin knows the difference between the
two modes. The port has to be handed over explicitly: matrix exposes metrics on
9000 while `services.synapse.ports.internal.http` is 8008, so the existing
internal-port lookup cannot serve this case on its own.
`scrape_target` now refuses `port` together with an explicit `port_kind`.
Before, `port` won silently and the error text still named the
`ports.internal.<port_kind>` path that the call had never consulted.
`native_metrics_apps` drops an app that pins itself to compose through
`compose_mode_force` while the cluster runs swarm, because the scrape target
would then name the wrong object. Only the literal case: matrix declares that
override as a Jinja expression over its flavour, which this lookup cannot
resolve from the scraping role's variable scope, so variant 0 on swarm stays a
latent mismatch.
The two comment lines claiming `container_name:port` went with the change;
after it they state the opposite of what the code does.
Verified by mutation: with the swarm branch of `scrape_target` disabled, the
new test emits `matrix-synapse:9000`, the exact string CI reported.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The entrypoint removed the old tree before it copied the new one. A
copy that failed left the volume without a working Moodle, and every
restart removed again instead of recovering. It now copies into a
staging directory inside the volume and swaps the trees once the copy
is complete, so a failed copy leaves the old tree in place.
An image that ships an older release than the volume records no longer
replaces it and exits with a message instead. Two instances on
different releases would otherwise overwrite each other's tree.
upgrade.php reports "No upgrade needed" on stderr, so the task counted
every run on a current site as a change. It reads stderr now.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
MCP Adapter 0.7.0 for WordPress refuses a request that follows
initialize without the MCP-Protocol-Version header: 'Invalid Request:
MCP-Protocol-Version header is required for a 2025-06-18 session'. The
contract probe never sent it, so the CLI test of web-app-wordpress
failed at tools/list.
The probe now keeps the version the server answers initialize with and
sends it on every later request of the streamable HTTP transport, which
the specification asks of a client since 2025-06-18.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pin moved from 4.5.11 to 5.3.0, and the role fitted neither a new
install nor an existing one.
Moodle 5 serves from public/ and routes every other path through
r.php. nginx now uses public/ as its root, hands unknown paths to the
router and passes PATH_INFO for scripts with slash arguments, and
config.php declares the router as configured. The paths the role checks
and writes (version.php, auth, webservice, the OIDC plugin) moved below
public/, the URL of the OIDC plugin stays auth/oidc. The source archive
no longer contains vendor/, so the image installs the Composer
dependencies it declares. The OIDC plugin is pinned to v5.2.1, the
newest release.
An existing site kept its old code: the entrypoint filled the code
volume once and never again. It now records the release it copied and
replaces the tree when the image carries another one, and a new task
runs admin/cli/upgrade.php. Plugins that were installed through the web
interface are not carried over, the README says how to ship them.
A site installed with 5.3 forces login by default, which 4.5 did not,
so its start page answers 303 to the login page and the webserver
health check refused it. The role declares 200 and 303 as the codes
that check accepts: 200 is what an upgraded site keeps answering.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A Redis sidecar whose consumer declares no services.redis.mem_limit
runs in a container that inherits the consumer's primary entity limit.
Since the lookup replaced the old filter, maxmemory followed that limit
and rose from 204 MiB to between 819 and 3276 MiB in 17 roles.
lookup('service_memory_mb', ..., provider=<role>) caps the result at
the same share of the provider's own limit when the application does
not declare one for that service. A declared limit is not capped, and a
container smaller than the provider keeps its own share. The Redis
sidecar passes provider='svc-db-redis': those 17 roles get 409 MiB, the
seven with a 256 MiB container stay at 204.
The lookup now rejects unknown options and a provider that is no role
or declares no limit for the service, so a misspelled option fails
instead of silently lifting the cap.
The memcached sidecar's literal -m 64 takes the same lookup and keeps
its value.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lookup('service_memory_mb', application_id, service_name, share) returns
a share of a service's mem_limit in whole MiB. It resolves the limit the
way sys-svc-container resolves the container's own: the named service,
then the role's primary entity, then RESOURCE_MEM_LIMIT.
It replaces the three filters of memory_filters, each with its own unit
parser and its own idea of the limit: Redis maxmemory at 0.8 of the
limit, and the Confluence and Jira heap at 0.7 and 0.35.
The central Redis (409 MiB), Confluence (5734/2867) and Jira (4300/2150)
keep their values. A Redis sidecar without a mem_limit of its own does
not: the filter fell back to a fixed 256m, which gave 204 MiB, while the
lookup takes the limit that container really runs with, the primary
entity's. That moves 25 roles to between 409 and 3276 MiB.
The heap clamps of the old filters are gone: the 1024 MiB floor, limit
minus 1024, the 12288 cap and Xms bounded by mem_reservation.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The guard required a register on every async kcadm job and nothing more.
A result that is registered but no longer handed to async_reap_jobs,
directly or through the list it is collected into, passed it, although
the reap then neither polls that job nor sees it fail.
The test now follows each registered result to an async_reap_jobs
expression, by variable name with word boundaries, so kc_declared_client
does not pass for kc_declared_client_jobs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stub answered every absolute name, so a guard that asked for the
backend twice and never for the exporter passed both tests. The stub now
records the names it is asked for, and the tests require the backend and
the exporter.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
COLLABORA_WOPI_HOST read the scheme and the port with
lookup('tls', 'web-app-nextcloud', ...) and the host with
lookup('domain', 'web-app-nextcloud'). The tls lookup aligns an app term
to its consumer: for a clearnet Collabora it answers for the clearnet
sibling of an onion-primary Nextcloud, while the domain lookup keeps the
onion name. Such a deployment renders https://<onion>:443, an origin
Nextcloud does not use, so the allowlist does not match its WOPI calls.
Nextcloud pins overwritehost and overwriteprotocol to its primary domain,
so the allowlist now asks the tls lookup for that domain instead of for
the app.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
