The ruff toolchain was unpinned, so host venvs, the lint container and CI
installed different releases over time. That drift is why lint-python only
failed on CI: locally ruff 0.15.x preview-gated ISC004/FURB192 and (falsely)
fired S310 on urllib.request.Request, while CI's 0.16.0 has ISC004/FURB192
stable and drops the S310-on-Request false positive.
Pin ruff==0.16.0 in pyproject.toml's dev extras (Dependabot keeps it current)
and have utils.install.lint.ruff read that pin and reinstall whenever the
found version drifts, so every environment resolves the same ruff. Then fix
what 0.16.0 flags: parenthesise the implicit string concatenations (ISC004),
use min() over sorted()[0] (FURB192), drop the now-unused S310 noqa on the
Request() builders (RUF100), and delete the narration comments that the
comment lint surfaces in the touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The mirror task staged package manifests for every role in the repo that ships
a package-lock.json, so a penpot deploy also pulled in unrelated apps such as
web-app-littlejs (nothing references its assets) and coupled every app's deploy
to their npm builds. Move the three staging steps into tasks/utils/stage_manifest.yml
and drive them from a single looped include gated on 'role_name in group_names',
so only the apps and dependent services actually deployed here are mirrored.
Collapses the per-task loop/when duplication to one include.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
generate_cache_override pinned deb.debian.org/archive.ubuntu.com/security.ubuntu.com
to INFINITO_CACHE_PACKAGE_FRONTEND_IP for every buildable service whenever that
env var was non-empty. The IP is a baked default.env value, so the hijack fired
even where no cache frontend exists: the guide job sources .env, replays a role's
compose build inside the DiD stack, and the moodle apt build hit 172.30.0.4 with
'No route to host' because no frontend is provisioned in CI.
Gate the override additionally on the frontend CA file existing non-empty, the
same signal ca.sh uses for an active cache profile and that registry_cache_active
produces. When the cache profile is inactive (CI/guide) the CA is absent, the
override is skipped, and the build resolves the mirrors via real DNS. compose.py
is copy-deployed and cannot import Profile, so a filesystem signal is required.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The asset-mirror task copied package-lock.json / package.json to
<CDN_ASSET_STAGE>/<role>/ but ansible.builtin.copy does not create the
parent directory, so every deploy failed with 'Destination directory ...
does not exist'. web-svc-cdn runs in every stack, so all swarm priority
jobs failed at deploy. Ensure the per-role staging dir exists first; the
<role> layout matches build_assets.sh globbing ${ROLES_DIR}/*/package-lock.json.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Formatter output from make quality: the git ls-tree argument list in
complexity.model and the xwiki job-name dict in the report_failures test
wrapped across lines. No semantic change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
report_failures.decisive_excerpt and new_in_branch._declares_application_id
now read via utils.cache.files.read_text instead of Path.read_text, and the
new_in_branch test constructs its vars path from ROLE_FILE_VARS_MAIN. Clears
the uncached-read and role-file-spot lint gates; behaviour is unchanged
(same UTF-8 decode, same exceptions, same resolved path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a main run, report-main-failures parses the failed deploy jobs, downloads
each role's rescue-diagnostics artifact, and opens a 'CI failure: <role>' issue
(label ci-failure) carrying the run link, the artifact names, and the decisive
rescue excerpt inline. An existing open issue for the role gets a comment
instead. Mode emojis are resolved through the symbol glossary, not hardcoded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
load.sh short-circuits on INFINITO_ENV_LOADED, so a shell that already
sourced the env kept a stale copy after 'make dotenv' regenerated .env
(make runs in a subshell and cannot reset the parent's guard), and a
freshly added key was missing until a new shell. Gate the short-circuit
on the .env mtime: re-source when it changed, still a fast no-op when it
did not. A stat failure falls back to the previous idempotent behaviour.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The compose and host discovery sorts already push clones (redundant
same-service-set variants) behind the budget cut so one representative
per dna cluster stays covered. Extend the same 'asc clone' prefix to the
swarm mode so its per-variant matrix drops redundant clone variants first
when the job budget is tight.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A push resolves the application roles that exist under roles/ locally but
not at origin/main and passes them as the orchestrator priority line, so
brand-new roles deploy and gate ahead of the regular matrix. The resolver
is stdlib-only (git ls-tree plus an application_id line scan) so it runs
on a bare runner without installing the resolver stack, and yields an
empty set when the baseline ref cannot be resolved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an in_main column (🌳) to the role complexity matrix: it resolves
the role directories present under roles/ at origin/main once and flags
every row true/false, so a role that is new on the current branch stands
out. When the ref cannot be resolved (a non-git checkout or an unfetched
main) the marker falls back to present for every role rather than
falsely reporting them new.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The packaging bootstrap and the main install/upgrade both download from
PyPI and fail hard on a connection reset. Wrap each in the standard
retry-until-succeeded loop (5 tries, 15s apart), matching the apt/dnf/
pacman/docker egress hardening already in place.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-package name/version guard used 'A && B || C', which shellcheck
flags because C also runs when A is true but B fails. Rewrite it as an
explicit 'if empty then continue' so the skip is unambiguous; behaviour
is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A forced rolling update keeps N-1 replicas' ORM sessions alive while the
replacement runs -u all, deadlocking on AccessExclusiveLock. Scale the
service to 0, wait until no running task remains, then scale back to the
topology replica count so the entrypoint's advisory-lock winner is the
sole database actor during the update window.
Renumber the task files to the dispatcher convention: 02_manager_ops
keeps its repo-wide name, its children follow as 03-07 in execution
order, and sub-steps move into a subdirectory named after their parent
task instead of letter suffixes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
host_deploy.sh drops its bespoke dump: the EXIT trap now runs
utils/diagnostics/container.py on failure before removing the boot
container, because the workflow's failure() collect step only fires
after this script has already torn the container down. That yields the
full snapshot (logs, inspect, in-container systemctl and journal, DiD
recursion) instead of three ad-hoc files, and successful runs skip the
capture entirely.
The compose-mode README replay moves out of test-guide.yml into
scripts/github/guide/compose_deploy.sh unchanged; the workflow step
becomes a one-liner.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The host-deploy EXIT trap now dumps failed units, the tail of the
journal, and the container log from the pkgmgr boot container into the
rescue-diagnostics base before removing the container, so a failed guide
run leaves a readable artifact instead of tearing down silently. The
base directory is hard-required from the workflow environment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
INFINITO_RESCUE_DIAGNOSTICS_BASE joins default.env as the only literal
of the runner-side collection root and is wired through the passthrough
handler into the generated .env. The four deploy workflows load it with
one grep bootstrap step after checkout instead of hardcoding the path in
their env blocks; upload paths and per-app DIR derivations reference the
variable. dedicated.sh sources it from .env alongside the existing
diagnostics keys and hard-requires it, no script-level default.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pacman's XferCommand curl gains --retry-all-errors --retry-delay 2 so
connection resets during package downloads retry instead of failing the
update; plain --retry does not cover curl exit 35. No --max-time here:
package downloads are size-unbounded and a ceiling would truncate large
transfers. dnf gets retries=5 via its native config, matching the apt
Acquire::Retries hardening already in place for Debian.
Settle the egress lint follow-ups: collapse the nested retry checks per
ruff SIM102 and reformat the curl filter tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A bare '/' does not recurse for manifest ecosystems, so the npm entry
missed every role-level files/package.json and composer/bundler were
pinned to single hardcoded paths. Use both glob spellings per entry:
'**' is proven in-repo to match root manifests (compose.yml PRs) and
'**/*' is the documented recursive form, so together they cover both
interpretations of the backend's glob expansion.
The commented-out ecosystem stubs stay as the intentional catalogue of
deliberately unscanned ecosystems; the file opts out of the comment lint
to keep them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add lookup('asset', app, pkg, path) and lookup('asset_host'): both resolve
the asset origin through one shared resolve_host, returning the internal
CDN domain when web-svc-cdn is deployed with flavor internal and
cdn.jsdelivr.net otherwise, so template URLs and CSP origins can never
diverge. Versions come from the role's committed files/package-lock.json.
Roles declare frontend dependencies in files/package.json; web-svc-cdn
aggregates every role lockfile, runs one npm ci per role in a node
container and mirrors the packages under /npm/<pkg>@<version>/ following
jsdelivr's path scheme, so internal and external URLs differ only by host.
Convert web-app-dashboard (keycloak-js 24.0.5 UMD) and web-app-littlejs
(bootstrap, fontawesome) to the asset lookup, drop the vendored keycloak
blob, and add a repository lint forbidding hardcoded external CDN hosts
in templates and CSP metas; pre-existing external CDN entries are checked
out with nocheck markers and a migration TODO.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two swarm defects in the same role. PeerTube has no upstream multi-instance support: the OIDC state cookie is encrypted with a per-process key and the RTMP live ingest binds a single instance with no VIP session affinity, so concurrent replicas broke the auth callback (AES-GCM 'Unsupported state') and would split live streams - pin replicas: 1 (jellyfin/xwiki precedent). Separately, port 1935 was published with no listener because live was off; enable live via a services.peertube.live.enabled config SPOT that gates both PEERTUBE_LIVE_* in the env and the rtmp port publish, so the published port and the RTMP listener can never diverge and the reachability probe passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
swarm's docker stack deploy drops the compose depends_on, and the penpot frontend nginx resolves its proxy_pass upstreams (backend, exporter) at config load, so when a peer service's VIP is not yet registered the frontend exits with [emerg] host not found in upstream and crash-loops under restart_policy any; the reschedules pile all three replicas onto the manager and the seed step finds no worker task. Override the frontend command to poll getent for both peers before exec-ing nginx, so all replicas start on the first pass and swarm spreads them one per node. The image entrypoint ends in exec "$@" and ships getent, so this is behaviour-preserving in both modes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Transient CDN/mirror TLS resets are a recurring hard-fail class (docker.com, PyPI). Harden all three egress paths at their SPOT: the curl filter now emits --retry-all-errors (retries=0 opts a call out), so every ansible curl retries; a new lint requires --retry-all-errors on file-downloading curl and --tries on wget in verbatim files/*.sh (which cannot use the filter), fixing the moodle/alias downloads it flagged; and the update role configures native retries for every package manager - apt Acquire::Retries, dnf retries=, pacman XferCommand over retrying curl - covering all five CI distros (ubuntu shares apt, centos shares dnf).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
download.docker.com intermittently resets the TLS connection; the shared apt_repo download helper carried curl --retry 3 without --retry-all-errors, so a transport-layer reset (exit 35) was never retried and hard-failed the play before the later variants ran. Add --retry-all-errors/--retry-delay to curl and --tries/--waitretry to the wget branch, and give the docker-package install and the post-repo apt-cache update the same retries/until hardening sys-svc-ssh already uses for nested CI egress.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pgAdmin under the role's 0.2 CPU / 256 MB limits needs ~144s to bind gunicorn, but the shared wget healthcheck renders start_period 30s, so swarm SIGKILLed the task as unhealthy and rescheduled in a loop until the converge wait expired at 0/N; compose never saw it because compose up runs without --wait. Set the template-local container_healthcheck_start_period to 10m, the same idiom a dozen other slow-boot roles use, which comfortably exceeds the ~343s converge cap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conflicts: keep both new passthrough env vars; adopt the swarm-nfs
ollama manager-ops split while porting our model-pull retry hardening
into it. Strip three narration comments the stricter comment lint
rejects.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When a run deploys both single-node modes, the host discovery redeployed roles the compose line already covers (e.g. web-svc-html in both matrices). The single-node workflow now signals a co-running compose line via INFINITO_HOST_EXCLUDE_COMPOSE (a run-context registry key like INFINITO_DEPLOY_MODE), and the host query excludes the compose line's actually-triggered selection before its own budget cut, so freed host slots go to roles compose does not cover. Priority and regular lines each filter against their own compose counterpart through the identical whitelist/blacklist pass-through; host-only runs stay unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The guide matrix rolled debian, whose image ships neither libnss_myhostname nor a myhostname nsswitch entry, so *.infinito.localhost fell through to DNS, coredns only knows the outer stack's domain, and hlth-webserver died on NXDOMAIN; arch passed because its base enables nss-myhostname out of the box. Install libnss-myhostname on the Debian family and append myhostname to the nsswitch hosts line idempotently, also inside containers - mirroring the host-mode guide fix. Probed across all five CI distros: fedora/centos/arch already resolve out of the box (both tasks no-op), debian/ubuntu gain exactly the missing pieces.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-node dnsmasq answered the app-domain wildcard with 127.0.0.1 on the manager; node dockerd hands that resolver to every task container, where loopback is the task's own netns, so any healthcheck reaching for a published service port (friendica's msmtp send to mail.<domain>:465 with email enabled) got connection-refused although the port was ingress-published and mailu healthy - manager-placed replicas could never become healthy and the stack sat at 0/3 until the converge wait expired. Answer with the manager's lab IP for every consumer; host-side processes reach published ports through it identically.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The playbook covers reset, reinitialisation and config debugging itself; the manual recipes used raw docker commands and hardcoded volume paths that no longer match the deployment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The upstream entrypoint rsyncs VERSION early inside the flock critical section it holds through the whole autoinstall, so non-slot-1 tasks passed the version-only gate mid-install, hit the still-held lock and died with an instant EIO from the blocking flock over the soft NFSv4 mount (six tasks inside slot 1's 45s install window). Strengthen the gate: require the config file and a non-blocking flock probe (single lock RPC, any error treated as not-ready) on top of the version match, which also covers the post-upgrade dbstructure window. The config path now flows from the FRIENDICA_CONFIG_FILE var through the compose environment into the gate and both container-side PHP helpers instead of three hardcoded copies.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The node-local matomo_tmp volume populated as root on any node that had not run the first-install entrypoint: the matomo image ships /var/www/html empty (code lives in /usr/src), and the entrypoint's chown -R www-data runs only when matomo.php is absent, which on a rescheduled task is never the case because matomo_data is shared NFS. The rescheduled task then died on 'tmp/cache/tracker could not be created', pausing every forced service update. Build a custom image whose /var/www/html/tmp is www-data-owned: docker volume populate replicates the image path's ownership into every fresh volume on every node (proven via moby populateVolume/CopyDir and a local reproduction), so reschedules get a writable tmp deterministically.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bootstrap task kept a hand-picked partial copy of the canonical nginx directory list that omitted conf.d/global and the servers/maps/streams/lua subtree. Roles that write http-includes before the front chain runs (prometheus's http-block in the bare variant 1, where only openresty itself is preloaded) hit 'Destination directory /etc/nginx/conf.d/global does not exist'; variant 0 masked the ordering because preloaded shared services pull sys-svc-webserver-core first. Loop over the canonical lookup('nginx','directories.ensure') SPOT plus the existing non-nginx extras, establishing the invariant that a provisioned openresty always owns its complete config tree.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Priority deploys gate via implicit success() on new lint/test needs.
Regular deploys carry always() in their if, so needs alone would only
order, not gate; explicit result checks make red lint/test skip them.
done keeps lint/test as unconditional backstop since the skip cascade
would otherwise turn red lint/test into a green pipeline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
crs-k/stale-branches rejects branches-filter-regex over 50 chars;
equivalent lookahead-only form is 49 and needs no trailing .* since
the action uses RegExp.test without anchoring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both the priority and the regular deploy line received the full per-mode slot budget, so a run with a priority input budgeted up to twice the mode share against the 256-job run limit. max_jobs now subtracts the jobs the priority line actually occupies - the regular line's blacklist is the priority whitelist, and the inner discover runs under the identical cap, so the deduction mirrors the real consumption including the priority line's own cut. A fully consumed budget short-circuits discover to an empty selection without invoking the complexity CLI (which rejects --max-jobs < 1), and the plan summary renders the reduced regular budget.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sort the new INFINITO_SWARM_* IP/subnet keys alphabetically in default.env, wire them through the static passthrough handler so make dotenv materialises them, and shorten the matomo DB-reset task name under the 80-char limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a fresh dedicated database (variant 1) the owner probe hit n8n's startup splash 22s after force-recreate: HTTP 200 text/html, which the probe read as 'owner exists', so owner setup was skipped vacuously and Playwright met the first-run wizard instead of the login form. Wait on /healthz/readiness (503 until the DB is connected and migrations ran) before bootstrapping, and accept only JSON responses in the owner probe - the splash is text/html, so the until-gate turns the previously inert retries into real ones and fails loudly if the splash persists.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mailu 2024.06 moved listener gating into socrate's PORTS env default (25,80,443,465,993,995,4190), which omits 110/143/587, so dovecot rendered port=0 for pop3/imap/submission and the front published those ports without listening on them; the swarm reachability check probing the real Endpoint.Ports failed exactly on that trio while the TLS twins worked. Render PORTS from the services.mailu.ports.internal SPOT that already feeds container_ports, keeping publish and listen in lockstep from one source (ssl=required stays, so STARTTLS is still mandatory).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The single-app compose job deploys nothing containerized (all svc-db/svc-prx skipped by the whitelist, DB seed off), so docker volume ls is empty and baudolo stamps an empty backup generation; verify_backup.sh then failed deterministically on 'no volume payload'. The test only ever passed piggybacked on app hosts with real volumes. Probe the host's backup subjects after the forced run: with zero volumes and zero compose containers, assert positively that the newest generation is empty (an empty generation is the correct outcome on a bare host) and skip the restore drill, mirroring the existing non-stack-host gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Matomo's swarm bootstrap failed rc=1: the high-churn tmp/templates_c Twig cache lived on the soft NFS state share, so a single mid-write RPC timeout surfaced as EIO and stalled the installer 300s; the DB was already seeded, so retries took the eraseAllTables path that wedged Matomo on the CoreUpdater screen the bootstrap image cannot click through. Mount a node-local matomo_tmp volume (nfs: false) for /var/www/html/tmp so caches never traverse NFS, and drop+recreate the app's own database in the pre-retry cleanup (via a container-run reset_database.php over the app credentials, gated exactly like the config-aside task on attempt>0 + no installed config) so every retry takes the clean-install path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The upstream entrypoint serialises its first-run init with flock on friendica-init-sync.lock inside the shared /var/www/html, which sits on NFSv4 on this branch; local_lock=flock is silently ignored on vers=4 mounts, the lock contends across replicas over server-side locking and the set -eu entrypoint dies with exit 65 on every worker, pushing all replicas onto the manager. Build a custom image whose entrypoint gate lets only task slot 1 run the init while other slots wait until the volume VERSION matches the image VERSION - the same pattern nextcloud already uses.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ansible-core 2.21 templates delegate_to before evaluating a task-level when, so the pre-pull task's delegate_to OLLAMA_HOST_NODE hard-failed as undefined on worker nodes even though its when would have skipped it; the resolver include with the identical condition had correctly skipped there and never set the fact. Move the resolver and the pre-pull into 02_manager_ops.yml behind a single gated include_tasks - a skipped dynamic include never queues children, matching the pattern mattermost, openproject, fider and decidim already use.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every DinD node is attached to a second network by the bootstrap cache connect, so the embedded DNS returns two A records per node name in undefined order; workers that resolved the manager onto the cache network could not reach the registry bound to the lab IP and rejected locally built images with 'No such image', pushing all replicas onto the manager. Pin a fixed lab ipv4_address per node and every node name via extra_hosts (files-first resolution wins over DNS); the subnet and IPs live in default.env as the INFINITO_SWARM_* SPOT consumed by compose interpolation and the topology exports. Proven by A/B: the identical friendica variant-2 swarm job went from worker rejects to fully green including the drain test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The map('split') chain returns an empty list on slurp-origin tagged
strings under ansible-core 2.20, so the dnsmasq forwarder was left with
no upstream and every clearnet lookup (image pulls, ACME) got REFUSED.
regex_findall reads the nameservers reliably.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
