Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A run-wide draw proved one distribution and said nothing about the other four, while reporting green. Both are matrix-row axes now, assigned by the same deterministic rotation mode and tor already use, so one sweep exercises every distribution and every filesystem.
The two read the row's position like an odometer, the distro as the low digit. Turning both on the position directly covers only n of the n x m pairs whenever the pools are the same length, and no sweep unlocks it.
Both are part of a row's identity: glyphs in the job label, columns in the plan table, shards in the artifact name. Both are selectable through %distro and /filesystem, and a retrigger replays the distro off a failed title. It leaves the filesystem out, because a title states the kind the matrix assigned and a deploy may have fallen back from it.
A row's filesystem is a preference unless a human names it, on the run or in a token. Reading the matrix's own narrowing as a demand would fail rows for every transient condition the applying step is built to tolerate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
i8cifail died twice on the same run, each failure hiding the next.
The retrigger recovers a source run's dispatch inputs from a job log, because the title truncates at 512 bytes and REST answers inputs: null for a finished workflow_dispatch. That reader picked the first depth-1 job outright. A skipped job never uploads a log blob, so its log endpoint answers BlobNotFound as a bare HTTP 404 -- and a force-cancelled run can leave every depth-1 job skipped, which is why deeper jobs are now read too rather than assumed inputs-free. Candidates are taken shallowest first, so the orchestrator's own inputs still win over what a caller forwarded. gh 2.97 also refuses to emit a log holding terminal escape sequences and exits 1; the log downloader read that as no log and silently wrote none. A missed fetch is now tolerated instead of aborting, so the guard in dispatched_priority reports an expired log in words rather than the command dying on gh: HTTP 404.
Behind that sat the resolver: it decoded whole priority entries, but an entry may pin axes (web-app-zammad#1@compose+tor), which the display-name codec does not parse. Every pinned entry aborted as corrupt. split_axes -- already the seam decode_list uses -- separates name from suffix, so only the name is resolved and the truncation guard stays sharp. The pin rides along into the retrigger: the entry never ran, so the run holds no evidence its axes were wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A recent refactor imported baudolo into utils.recovery for three layout strings, and cli.administration.recover.__main__ imports full unconditionally - so the whole recovery CLI, --help included, died at import on every host without baudolo. That is most of them: recover device|nfs|secrets runs on swarm nodes, the NFS server and bare rescue hosts, while only the backup node carries svc-bkp-volume-2-local. utils/recovery had been deliberately stdlib-only for exactly that reason.
baudolo 5.0.0 writes a manifest into every generation stating the layout it used, so the names are now asked of the generation being read rather than of the installed package. A tree written by any version is walked with the names that version used, and utils.recovery.layout answers only for generations older than the manifest, which cannot describe themselves. That also retires two of core's guesses: the '<entity>_database' naming convention now yields to the engine the run actually recorded.
A repo whose manifest cannot be read is skipped rather than guessed at, but no longer takes the other repos with it - recovering two of three beats recovering none because one manifest is corrupt. The exit code reports the gap.
The pin lint compared two declared version strings and was therefore green while a different distribution was installed. It now checks the installed distribution and the layout literals against baudolo's own BackupPaths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
utils/recovery restated three things baudolo owns: the generation layout (suffixes, sql/ and files/ directories), the \connect parser for cluster dumps, and the databases.csv reader. Each was a second implementation of a contract whose author is one pip install away - the connect parser had even received the same quoted-identifier fix twice, once per repo.
The layout constants are now derived from BackupPaths, the inventory comes from dump_inventory, and the CSV goes through read_rows. dump_inventory is stricter than the parser it replaces: it also reads CREATE DATABASE lines, so a database a stream creates but never reconnects to is found - a new test pins that. recover/full.py and the drill's token.py take their globs from the same constants instead of spelling them out.
The dependency this creates is declared: backup-docker-to-local==4.0.0 in the base dependencies, because the deploy CLI must import from the base set alone. The role installs the same version, and a lint holds the two pins together - they drifted apart is exactly the failure this whole commit removes, so it must not come back one level up.
The generation-layout tests now write their fixtures through BackupPaths itself; before, the fixture hand-built the paths with the same assumption the test claimed to verify.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A backup generation stores databases as sql dumps, not as a file tree
(--dump-only-sql), and the recover chain only ever restored trees:
`git grep sql HEAD -- cli/administration/recover utils/recovery` was
empty. An operator running `recover full` after a real loss got their
files back, their databases untouched, and an exit code of 0.
Adds the missing leg as a fifth recoverer, planned right after the
volume steps so the dumps land once the volumes are back. Three
properties make it a restore rather than an attempt:
- It refuses while a consumer of that database runs. `--empty`
pre-cleans the schema in one psql session and replays in the next; a
booting consumer recreates it in between and the dump's own CREATE
TABLE fails. That is what turned CI red with `relation
"ar_internal_metadata" already exists`.
- The engine is read, not guessed: the service key in the consuming
role's meta/services.yml, the same value that fed the backup seed.
The two previous call sites each sniffed it from the container image
and could disagree.
- The dump's own header states which server it came out of, and that is
compared against the running engine before the pre-clean drops
anything. Forward across a major version is the upgrade path and
stays allowed; backward is refused. The version deliberately does not
travel in databases.csv: that file is written at seed time and would
state the version deployed back then, and a generation copied to
another host carries its header along while a csv row stays behind.
Mind the header formats, they are not symmetric - postgres puts
`Dumped from database version` on line 7, while mariadb's line 2 opens
with mariadb-dump's own version and only the tab-separated `Server
version` line names the server. Both are pinned as test fixtures,
captured from postgres:17-alpine and mariadb:11 themselves.
baudolo is untouched: nothing beyond the credentials it already carries
is read from databases.csv, and no upstream format changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four findings the quality gate raised against the offset and resume work, all
of them mine.
INFINITO_CI_OFFSET sat next to INFINITO_CI_CHUNK, where it reads well and
sorts wrong, and no handler claimed it -- so `make dotenv` would never have
materialised the default and every run outside CI would have resolved the
offset from an unset variable. It is in ASC order now and passes through the
static handler.
runs.py crossed 500 lines. The three helpers that translate job titles back
into selection tokens move to selections.py, which is a seam rather than a
quota: reading a finished run and dispatching a new one is one subject,
turning a job title into `role#variant@mode+tor` is another, and only the
latter is what lets a retrigger replay exactly what failed.
The sshd handler still told operators to look for a handler named
'sshd restart' after the topic became sshd-restart, and three lines from the
offset commit were past the format width.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A retrigger started the regular line at the head again, redeploying the exact
window the source run had already judged. The red rows come back on the
priority line anyway, so that window was runner time spent on questions with
an answer.
The offset is computed now: the last row of the leading run of rows the source
run reached, as a selection token. Green or red does not matter -- what matters
is that the row ran. The scan stops at the first gap, because a hole inside the
window means that row was filtered out, not that the run got further, and
resuming past it would skip everything behind it.
The ranking is recomputed under the configuration the retrigger carries, since
that is the list the offset resolves against. The priority line is left out of
that computation on purpose: it only blacklists rows from the regular query,
and a token in it that no longer resolves would abort the very helper whose job
is to save time.
offset therefore joins priority as an input the retrigger decides rather than
carries. The integration test forced that: it asserts every carried input
arrives unchanged, and offset no longer did. The category was wrong, not the
test.
Costs a full local discovery per retrigger. There is no cheaper answer -- jobs
inside a chunk are sorted by name, so how far a run got cannot be read off the
job list without the ranking.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
runs.py stood at 568 lines against the 500-line cap from 4b37e123d. The git and gh helpers were the clearest seam in it: eight functions that know nothing about deploy runs and only drive subprocess and the gh CLI. They move to cli/administration/deploy/ci/gh.py, leaving 482 lines behind. The module is deliberately not named runs_gh.py, which would have formed a prefix group with runs.py and tripped the structure lint.
Re-exporting the moved names from runs.py does not survive the formatter: autoformat strips them as unused imports, and seven call sites in status/__main__.py and trigger/__main__.py would then raise AttributeError at runtime. The call sites therefore reach into gh directly, and the mock targets move with them -- patching a name in the module that no longer resolves it makes a test green without testing anything.
fetch_jobs stays reachable through runs, because find_last_deploy_run calls it internally and test_status.py covers both the URL and the branch path with a single patch. Splitting that name would have left one of the two paths unpatched.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A retrigger answered a red run with role names. Three things were lost on the
way, and each of them let the rerun test something other than what broke.
The mode was a scope: --failed swarm dropped whatever fell in compose, and two
aliases passed that scope permanently. Every mode is read now, and a leftover
scope argument is accepted and ignored rather than breaking the callers.
The selection was a role id, so the rerun deployed whichever variant, mode and
onion state the sweep rotation picked next -- not necessarily the combination
that failed. Each red job now contributes its own token
(web-app-x#0@swarm+tor); a role that failed in two modes comes back twice.
The carried configuration was a hand-written list of eight inputs against the
thirteen the form declares, so resume_from_chunk, sweep, workspace_track and
whitelist silently fell back to defaults. The list is read off
entry-manual-steer.yml now, and everything but the priority line travels
verbatim, values taken from the job log that records them all.
That rule is deliberate and total: it supersedes the suite-skip behaviour on
fork/feature/ai-agent-employees (b1ca39e9d), which turned workspace and
instructions off when the source run had passed them. A retrigger reproduces
its source; deciding which suites are worth re-running is not its job.
The integration test walks the workflow's own input declaration, so an input
added to the form without being carried fails here rather than in a run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The node identity is minted during the first distro leg's deploy, as root inside the DiD on the bind-mounted workspace, mode 0600. The workspace survives between legs, so the second leg's 'pack the repo once on the controller' runs tar as the runner user and aborts on all three key files with EACCES -- 13 seconds into the play, before anything of the role under test has run.
Five swarm jobs of run 31864448027 die exactly there: svc-db-postgres, svc-db-openldap, svc-db-redis, web-app-matomo and web-svc-cdn. First leg 3313-5778s and green, second leg 125-140s and dead, whichever distro it happens to be. The remaining swarm failures are not immune, only shadowed: they die in their first leg for unrelated reasons and never reach a second.
Ownership follows the tree, not the process that happened to mint it. The repair runs on the reuse path too, where ensure_node_onion returns early and a root-owned key set from an earlier run would otherwise never be corrected. Modes stay 0600. lchown so a symlink cannot redirect the change, and a path that vanishes mid-walk is skipped instead of aborting the provisioner.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The applications: host_vars block is the config namespace keyed by role id, so it also carries system services (sys-ctl-hlth-csp, sys-svc-container). Those declare system_service_id and match no application role, and the resolver died on them instead of reporting their absence.
role_name now raises RoleNotFoundError and exits 3, distinct from argparse's usage error (2) and from a genuine failure (1). run_subprocess grew ok_returncodes so a caller can accept an expected non-zero status while every other failure still surfaces, and resolve_role_path returns None for that code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CONFIG_INPUTS lost modes, sequencing and mode_fail_fast when the deploy
modes collapsed into one chunked sweep, but two mirrors of it did not
follow.
The --run help text enumerates what a retrigger carries over, so it was
telling the operator about three inputs the dispatch no longer has. The
job-log fixture described a run recording mode_fail_fast, a state the
dispatch can no longer produce, which left the log half of
config_from_run asserting on an input nobody writes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
report_failures matched job names against a raw role id
((?:web-app|web-svc|svc|sys)-[a-z0-9-]+), but job titles have carried
display names for a while, so the regex matched nothing and every failed
deploy on main went unreported. runs.py had the same shape and stopped
at the first mode glyph, which swallowed a caller path like
'z / Host / sys-front-proxy' into the role name.
Both now go through axes.parse_label, kept next to the function that
builds the label so the two cannot drift, and resolve the middle through
the display-name codec. The parser anchors on the LAST mode glyph so a
reusable-workflow caller path carrying one of its own cannot capture the
role.
The onion state is part of what identifies a job, because a priority row
runs the same mode and variant twice; artifact_name takes it so the
reporter looks for the artifact the run actually uploaded.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
compare_application_keys skips generated credentials because no default exists
for a secret, but it recognised them by the pre-rename prefix. Once the
generator started writing them again, every credential arrived as
secrets.credentials.<key> and the exemption stopped matching, so validation
rejected three keys per credential -- the leaf and both intermediate nodes,
which recursive_keys emits separately -- and aborted the deploy before the
playbook ran. The condition now admits the bare secrets node and the
secrets.credentials subtree, leaving a sibling such as secrets.other flagged.
The check in the inventory manager stays on the bare prefix on purpose: it
walks meta/secrets.yml, whose own root key is credentials.
test_credentials_keys_are_skipped fed the validator the old shape, so it kept
passing while real inventories were being rejected.
Evidence: run 31655559799, job 94311023723,
"Missing default for web-app-dashboard: secrets.credentials.sso_proxy_cookie_secret".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
provision and credentials are sibling packages, so reaching the emit helpers
with a parent-relative import tripped TID252. Same-package imports stay
relative, as everywhere else in the package. The blank line before the
role_resolver import keeps the first-party and relative groups apart, which
I001 requires.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The host_vars merger still wrote applications.<app>.credentials and read
"credentials" out of the generated snippet. f713ba835 moved the emitter to
secrets.credentials, so the read resolved to {} on every app: nothing was ever
copied, and _ensure_ruamel_map left an empty credentials: {} shell behind as
the only trace. Every credential generated since that commit was discarded on
its way into the inventory.
Both sides now go through credentials_map() and declared_credentials(), the
accessors the emitter already uses, so the two ends cannot drift again.
_ensure_ruamel_map was a copy of emit.ensure_map and is gone.
The failure surfaced in two shapes from the one cause. Templates reading
through lookup('config', ...) raise, because get() defaults to strict:
"Key 'secrets' not found in dict at 'secrets'" for
web-app-mastodon.secrets.credentials.secret_key_base, with the dumped payload
showing 'credentials': {} and no secrets node. Templates reading through
lookup('database', ...) stay quiet, because that plugin passes strict=False
with default="": friendica's MariaDB received an empty MYSQL_ROOT_PASSWORD and
refused to initialise.
The generator's own tests pinned the pre-rename shape in both the snippet and
the assertion, which is why they stayed green through the regression.
Evidence: run 31649221474, job 94303863545.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
PASS 2 read the exact values PASS 1 wrote, so no deploy ever had to propagate a secret that moved. Every role that only writes a credential at creation time therefore stayed green without the update path being exercised once.
Both orchestrators now call `inventory credentials reset` after the recover drill and before the async pass: compose through the container in the new .rotate module, swarm through the matrix orchestrator with --mirror, because its inventory keeps one host_vars copy per node. --backup keeps the pre-rotation file for the post-mortem, since after a rotation nothing records what PASS 1 deployed with. `administrator` stays exempt in both: its password is ansible_become_password, and rotating it would lock the deploy out of its own host.
Expect roles that never re-apply a changed secret to fail PASS 2 from here on; that is the gate working. Both database engines already converge their root password on every run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Provisioning fills a missing credential and leaves an existing one alone, so every deploy pass reads the value the first pass wrote and the update path is never exercised. Rotating the generated values makes a redeploy carry a secret that moved.
`infinito administration inventory credentials reset` drops every vault-encrypted leaf under applications.*.credentials.* (--schema) and the pinned password of every declared user (--users), then lets the existing generators refill the gaps, so no schema logic is duplicated. Plain credentials stay unencrypted in host_vars and are therefore left alone, as are users that exist only in the inventory and would get nothing back. --except exempts application ids and user keys, --backup keeps the pre-rotation file as <name>.YYYYMMDDTHHMMSSZ.backup, and --mirror realigns the other hosts of a multi-host inventory.
Adds the command only; no deploy path calls it yet. Measured against web-svc-coturn and web-app-flowise: eleven generated secrets move between two runs while ansible_become_password and the administrator password stay pinned.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.
The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.
--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.
Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.
tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.
Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A user password is pinned once by inventory creation from a shell-safe
alphabet, because it travels through Ansible, a shell and container exec.
An application whose registration demands a special character then had no
way to say so, and roles worked around it with a second credential beside
the account they already owned. The account now carries algorithm and
validation itself, two roles disagreeing about one account abort instead of
one silently winning, and an unsatisfiable pair fails naming both halves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.
Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.
Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:
- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
Same value, so nothing breaks today; editing the documented one would have
silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
rename never saw it. Its credential would not have resolved and the role
would have deployed with an unusable administrator account.
The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.
make quality green on the merge result.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A retrigger reproduces its source run's configuration, and the tor axis was
missing from it: rerunning a run dispatched with 'exclusive' silently fell back
to 'auto' and tested a different matrix than the one whose failures it was meant
to chase.
The mode is read from the job log's Inputs block, not from the run title. The
title renders it as a glyph, so recovering it there would mean translating a
symbol back into a word on every retrigger; the log already holds what the
operator picked, and inputs_from_jobs reads that same block for the priority
list today. LOG_INPUTS names the inputs that take this route.
That also removes a trap the glyph created. The tor segment renders through
format('{0} ', ...) with no opening literal, and run_name locates a value by
searching for its head - str.find('') matches at position 0, so the parser would
have handed back the title's own prefix as the tor value, plausibly and without
raising. segments() now skips a headless segment; its glyphs stay openings, so
they still terminate the value before them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A retrigger carried the workspace and instructions toggles over from the
run title, so a suite that had already gone green was dispatched again
and spent runners on work with a known answer.
Derive each toggle from the source run's job record instead: a suite is
turned off only when every one of its jobs reports success. A suite that
was skipped, cancelled or never dispatched has not passed, so it is still
asked for; absence of a failure is not evidence of a success. One failing
shard keeps the whole suite on.
Match against the job path rather than the run title by dropping the
workflow-name segment, and fail open when nothing matches, so an unknown
job layout re-runs the suite rather than silently skipping it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The docstring promised the narrow form "{{ '<role>' in group_names }}" and
pointed at tests/integration/roles/meta/test_services_dynamic_flags.py, a path
that does not exist. The helper substring-matches "in group_names", so a
negated predicate matches too -- and that is deliberate: such a flag is true on
some node shape, and the static closure is the co-deploy superset, so the
dependency must resolve at tooling time either way.
tests/integration/roles/meta/services/test_dynamic_flags.py:194 asserts the same
substring rule, which is the real contract.
Reported during triage as a latent defect (seaweedfs' frontend is gated on
"not in group_names" and reads as enabled on a tor node). It is not one; the
description was. Tightening the matcher would have changed behaviour against the
test that pins it, so that is filed as won't-fix and only the wording changes
here. The README table gains the row it was missing.
No executable line in this diff.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A branch with no tracking remote fell straight through to origin, so on a fork checkout every run lookup asked the upstream repository and answered HTTP 404. i8cifail could not reach a single run of this branch.
Consult remote.pushDefault between the tracking remote and origin. That is git's own answer to where the branch is pushed, hence where its runs live, and the i8puma alias already reads exactly this key — the two resolution paths now agree instead of contradicting each other.
The fallback chain had no test at all: every existing case mocks resolve_repo away. Four cases now pin it, including a pushDefault naming a remote that no longer exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 123794c7f4)
A branch with no tracking remote fell straight through to origin, so on a fork checkout every run lookup asked the upstream repository and answered HTTP 404. i8cifail could not reach a single run of this branch.
Consult remote.pushDefault between the tracking remote and origin. That is git's own answer to where the branch is pushed, hence where its runs live, and the i8puma alias already reads exactly this key — the two resolution paths now agree instead of contradicting each other.
The fallback chain had no test at all: every existing case mocks resolve_repo away. Four cases now pin it, including a pushDefault naming a remote that no longer exists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
mkstemp creates its file 0600 and Path.replace carries that mode into the target, so every freshly rendered Corefile was owner-only while coredns runs as nonroot. The container exits with 'open /Corefile: permission denied', name resolution dies for the whole stack, and deploys then fail at whatever they try to fetch first.
Long-running stacks hid it: coredns reads its config once at startup, so a container created before a re-render keeps working. Only new worktree stacks broke.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit efc281e327)
mkstemp creates its file 0600 and Path.replace carries that mode into the target, so every freshly rendered Corefile was owner-only while coredns runs as nonroot. The container exits with 'open /Corefile: permission denied', name resolution dies for the whole stack, and deploys then fail at whatever they try to fetch first.
Long-running stacks hid it: coredns reads its config once at startup, so a container created before a re-render keeps working. Only new worktree stacks broke.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GitHub cuts display_title at 512 UTF-8 bytes and closes it with a
literal '...'. The run-name template renders priority second-to-last
and whitelist last, and on the --failed path the whitelist is empty, so
the cut always lands inside the priority list: 21 of 69 roles survived
the source run's title, the 21st a half-token.
untriggered_priority parsed that title and passed undecodable names
through verbatim, so the fragment reached the dispatched priority
input, became the orchestrator's whitelist, and hit the complexity
filter's lexer, whose word class has no rule for CJK. Sequencing died
with argparse exit 2 and took the run with it. The retrigger that is
supposed to repair a red run was breaking its successor.
Read the priority from the ##[group] Inputs block a called job's log
records instead: it carries all 69 roles. Depth-1 jobs only -- a deeper
one echoes its own workflow's inputs, which have no priority.
Drop the decode passthrough now that the source is lossless. The loud
failure is only affordable once a whole value exists; before, it would
have aborted every large retrigger.
dispatched_priority cross-checks against the title: it still proves a
priority line existed even when truncation mangled it, so an empty log
read is a broken reader -- expired logs, a moved block -- and not a run
without priority. Without that check the carry-over would vanish
silently, which is what the rest of this change exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GitHub cuts display_title at 512 UTF-8 bytes and closes it with a
literal '...'. The run-name template renders priority second-to-last
and whitelist last, and on the --failed path the whitelist is empty, so
the cut always lands inside the priority list: 21 of 69 roles survived
the source run's title, the 21st a half-token.
untriggered_priority parsed that title and passed undecodable names
through verbatim, so the fragment reached the dispatched priority
input, became the orchestrator's whitelist, and hit the complexity
filter's lexer, whose word class has no rule for CJK. Sequencing died
with argparse exit 2 and took the run with it. The retrigger that is
supposed to repair a red run was breaking its successor.
Read the priority from the ##[group] Inputs block a called job's log
records instead: it carries all 69 roles. Depth-1 jobs only -- a deeper
one echoes its own workflow's inputs, which have no priority.
Drop the decode passthrough now that the source is lossless. The loud
failure is only affordable once a whole value exists; before, it would
have aborted every large retrigger.
dispatched_priority cross-checks against the title: it still proves a
priority line existed even when truncation mangled it, so an empty log
read is a broken reader -- expired logs, a moved block -- and not a run
without priority. Without that check the carry-over would vanish
silently, which is what the rest of this change exists to prevent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
mkstemp creates its file 0600 and Path.replace carries that mode into the target, so every freshly rendered Corefile was owner-only while coredns runs as nonroot. The container exits with 'open /Corefile: permission denied', name resolution dies for the whole stack, and deploys then fail at whatever they try to fetch first.
Long-running stacks hid it: coredns reads its config once at startup, so a container created before a re-render keeps working. Only new worktree stacks broke.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Parallel `make test` targets each auto-ran `compose-up`, which rendered
into a fixed Corefile.tmp; the first replace() stole the file from the
others and surfaced as "envsubst did not produce an output file".
mkstemp per process plus a flock'd re-check in the test wrapper so only
one target brings the stack up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The workflow directory mixed entry points, reusable stages and scheduled jobs into one flat list sorted by topic, so the first question about any file - what starts it - was the one the name did not answer. Every workflow now carries one of three prefixes, derived from its on: block: cron- for a schedule trigger, call- for one only reachable through workflow_call, entry- for everything a repository event or a human starts. Scheduled workflows additionally carry an alarm clock as the first character of their name.
Two lints keep it that way. One derives the expected prefix from the triggers and compares it against the file name; it flagged all 17 offenders before the rename and none after. The other resolves every .github/workflows/<file> string in the repository against the directory, which is what a rename silently breaks in Markdown links, Python constants and shell API URLs alike.
Renaming pulled 69 pre-existing comments into the comment lint's changed-file scope. The vendored CodeQL template got the nocheck marker its scorecard sibling already carried, two workflow headers moved above name: where they count as file headers, and the remaining banners were deleted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every conflicted hunk resolves onto the swarm-nfs side, because on all six
files it is the strict superset: it carries the display-name codec
(utils/roles/display.py) the CI role selections now encode through, and the
sequencing glyph move from 910748ec4. svc-net-tor holds no unique content on
any of them -- runs.py even had non-conflicted hunks already calling `codec`,
so keeping its side would have left a NameError.
The lint fallout is settled in the same commit, as 4afc9b770 did for the
previous merge: the shell-or-true lint lives on svc-net-tor (c505ce746) and
swarm-nfs brings two unmarked `|| true` sites with it. Both get a nocheck
reason rather than a rewrite, because neither swallows anything -- the
slaptest call is the measured exception of c4bbbba93 and has_cn_config decides
right below it, and the dockerd restart is followed by an is-active poll that
fatals. In both the exit code was never the authority.
make quality green end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One conflict, in the category catalogue: this branch added a virt category for
the sandboxed container runtimes, the other did not.
Keeping our side alone would have closed the conflict and broken the build. The
same merge introduces a test requiring every category to declare a hanzi label,
unique among its siblings, and virt predates that rule and carries none. It now
declares 沙箱, checked against its ten siblings for a collision.
A user password reaches its application through Ansible, a shell, a container
exec -e and finally the runtime's own environment reader. The generator seeded
it from the same alphabet as the become password, punctuation included, and
that does not survive the chain intact.
Three roles had already worked around it by declaring a separate alphanumeric
credential for an identity that their meta/users.yml already declares, and
saying so in the description. Removing those without fixing the generator would
have reintroduced exactly the bug they were built against, so the generator is
fixed first and all three now read the user's own password. At length 64 the
smaller alphabet costs nothing worth measuring, and the become password keeps
the old one because it travels a different path.
Zammad additionally used the bot's password for its wizard-bypass admin, one
secret for two users. Each has its own declaration and now its own value.
A test asserts the generated password is alphanumeric, so a future change to
the alphabet fails there rather than inside a Ruby ENV.fetch in a container.
The category labels now run together into a single block instead of being
separated from each other, the glyph sits flush against that block, and the
only `·` left is the one between the categories and the role's own fragment:
`🐳网络应用·Nextcloud` where it used to read `🐳 网络·应用·Nextcloud`.
The deploy-job parser drops the whitespace requirement behind the glyph in
the same move. Without it no job name from a fresh run matches any more, and
the retrigger would find zero roles without reporting an error.
The name still never holds a space, so the heading keeps collapsing its own
whitespace to `·` and the space-separated whitelist and priority inputs stay
parseable with more than one role. Empty parts are dropped when joining, so a
role with no category at all (pkgmgr, test-e2e-*) does not gain a leading
separator.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI deploy jobs were titled with the raw role id, so a run page read as a
column of dash-joined prefixes. They now carry a display name: the category
path as hanzi labels, then the role's README heading or the part of the id
the categories do not already say, whichever is shorter.
utils/roles/display.py owns both directions. Encoding happens once, at
discovery time, as a `label` key on every matrix entry; the workflows render
`<glyph> ${{ matrix.label }}` and keep the glyph as a literal because the
retrigger reads the deploy mode off it.
Decoding is what makes the retrigger keep working. It accepts a display name
with or without its variant shard and a bare role id unchanged, so job names
from runs that predate this still parse and a hand-typed dispatch never has
to spell hanzi. The parser therefore stopped matching an id-shaped tail and
asks the codec instead; a job the codec does not recognise is skipped rather
than counted as a role.
The role selections go out as display names too, so the run title reads like
the jobs it spawns. Four consumers decode them back, each the single place
the value is actually used: the whitelist resolver before it checks the role
directory, the discovery query on its env, and the sequencing and plan CLIs
on their arguments.
Every fragment is joined by `·`, including before the title, and the heading
loses its whitespace and any emoji variation selector. A display name holding
a space would make the space-separated whitelist and priority inputs
unparseable with more than one role.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A user definition that declares no password falls back to
{{ 42 | strong_password }}. That is a template, not a value: it is re-rendered
at every use site, so the task that creates an account and the task that later
authenticates as it read two different secrets. The account exists and the
login fails. Inventory creation now decides the value once, after the roles are
resolved and next to the role credentials that were already generated there.
The scope follows the deployment: only roles resolved into this inventory
contribute users, an existing password is never overwritten, and a second run
writes nothing.
Building it duplicated what the package already carried three to seven times
over: an ensure-map helper defined three times, the same fifteen-line
round-trip load repeated seven times, and the vault encrypt-and-reparse dance
twice. They now live in one module, the round-trip counterpart to the existing
read-only yaml_io. host_vars drops from 213 to 130 lines, and its two override
entry points share the merge they used to copy. Behaviour is unchanged, and
that is measured rather than asserted: the committed versions and the new ones
were driven against the same host_vars and produced byte-identical output for
all six writers, comment placement and vault block layout included.
The adapter probes move into their own folder because a shared file-name prefix
reads as a folder in disguise, and the lint that enforces it scans the tracked
file list, so the repository was failing on it. A test that was defined at
module level instead of inside its class, and therefore never ran, is indented
back in.
The REST API answers inputs: null for a finished workflow_dispatch, so
the run name is the only record of what a manual run was dispatched
with. It carried one value, distros; everything else silently fell back
to the workflow defaults, and a failure that only reproduces on swarm
with btrfs came back green from a run that deployed all three modes on
whatever filesystem it drew.
Render every input as its own <emoji><value> segment instead, omitted
when it holds its default, so the title carries exactly what deviates
and an absent input means 'leave it on the default'. run_name derives
those segments from the workflow rather than knowing them, so the format
cannot drift away from the parser without a failing test.
distros no longer defaults to debian -- empty draws one at random, the
way untagged pushes already do. That leaves nothing in the title, so the
effective list is recovered from the discover job names, matched by
content against the meta/distros.yml SPOT rather than by job-name shape.
Priority roles that fell behind the budget cut in every mode hold no job
at all, so failed_roles cannot see them: they are neither green nor red,
they never ran. Carry them along or they never deploy.
--run now resolves before the selection, so it reproduces a run whatever
picks the roles, not only with --failed.
tests/utils/ci_*.py move into tests/utils/ci/ -- two files sharing a
delimiter-bounded prefix are a folder in disguise, and the structure
lint says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The REST API answers inputs: null for a finished workflow_dispatch, so
the run name is the only record of what a manual run was dispatched
with. It carried one value, distros; everything else silently fell back
to the workflow defaults, and a failure that only reproduces on swarm
with btrfs came back green from a run that deployed all three modes on
whatever filesystem it drew.
Render every input as its own <emoji><value> segment instead, omitted
when it holds its default, so the title carries exactly what deviates
and an absent input means 'leave it on the default'. run_name derives
those segments from the workflow rather than knowing them, so the format
cannot drift away from the parser without a failing test.
distros no longer defaults to debian -- empty draws one at random, the
way untagged pushes already do. That leaves nothing in the title, so the
effective list is recovered from the discover job names, matched by
content against the meta/distros.yml SPOT rather than by job-name shape.
Priority roles that fell behind the budget cut in every mode hold no job
at all, so failed_roles cannot see them: they are neither green nor red,
they never ran. Carry them along or they never deploy.
--run now resolves before the selection, so it reproduces a run whatever
picks the roles, not only with --failed.
tests/utils/ci_*.py move into tests/utils/ci/ -- two files sharing a
delimiter-bounded prefix are a folder in disguise, and the structure
lint says so.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, each resolved by combining both sides rather than picking one.
compose.yml passes both env vars through to the container: swarm-nfs adds
INFINITO_SWARM_MGR_IP, this branch adds INFINITO_TOR_SOCKS_PORT, and neither
substitutes for the other.
daemon.json.j2 takes the data-root block from swarm-nfs and keeps HEAD's dns
key. The incoming side writes networks.internet.dns directly; lookup(
'container_dns') already returns that value as its clearnet entry and prepends
the docker0 bridge on a Tor node in compose mode, so the incoming form is a
strict subset and dropping it loses nothing.
The WordPress CA-trust addon spec gains skipUnlessServiceEnabled("sso") from
swarm-nfs alongside HEAD's gotoOnion and resolveTimeout, and asserts on the
Plugin Name the mu-plugin header declares - the mustuse screen renders that,
not the filename - with resolveTimeout applied to the literal the incoming
side left bare.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
