- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lifecycle.md said a deprecated role is still covered by the test suite while the tested envelope already left it out. A deprecated role is now untested: the matrix-deploy MUST NOT exercise it and a failure in it MUST NOT block a release, while planned, pre-alpha and eol stay open to best-effort runs. The complexity legend says untested too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An image bump in a provider redeployed every variant of every consumer,
including the variants that switch that provider off and therefore never
deploy the bumped image at all. The diff-derived whitelist now narrows each
affected role to the variants whose own round closure still reaches a seed,
emitted as `role#0,2` selection tokens. That closure is walked on the same
variant-merged services maps the matrix planner builds its rounds from, so the
narrowing follows a parent's variant choice across every generation of the
chain. It covers every auto-derived whitelist: the weekly update PRs,
Dependabot, and any other diff-driven run.
The narrowing never shrinks the matrix on partial information. A seed role
keeps every variant, because its own files changed. A role reached only
through a run_after edge keeps every variant too, because the round closure
does not follow that edge and a role whose hits are empty is therefore
unnarrowed rather than dropped.
A pinned token naming a role the discovery query does not return now warns
instead of aborting the run. The diff-derived whitelist legitimately names
roles the lifecycle envelope filters out, and once it pins variants such a
token would abort every chunk; role names are still checked against roles/
before the run starts, so a typo is caught one layer earlier.
The round's variant merge moves out of the deploy inventory package into
utils/roles/applications/variants.py. The CI selection has to build the same
maps, and importing the deploy package for them pulls in a module that reads
INFINITO_SRC_DIR at import time, which the runner host does not set.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four module-level constants were defined and never read: _OPERATORS in the complexity filter, superseded by the op group of _TOKEN_RE two lines below; _VALID_SWARM_CONDITIONS, which swarm_restart_condition never checks because it maps to one of three literals by construction; _MISSING in services/config.py, where resolution already goes through get(strict=False, default={}); and _DEFAULT_FLAG_VALUES_THAT_ARE_NOT_NAMES in the exec-resolver lint, referenced by no rule in it. The nfs_fstype lookup rebound its variables parameter and returned without reading it.
Every name was private to its module and a repo-wide search over every file type finds no reader. CodeQL flagged them as unused; ruff cannot, because pyflakes has no rule for module globals and treats a rebound parameter as used.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CodeQL reports cmd and sort_spec as possibly uninitialised after the except branch that calls parser.error, because it does not model the argparse error method as NoReturn. It is: usage is printed and the process exits with status 2. An explicit return 2 on that branch states the exit status the call already produces, so the analyser and a reader see the same path.
The retry loop in the registry client counted two attempts in a variable the body never read; the underscore prefix was the convention for that, but CodeQL only honours the bare name.
No path changes behaviour; the three alerts stop pointing at correct code so the scanner stays readable for the ones that are not.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two leftovers of my own. The sweep that removed the reverse client entries
appended a newline where it cut a block, which at end of file leaves a blank
line after the last key in four services.yml. And the two cache-read nocheck
comments in the bond editor were written past the line limit, so the
formatter breaks their read_text calls across three lines and carries the
comment to the closing paren.
The wrap does not cost the suppression: test_no_uncached_file_reads matches
the statement, not the line the directive sits on, and passes on this form.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A bond lives on the consumer's side of a service relation, so seeing how two
roles couple means opening both roles' meta/services.yml and comparing, and
retuning one means editing a scalar buried among dozens of unrelated keys.
Add `infinito meta roles applications bond` (make bond / i8bond): it resolves
every services.<key>.bond to a role-to-role edge through the service registry
and serves the result as a matrix, each cell carrying both directions because
the two need not agree. Brightness is the bond, black for none and white for
1, so the coupling of the whole repository reads at a glance.
Clicking a cell writes the new value back to the declaring role. The write is
line surgery rather than a YAML round-trip: these files carry `# nocheck:`
directives and hand-tuned ordering that re-dumping would flatten, so only the
bond scalar moves and a trailing comment on that line survives. The browser
sends a role pair, never a path, and the server resolves the service key from
its own edge map, so a pair it does not know is refused before any write.
`--out FILE` exports a read-only page and `--format json` the raw edges, for
callers that want to post-process rather than edit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The greedy set-cover retires a role's variant-0 row when an earlier row of another role embeds it, on the premise that embedding pulls the provider at variant 0. The premise is a property of the pair, not of the coverer: a job's round index is the primary role's own variant index, and plan_dev_inventory_matrix hands each pulled-in dependency 'round_index if round_index < its variant count else 0'. A variant 3 coverer therefore pulls a 3-variant provider at 0 and a 14-variant provider at 3.
Coverage now asks that question per pair instead of assuming the answer. Nothing changes on the current tree, where 53 rows are covered before and after: the discovery sort ends on 'asc variant', so a role's variant-0 row is always green before its higher ones and gets to cover first. That is a property of a sort key in default.env, not of this code, and 2633 embed pairs would fall the wrong way without it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A round deploys a provider once, so the role that pulls it in is the only place that can say how lean it has to be. A variant entry may now carry any application config topic under a service it depends on: a mapping deep-merges, so naming one key changes that key alone, and null empties the topic. Empty was the case that needed a marker of its own, because merging an empty mapping is a no-op and would leave exactly the services the round was trying to shed.
The topic vocabulary and the merge semantics live in utils.roles.applications.topics, which is also where resources.py now gets them rather than spelling the same null-or-merge out a second time. Collecting who demands what stays in the inventory tree, refuses two roles demanding different values for one provider in the same round, and skips a role's own entity entry so the existing services.<own>.users declarations are not read as overrides.
The CI set-cover learns the same fact. Its greedy cover retires a role's variant-0 row when an earlier row of another role embeds it, on the stated premise that embedding pulls the provider's variant 0. A variant that dictates that provider's config breaks the premise from the other side: whatever ran, it was not the provider as the provider declares it. Such a pair now covers nothing, so the retired row keeps its job, its artefact and its verdict.
Measured on the current tree: 20 override pairs, 41 covered rows before and after, so nothing is revoked today. The rule bites the moment an override lands in a row that covers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Output of 'make autoformat': ruff collapses expressions that were wrapped across lines without need, shfmt and the yaml formatters drop surplus blank lines. No semantic change - the whole diff is 10 insertions against 39 deletions, and every hunk joins lines that already said the same thing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
clone and covered_by both name a stand-in row: another row carries this one's coverage, so the sweep may drop it. Both were decided over every row and then applied to a filtered set, so a run that selected a subset deferred to a stand-in the filter had already removed - the row was cut in favour of a deploy that never happens. --whitelist svc-db-mariadb yielded no row at all, and the swarm-zombie job passed green while drilling nothing.
The sort's last-resort tie-breaker was drawn per process, so discovery and the plan renderer could name different assignments for the same run. INFINITO_DISCOVERY_SEED derives it from the row's own identity instead; the workflow feeds github.run_id, so the rotation still happens, once per run rather than once per process.
Measured against the previous order: on the mode-only filter both cut sets are identical (swarm 220 rows, 44 cut; compose 243 rows, 54 cut), so the full sweep is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The applications: host_vars block is the config namespace keyed by role id, so it also carries system services (sys-ctl-hlth-csp, sys-svc-container). Those declare system_service_id and match no application role, and the resolver died on them instead of reporting their absence.
role_name now raises RoleNotFoundError and exits 3, distinct from argparse's usage error (2) and from a genuine failure (1). run_subprocess grew ok_returncodes so a caller can accept an expected non-zero status while every other failure still surfaces, and resolve_role_path returns None for that code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Variant bundling packed several variants onto one runner and made the
job count depend on storage sums. Every variant now gets its own matrix
row instead, and the two things that used to be fixed per workflow --
which deploy mode a row runs in, and whether it sits behind the node
onion -- become axes assigned per row.
Both axes are a deterministic rotation over the row's position and the
sweep number, never random, so a red job reproduces by re-running the
same sweep. Tor turns on sweep//2 so it does not flip in lockstep with
the mode: a row walks all four mode/tor combinations over four sweeps
instead of only two.
A role offers at most two modes in practice, since swarm requires its
own stack and host requires the absence of one.
Drops with the bundler: limits.py, the variant_bundle env handlers,
INFINITO_VARIANT_BUNDLE_SIZE/_MAX_STORAGE, the per-role
meta/tests.yml variant_bundle_size key and its only user, and the
ComplexityRow bundles/jobs fields with the --deploy-mode and --max-jobs
flags that consumed them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`cli build include` generated one tasks/groups/<group>-roles.yml per invokable category during setup, and the stage plays included those files. They were written by the in-container UID, which is why a dedicated clean-container-owned helper had to delete them from inside the container.
The stage plays now loop over lookup('group_roles', <group>) and include tasks/utils/setup/{group,role}.yml directly. ordered_roles() in utils/roles/order.py holds the run_after topological sort the generator used to perform at build time.
utils/roles/stage.py:_group_names() no longer globs the generated files; it reads get_invokable_paths(), the same categories.yml source those files were derived from.
Measured: handlers still flush after every role (ansible-core 2.20.2, three loop items, three separate handler runs), and resolving all 18 groups costs 0.60s cold against 1.4s for the generator.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
07_compose_minimal.sh claimed to disable 'all variable (group-conditional)
services' and then named seven by hand. tor was not among them, so a
deploy meant to be minimal pulled svc-net-tor, which hard-depends on
web-svc-mirror, whose group-conditional prometheus flag the closure
resolver treats as true — dragging the whole web stack in. Steps 12 and
13 passed no disable at all, which is why a mariadb swarm test deployed
keycloak, mailu, matomo, prometheus and dashboard.
The set is now derived from the contract the repo already enforces
(tests/integration/roles/meta/services/test_dynamic_flags.py): a key is
group-conditional when enabled OR shared is a Jinja expression. Checking
only enabled would miss mariadb, postgres, redis, memcached, typesense.
Exclusions are load-bearing, not cosmetic: dashboard because disabling
the primary app aborts init; javascript because it aliases cdn, which
web-app-dashboard declares a hard dependency with no runtime fallback;
mariadb and postgres because steps 12 and 13 test them.
variable_services assigns to a local first — disable="$(cmd)" does not
set the exit status of a simple command, so a failing CLI would have
yielded an empty disable set and steps 12/13 would have run the full
deploy and reported green.
roundtrip could not take disable at all; it now reaches both the compose
and the swarm leg.
Not verified by a deploy: 33 of the keys are new to step 07, and the
variant axis of svc-db-mariadb/svc-db-postgres collapses because tor is
its only dimension.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twelve conflicts, resolved by intent rather than by side.
Six shared one shape: the incoming side adds the new rescue_index.sh call but
drops this branch's `# nocheck: shell-or-true` markers. The markers carry a
lint contract here, so the incoming structure was taken and the markers put
back — dedicated.sh, guide/one.sh, guide/host_deploy.sh, swarm/routine/00_one.sh
and trigger_units.sh, where the incoming version additionally writes the full
journal to a file instead of only echoing a tail.
web-app-matrix keeps HEAD's _synapse_extra_args accumulator (the superset: CA
mount plus the onion case) but adopts the incoming improvement inside it — the
inlined regex_replace gives way to lookup('subnet_gateway').
handlers/__init__.py drops rescue_diagnostics_dir from the old import group
(the incoming side moved it under .infinito.rescue, keeping both would import
it twice) and keeps tor_socks_port, a HEAD-only handler ORDERED_HANDLERS still
needs.
svc-bkp-volume-2-local takes the incoming side throughout: both branches landed
the same feature, but swarm-nfs evolved it — the knob moved from the global
STORAGE_BACKUP_SNAPSHOT_MODE to services.volume-2-local.snapshot_mode, plus the
subvolume helper, 04_snapshot_subject.yml, meta/packages.yml and baudolo 3.2.2.
Three files that left behind were removed: files/baudolo_snapshot.py and the two
worktree tests at their pre-refactor paths, all byte-identical to their moved
counterparts.
Six follow-on failures the merge produced, fixed at the root:
- STORAGE_BACKUP_SNAPSHOT_MODE lost its last reader with the config-lookup
design and is gone from group_vars.
- utils/github/variant_bundle_limits.py sat next to variant_bundles.py, which
the prefix lint forbids; the pair now lives in utils/github/variant/ as
bundles.py and bundle_limits.py, with the -m entry point, the ruff
per-file-ignore, the design-doc link and the mirroring test path updated.
- The incoming WordPress addon spec navigated with a raw page.goto and carried
three bare timeout literals; both route through gotoOnion and resolveTimeout
now, which is what makes navigation survive onion transport.
- Six unmarked `|| true` got a concrete reason each: docker.socket is absent on
distros that ship docker without it, the loop device is already detached when
the probe failed before attaching, and modprobe zfs may fail because the
/dev/zfs check decides.
make quality green across all four targets. The cold-domains budget test that
had to be waived here passes on its own now — 19d56f07e raises it to 100s.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
variant_bundle_limits.py next to variant_bundles.py formed the prefix group
'variant', which the structure lint reads as a folder in disguise. It stayed
green locally because that lint asks git ls-files: the new file was untracked
through every quality run and only became visible once committed. CI saw it
immediately - the sole failure of run 30675284663.
Both move to utils/github/variant/ as bundles.py and limits.py, and the test
mirrors along.
The BLE001 per-file ignore is removed rather than repointed. parse_size raises
humanfriendly.InvalidSize for a malformed string and TypeError only for a
non-string, and the value reaching it is always str - so the catch-all was
never needed. Measured, not assumed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The compose and host deploy jobs tee the whole deploy stdout into the file
ANSIBLE_LOG_PATH names, and both downstream steps parse that stream. Only
stdout carries the host-side "matrix-deploy: round N/M ... PASS n" markers,
so it is the sole source the per-variant runtime tables can come from.
dedicated.sh's cleanup then copied the container-side ansible log onto that
same path. docker cp unlinks and recreates the destination, so tee kept
writing into an orphaned inode and the visible file became ansible's own
log_path output — whose "<ts> p= u= n= LEVEL| " line prefix defeats the
anchored recap matchers in logparse.py while the unanchored has_sections
substring test still fires. in_roles_recap never flips, every role-time line
is skipped, and the step dies with "no profile_roles entries found" although
the deploy succeeded. Run 30578244959: 14 of 14 compose+host jobs.
Why it looked like a flake: docker cp cannot read a /tmp that is a tmpfs
mounted inside the container, and the Debian image enables systemd's
tmp.mount while CentOS Stream 10 does not. The copy silently failed on
debian (host file survived, jobs green) and landed on centos (host file
destroyed). feature/swarm-nfs never had the block; it returned here through
merge drift from main, where the copy is still load-bearing because those
workflows do not tee.
The copy is dropped: stdout is a strict superset of what it provided, and
the in-container verifier reads the log inside the container. Alongside it:
the swarm workflow's file-exists guard is adopted so an aborted deploy does
not paint the summary step red on top; the log_path prefix moves to
utils/ansible_log.py and is applied in logparse.py, so an ansible run log
parses instead of yielding an empty summary; a lint pins that nothing writes
onto ANSIBLE_LOG_PATH again; and the parser regression test now covers the
recap headers, not just the role rows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.
meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.
Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.
tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
categories.yml moves from roles/ to the repository-root meta/, joining packages.yml. utils/roles/categories.py owns its location plus the tree-walking helpers that were stranded in entity/name.py, so no caller spells the path or duplicates the lookup.
meta/distros.yml becomes the SPOT for every distribution: os_family, Galaxy platform name, dev-environment base image, and the pkgmgr/pkgmgr-virgin/environment reference templates. It replaces seven scattered copies (DISTRO_FAMILY, VALID_DISTROS, _DISTROS, _CANONICAL_PLATFORMS, two shell arrays, a Makefile literal). INFINITO_DISTROS is no longer a static default but generated by utils/env/handlers/infinito/distros.py; the shell consumers read it through the env layer.
Drops dead CLI surface found along the way: --categories-file and --roles-dir on the provision CLI plus three unused parameters of generate_dynamic_inventory, and the positional roles_file of the invokable categories command.
tests/lint/repository/test_distros_spot.py pins the copies that cannot call the SPOT (workflow literals, docs table) and forbids composing a pkgmgr reference outside it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Formatter output from make quality: the git ls-tree argument list in
complexity.model and the xwiki job-name dict in the report_failures test
wrapped across lines. No semantic change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
report_failures.decisive_excerpt and new_in_branch._declares_application_id
now read via utils.cache.files.read_text instead of Path.read_text, and the
new_in_branch test constructs its vars path from ROLE_FILE_VARS_MAIN. Clears
the uncached-read and role-file-spot lint gates; behaviour is unchanged
(same UTF-8 decode, same exceptions, same resolved path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A push resolves the application roles that exist under roles/ locally but
not at origin/main and passes them as the orchestrator priority line, so
brand-new roles deploy and gate ahead of the regular matrix. The resolver
is stdlib-only (git ls-tree plus an application_id line scan) so it runs
on a bare runner without installing the resolver stack, and yields an
empty set when the baseline ref cannot be resolved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an in_main column (🌳) to the role complexity matrix: it resolves
the role directories present under roles/ at origin/main once and flags
every row true/false, so a role that is new on the current branch stands
out. When the ref cannot be resolved (a non-git checkout or an unfetched
main) the marker falls back to present for every role rather than
falsely reporting them new.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Read INFINITO_LIFECYCLES from default.env via a new tested_lifecycles() helper; derive TESTED_LIFECYCLES, the README overview envelope, the guide-select role filter, and the invokable-lifecycle lint from it. Add "maintained" to the envelope and drop the redundant hardcoded "alpha beta rc stable" strings from workflows, CLI help, and docs.
Guide selection now skips pre-alpha/eol roles (153 -> 117 testable).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The discovery query read every INFINITO_* knob with a silent .get default, which the env lint forbids: defaults live in default.env, and an unset key must fail loudly instead of masking a broken environment. All five accesses go through os.environ[...] now; the documented empty-value branches (sort spec falling back to default.env, auto job slots) stay explicit logic. The complexity model's random tie-breaker suppression carries its required justification.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The discovery SPOT (cli.meta.ci.query) now runs one query shape per mode:
compose and host stay whole-role with variants packed into bundles, swarm
ranks and budget-cuts on per-variant rows and emits role#variant tokens
through apps.sh into variant_bundles, so the cut can pick a subset of a
role's variants and the --matrix view is the selection priority.
Replace the --unique flag with visible dna/clone columns (heaviest role
per dna cluster stays ahead of the cut), add the integrated column and
sort, count host bundles like compose, and give the deploy plan an id
counter, per-variant weights and display-width-aware alignment matching
the matrix renderer. Add i8ma*/i8pa* inspection aliases per mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add cli.build.readme.overview (make readme-index / make docs) rendering a
roles overview in the root README: Name, Lifecycle, Homepage, Video,
Description, Integrated with, Install. New info-media lint requires
homepage + video in every invokable role's meta/info.yml (opt out via
'# nocheck: info-media' in meta/main.yml); backfilled 147 roles with
researched, adversarially-verified homepages and official videos.
Retire the 'unsupported' lifecycle value: fold it into 'eol' as a valid
live end-of-life status (shipped but untested/unmaintained) across the
stage lists, symbol glossary, allow-list lint and docs. Retag confluence,
jira, minio, phpldapadmin to eol and magento to pre-alpha.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
roles/categories.yml now declares the ordered 'stages' and a per-category 'stage' (plus run_after and bootstrap). utils/roles/stage.py resolves a role to its stage and exposes stage_groups(); the new stage_groups lookup feeds the 01_constructor / 02_server / 02_workstation / 03_destructor group loops in place of their hardcoded lists. A static invariant test proves the lookup reproduces the exact previous loops, so the rendered plays (and deploy behavior) are unchanged.
cli.meta.roles.order.run orders its main phase by stage as a run_after tiebreak. plugins/filter/invokable_paths tolerates the new sibling 'stages' key, and the invokable-roles wiring test recognizes stage_groups-driven wiring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).
The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.
test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.
Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.
Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
meta/services.yml modes states where a role RUNS; a new meta/tests.yml
skip list deactivates TESTING a mode without touching that capability.
The table gains test_compose/test_swarm/test_host columns (base column
minus skip), --filter/--sort and the JSON/YAML payloads know the new
fields, and CI discovery (apps.sh) now filters on test_<mode> == true.
svc-storage-nfs-client is the first consumer and skips all three modes.
Also replaces the nine VS16-based header/lifecycle emojis (like the
old host/jobs/weight symbols) with East-Asian-Wide ones: terminals
render VS16 sequences 2 cells wide while the width model said 1,
shifting every header right of them off its column. A guard test pins
all table symbols to wide, VS16-free codepoints.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
compose and swarm columns go through _tested_apps (invokable +
lifecycle envelope + per-mode skip) but host only checked non-stack +
modes.host.enabled, so host CI discovery reacted to every lifecycle:
61 roles instead of the 24 inside the alpha/beta/rc/stable envelope.
Route host through _tested_apps("host") in both the role-level and
per-variant row builders.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surfaced by make quality: alias comments on i8matiso/i8mativaso/i8s; self-path nocheck + drop two narration comments in load.sh (venv note kept in the header); shorten the host deploy job name to <=80 chars; move the legend rationale from an inline comment into the docstring. Also move utils/env/handlers/infinito/fork_{account,repository_url}.py into a fork/ subfolder (shared-prefix rule) and update the handler imports.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Host lane for invokable non-stack roles (drivers, backup, wireguard, swapfile, ...). modes.host.enabled declared on 56 such roles; new lint test_role_host_mode enforces it. The complexity matrix gains a host column (not stack and modes.host.enabled), --deploy-mode host, and host bundles (one per variant); apps.sh accepts INFINITO_DEPLOY_MODE=host and filters host == true. New test-deploy-host.yml workflow (INFINITO_MAX_JOBS=10) deploys discovered host roles via the mode-agnostic ci/all.sh path, wired into ci-orchestrator (modes default 'swarm compose host').
Also: complexity -s/--symbol compact view (emoji-only headers, true/false as check/cross, lifecycle stage symbols, grouped legend with per-entry explanations, display-width aligned); stack column moved after host. scripts/meta/env/load.sh made zsh-compatible by delegating to bash and importing INFINITO_/PYTHON/VENV/PATH back.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per-role deploy-mode opt-out moves from meta/tests.yml 'skip:' to meta/services.yml <entity>.modes.{compose,swarm}.enabled on the primary entity. get_role_skip now derives from it; get_role_mode_enabled added. A new lint requires every role shipping templates/*compose*.yml.j2 to declare both mode flags. Migrates 107 services.yml and drops 11 skip-only tests.yml. Also removes the variant_status deployability gate so swarm runs one job per variant unconditionally (variant_status.py and deployable_variant_indices deleted, complexity model simplified). Adds the i8matix alias.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolve every 3+-file same-prefix group flagged by the new prefix lint
by moving the files into a subfolder named after the shared prefix and
dropping the prefix from the file names, updating all references:
- utils/env/handlers/infinito_*.py -> infinito/ (dir/, package_cache/
nested a second level); registry imports and the four handler-dir
globs switched to iter_project_files
- utils/install/lint/ansible_*.py -> ansible/ (aliased imports)
- roles/sys-svc-compose/files/swarm_*.{py,sh} -> files/swarm/
- web-app-erpnext/web-app-zammad files/scripts/apply_*.{py,rb} -> apply/
- web-app-odoo files/sql/{select,update}-* and sql/oidc/{grant,insert,
select,update}-* -> matching subfolders
- web-app-keycloak tasks/update/05_ldap*.yml -> 05_ldap/ (internal
includes made role_path-absolute)
- plugins/filter/get_*.py -> plugins/filter/get/ (filter names are the
filters() dict keys, so Jinja usage is unaffected; ansible discovers
the one-level-deep subdir)
Ansible lookup dirs, addon dirs and .github/workflows keep their flat
layout (the filename is the public id / GitHub only discovers flat).
Unit tests mirror the new source layout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
callorder moves to 'order run' unchanged in behaviour (group-file call
order of the main pass); the new 'order preload' prints the
sys-service-loader preload order from the service registry, run_after
edges included, so both scheduling layers have their own inspector
instead of one tool mixing them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
meta/tests.yml becomes the SPOT for per-role test metadata: the
test-deploy discovery skip list moves there (top-level skip:) from the
primary service entity in meta/services.yml, next to the existing
cli.timeout. get_role_skip reads the new location; the drv lint,
--skip-mode help and unit fixtures follow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a `stack` column to the complexity matrix: true when a role renders
its own container stack (ships templates/*compose*.yml.j2), false for
host-only roles (backup crons, wireguard, swapfile), the swarm
infrastructure roles themselves, and central-webserver content
providers. Exposed as a sort key and filter field.
Make `swarm` imply `stack`: a role with no compose template of its own
is never a swarm stack-deploy target, so swarm is forced false when
stack is false. Because scripts/meta/resolve/apps.sh selects the swarm
deploy jobs via `--filter "swarm == true"`, this drops 17 stackless
roles (106 -> 89) from the swarm job list; all keep compose testing and
their swarm overlay coverage still runs through the webserver/proxy
role, so no real deploy coverage is lost.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Inventory include sets ignored the active variant: swarm
derive_includes walked the variant-free base config (dynamic service
flags count as enabled) and the compose CombinedResolver followed
run_after as an include edge. Providers a variant pins to false
(e.g. mobilizon variant 1 sso) were still deployed.
- derive_includes: swap every app listed in INFINITO_APP_VARIANTS to
its variant config, index 0 included (nextcloud variant 0 disables
11 partner services)
- CombinedResolver: follow_run_after flag; include resolution passes
False so run_after stays a pure ordering hint
- drop dead deps.py chain (resolve_deploy_ids_for_*) and unused
resolve_with_edges; rewiring them would resurrect variant-blind
run_after-following includes
- inventory.md: document the real include pipeline and
is_explicit_truth semantics
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lifecycle envelope for test-deploy discovery is now driven by
INFINITO_LIFECYCLES (default: alpha beta rc stable) instead of a
hardcoded constant. apps.sh forwards it to the complexity CLI via a
new --lifecycles flag (comma- or whitespace-separated); the model
falls back to the built-in default when unset.
Manual CI (entry-manual) gains two override fields: 'lifecycles' to
override the envelope per run, and 'modes' (default 'swarm compose')
to pick which deploy jobs run. ci-orchestrator gates test-deploy-compose
and test-deploy-swarm on the selected modes and threads lifecycles into
both. The new key is registered in the passthrough handler so it lands
in the generated .env.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
autoformat reformats (settings sort, complexity/model.py, test_database_query.py, keycloak vars wrap); shorten akaunting swarm task name to <=80 chars; delete narration comments from keycloak vars; remove 'invariant' from the comment linter's valid marker set. make autoformat + make test verified green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drive the test-deploy matrix discovery through the complexity report instead of the type CLI: apps.sh selects the mode's tested roles via the 'compose'/'swarm' column (which already bakes in invokable + tested-lifecycle + per-mode skip), orders them coverage-first, and hard-caps the cumulative per-mode runner budget.
- complexity gains --deploy-mode {compose,swarm} (makes 'bundles'/'jobs' the runner count for that mode: compose packs variants into size/storage bundles, swarm runs one per deployable variant) and --max-jobs N (keep only the leading rows whose cumulative jobs stay below N).
- apps.sh emits whole role names; variant_bundles/output_apps.sh expand them into the matrix unchanged. Manual INFINITO_WHITELIST becomes a 'name %% {…}' filter clause applied before the cut, so a manual run only counts the apps it actually invokes.
- INFINITO_DEPLOY_MODE is now required; INFINITO_MAX_JOBS, INFINITO_DISCOVERY_SORT and INFINITO_REQUIRED_STORAGE move to default.env (the SPOT) + the passthrough handler, so no shell-side defaults remain.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend the 'meta roles applications complexity' report:
- --sort takes comma-separated multi-key clauses with per-key direction (e.g. 'desc embeds, asc weight'); replaces --order.
- --variant lists one row per meta/variants.yml variant; --format adds yaml.
- New columns: weight (was total), random, id, row, variants, bundles, jobs, lifecycle, covered_by, compose, swarm.
- covered_by marks the first green (1-based id) different-role predecessor that embeds the row; variant>0 never covered.
- bundles is the compose bundle/job count (whole-role) or 1 (per variant); jobs is the running sum.
- compose/swarm reflect the CI test-deploy skip logic (invokable + tested lifecycle + per-mode skip + swarm deployable variant).
- --filter is a boolean expression DSL (%% == \!= < > <= >=, and/or/xor/not, parens, set literals) over row fields.
Add shared SPOTs: variant_bundles.compose_bundle_counts and variant_status.deployable_variant_indices, reused by the swarm/compose matrix. Update roundtrip.sh/md to --sort "desc weight".
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collect and aggregate each service's min_storage (summed like mem) and surface it as a column in the detail, per-variant and per-role views, the JSON output and the filter/order fields.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
