Resolves 68 conflicted files. The governing rule was our structure with their
network semantics: the tor boolean becomes a network mode (clearnet | tor |
multi) throughout the variant layer, while this branch's architecture and mesh
axes, its extracted modules and its i18n content stay.
Where the two sides covered different ground, both are kept rather than one
picked:
- ca_injected answers a ca_client on the deployment's TLS mode OR on an SSO
issuer that resolves self-signed. 3a3e9ed2c9 records why: an issuer is not
an object store, so keeping only one side drops the other's coverage.
- get_merged_domains keeps the loud refusal when svc-net-tor is deployed
without a provisioned node, now gated on their tor_serves_host; their
node_onion_of returns an empty string there, which would have served every
onion-exclusive app on clearnet in silence.
- sso_cookie_domains stays the single source for oauth2-proxy's cookie scope.
Their per-network branch omitted the issuer domain on the onion arm, which
breaks the callback leg.
- The clearnet CSP batch carries --clearnet-timeout again; their batch dict
hardcoded zero while the role still passes the flag.
- The dashboard keeps its i18n content dir, image, version and design
resolution; their side reduced the logo sources to the bare service
provider values.
Their side wins where it is measurably better: the onion mirror rewrite is
gated per vhost rather than per app, the Playwright onion decision reads the
pass's own domain instead of the role's canonical one, and the matomo
reinjection and dashboard config render per network sibling.
Eight files were broken by the merge without ever carrying a conflict marker,
because git marks only the lines both sides touched:
- utils/cache/domains.py lost the onionize block that sat between two hunks
- roles/sys-front-inj-all/templates/body_filter.lua.j2 called it
- web-app-bluesky, keycloak sso_proxy: the get_entity_name filter was renamed
- trigger/__main__.py, playwright_coverage.py and its test: the deleted tor
module
- cli/administration/deploy/ci/timings.py read Label.tor
- plugins/filter/csp_filters.py called _aligned_url with four of six arguments
- tests/unit/python/utils/cache/test_domains.py referenced a renamed class
- tests/.../sys-ctl-hlth-csp/files/test_script.py mocked a function main no
longer calls
Verified: the variant layer (155 tests, 65 subtests), ca_injected (18),
the CI layer (175), the domains cache including the restored guard (32),
retrigger inputs (9), timings and the CSP health script (53). make
quality-high clears autoformat; its remaining findings are tracked separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Without a swarm manager the mesh planner only built full meshes, whose
handshake gate waits for every peer, so laptops behind NAT could not join.
An inventory may now name a hub in networks.hub: the planner builds every
declared mesh around it, the hub forwards between its spokes, a spoke marked
networks.roaming gets no endpoint in the hub's peer list and dials the hub
with PersistentKeepalive, and the hub's gate skips roaming spokes while a
roaming spoke gates only on the hub. The mesh CLI takes the hub's public
address per host (--endpoint) and a vault password per host
(--host-vault-password-file). Full meshes and the swarm manager hub plan
exactly as before; NETWORK_MESH_POOL keeps the operator's subnet.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Combine main's WireGuard mesh axis with this branch's network axis in the CI
variant code. Every matrix row now carries both: network (clearnet, tor,
multi) from utils.github.variant.network and vpn (meshed or direct) from
utils.github.variant.vpn. Label parsing, which main moved to label.py, reads
the network glyph instead of the old tor flag and keeps the vpn glyph; the
selection token, plan table, matrix and trigger CLIs, the workflows and the
swarm routine forward both axes. The test of an incapable priority variant
now compares the set of network modes, since a swarm priority row also
covers both mesh arms.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Some servers build every absolute URL from one site URL, so a second
network cannot be answered under its own origin. reachability.handover
keeps the siblings but turns each one outside the canonical network into a
302 to the same path on the canonical URL (basic and ws_generic vhosts).
The redirect target comes from the canonical domain, not the application
id, because lookup('tls', application_id) follows the network of the vhost
being rendered and would point the sibling at itself.
The flag also drops Onion-Location for the role (Tor Browser would bounce
between the two), skips those siblings in the CSP crawl, expects 302 from
the webserver health check and keeps Playwright on the canonical domain.
The key needs a nocheck reason like single_mode.
Mattermost opts in: it builds every OAuth callback from its one SiteURL, so
an onion login came back on the clearnet callback and failed with 400.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A process that renders network-bound URLs once, such as oauth2-proxy or the dashboard, can serve only one network, so a multi-network role runs one per network. network_instances lists those networks with a label, the first one unlabelled so a single-network role keeps its names, and labelled and labelled_file suffix service and file names with it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The onion pick only bypassed the SSO issuer, so a clearnet CDN, asset or dashboard host an onion page loads went through Tor, which the lab network cannot exit. browser_proxy_bypass derives the list from the deployed applications' domains plus the issuer; it is empty on a clearnet pick and on a tor-only node. Specs that passed their own proxy to newContext dropped the bypass and now inherit use.proxy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The entity name keys a role's instance directory, its compose project and its container name, and was derived only by stripping the longest matching category prefix. Two roles under different categories whose names share a suffix therefore collapse onto one entity and overwrite each other's rendered stack, which no provides: can repair.
A literal entity_name in vars/main.yml now wins over the derivation. web-app-seaweedfs-console takes its shorter name back and declares the entity seaweedfs-console, which frees the seaweedfs key in its own services map for the engine consumer and removes the provides:, covers: and alias entries that stood in for it. A new lint pairs the declaration with the primary key it has to match, because a drift between them would drop the role from the provider set in silence.
Compared across all 288 roles, exactly one entity changes, and it restores what main already keeps on disk, so no instance directory moves.
The function and the filter lose their get_ prefix. Two lints read that prefix as the discriminator between the variable and the filter that fills it; both discriminate on the pipe now. Ansible's templar resolves a filter and a same-named variable independently, verified against ansible-core 2.20.2.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
web-app-seaweedfs becomes two roles. web-svc-seaweedfs keeps the storage
engine: the container, the seaweedfs_data volume, the per-consumer S3
identities and the single canonical domain api. web-app-seaweedfs-console
takes the filer and master web UIs into its own compose stack behind the
admin-only oauth2 proxy, reaching the engine over the shared object-store
network. A node can now run the store without the consoles.
The console needs a distinct role name because get_entity_name strips the
category prefix: both halves would have resolved to the entity seaweedfs and
collided on the compose project, the instance directory and the container
name.
The filer and master ports move into the engine's services.seaweedfs.ports
.internal as their single owner; 01_init_consumer.yml consumes the master
port and would otherwise have resolved it only by role-var leakage from the
console, breaking bucket provisioning on exactly the console-less node the
split makes possible.
The engine's api vhost gains the prometheus locations. It never had them: the
override satisfied the cross-cutting-include lint only because the strings sat
in the SSO branch that the api domain never took.
Consumers, the objstore lookup and the catalogs follow the role id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hardcoded-primary-domain lint exempted tests/ and scripts/tests/, so test
fixtures kept their own copy of the domain and would drift from
INFINITO_DOMAIN the moment it changes. The lint now scans both trees, and a
unit test pins its scope: only default.env, Markdown, inventories/, .claude/
and build/ may name the domain.
Python tests take the value from default_domain_primary(), and the workspace
helper builds its dashboard and Matomo URLs from INFINITO_DOMAIN once the repo
.env is loaded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A node runs in NETWORK_MODE clearnet, tor or multi; empty derives multi when svc-net-tor is deployed. meta/networks.yml reachability narrows a role (modes, single_mode, each justified by a network-reachability nocheck), and utils/networks/reachability.py is the one registry every network decision goes through. services.tor keeps only the dependency bond; exclusive and primary are gone and Matrix is single_mode.
Every canonical domain gains one sibling per effective network, clearnet first. sys-stk-front-proxy renders one vhost per sibling, and TLS, CSP, injected snippets, redirects, the web health probe and the CSP crawler are decided per vhost. Cross-app URLs follow the network of the vhost being rendered, the SSO issuer stays single, clearnet vhosts of multi roles send Onion-Location, and the shared location forwards X-Forwarded-Host, which lets Wazuh derive its OIDC redirect per request again. The to_onion_url filter is removed; its two consumers render per vhost instead.
Measured before commit: on a clearnet node the domain map of all 108 apps and every CSP header are identical to HEAD; on a multi node every role keeps its clearnet domains first and only gains onion siblings, and no CSP source crosses networks except the SSO issuer.
Refs: docs/requirements/037-multi-network-serving.md
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Run 34421944328 web-app-litellm#1 (README host-mode replay) answered
every health probe on llm.ai.infinito.test with 502: litellm
crash-looped on "P1000: Authentication failed against database server
at postgres" while postgres logged 'Role "litellm" does not exist'. The
README inventory has no svc-db-postgres group, so sys-svc-rdbms skipped
the central provisioning, yet svc-ai-litellm pinned postgres.shared to
true and kept consuming a shared database nobody had created its role in.
postgres.shared now reads "{{ 'svc-db-postgres' in group_names }}":
an inventory with the central group keeps the shared database, the
README inventory gets a dedicated one.
That dedicated instance, rendered by sys-svc-rdbms postgres.yml.j2,
joins only the compose default network, and the networks renderer
dropped default from every svc-db-*/svc-ai-* role. A local variant-1
deploy measured litellm-database on litellm_default alone and litellm on
the shared litellm network alone. _suppress_default now keeps default
when the role's database is local. The variants pin shared to true (0)
and false (1) so the matrix exercises both paths, and the generated
READMEs show the postgres relation as 0..1.
The three attachment helpers move from render.py to attachments.py,
which render.py re-exports, and test_render.py splits along the same
seams into five files plus _render_helpers.py with every test id kept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
utils/roles/applications/config.py turns a missing key with a None default into False, so the provider-side probe in attachments.py read an omitted mcp_consumer flag as an explicit refusal. No MCP client ever joined a provider's shared_net: open-webui rendered no gitlab network, http://gitlabmcp:8080/mcp did not resolve from its namespace, and Open WebUI's verify call answered 400 in both GitLab#1 jobs of run 33698844864, clearnet and tor alike. Measured on that run's own inventory: without the change open-webui does not join the gitlab overlay, with it it does, and a sentinel probe shows the provider carries no mcp_consumer key at all.
utils/roles/applications/mcp.py:295 already states the rule this restores, that an omitted flag inherits the client's own declaration and only an explicit false excludes.
The stub in the render tests answered an unset path with the caller's default, which is why the regression passed them; it now coerces the way the accessor does, and the new case fails without the fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
allowed_consumers was twenty hand-kept lists of which fourteen were identical, so every new client meant an edit in every provider. A client now declares services.<own-entity>.mcp_consumer: true once and a provider refuses one with mcp_consumer: false, which is also what tells a deliberate exclusion apart from a forgotten entry.
The group_names-gated consumer entries give the inventory resolver the co-deploy edge meta/mcp.yml never had: one of sixty-nine matrix rounds carried an MCP client before, twenty-three role/round pairs do now. bond 0.5 keeps the clients out of round 0 so the host memory budget still holds.
BREAKING CHANGE: meta/mcp.yml no longer accepts allowed_consumers; declare services.<entity>.mcp_consumer on the client instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both renderers build their output as a list of lines seeded with the "networks:"
header and append one line per attachment. With no attachment to append they
still returned the bare header, and a mapping key with no children parses as
null, not as an empty mapping. Compose then rejects the whole file instead of
ignoring an empty section.
The two functions do not share their non-empty paths and disagree on where the
newline goes -- render_compose_networks terminates with one, render_container_networks
leads with one -- so the guard is written out in each rather than factored into a
helper that would have to know both shapes.
Both callers already treat an empty string as "insert nothing", so returning it
needs no change on their side.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
