Commit Graph
11 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 0af8ee5992 Merge branch 'feature/updater' into feature/store-i18n
Five conflicts, resolved by what each side is for rather than by taking a
side:

- sys-service/01_prune.yml and sys-service-terminator/02_unlock.yml take
  the incoming single `systemctl` invocation over four module loops. The
  unlock side also drops an unmask step: the list it runs on is built with
  `reject('search', ' masked')`, so it never held a masked unit and the
  step did nothing.
- sys-stk-front-proxy/tasks/main.yml keeps this branch's version. The block
  the other side still carries there moved into vhost.yml, which runs it
  once per network sibling; taking theirs would have duplicated it.
  vhost.yml holds every task of that block, the global injection and the
  TLS layer included.
- web-app-matomo/08_reinject_consumers.yml likewise keeps the per-vhost
  loop, which re-renders the injection for every network sibling and passes
  matomo_token_override.
- utils/cache/domains.py combines both: the incoming side rendered
  DOMAIN_PRIMARY with the real templar and variables, which is right, but
  added a second renderer beside the module's own. _render_plain now takes
  that context optionally, so the other two call sites stay as they were.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-24 01:57:56 +02:00
kevinveenbirkenbachandClaude Opus 5.5 57eb64e0af fix(cache): render a templated DOMAIN_PRIMARY against the play variables
get_merged_domains rendered DOMAIN_PRIMARY with no templar and an empty variable scope. A primary declared as a template, such as echoserver's "{{ SOFTWARE_NAME | lower }}", therefore came out as 'none' inside the applications SPOT, while direct lookups saw the domain Ansible had already resolved. Every SPOT value built through it named file.none, and the dashboard waited for https://file.none/assets/img/logo.png until full-2 failed. The render now gets the templar and the play variables, which also covers services.tor.node.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-24 00:40:32 +02:00
kevinveenbirkenbachandClaude Opus 5 677a63931c Merge branch 'feature/network/multi-network-serving' into feature/store-i18n
Resolves 68 conflicted files. The governing rule was our structure with their
network semantics: the tor boolean becomes a network mode (clearnet | tor |
multi) throughout the variant layer, while this branch's architecture and mesh
axes, its extracted modules and its i18n content stay.

Where the two sides covered different ground, both are kept rather than one
picked:

- ca_injected answers a ca_client on the deployment's TLS mode OR on an SSO
  issuer that resolves self-signed. 3a3e9ed2c9 records why: an issuer is not
  an object store, so keeping only one side drops the other's coverage.
- get_merged_domains keeps the loud refusal when svc-net-tor is deployed
  without a provisioned node, now gated on their tor_serves_host; their
  node_onion_of returns an empty string there, which would have served every
  onion-exclusive app on clearnet in silence.
- sso_cookie_domains stays the single source for oauth2-proxy's cookie scope.
  Their per-network branch omitted the issuer domain on the onion arm, which
  breaks the callback leg.
- The clearnet CSP batch carries --clearnet-timeout again; their batch dict
  hardcoded zero while the role still passes the flag.
- The dashboard keeps its i18n content dir, image, version and design
  resolution; their side reduced the logo sources to the bare service
  provider values.

Their side wins where it is measurably better: the onion mirror rewrite is
gated per vhost rather than per app, the Playwright onion decision reads the
pass's own domain instead of the role's canonical one, and the matomo
reinjection and dashboard config render per network sibling.

Eight files were broken by the merge without ever carrying a conflict marker,
because git marks only the lines both sides touched:

- utils/cache/domains.py lost the onionize block that sat between two hunks
- roles/sys-front-inj-all/templates/body_filter.lua.j2 called it
- web-app-bluesky, keycloak sso_proxy: the get_entity_name filter was renamed
- trigger/__main__.py, playwright_coverage.py and its test: the deleted tor
  module
- cli/administration/deploy/ci/timings.py read Label.tor
- plugins/filter/csp_filters.py called _aligned_url with four of six arguments
- tests/unit/python/utils/cache/test_domains.py referenced a renamed class
- tests/.../sys-ctl-hlth-csp/files/test_script.py mocked a function main no
  longer calls

Verified: the variant layer (155 tests, 65 subtests), ca_injected (18),
the CI layer (175), the domains cache including the restored guard (32),
retrigger inputs (9), timings and the CSP health script (53). make
quality-high clears autoformat; its remaining findings are tracked separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 09:53:41 +02:00
kevinveenbirkenbachandClaude Opus 5.5 0d205b383f fix(networks): let a swarm node see the Tor provider of its cluster
Only the node that runs svc-net-tor had it in group_names, so every other swarm node resolved no node onion, served no onion siblings and scheduled no onion apex redirect. tor_serves_host also accepts a provider anywhere in a cluster of more than one svc-swarm-node; a single compose host keeps the old answer.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 04:47:38 +02:00
kevinveenbirkenbachandClaude Opus 5 e16d807e62 fix(cache): refuse a tor deploy whose node onion is empty
get_merged_domains gated the onion injection on node_onion being set AND
svc-net-tor being in group_names. When the node was empty it answered the
clearnet map with no error and no log line, and because group_names is
part of the cache key, a clearnet map and an onion map coexist in one
process under separate keys. One render then serves onion while another
freezes a clearnet asset URL into a stylesheet.

That URL is unrecoverable at request time: body_filter.lua returns a CSS
response before onionize(), so the <link href> in the head is rewritten
while the url(...) inside the stylesheet body is not. The symptom is a
single toHaveCSS mismatch and nothing else.

An empty node under a deployed svc-net-tor now raises. A scope carrying no
group_names at all stays silent, because that is indistinguishable from a
legitimate non-tor deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-14 00:21:24 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e2beab9f1c test(domains): read the primary domain from default.env in every test
The hardcoded-primary-domain lint exempted tests/ and scripts/tests/, so test
fixtures kept their own copy of the domain and would drift from
INFINITO_DOMAIN the moment it changes. The lint now scans both trees, and a
unit test pins its scope: only default.env, Markdown, inventories/, .claude/
and build/ may name the domain.

Python tests take the value from default_domain_primary(), and the workspace
helper builds its dashboard and Matomo URLs from INFINITO_DOMAIN once the repo
.env is loaded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 22:53:54 +02:00
kevinveenbirkenbachandClaude Opus 5.5 66c8df6729 feat(network): serve roles on clearnet and onion from one registry
A node runs in NETWORK_MODE clearnet, tor or multi; empty derives multi when svc-net-tor is deployed. meta/networks.yml reachability narrows a role (modes, single_mode, each justified by a network-reachability nocheck), and utils/networks/reachability.py is the one registry every network decision goes through. services.tor keeps only the dependency bond; exclusive and primary are gone and Matrix is single_mode.

Every canonical domain gains one sibling per effective network, clearnet first. sys-stk-front-proxy renders one vhost per sibling, and TLS, CSP, injected snippets, redirects, the web health probe and the CSP crawler are decided per vhost. Cross-app URLs follow the network of the vhost being rendered, the SSO issuer stays single, clearnet vhosts of multi roles send Onion-Location, and the shared location forwards X-Forwarded-Host, which lets Wazuh derive its OIDC redirect per request again. The to_onion_url filter is removed; its two consumers render per vhost instead.

Measured before commit: on a clearnet node the domain map of all 108 apps and every CSP header are identical to HEAD; on a multi node every role keeps its clearnet domains first and only gains onion siblings, and no CSP source crosses networks except the SSO issuer.

Refs: docs/requirements/037-multi-network-serving.md

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 19:42:05 +02:00
kevinveenbirkenbachandClaude Opus 5 5b1ac73dd9 fix(domains): render the tor flags before the onion map reads them
services.tor.enabled is Jinja in 99 of the 105 roles that declare it. A lookup('tls') or lookup('domains') inside a meta/*.yml value runs while the applications render is in flight, the re-entry guard hands it the unrendered tree, and _as_bool read the flag's own source text as False. The onion injection then did nothing for any app and the clearnet domain was returned.

fa3797540a already described this and quarantined it: the render state joined the domains cache key so the poisoned map answers no later read. The value stayed wrong, which is enough until a call site bakes it into applications. roles/web-svc-asset/meta/services.yml:15 is the only one in the repo that does, and in run 34922994823 it froze https://file.infinito.test/assets into the payload while every vhost of that round was published under the node onion. web-app-dashboard then waited 60 times on a host openresty answers with ssl_reject_handshake, and the deploy died on TLSV1_UNRECOGNIZED_NAME.

_resolved_flag renders enabled, exclusive and primary and raises when template text survives, the shape plugins/lookup/proxy_pass.py already uses. plugins/lookup/config.py and plugins/lookup/email.py render before coercing for the same reason; this module was the one that did not.

Measured across the whole map: 44 of 12819 leaves change, none added, none removed, 17 roles. 42 are clearnet to onion. The two others are repairs as well: keycloak's csp frame-ancestors moves from https to http, which is what an onion node serves, and the nextcloud sociallogin provider name stops being a hostname and becomes the bounded name that test_sociallogin_provider_name.py documents.

The new tests drive the same function with Jinja instead of the literal booleans the existing ones pass, which is why those never saw this. Three of the four fail without the fix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 11:20:40 +02:00
kevinveenbirkenbachandClaude Opus 5 fa3797540a fix(cache): keep a mid-render domain map out of the finished one
26 roles resolve a domain in their metadata, so lookup('tls') and lookup('domains') run inside the applications render. Under the re-entry guard those calls receive the unrendered tree, whose service flags are still Jinja strings, and _as_bool reads "{{ 'svc-net-tor' in group_names }}" as false. The resulting map skipped onion injection for every app that does not enable tor literally, and it was cached, so it answered every later read in that process.

Before the applications carrier each worker rendered and poisoned its own cache the same way, which is why the shared Keycloak client registered 5 onion origins then and 65 now. The render state joins the cache key, so the render's own call sites still share one map while a later read builds the finished one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-03 09:31:25 +02:00
kevinveenbirkenbachandClaude Opus 5 af605b049f refactor(domains): give local testing a .test domain, leave .example to docs
RFC 6761 reserves .test for testing and .example for documentation, and the
project's own domain was doing both jobs under the documentation name. It is
what default.env hands every local deploy and what the test fixtures assert
against, so it is a testing name.

Rename it wherever it stands for the running machine: the environment
defaults, the development inventory, the workspace and trust scripts, the test
suites, and the four documents that describe real behaviour - the DNS mapping
to loopback, the browser CA trust, WSL2 resolution. Those documents would have
started lying the moment default.env changed.

The sandbox allowlist moves with them, since egress is matched per pattern and
nothing resolves the old name any more, and the sandbox document that mirrors
that list follows it.

Two places keep .example on purpose: the link-style sample in the
documentation guide, which is documentation about documentation, and every
generic fixture name elsewhere in the suite - a.example, foo.example and their
kin. Those are a larger sweep of their own and are not touched here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 13:10:34 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00