Commit Graph
87 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 ccb2c82b8b test(svc-net-tor): let the probe report why a domain failed
The apex infinito.example has failed six compose jobs with HTTP 000 and no
explanation, because the probe throws curl's stderr away:

    curl ... "${scheme}://${domain}/" 2>/dev/null || true

--show-error was already set, so curl had the answer the whole time and the
redirect discarded it. Command substitution captures stdout only, so
%{http_code} and the accept regex at :61 are untouched, and || true still
swallows the exit code. test-e2e-cli/tasks/run_one.yml:85-92 prints stdout and
stderr when rc != 0, so the text reaches the job log.

This is NOT the fix for that failure. Three rounds of reasoning about it rested
on the premise that the apex answers 301 and curl follows it off-instance;
openresty.log:126 refutes it. That line logs the successful clearnet probe
"GET / HTTP/2.0" 200 with "Scheme: https" and User-Agent curl/8.18.0, so https
requests do appear in that log -- and the apex left no entry at all. curl never
completed hop 1, and there never was a 301 to follow. Whether the cause is
NXDOMAIN, a refused connect, or the 443 default_server's ssl_reject_handshake
(sys-svc-webserver-core/templates/default_server.conf.j2:17-20, which produces
exactly this signature: rejected handshake, 000 at curl, no access-log entry)
is what one curl error line will now say.

Costs up to six extra stderr lines per failing domain, RETRIES=6.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:28 +02:00
kevinveenbirkenbachandClaude Opus 5 4739b2de99 fix(svc-net-tor): stop imposing a bind policy on a foreign dnsmasq
bind-dynamic sat outside the {% if TOR_DNSMASQ_OWNS_LISTENER %} guard that
gates the listen-address lines, and router.yml:72 forces that flag false in
swarm. So on a stack host the drop-in declared it does not own the listener
and then dictated its bind mode anyway -- naming no address at all.

That converts a wildcard-bound instance's sockets into enumeration-derived
ones. When an IPv6 netlink dump aborts, dnsmasq skips the AF_INET pass and
its GC closes every IPv4 listener, silently: the message is LOG_DEBUG and
bind-dynamic bind failures are non-fatal. The node was left with 10 IPv6
listeners and zero IPv4 while four IPv4 addresses existed, and router.yml
had already pointed /etc/resolv.conf at 127.0.0.1 -- so dockerd got ICMP
port-unreachable and could not resolve ghcr.io. 33 failures in 11 groups,
none before the drop-in landed. The two worker nodes, which never received
it, kept 0.0.0.0:53 bound throughout.

Ownership is the right axis, not deployment mode. Where the role owns the
listener, bind-dynamic is load-bearing: a listen-address naming docker0 can
disappear across a docker restart, and without OPT_CLEVERBIND that bind is
fatal. Where it does not own the instance, the directive is pure collateral
-- and next to a base config carrying bind-interfaces it aborts startup
outright.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 10:10:12 +02:00
kevinveenbirkenbachandClaude Opus 5 20ddda8c1c fix(nextcloud): reach the onion OIDC issuer over Tor SOCKS again
libcurl refuses a .onion host by name (RFC 7686, error 6) before it
consults /etc/hosts, extra_hosts or DNS, so the platform's DNS+iptables
onion egress cannot serve a server-side backchannel at all. Measured:

    curl --noproxy '*'                          -> exit 6
    curl --resolve auth.<onion>:80:127.0.0.1 …  -> exit 6

--resolve fills curl's own cache strictly earlier than getaddrinfo and
is still refused, which makes sys-svc-container's onion_oidc_socks
template -- extra_hosts, despite its name -- a no-op for every libcurl
client. Only a configured proxy makes curl skip the check.

Since 9d91fbea8 emptied proxy.config.php.j2 there is no such proxy, while
svc-net-tor's exclusive/primary defaults replace Keycloak's clearnet
domain and KC_HOSTNAME pins the issuer to the onion. Nextcloud's
oidc_login discovery call therefore dies inside PHP, /apps/oidc_login/oidc
returns 500, and oidc_login_auto_redirect sends /login straight there.

Restoring the old socks5h://host.docker.internal:9050 alone would make it
worse: tor runs as a single manager-pinned replica publishing SOCKS with
mode: host, so only the replica on that node could connect and a
deterministic failure would become a 2-in-3 flake. In swarm, though, tor
is NOT host-networked -- that branch is compose-only -- and it already
binds SocksPort 0.0.0.0, so the shared overlay is a usable path from
every node.

Give svc-net-tor a shared_net overlay with alias 'tor' and dial it by
service name. Attachment is gated by a new consumer kind, onion_sso: the
consumer's services.sso.enabled AND the provider's services.tor.enabled.
Both are group-membership derived, so the conjunction is exactly "the
issuer is an onion" and stays false in clearnet deployments -- which
matters because the service registry is not filtered to deployed roles.
The existing kinds could not express it: keying on sso alone would also
hold without tor, and keying on tor would miss Nextcloud's variant 2,
which disables its own onion yet still needs the client path.

tor_socks_proxy keeps the swarm/compose split in one place so the six
roles sharing this defect need not restate it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 00:50:48 +02:00
kevinveenbirkenbachandClaude Opus 5 afb78286ef refactor(svc-net-tor): drop the now-callerless external probe
Deleting the external onion probe left check_external, _CURL_REACHED_RCS,
the internal|external|both dispatch and the optional socks argument with
no caller, while the docstring still promised 'Exit 0 only when both
succeed' and that the external path 'must actually work end to end' —
the exact contract that deletion removed.

The script now takes one argument. Real-network reachability stays gated
where it blocks something: test-e2e-playwright's wait_onion_reachable.sh.

Wiring the CLI test into e2e-cli also armed its default retry budget:
20 attempts x curl --max-time 30 plus 19 x 15s sleep is 885s per failing
domain, serial. TEST_E2E_CLI_TIMEOUT is 3600s, so four failing domains
exhaust it and a tor deploy serves about twelve vhosts. test.env.j2 now
pins RETRIES=6 and SLEEP_SECONDS=10 — 230s per domain.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 10:29:10 +02:00
kevinveenbirkenbachandClaude Opus 5 e5617917e8 fix(svc-net-tor): wire its CLI test into e2e-cli
The role shipped a complete probe script at files/test.sh and no
templates/test.env.j2. discover_cli_roles globs the env template alone,
so the role was never discovered and the script had never run — and the
script hard-requires TOR_SOCKS and NGINX_SERVERS_DIR via ${VAR:?}, so it
could not have run without one either. Its own usage hint pointed at a
path that does not exist.

Moving the script to the discovered path and adding the env template
makes both halves line up. Verified against the live stack: the onion
answers through the Tor SOCKS proxy, 1 probed, 0 failed.

The new lint keeps the pair together and rejects a script left at
files/test.sh. The two halves fail differently and only one is loud: an
env template without a script aborts the play, a script without an env
template is never discovered and its silence reads like a pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 19:02:24 +02:00
kevinveenbirkenbachandClaude Opus 5 3cf7c8f3ed test(svc-net-tor): add public-flavor CI variant
variants.yml held a single entry, so the CI coverage the previous commit
unlocked would have exercised chutney only — a private, offline Tor net
built for tests. The public flavor is the production path and stayed
uncovered.

Variant 1 overrides flavor and use_entry_guards and nothing else:
resources fall through to meta/services.yml, which is sized for the one
tor container, whereas variant 0 raises them for chutney's ~9 in-container
nodes. Guards stay off for the same reason variant 0 turns them off — a
degraded entry guard must not stall the node.

Swarm splits per variant and now runs two jobs for this role; compose
bundles both into one.

Unproven: whether a public-flavor container bootstraps into the real Tor
network from a GitHub runner. The internal loopback probe is fatal, so a
failed bootstrap fails the job hard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 19:02:24 +02:00
kevinveenbirkenbachandClaude Opus 5 946b3769cb feat(svc-net-tor): let CI deploy the role in compose and swarm
modes.compose.enabled and modes.swarm.enabled were both false, and a
stack role is forced host=false, so all three test_<mode> columns were
false and the CI discovery filter (test_<mode> == true) dropped the role
from every matrix. It was resolved into the priority whitelist as a new
role and then silently filtered out again — svc-net-tor was only ever
exercised indirectly, as a dependency other roles pull in.

The flag is read solely by the discovery model and the README generator;
nothing in the deploy path consults it, so this has no runtime effect.

meta/categories.yml pins the whole net category to compose/swarm false,
which fits svc-net-wireguard-{core,plain,firewalled}: none of them ships
a compose template. svc-net-tor does, and is the only role in the
category that does, so it takes the category-modes exemption the lint
provides rather than bending the category default for three host roles.

Regenerated READMEs follow: svc-net-tor now carries its mode symbols in
every cosmos diagram that names it as a dependency.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 19:02:24 +02:00
kevinveenbirkenbachandClaude Opus 5 f486269051 fix(svc-net-tor): drop dead external onion probe
The probe carried retries: 90, delay: 5 but failed_when: false, so it
could never gate a deploy — its only consumer was the debug warning
below it. Measured in CI run 30963648828: 4 blocks at 7:40/7:43/1:08/8:03,
283 retries, 3 of 4 ending non-fatal 'not reachable'.

Onion reachability is already gated where it matters, by
test-e2e-playwright's wait_onion_reachable.sh: it fails hard, budgets
2 x 40 x 15s, and force-recreates Tor for fresh circuits in between. In
the same job that gate ran 28 times for 6s total, while this probe was
reporting the onion unreachable.

The internal loopback probe stays and remains fatal.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 19:02:23 +02:00
kevinveenbirkenbachandClaude Opus 5 d4f73dac7f fix(svc-net-tor): tie listener ownership to the deploy mode, not to config contents
The previous probe counted effective directives in /etc/dnsmasq.conf and read
any of them as a foreign owner. Every distro ships a stock file with some, so
on centos and fedora the drop-in rendered without listen-address, dnsmasq bound
every interface via bind-dynamic and refused to start — both workspace jobs died
at router.yml:80 five minutes in, with byte-identical 244-byte drop-ins.

Deploy mode separates the two real cases: the swarm lab writes its own full
config that already serves every interface, so the drop-in must not narrow it;
everywhere else the role owns the listener as before. Same variable that already
gates TOR_CONTAINER_DNS_HOST.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 01:00:13 +02:00
kevinveenbirkenbachandClaude Opus 5 439d06ff7e style(svc-net-tor): autoformat rewrap of the dnsmasq ownership expression
No semantic change; the formatter redistributes the same filter chain.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:49:05 +02:00
kevinveenbirkenbachandClaude Opus 5 c772d6537b fix(svc-net-tor): only claim the dnsmasq listener when nothing else owns it
The foreign-config probe looked in /etc/dnsmasq.d/ only, so a host whose
resolver is configured in /etc/dnsmasq.conf — the swarm lab — read as
unowned. The role then wrote listen-address=127.0.0.1 and narrowed a
bind-dynamic resolver that had been answering on every interface, leaving
nested image builds without a nameserver. Count the main config's effective
directives too, ignoring comments and the conf-dir line the role adds itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 15:09:59 +02:00
kevinveenbirkenbachandClaude Opus 5 52b798094a style(container_dns): satisfy ruff on the onion dns bundle
Drop a return-only temporary and state autoescape explicitly — the test
renders a dnsmasq config, where html escaping would corrupt the output.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 09:24:38 +02:00
kevinveenbirkenbachandClaude Opus 5 989847a7cb fix(svc-net-tor): install dnsmasq through the package registry
The egress router reached for ansible.builtin.package with a literal
name, which bypasses the per-distro mapping the registry exists for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 09:18:37 +02:00
kevinveenbirkenbachandClaude Opus 5 9d91fbea83 feat(svc-net-tor): route onion names through tor via dns, not per-app proxies
Reaching an onion address required each role to point its own http client
at the tor socks proxy, which then carried clearnet traffic too — fatal
in the test network, which has no exit, and the reason nextcloud needed a
per-host exclusion list.

The egress router already redirects tor's virtual range to the TransPort
and dnsmasq already forwards .onion to tor's DNSPort; the piece missing
was reachability. dnsmasq listened on loopback only, which docker
discards, so containers never saw it. It now also answers on the docker
bridge, the daemon points there with the clearnet resolver kept as
fallback, and the role's own onion resolves to a container-reachable
backend instead of loopback.

Nextcloud drops its proxy block and the exclusion list; every other role
gets the same behaviour without configuring anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 01:13:33 +02:00
kevinveenbirkenbachandClaude Opus 5 b2fb06cebe fix(svc-net-tor): move the chutney control ports out of the app range
Chutney pins controlport_base at 8000 and the tor container runs in the
host network namespace, so its nodes squatted 8000-8006 — right inside
the app port block that starts at 8001 and runs near-contiguously to
8088. nextcloud, wordpress, mediawiki and confluence could not bind
their proxy port. Shifting the base to 18000 frees the range; the grep
fails the build if upstream ever rewords the line.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 23:51:17 +02:00
kevinveenbirkenbachandClaude Opus 5 b442a670f9 refactor(svc-net-tor): extract onion reachability tasks
The three onion-reach tasks repeated the same egress-router and
node-onion guard; they now live in onion_reach.yml behind a single
include condition.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-25 19:12:38 +02:00
kevinveenbirkenbachandClaude Fable 5 5399b48482 feat(svc-net-tor): add a chutney flavor for private-net test deployments
The test variant (meta/variants.yml, variant 0) switches the tor
service to the chutney flavor: the image ships chutney and the
entrypoint boots a private Tor network inside the container (all nodes
on 127.0.0.1, so the design is identical under compose host-networking
and swarm). The entrypoint bootstraps from /opt/chutney (chutney
creates its net/ state CWD-relative), extracts the DirAuthority/
TestingTorNetwork lines into /run/tor/chutney-net.conf and fails fast
when that file ends up empty, since the node would otherwise silently
join the public network. torrc %includes the conf only for the chutney
flavor.

Descriptor publish and rendezvous now run at localhost speed: per-app
onion e2e suites dropped from minutes to seconds in the compose
validation. The test variant also raises the footprint to
2g/1.0cpu/512pids (measured: at 1g/0.25 the ~9 extra tor nodes pin the
caps and desc publishing stalls); meta/services.yml stays static on
the public flavor and the old production footprint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 19:31:16 +02:00
kevinveenbirkenbachandClaude Fable 5 cdd2bbf81c refactor(tasks): apply the 00_core numbering scheme to branch-local roles
svc-net-tor and web-svc-mirror predate the task-file numbering lint the
swarm-nfs merge brought in (a numbered *_core.yml must be 00_core.yml);
rename their 01_core.yml and retarget the main.yml includes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-24 19:00:42 +02:00
kevinveenbirkenbachandClaude Fable 5 e7834013d1 fix(svc-net-tor): extract resolv upstreams with regex_findall
The map('split') chain returns an empty list on slurp-origin tagged
strings under ansible-core 2.20, so the dnsmasq forwarder was left with
no upstream and every clearnet lookup (image pulls, ACME) got REFUSED.
regex_findall reads the nameservers reliably.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 17:41:21 +02:00
kevinveenbirkenbachandClaude Fable 5 62a46e472f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 16:27:25 +02:00
kevinveenbirkenbachandClaude Fable 5 9ae0c7eff6 fix(svc-net-tor): run the node onion natively in swarm
Pin tor to the manager (host-published SOCKS/DNS live on the stack
host), forward HiddenServicePorts and the dnsmasq onion fast path to
the node address instead of loopback (dead-ends in task netns),
tolerate an empty clearnet-upstream capture, detect foreign dnsmasq
drop-ins before claiming the listener, and transfer onion_self_reach
to remote targets via script.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 15:19:52 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7511852bd1 fix(e2e): never skip onion suites; unpin entry guards in test envs
The Playwright onion gate probed the app onion over Tor and silently
SKIPPED the whole suite when it did not answer — a broken onion could
go green. Remove the gate: the probe now waits hard (40x15s, covering
normal 2-5 min descriptor publication) and an unreachable onion fails
the deploy loudly. Reachability is part of what is under test.

The readiness probe treated transport-level failures (status: -1) as
'< 500', satisfying its until condition on attempt 1 and never using
its retries (nextcloud [0,1] died to TLSV1_UNRECOGNIZED_NAME this way).
Require a real HTTP status >= 100 so transport errors retry.

Tor pins one long-lived entry guard; its own log showed ours degraded
('failing more circuits than usual ... 38 collapsed, 21 timed out'),
which throttles every flow of the node at once — the most plausible
cause of the 89-minute descriptor-publication outage and the
intermittent ERR_SOCKS_CONNECTION_FAILED in E2E. New config key
services.tor.use_entry_guards (default true keeps the Tor default and
production anonymity posture); the matrix variant pins it false so test
deployments render UseEntryGuards 0 and a single bad relay cannot stall
the node. Anonymity is not under test there, only reachability.

Verified: full svc-net-tor closure deploy over onion with the gateless
semantics, failed=0, every onion suite executed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-16 20:02:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6b40d63d30 fix(svc-net-tor): portable dnsmasq filter + non-fatal real-Tor probe
filter-rr was added in dnsmasq 2.86, so it aborts dnsmasq startup on distros shipping older builds (CentOS Stream 9 has 2.85). Use filter-AAAA instead, which parses on every CI distro (dnsmasq 2.85 through 2.93, verified per-distro with dnsmasq --test).

Split onion_self_reach.py into internal/external/both modes. Keep the internal loopback fast path as the hard gate (later host-side steps resolve the onion through it), and probe the real Tor path best-effort (retries the full ~450s publish window, failed_when: false, warns): descriptor publication is async, needs an app backend behind the onion, and is blocked on CI runners with no Tor egress, none of which should fail a deploy. Playwright retries the real onion with its own onion-scaled timeouts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 16:05:00 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8a2e0f6a2e fix(svc-net-tor): portable dnsmasq drop-in + tolerant self-reach gate
CI deploys on Arch runners died at the dnsmasq drop-in render: the pacman dnsmasq package ships neither /etc/dnsmasq.d nor an active conf-dir, so create the drop-in directory and enforce conf-dir in /etc/dnsmasq.conf (restarting dnsmasq when it changes). The self-reach gate then blocked every node without a web server on :80 (standalone svc-net-tor, dev environments): with the loopback shortcut an instant connection refusal proves the fast path works, so treat it as reached; likewise curl rc 52/56 over the Tor SOCKS require a completed rendezvous (descriptor published) and count as reached, while SOCKS failures and timeouts still fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 13:58:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 dcd7f9861c fix(svc-net-tor): gate deploy on both onion paths + node-onion loopback shortcut
Replace the internal-only reachability one-liner with onion_self_reach.py, which gates the role until the node onion answers both internally (host resolver loopback fast path) AND over the real Tor SOCKS (descriptor published — the path Playwright/users take). Add a dnsmasq address= shortcut mapping the node's own onion suffix to loopback so host-side deploy steps skip the Tor circuit; other .onion still egress via Tor and container/Playwright traffic (Tor SOCKS) is unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 10:50:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 10a11fc570 fix(svc-net-tor): stabilise the .onion egress path for downstream deploys
- dnsmasq: filter-rr=AAAA on the .onion split-DNS. Recent Tor answers the
  AAAA automap with a link-local-shaped v6 address that has no REDIRECT rule,
  so clients that prefer v6 (getaddrinfo) black-holed every onion request.
- self-reach gate: after wiring the egress router, block the role until the
  node onion actually answers over Tor. A (re)started Tor republishes its
  hidden-service descriptor for ~2-5 min; without the gate every later step
  touching an onion URL (front-proxy checks, injections, Playwright) failed
  as if transient. Any HTTP reaction — including OpenResty's 444
  connection-close for the bare node onion — counts as reachable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 02:34:00 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bbb1220d7d feat(web-svc-mirror): first-party privacy proxy for third-party assets, coupled to tor
New web-svc-mirror role: an nginx vhost that proxies and caches exactly
the external asset hosts aggregated from all deployed applications' CSP
whitelists (external_asset_origins filter) under first-party paths
(/<origin-host>/...), refusing every other upstream. svc-net-tor
consumes the service, so any tor deployment automatically serves the
mirror on its onion.

For apps with services.tor.enabled the shared body filter rewrites
external asset URLs in HTML to the onion mirror (onion sessions only;
clearnet untouched) and the CSP builder allows the mirror origin in the
asset directives. The dashboard's keycloak-js vendoring is reverted:
onion sessions now load the adapter through the mirror, clearnet loads
jsdelivr directly as before.

Verified: variant=0 deploy green (failed=0), dashboard 14/14 over onion
with keycloak-js served through the mirror, mirror spec green, health
check accepts the 404 vhost root.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 10:56:13 +02:00
kevinveenbirkenbachandClaude Opus 4.8 84b22b37a8 fix(web-app-n8n): complete tor service onboarding + resolve subnet collision
n8n was merged (from main) serving a canonical domain without a tor service block, and its local subnet 192.168.106.16/28 collided with svc-net-tor. Declare the complete tor block (meta/services.yml with the playwright-service-flag nocheck, meta/variants.yml pinning services.tor.enabled true in variant 0 and false in the no-auth variant) and move svc-net-tor's nominal (host-networked) subnet to 192.168.106.32/28. Fixes the tor-contract, variant-coverage, bond-baseline, playwright-env and subnet-overlap/duplicate test failures.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 21:02:27 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c09eb61dc6 style: drop explanatory comments from the Tor onion changes
Remove the inline/trailing comments added across the Tor egress-router, onion-domain, mailu-notls and vault round-trip changes, per the repo no-comments convention. Comment-only: no code behaviour changes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:25:44 +02:00
kevinveenbirkenbachandClaude Opus 4.8 32daff8773 fix(svc-net-tor): idempotent egress-router upstream capture
The upstream-capture copy re-read /etc/resolv.conf on every run and filtered out 127.0.0.1. Once the router has pointed the resolver at dnsmasq (127.0.0.1), that filter yields an empty list, so the copy failed with "src (or content) is required" and aborted the deploy on re-runs. Guard the resolver read and the persist copy on the persisted upstreams file being absent (via a stat), so the capture runs only on first setup and later runs reuse the persisted upstreams.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 11:24:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 62ffb10d8e feat(svc-net-tor): host-level transparent Tor egress router for .onion
Route any outbound TCP to a *.onion address through Tor, protocol-agnostically (HTTP, SMTP, IMAP, ...), while clearnet traffic resolves normally and goes direct. Implemented on the target host and gated on TOR_EGRESS_ROUTER_ENABLED: torrc gains AutomapHostsOnResolve + AutomapHostsSuffixes .onion + VirtualAddrNetworkIPv4 + TransPort/DNSPort; a dnsmasq drop-in split-resolves *.onion via the DNSPort (clearnet upstreams embedded inline, since an AppArmor-confined dnsmasq may only read /etc/dnsmasq.d/**) and the host resolver points at it; an iptables nat OUTPUT rule redirects the automapped virtual range to TransPort. Verified in-container: http://<app>.<node-onion>/ returns 200 with no SOCKS wrapper, clearnet DNS + HTTPS stay direct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 08:22:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 005f649498 refactor(onion): the inventory provisioner is the SPOT for the node onion
Mint the node onion during provisioning and write its address straight into applications.svc-net-tor.services.tor.node, dropping the INFINITO_TOR_NODE_ONION env indirection: ensure_node_onion (was init_env) mints/reuses the .onion-identity key files + returns the address (no env write); identity_hs_dir/IDENTITY_DIRNAME move to utils.tor_onion so cli and utils share the path SPOT; provision.apply_tor_node_onion sets services.tor.node in host_vars when svc-net-tor is deployed (run by 'inventory provision' at deploy time, and the svc-net-tor role copies the same key files into the daemon); domains.py reads services.tor.node; nginx_vhosts reads the key file instead of os.environ; INFINITO_TOR_NODE_ONION removed from default.env/passthrough/compose + the inventory env-lookup. Bonus: the address now lives in the persistent key files, surviving .env regeneration by make test/autoformat.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 221ea9bd71 feat(svc-net-tor): declare node onion + SOCKS port SPOTs
meta/services.yml gains tor.node (node onion address, filled per environment by the inventory) and tor.ports.local.socks (SOCKS port single source of truth). torrc renders SocksPort via the tor_socks lookup; the role test.sh takes TOR_SOCKS from the environment instead of a hard-coded 127.0.0.1:9050.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0a7164af48 feat(tor): per-role onion service block in meta/services.yml
Domain-serving roles reference the svc-net-tor provider via a tor: block. Consumers declare enabled (dynamic group_names gate) and shared: true (static — onion is always the shared node model, inline nocheck: dynamic-flag). The provider svc-net-tor additionally declares exclusive/primary defaults that consumers inherit via resolve_service_config unless overridden.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 945cf1a911 fix(svc-net-tor): apply torrc/key changes without the cross-role compose-up handler
The key/torrc tasks notified the sys-svc-compose "compose up" handler, which is
only registered when this role's later include_role of sys-svc-compose runs.
When svc-net-tor is loaded through sys-service-loader, its per-service
`meta: flush_handlers` fires before that include, so the notification resolves
to an unregistered handler and the play aborts ("handler compose up not found").

Register the key/torrc results instead and force-recreate the Tor compose
project inline when either changed (sys-svc-compose only recreates on
compose/env change, and the bind-mounted torrc is re-read at startup). No
cross-role handler dependency, so loader ordering can no longer break it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 03:26:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e6538e4f61 feat(svc-net-tor): Tor onion-service provider (full onion node)
Turn a NAT/edge node into a full Tor v3 onion node: DOMAIN_PRIMARY
becomes the node's .onion, so the whole stack (web, SSO, LDAP, CA)
resolves onion domains and is reachable over Tor without a public IP,
port forwarding, VPS or DynDNS.

- utils/tor_onion.py: offline v3 key mint (address + HiddenServiceDir
  key files), verified against a known key->address test vector.
- cli .../inventory/onion: `init-env` pre-mints the node onion, writes
  INFINITO_DOMAIN(+_RE) into .env and authoritative key files into
  .onion-identity/hs (idempotent). The onion is a deploy-time input;
  no in-deploy random minting (that would not match the pre-set domain).
- roles/svc-net-tor: Tor sidecar (Debian-built, network_mode: host),
  installs the minted keys, torrc with one node HiddenService + a SOCKS
  proxy restricted to loopback/DiD ranges. files/test.sh probes every
  served onion domain over the Tor SOCKS.
- tls: .onion -> TLS off (onion v3 crypto is the transport auth; LE
  cannot sign .onion) + app-level flavor override.
- harness/DNS: coredns resolves the onion zone via INFINITO_DOMAIN;
  skip the clearnet DNS sanity check for onion; raise nginx
  server_names_hash_bucket_size to 128 for long onion server_names.
- test-e2e-playwright: gated browser SOCKS proxy (PLAYWRIGHT_PROXY) +
  navigation/action timeouts so specs reach .onion over Tor and fail
  fast; the onion node points it at the Tor SOCKS.
- docs/requirements/031: full design.

Verified: dashboard + SSO (Keycloak) + 9 apps reachable over Tor
(HTTP 200 through the node hidden service). make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 12:38:11 +02:00