The onion port probe gave every SOCKS connect 60s, half of Tor's
SocksTimeout (120s by default; torrc does not set it), although its own
docstring argued that abandoning an attempt before SocksTimeout is
destructive. Every attempt authenticates with fresh credentials, so Tor
builds a whole new rendezvous for it and keeps working on it until
SocksTimeout. The probe walked away first and called the port dead. The
other raw onion connects in the repository, the svc-db-* CLI tests and
test-e2e-playwright's wait_onion_reachable.sh, wait 120s per attempt.
Run 40128751852 shows how often an attempt needs more than that. Of 178
svc-net-tor CLI runs in its 89 compose onion jobs, three lost their
first attempt to the 60s timeout and connected on the second within
seconds (Shopware#1, Hugo#1, n8n#1; the CLI test took 77-79s against
the usual 1-16s), and Hugo#1's other run took 40s on a single attempt. When
the stall outlasts the 70s until the second attempt, the probe fails
while the HTTP probe reaches the same onion port seconds earlier:
OpenBao#1 (run 39166850273, job 118050491920, the original failure of
#1048) and Socialhome#1 (run 40088473634, job 121019758461). In the
Socialhome job tor.log holds no line for either attempt, because Tor
never reached its own verdict.
ONION_PORT_TIMEOUT is now 130: SocksTimeout plus room for Tor's
once-per-second expiry sweep, so an attempt ends with Tor's own answer
(reply 0x06, and a "Giving up" notice in tor.log, which the rescue
artifact captures) instead of the probe's. The second attempt now starts
at 140s and ends at 270s, so the probe rides out every stall the old
budget did, and any stall shorter than about 260s instead of about 120s.
A green run still costs one connect per port, and a refused or unmapped
port still fails at once. A unit test pins the window above Tor's
SocksTimeout and fails as soon as torrc sets its own.
That doubles what a dead port costs, and meta/tests.yml caps the whole
CLI test. The largest compose onion deploy in run 40128751852 probes 19
domains and 5 ports; with everything dead that is 19 x 350 + 5 x 260 +
10 = 7960s, beyond the 7200s that 069df39027 set for the same reason.
9000 keeps the full failure output for up to 22 domains.
Closes#1214
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The check listed the torrcs without the limit through "grep -L ...; true".
With no torrc under /opt/chutney/net/nodes, or with one that cannot be
read, grep printed nothing, the trailing true turned its error into
success, and the test reported that every node runs one worker.
The check now counts the limit per file with "grep -cH" and reads the exit
status. A status above 1 or an empty answer fails the test, and a file
with a count of 0 is named as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Tor starts its worker threads by the number of CPU cores it detects, and
the cpus limit of the container does not change that number. On a host
with 20 cores every node of the private chutney network ran 21 threads and
grew to about 0.6 GB. The seven relays and authorities filled the 2g
limit and its swap within ten minutes, the kernel killed them one by one,
and no circuit could be built any more.
The entrypoint now writes NumCPUs 1 into the torrc of every chutney node
before it starts the network. Measured on that host over ten minutes: the
nodes run 3 threads and stay at 65 to 86 MB, the container at 0.47 GB
without swap, where it had filled 2 GB plus 2 GB of swap before.
The CLI test of the role fails when a chutney node runs without the limit.
Closes#824
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Tor attaches a stream to an onion to the rendezvous circuit that matches its
isolation and keeps it there until SocksTimeout; it never moves the stream to a
fresh circuit. The probe connected without SOCKS authentication and gave up
after 60s, so a dead rendezvous circuit would absorb both passes. That is the
most likely reading of run 35661520542 (web-app-mastodon#1, public Tor), where
<onion>:80 timed out twice right after curl had reached the same onion port.
Authenticate every connect with fresh random username/password credentials.
Tor accepts any and, under its default IsolateSOCKSAuth, builds a separate
rendezvous circuit per credential set, so a retry can no longer land on the
circuit that failed the attempt before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
svc-net-tor rendered its own guard script, its own unit and its own handler, and hooked the rules into DOCKER-USER with an insert, because a rule appended to a chain docker owns lands behind the RETURN docker puts there. The script flushed its chain and refilled it rule by rule, so the boundary was gone for the duration of the refill.
A role that needs packet filter rules now writes templates/nftables.conf.j2 declaring its own table and includes svc-net-firewall, which renders it to /etc/nftables.d/<role>.conf, rejects it with nft -c, loads it, reads the table back and refuses one that carries no chain. Deployed on its own the role installs nftables and the ruleset that loads every fragment at boot. That ruleset is a distribution decision, so its path follows os_family: RedHat reads /etc/sysconfig/nftables.conf, the others /etc/nftables.conf. The unit is only enabled, never started, because on Debian its ExecStop flushes the whole ruleset, docker's tables included.
One table per role removes the shared path: a fragment replaces its own table and touches no other, and docker never touches either. nft -f on an existing table merges, so each fragment creates, deletes and redeclares its table in one transaction. The tests load every fragment twice into a throwaway network namespace and require the listing to match, because a merging fragment loads twice without error and shows up only as a duplicated rule.
The tor guard now applies in both deploy modes. It was pinned to swarm while torrc binds TransPort and DNSPort on the bridge address in compose too, so on a single-host install those ports answered every on-link neighbour. The guard drops rather than admits, so nothing short-circuits another table, and ip saddr carries an implicit ipv4 dependency, which leaves the ipv6 exposure exactly where the iptables rules left it. svc-net-tor reads its own chain back and refuses a guard that drops nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sending to a .onion needs two things of the node: a resolver that answers the suffix, and a route from the container to Tor's transparent port. Neither existed. Compose pointed Mailu at unbound, which knows nothing about .onion, and in swarm Tor's DNSPort and TransPort listen inside the container where the host's dnsmasq and nat rules cannot reach them, so the forwarding rules that were already there pointed at a closed port on every swarm node.
Compose resolves through unbound, which now forwards the onion zone to Tor's DNSPort on the bridge it already owns. The forward alone would have answered SERVFAIL: the signed root proves that onion does not exist, so with auto-trust-anchor-file set every answer below it validates as bogus, and Tor's DNSPort can supply no DS. The zone is therefore declared insecure. Swarm resolves through the node dnsmasq instead, which every container already uses because container_dns puts the bridge resolver into daemon.json wherever svc-net-tor is deployed, and which now reaches Tor because its ports are published host-mode next to the SOCKS port that was published there already.
The transport rule changed target rather than mechanism. REDIRECT rewrites the destination to the primary address of the interface the packet arrived on, which is a different project bridge for every role, so Tor would have had to listen on all of them. A DNAT naming the bridge keeps one listener, and in swarm the same address reaches the published port through the proxy docker runs there.
That leaves the exposure. SocksPort has SocksPolicy; DNSPort and TransPort have no equivalent, and a swarm-published port carries no host address to bind to, so in swarm the packet filter is their only boundary. Compose needs none, because there Tor binds loopback and the bridge and nothing else. The swarm guard admits this deployment's own address space and drops the rest, and it is installed as a unit ordered after docker rather than as ansible-applied rules: nothing here persists netfilter state, so docker recreating its chains at boot would have left Tor back on the wildcard with the guard gone. It rebuilds its chain instead of appending to it, because the appending version could only ever be loosened. It hooks INPUT and DOCKER-USER both, since host-local traffic reaches a published port through docker's proxy and external traffic through FORWARD.
The deploy now proves all of it back and fails when a rule is missing, the router no longer runs on swarm nodes that carry no Tor, and the CLI test probes the forwarded ports over SOCKS instead of assuming a HiddenServicePort line means a reachable service.
Measured so far: Tor accepts the non-loopback binds and reports circuit-established, and the chain, its order and the DOCKER-USER hooks read back correctly from a deployed host. The swarm path is not yet exercised.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Run 34421944328 Nextcloud#13 lost the dashboard's Matomo tracker. The
dns53 sampler on mgr-01 shows no :53 socket from 01:39:52 to 01:48:44
while the dnsmasq PID stayed the same, six seconds after a short-lived
container's veth on docker0. In that window the manager could not
resolve its own onion vhosts, so sys-front-inj-matomo's site checks for
web-svc-logout (01:45:35) and web-app-dashboard (01:47:18) never reached
OpenResty, the injection was disabled as "API not ready", and the
Playwright check found no matomo.js. The other four swarm managers of
the run that carried the same listener kept it throughout.
Under bind-dynamic, dnsmasq 2.91 re-enumerates interfaces on every
netlink address event and then garbage-collects every listener it did
not re-find (network.c enumerate_interfaces), also when the enumeration
failed; release_listener has no other caller. 5aa7506e36 reproduced this
on the lab nodes and 11c98825aa already moved the bridge-less case to a
static bind. The bridge case now binds statically as well.
This supersedes two earlier rationales, both measured on the local stack:
- bb0e098a9f kept bind-dynamic because dnsmasq starts at boot before
docker0 exists. A unit drop-in (files/dnsmasq-restart.conf,
Restart=on-failure, RestartSec=5) now retries the start: with a missing
listen-address the start exits 2, and dnsmasq came up 4s after the
address appeared.
- 4739b2de99 called bind-dynamic load-bearing because docker0 can
disappear across a docker restart. Only a start without the address is
fatal; a bound socket survived the address being removed and re-added
and the interface being recreated with a new ifindex, answering on the
same socket inode each time.
The docker0 fact is re-gathered before torrc and the dnsmasq drop-in are
rendered. The play-start fact predates default-address-pools moving the
bridge (285ba1e76c), and a static bind on a stale address would make
dnsmasq exit. container_dns reads only the pool, so daemon.json is not
affected by the refresh.
The host resolv.conf moves from an inline copy content to
templates/resolv.conf.j2; both it and the drop-in render byte-identical
to what the local stack already runs.
A veth churn of 13.8k cycles did not reproduce the loss locally, under
either bind mode.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The healthcheck ran `tor --verify-config`, which parses the torrc and never
speaks to the daemon. Ansible renders that file from a template, so it is valid
by construction and the probe could not fail once it existed: bootstrap
progress, circuit establishment and descriptor publication were all invisible,
and `retries` was a dead parameter. Swarm therefore reported the service healthy
while Tor was still coming up, and the readiness that actually gates deploys had
to be rebuilt beside it, in onion_self_reach.py and playwright's
wait_onion_reachable.sh.
The torrc already opens a cookie-authenticated control port, so the probe now
authenticates there and asks GETINFO status/circuit-established. That is the
question the container is supposed to answer. It needs no package the image
lacks: bash carries /dev/tcp and od hex-encodes the cookie, both present in
debian:trixie-slim.
Measured against real Tor, the public flavor reports healthy 6.1s after start
and chutney 54.6s, each confirmed by "Bootstrapped 100% (done)" in the container
log. Chutney is the slower of the two because its entrypoint brings up a whole
private network and waits for consensus before the node tor starts; its
localhost advantage only pays off afterwards, at publish and rendezvous. Both
fit ten minutes with margin to spare, so the flavor-dependent start window goes
and every timing is now the platform default. Three nocheck markers go with it,
including an interval of 60s whose own reason said the probe was "worth running
rarely" while setting twice the default frequency.
Two paths and one condition that had grown copies are pinned in vars/main.yml.
The Dockerfile takes the script locations as build args rather than repeating
them, reaching the entrypoint through a fixed symlink because ENTRYPOINT in exec
form would not expand the arg and would silently keep pointing at the old path.
One behaviour change worth stating plainly: a Tor that never establishes a
circuit is now reported unhealthy after the start window, where before it was
reported healthy forever. That is the point of the change, and it moves such a
failure from the later Playwright run to the service itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The probe abandoned each attempt at --max-time 30 while Tor's SocksTimeout
sits at its 120s default (no SocksTimeout anywhere in the repo; positive
control SocksPort hits templates/torrc.j2:1). It gave up at a quarter of
Tor's working window, and destructively: every retry opens a fresh SOCKS
connection, discarding the descriptor fetches and rendezvous retries Tor
was still working through.
Run 32284541284 job 96178512465 shows the cost. Ansible recorded start
07:22:17.260, end 07:26:33.366, delta 0:04:16.105994. The first domain
burned its deterministic 6x30 + 5x10 = 230s and failed; the second then
succeeded on its first attempt but needed ~23s of that 30s window - 77%
of the limit. The transient had cleared at t~233s. The first domain's
budget ran out three seconds short of it.
Two round numbers, one behaviour: 5 attempts at 60s is 340s of patience
against today's 230s, so the same transient now has 107s of margin
instead of -3s, and a 23s connect sits at 38% of the window instead of
77%. LAM#1 (job 96179137200) is the near-miss that corroborates it - four
dead attempts, then a ~4s success on the fifth.
meta/tests.yml is required, not optional. roles/test-e2e-cli/tasks/run_one.yml:74
runs the script under `timeout {{ _cli_timeout }}`, and with no meta/tests.yml
this role fell through to the 3600s default in roles/test-e2e-cli/defaults/main.yml:3.
Widening the window moves the all-domains-fail ceiling from 16 domains to
11, and a 13-domain role demonstrably exists (Jellyfin#0, four passes of
"13 probed, 0 failed"). 7200 restores the headroom.
The `|| true` goes with it: the script runs under `set -uo pipefail` with
no -e, so the command substitution's exit code was already discarded and
line 63 branches on ${code}. tests/lint/filesystem/shell/test_no_or_true.py
never saw it because its regex wants #, ; or EOL after `true` and a `)`
follows here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Job 94849355639 of run 31823845576 failed on exactly one name: the bare apex
infinito.example, HTTP 000, "Could not resolve host", while
espo.crm.infinito.example answered 200 in the same second through the same
resolver. The probe printed dns= and stopped there, so the log named the
symptom and no artifact carried the cause. There is no dnsmasq log
(log-queries is off, the journal has zero dnsmasq lines), no CoreDNS log in
the compose artifact set, and no query output from the host at failure time.
Reproducing the resolver stack locally did not close it either. CoreDNS on the
CI Corefile, dnsmasq on the drop-in pulled byte-identical from that job's own
--diff at 20:58:35, and the full dnsmasq -> 127.0.0.11 -> CoreDNS chain with a
live forward path all answer the apex with 172.30.0.10. The configuration is
exonerated; what is missing is the transaction itself.
So the failure branch now prints the active resolver, any /etc/hosts match,
what the local dnsmasq answers, and what each upstream answers. The last two
are the split that matters: they separate "dnsmasq dropped it" from "upstream
dropped it", which is the fork the bench could not settle. Upstreams are read
from the drop-in directory rather than one file, so a shared dnsmasq carrying
foreign drop-ins is covered, and matching only bare-IP server= lines skips
server=/onion/127.0.0.1#9053.
TOR_DNSMASQ_CONF arrives through test.env instead of a hardcoded
/etc/dnsmasq.d, using the same :? idiom 8d7fff4d2 gave the other four values,
so the path keeps its single source in vars/main.yml. dig is opportunistic:
without bind-tools the resolver and hosts lines still land.
Verified: bash -n and shellcheck clean, the failure path exercised end to end
against a CI-shaped drop-in with 127.0.0.11 extracted and the /onion/ line
correctly skipped, test_comments_valid passed.
Two things this does not close. dig is probably absent on the Arch target that
failed, since the bind9-dnsutils lines in that job come from a Debian build
stage, not the target; adding bind-tools to the role would fix that and is a
separate concern. And the trigger is now rare, because pinning tor true across
the espocrm variants removed the mixed round that put the clearnet apex into
the vhost list at all. This is insurance, not a plan that pays out next run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
roles/svc-net-tor/templates/test.env.j2 sets RETRIES=6 and SLEEP_SECONDS=10, and
test-e2e-cli/tasks/run_one.yml renders that file unconditionally at :66 before
sourcing it at :74. The ${RETRIES:-20} / ${SLEEP_SECONDS:-15} fallbacks could
therefore never be reached -- a second source of truth that additionally
documented the wrong numbers: the header promised "default 20 / default 15"
while every run used 6 and 10.
They now take the :? form the two variables above them already use, so
test.env.j2 is the only place the values live, and a missing one aborts loudly
instead of silently probing three times longer than intended. The header stops
claiming defaults.
The sibling probes in svc-db-mariadb, svc-db-openldap and svc-db-postgres keep
${RETRIES:-20}: their test.env.j2 does not set the pair, so there the fallback is
live, not dead. Converting them would mean adding the values to those templates
first, and is deliberately left out.
Measured: a missing RETRIES exits 1 with "line 24: RETRIES: pass RETRIES as env
(attempts per domain, from svc-net-tor test.env)"; a complete env leaves the
probe output unchanged. shellcheck is clean against the staged content, and the
comment, shell and cli-asset-pairing lints pass.
The message wording avoids an apostrophe on purpose: inside ${VAR:?...} bash
still treats one as a quote character, which splits the abort message across the
next source line.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
HTTP 000 cannot tell a name that never resolved from a listener that never
answered. Both leave curl without a status and openresty without an access-log
entry, which is exactly the signature the apex infinito.example has produced in
every compose job of run 31543714113.
2a60ddb3c already stopped discarding curl's stderr, so the message now names the
layer -- (6) could not resolve, (7) failed to connect, (35) handshake. It never
names the address, and that is the half that decides between "the resolver
answered wrong" and "the resolver did not answer".
getent hosts goes through the same NSS path curl uses, so the verdict carries
what the node itself resolved:
[FAIL] localhost HTTP 000 dns=::1
[FAIL] no-such-host-xyz.invalid HTTP 000 dns=
Empty means resolution failed. An address together with a connect or TLS error
means the resolver answered and the far side did not. An address other than the
node's own means the resolver answered wrongly. Read next to curl's line, those
three branches separate.
Only the failure path changes; [OK] keeps its format, and getent's own noise
goes to /dev/null so a probe never fails on the diagnostic.
Measured: both branches above; getent and head present in all four node images
(fedora, arch, centos, debian) via docker run; shellcheck clean against the
staged content; tests/lint/repository/test_comments_valid.py,
tests/lint/filesystem/shell/ and test_cli_test_assets_paired.py pass.
This is not the fix for the apex failure. Nine mechanisms were eliminated by
measurement today -- CoreDNS, dnsmasq and Docker's 127.0.0.11 each resolve both
the apex and its sibling to 172.30.0.10; the vhost exists in http and https;
nginx validated six times; no server-name conflict, no proxy env, no conntrack
pressure. What remains needs the one fact the log has never carried, and this
line carries it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The apex infinito.example has failed six compose jobs with HTTP 000 and no
explanation, because the probe throws curl's stderr away:
curl ... "${scheme}://${domain}/" 2>/dev/null || true
--show-error was already set, so curl had the answer the whole time and the
redirect discarded it. Command substitution captures stdout only, so
%{http_code} and the accept regex at :61 are untouched, and || true still
swallows the exit code. test-e2e-cli/tasks/run_one.yml:85-92 prints stdout and
stderr when rc != 0, so the text reaches the job log.
This is NOT the fix for that failure. Three rounds of reasoning about it rested
on the premise that the apex answers 301 and curl follows it off-instance;
openresty.log:126 refutes it. That line logs the successful clearnet probe
"GET / HTTP/2.0" 200 with "Scheme: https" and User-Agent curl/8.18.0, so https
requests do appear in that log -- and the apex left no entry at all. curl never
completed hop 1, and there never was a 301 to follow. Whether the cause is
NXDOMAIN, a refused connect, or the 443 default_server's ssl_reject_handshake
(sys-svc-webserver-core/templates/default_server.conf.j2:17-20, which produces
exactly this signature: rejected handshake, 000 at curl, no access-log entry)
is what one curl error line will now say.
Costs up to six extra stderr lines per failing domain, RETRIES=6.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deleting the external onion probe left check_external, _CURL_REACHED_RCS,
the internal|external|both dispatch and the optional socks argument with
no caller, while the docstring still promised 'Exit 0 only when both
succeed' and that the external path 'must actually work end to end' —
the exact contract that deletion removed.
The script now takes one argument. Real-network reachability stays gated
where it blocks something: test-e2e-playwright's wait_onion_reachable.sh.
Wiring the CLI test into e2e-cli also armed its default retry budget:
20 attempts x curl --max-time 30 plus 19 x 15s sleep is 885s per failing
domain, serial. TEST_E2E_CLI_TIMEOUT is 3600s, so four failing domains
exhaust it and a tor deploy serves about twelve vhosts. test.env.j2 now
pins RETRIES=6 and SLEEP_SECONDS=10 — 230s per domain.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The role shipped a complete probe script at files/test.sh and no
templates/test.env.j2. discover_cli_roles globs the env template alone,
so the role was never discovered and the script had never run — and the
script hard-requires TOR_SOCKS and NGINX_SERVERS_DIR via ${VAR:?}, so it
could not have run without one either. Its own usage hint pointed at a
path that does not exist.
Moving the script to the discovered path and adding the env template
makes both halves line up. Verified against the live stack: the onion
answers through the Tor SOCKS proxy, 1 probed, 0 failed.
The new lint keeps the pair together and rejects a script left at
files/test.sh. The two halves fail differently and only one is loud: an
env template without a script aborts the play, a script without an env
template is never discovered and its silence reads like a pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Chutney pins controlport_base at 8000 and the tor container runs in the
host network namespace, so its nodes squatted 8000-8006 — right inside
the app port block that starts at 8001 and runs near-contiguously to
8088. nextcloud, wordpress, mediawiki and confluence could not bind
their proxy port. Shifting the base to 18000 frees the range; the grep
fails the build if upstream ever rewords the line.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test variant (meta/variants.yml, variant 0) switches the tor
service to the chutney flavor: the image ships chutney and the
entrypoint boots a private Tor network inside the container (all nodes
on 127.0.0.1, so the design is identical under compose host-networking
and swarm). The entrypoint bootstraps from /opt/chutney (chutney
creates its net/ state CWD-relative), extracts the DirAuthority/
TestingTorNetwork lines into /run/tor/chutney-net.conf and fails fast
when that file ends up empty, since the node would otherwise silently
join the public network. torrc %includes the conf only for the chutney
flavor.
Descriptor publish and rendezvous now run at localhost speed: per-app
onion e2e suites dropped from minutes to seconds in the compose
validation. The test variant also raises the footprint to
2g/1.0cpu/512pids (measured: at 1g/0.25 the ~9 extra tor nodes pin the
caps and desc publishing stalls); meta/services.yml stays static on
the public flavor and the old production footprint.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
filter-rr was added in dnsmasq 2.86, so it aborts dnsmasq startup on distros shipping older builds (CentOS Stream 9 has 2.85). Use filter-AAAA instead, which parses on every CI distro (dnsmasq 2.85 through 2.93, verified per-distro with dnsmasq --test).
Split onion_self_reach.py into internal/external/both modes. Keep the internal loopback fast path as the hard gate (later host-side steps resolve the onion through it), and probe the real Tor path best-effort (retries the full ~450s publish window, failed_when: false, warns): descriptor publication is async, needs an app backend behind the onion, and is blocked on CI runners with no Tor egress, none of which should fail a deploy. Playwright retries the real onion with its own onion-scaled timeouts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CI deploys on Arch runners died at the dnsmasq drop-in render: the pacman dnsmasq package ships neither /etc/dnsmasq.d nor an active conf-dir, so create the drop-in directory and enforce conf-dir in /etc/dnsmasq.conf (restarting dnsmasq when it changes). The self-reach gate then blocked every node without a web server on :80 (standalone svc-net-tor, dev environments): with the loopback shortcut an instant connection refusal proves the fast path works, so treat it as reached; likewise curl rc 52/56 over the Tor SOCKS require a completed rendezvous (descriptor published) and count as reached, while SOCKS failures and timeouts still fail.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the internal-only reachability one-liner with onion_self_reach.py, which gates the role until the node onion answers both internally (host resolver loopback fast path) AND over the real Tor SOCKS (descriptor published — the path Playwright/users take). Add a dnsmasq address= shortcut mapping the node's own onion suffix to loopback so host-side deploy steps skip the Tor circuit; other .onion still egress via Tor and container/Playwright traffic (Tor SOCKS) is unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
meta/services.yml gains tor.node (node onion address, filled per environment by the inventory) and tor.ports.local.socks (SOCKS port single source of truth). torrc renders SocksPort via the tor_socks lookup; the role test.sh takes TOR_SOCKS from the environment instead of a hard-coded 127.0.0.1:9050.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turn a NAT/edge node into a full Tor v3 onion node: DOMAIN_PRIMARY
becomes the node's .onion, so the whole stack (web, SSO, LDAP, CA)
resolves onion domains and is reachable over Tor without a public IP,
port forwarding, VPS or DynDNS.
- utils/tor_onion.py: offline v3 key mint (address + HiddenServiceDir
key files), verified against a known key->address test vector.
- cli .../inventory/onion: `init-env` pre-mints the node onion, writes
INFINITO_DOMAIN(+_RE) into .env and authoritative key files into
.onion-identity/hs (idempotent). The onion is a deploy-time input;
no in-deploy random minting (that would not match the pre-set domain).
- roles/svc-net-tor: Tor sidecar (Debian-built, network_mode: host),
installs the minted keys, torrc with one node HiddenService + a SOCKS
proxy restricted to loopback/DiD ranges. files/test.sh probes every
served onion domain over the Tor SOCKS.
- tls: .onion -> TLS off (onion v3 crypto is the transport auth; LE
cannot sign .onion) + app-level flavor override.
- harness/DNS: coredns resolves the onion zone via INFINITO_DOMAIN;
skip the clearnet DNS sanity check for onion; raise nginx
server_names_hash_bucket_size to 128 for long onion server_names.
- test-e2e-playwright: gated browser SOCKS proxy (PLAYWRIGHT_PROXY) +
navigation/action timeouts so specs reach .onion over Tor and fail
fast; the onion node points it at the Tor SOCKS.
- docs/requirements/031: full design.
Verified: dashboard + SSO (Keycloak) + 9 apps reachable over Tor
(HTTP 200 through the node hidden service). make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
