feature/updater's server.test.js starts the login broker without
SITE_TITLE, which the broker on this branch reads through requireEnv. After
the merge (9df6619903) the broker exited with 1, startBroker rejected, the
after hook called stopBroker(undefined) and node waited forever, so the
whole JavaScript unit suite hung.
feature/iam-profile-rbac carries a combined version of this file
(c3221a3e1c) that also sets SITE_TITLE; it wins this hunk when that branch
merges back, together with the broker change its extra test needs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts resolved:
- sys-front-inj-matomo: inject.yml keeps the store-i18n split; the cached
failed site list load (5b7284c4a4) is ported into resolve_site.yml next
to its token guard, and test_site_cache.py reads that file.
- web-app-bluesky: platform_administrator from store-i18n with the
prefix-free handle and its i8- reserved variant from feature/updater;
the broker CONFIG keeps siteTitle, storePath and chooseHandlePath.
- web-app-stalwart README: the managed Quick Setup section is dropped,
the docs role renders it.
- README.md: the role overview is regenerated from the merged tree.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The broker built every handle as kc-<username>. The PDS (0.4.66) caps the
first handle label at 18 characters, so kc-kevinveenbirkenbach (22) failed
createAccount with InvalidHandle: Handle too long, and the prefix only
existed to dodge the PDS's reserved names.
handle.js derives the label from the Keycloak username without a prefix and
adds i8- only after the PDS answers Reserved handle (1029 names, among them
administrator). A label that fits neither way leads to a form at /sso/handle
where the user picks 3 to 18 characters of [a-z0-9-] that contain a hyphen,
which no Keycloak username holds, and never contain i8-. The POST must come
from the proxied host's origin; the hostname is compared, since nginx passes
Host without its port.
store.js keeps username -> {handle, did, encrypted app-password} in a JSON
file (0600, written through a rename) in the new broker_data volume, so a
broker restart neither creates a second account nor resets a password. A
stored entry whose login the PDS refuses gets its password reset by DID; when
that login still fails the account is gone (a single-volume PDS rollback) and
the user is provisioned again. Concurrent first visits share one
provisioning. A derived handle that is taken is only recovered when no other
user holds it in the store, and an e-mail address a legacy kc- account still
holds ends in a 409 page instead of a retry loop.
Without the broker, 04_native_admin.yml follows the same rule: the plain
handle first, i8- on Reserved handle. The native Playwright tests sign in by
e-mail, which the PDS resolves, and accept either handle.
Evidence: the PDS rules and messages were read from the 0.4.66 image.
server.test.js runs the broker as a child process against a PDS stub that
answers with those messages: plain, i8-, form, 403/400/409, the takeover
guard, restart, lost account, reset and the concurrent visits, whose test
fails without provisionOnce. Unit tests 21/21, JavaScript suite 102/102,
make test-lint and make lint pass. The image builds and the node user writes
the store into a fresh volume.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Prometheus, Baserow, Checkmk and Jitsi inject their own logout button, mark it with data-injected-logout and send it to /oauth2/sign_out?rd=<Keycloak logout>. The shared logout script matched the same button through that attribute name and bound a second handler that navigates straight to Keycloak, which cancelled the sign_out request: on the prometheus+tor pass of fork run 40042690056 (job odoo#0) the request shows as ERR_ABORTED 12 ms before the Keycloak logout page loads, the onion oauth2-proxy log records no sign_out, and the session survived the logout. On clearnet the logout service's per-domain sweep hid it.
The scan now skips any element carrying data-injected-logout. All four roles set the link and the handler themselves from lookup('sso', ..., 'logout_url'), so their destination does not change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves 68 conflicted files. The governing rule was our structure with their
network semantics: the tor boolean becomes a network mode (clearnet | tor |
multi) throughout the variant layer, while this branch's architecture and mesh
axes, its extracted modules and its i18n content stay.
Where the two sides covered different ground, both are kept rather than one
picked:
- ca_injected answers a ca_client on the deployment's TLS mode OR on an SSO
issuer that resolves self-signed. 3a3e9ed2c9 records why: an issuer is not
an object store, so keeping only one side drops the other's coverage.
- get_merged_domains keeps the loud refusal when svc-net-tor is deployed
without a provisioned node, now gated on their tor_serves_host; their
node_onion_of returns an empty string there, which would have served every
onion-exclusive app on clearnet in silence.
- sso_cookie_domains stays the single source for oauth2-proxy's cookie scope.
Their per-network branch omitted the issuer domain on the onion arm, which
breaks the callback leg.
- The clearnet CSP batch carries --clearnet-timeout again; their batch dict
hardcoded zero while the role still passes the flag.
- The dashboard keeps its i18n content dir, image, version and design
resolution; their side reduced the logo sources to the bare service
provider values.
Their side wins where it is measurably better: the onion mirror rewrite is
gated per vhost rather than per app, the Playwright onion decision reads the
pass's own domain instead of the role's canonical one, and the matomo
reinjection and dashboard config render per network sibling.
Eight files were broken by the merge without ever carrying a conflict marker,
because git marks only the lines both sides touched:
- utils/cache/domains.py lost the onionize block that sat between two hunks
- roles/sys-front-inj-all/templates/body_filter.lua.j2 called it
- web-app-bluesky, keycloak sso_proxy: the get_entity_name filter was renamed
- trigger/__main__.py, playwright_coverage.py and its test: the deleted tor
module
- cli/administration/deploy/ci/timings.py read Label.tor
- plugins/filter/csp_filters.py called _aligned_url with four of six arguments
- tests/unit/python/utils/cache/test_domains.py referenced a renamed class
- tests/.../sys-ctl-hlth-csp/files/test_script.py mocked a function main no
longer calls
Verified: the variant layer (155 tests, 65 subtests), ca_injected (18),
the CI layer (175), the domains cache including the restored guard (32),
retrigger inputs (9), timings and the CSP health script (53). make
quality-high clears autoformat; its remaining findings are tracked separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A PDS that meets migrations it does not ship refuses to start with
"previously executed migration ... is missing", after the old container is
already gone. Before the stack starts, the role now runs
files/javascript/pds_migrations.js inside the pinned PDS image against the
existing data volume: it compares the migrations recorded in every actor
store and the service database with the ones the pinned @atproto/pds
ships, records the pinned version on a match, and stops the deploy with a
report naming both versions when the data is newer. A fresh volume skips
the check.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
b6c0f5e1a2 split the transport helpers out of env.js into onion.js and left this test requiring the old module. apiFetchOnion and apiGetOnion appear nowhere in env.js, so all four cases died on 'apiFetchOnion is not a function' rather than exercising the onion retry they describe.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A consumer proved its upload by logging the administrator into the Filer UI in
a browser and walking its JSON listing. The filer moved into the optional
console role with the split, so that path depended on a role a node need not
deploy, and since the engine declares only the api canonical, the lookup for
the filer URL raised outright for every consumer with seaweedfs enabled.
The shared helper now lists the bucket through ListObjectsV2 on the engine,
signed with the consumer's own identity rather than the administrator's. That
drops the console dependency and the Keycloak round trip per check, and proves
the per-consumer identity in s3.json reaches its own bucket.
SigV4 for the browser side is hand-rolled on node:crypto; its vectors are the
ones the engine probe's Python signer is pinned to, and a test fails if the two
drift apart. The signer deliberately does not set Host: Playwright derives it
from the URL and that is the value signed.
isAuthChain moves to the keycloak helper, where it belongs; web-app-funkwhale
pulls it through the personas barrel.
Measured against SeaweedFS 4.47: ETag and LastModified are returned, nested
prefixes list flat and complete, an in-place overwrite changes the token while
an untouched object keeps it, and s3.js completes the round trip end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every role README carries a ```mermaid flowchart and the documentation
showed it as source. A post-transform now replaces those literal blocks with
<pre class="mermaid"> in html builds, which the runtime picks up; every other
builder keeps the fence as it was.
sphinxcontrib-mermaid injects an inline init script, which this deployment's
CSP refuses, so the runtime and its initialisation are two linked files
instead: mermaid-init.js calls mermaid.initialize with securityLevel strict
and follows the reader's colour scheme. The diagram source is html-escaped
before it is embedded, or an & or < in a node label would break the page.
The runtime is fetched at image build time through ADD --checksum, so a
changed upstream file fails the build rather than shipping. Version and
digest are not ARG defaults, which test_arg_without_default forbids; they
travel meta/services.yml -> vars/main.yml -> compose.yml.j2 the way
DOCS_IMAGE and DOCS_VERSION already do, and an update block registers the
pin with the npm version watcher instead of exempting it.
Verified by building a page with sphinx: the diagram arrives as
<pre class="mermaid"> with its source escaped, a bash fence stays a
highlighted block, and both scripts are linked. A missing mermaid.min.js does
not fail that build, so a local run without the image stays green.
scripts/lint/mermaid.sh keeps validating diagram syntax through mmdc, which
npm installs unpinned. It therefore checks against a newer mermaid than the
11.4.1 readers get, and a diagram using syntax from in between would pass the
lint and render blank.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
When a node serves a role on several networks, the runner picks one of the
role's vhosts at random and keeps that pick for the flake retry. The .env is
rendered for the picked vhost, so APP_BASE_URL follows its network, and
CANONICAL_DOMAIN moves to its sibling there. Reports of a pick outside the
canonical network land under <application_id>+<network>.
network-leak.spec.js is staged next to every role spec and skipped unless the
role is served on more than one network. It fails when a request leaves the
network of the document that issued it, or when the browser blocks mixed
content. Only the SSO issuer may cross networks, so an onion pick reaches it
past the Tor proxy.
Onion detection reads the host of APP_BASE_URL before CANONICAL_DOMAIN, so
roles without CANONICAL_DOMAIN also get the onion timeouts and Tor retries.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every version whose tree holds a locale/<code>/LC_MESSAGES/docs.po with at
least one translation now gets one site per such language next to the
English one, built with `-D language=<code> -D html_copy_source=0` after
the English site is published. Sphinx reads one docs.po per language
(locale_dirs, gettext_compact = "docs"); YAML pages stay untranslated
through an extension that runs before the Locale transform.
The deployed working tree is built as the unlisted version `deployed` from a
snapshot baked into the image, because the git mirror only holds what is
pushed to main. The server answers /<version>/<code>/ and
/api/languages/<version>, a language without a site is a 404 rather than the
English page, and a switcher on every page lists the built languages and
keeps the page path. min_storage grows to 13GB for the translated sites.
The 47 docs.po catalogs hold the 24869 documentation messages; German is
partially machine-translated so far. The raw-docker and primary-domain lints
skip .po files, which carry the Markdown's own commands and hosts as
translation sources.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
locale/<code>/LC_MESSAGES/core.po now exists for all 183 non-source
languages with the 407 core messages. German is machine-translated in full;
the 29 languages the logout panel shipped are carried over from
logout_i18n.yml.
The keycloak logout panel reads every language but English from core.po
through the new i18n_catalogue lookup, so logout_i18n.yml keeps only the
English source strings. Each language takes its text direction from
meta/languages.yml. The Playwright spec asserts the German catalogue on the
served logout page.
Catalogs no longer carry Babel's placeholders: render() writes the author,
kevinveenbirkenbach@infinito.nexus, https://infinito.nexus, the Infinito.Nexus
Community License and contact@infinito.nexus as the bug address into every
header it writes, from constants in utils/software.py.
tests/lint/repository/test_i18n_catalogs.py holds that every language has a
core catalog, none exists for an unknown language, the core catalogs hold
exactly the current sources, and every translation keeps its protected
spans.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Sphinx configuration, extensions and generators lived in the separate
infinito-sphinx repository, whose image baked a snapshot of core at build time
from a base image name core no longer publishes. Only core consumed it, and its
generators read core's role layout, so the two repositories drifted without any
CI covering the pair.
The tooling now ships in roles/web-app-docs/files/, and web-app-sphinx is
renamed to web-app-docs. The container is a small service instead of a static
image:
- it keeps a git mirror of the repository in the docs_sites volume;
- /latest/ is the last commit of the default branch, rebuilt whenever a fetch
finds a new one while the previous build stays online;
- every release tag is selectable in the sidebar switcher and on /versions/,
and is built the first time someone opens it, behind a progress bar;
- replicas share the volume: the one holding builder.lock fetches and builds,
the queue and the build states are files beside the sites, so another
replica resumes the queue when the builder dies.
The theme rewrites every page serially after the build, which took about 75
minutes for the 4544 pages; that pass now runs on the -j workers with the
theme's own function. sphinx-apidoc skips tests/, because
tests/utils/bare_bootstrap_probe.py installs an import blocker when autodoc
imports it and killed the Sphinx workers. The roles overview is generated by a
separate process, so autodoc resolves utils from the version being built, not
from main. The role pages now show galaxy_info.description instead of the
top-level key that never existed.
The Sphinx packages join the dev extra, because the mirrored unit tests of the
tooling import them. min_storage now budgets the built image, the latest site
and a running build's scratch instead of the old image's size. The Playwright
suite is split into front page, versions and on-demand build files; the last
one selects an unbuilt tag in the overview and asserts the build gets queued.
Operators must rename the web-app-sphinx inventory group to web-app-docs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
0a52433dcb gave the standalone request fixture a Tor-resilient GET,
because the bundled socks client caps the CONNECT at 30s and a cold
onion circuit fails a request regardless of its timeout. POSTs and calls
made through context.request still went out once, and a circuit Tor
rejected ("Socks5 proxy rejected connection") was not recognised as
transient at all.
apiFetchOnion now takes the verb from opts.method and calls
request.fetch, so it serves both APIRequestContexts; apiGetOnion becomes
its GET shorthand. The rejection joins the transient patterns. Retries
stay onion-only and transport-only: clearnet keeps a single attempt and
HTTP or other errors re-throw at once.
The Mattermost LiteLLM spec routes its login, users/me, users/username
and channels/direct calls through it. The /posts call that sends the
prompt stays a single attempt, because a replayed post would message the
bot twice. The Hermes administrator spec reads /v1/models through
apiGetOnion.
env.test.js pins a POST surviving a rejected circuit, the single
clearnet attempt, no retry of a non-transport error over Tor, and the
GET shorthand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
performKeycloakLogin polled the page URL with toContain(canonicalDomain).
On the Keycloak auth page the app domain already sits in redirect_uri,
so the poll was satisfied before the login had completed. The denial
path and the biber/admin iframe checks used the same substring test.
All of them now compare hostnameOf(url) with the canonical domain, which
is a bare hostname (env.js reads it with /\.onion$/).
LOGIN_CONTROL_NAME never matched a login link that carries an
icon-font glyph: Playwright's accessible name includes the private-use
character, and the pattern only allowed whitespace before the text. The
post-login waitFor({state: "hidden"}) therefore passed at once, even
before oidc.js had swapped the link. The pattern now admits \p{Co}
before the text, so that wait really waits for the swap.
dom.test.js pins both: glyph-prefixed login controls match while Logout
and Accessories do not, an IdP URL carrying the app domain in
redirect_uri is rejected, a look-alike suffix is rejected, and
about:blank yields an empty host.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
confirmKeycloakLogoutIfPrompted waited the full 2s times the onion
multiplier for Keycloak's confirmation button even when Keycloak had
already redirected away because the end-session request carried an
id_token_hint. It now races the button against the page leaving the
logout endpoint (waitUntil: commit) and still clicks the button when
Keycloak renders the prompt. The dashboard's own confirmLogoutIfNeeded
stops the same way, and its poll interval no longer scales, so a slow
target is noticed sooner rather than later.
inAppLogout's first candidate poll ignored a visible settings link, so a
surface without a recognisable logout control or menu trigger, such as
the Mastodon home timeline, burned 30s times the multiplier before the
existing settings fallback ran. A visible settings link now ends that
poll as well; logout controls and menu triggers are still checked first
in every round. The settings selector moves to settingsLinksOn so both
callers share it.
Traces of run 34421944328: the dashboard's confirm probe expired after
2.03s on clearnet and 10.07s over Tor, and Mastodon's biber logout
polled 165s over Tor before reaching the settings page. Replayed in
Chromium, a Mastodon-shaped surface logs out in 3.24s instead of 33.45s
and a redirecting Keycloak logout returns in 0.20s instead of 2.00s,
while a prompting one is still confirmed. In the traced persona logouts
of six artifacts every other poll found a candidate in its first round,
so their path does not change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three waits in the logout flow were blind sleeps scaled by the onion multiplier.
A timeout is an upper bound and shrinks to nothing when the answer arrives early,
but page.waitForTimeout always burns its full duration, so multiplying it by five
spends the time whether the target is slow or not. The settle before the logout
sweep cost three seconds on clearnet and fifteen on an onion, every persona, every
role; the dropdown settle cost that again per menu trigger tried; and the poll
interval of waitForAnyLogoutCandidate was multiplied too, which made detection
five times coarser exactly where the target is slowest.
So the sleeps become waitForLogoutControl, a bounded poll that keeps the scaled
deadline and drops the scaling from the interval. It returns the moment a
logout-shaped element is visible, which is the condition the click that follows
needs anyway. Worst case is unchanged: with nothing to find it runs the deadline
out, exactly what the sleep did unconditionally. The first of the two
waitForAnyLogoutCandidate calls around the settle went with it; its result was
discarded, and the second still carries the thirty second deadline that covers a
navbar arriving late.
Four unit tests pin the contract, since the argument for the change is entirely
about when it stops waiting: it returns without sleeping when the control is
already there, returns as soon as one appears, costs at most the deadline when
none ever does, and polls on a fixed 250ms interval no multiplier touches.
Measured context, from run 34348821577: the same dashboard logout test takes
161.0 to 164.0s over Tor and 34.1 to 34.8s on clearnet across five jobs spanning
two distros, both deploy modes and three filesystems. A 1.9% spread across five
runner VMs is not network latency, it is a constant being multiplied, and the
ratio 4.73 is the multiplier itself. Solving the pair puts 32.0s of that test
inside scaled waiting against 2.3s of work. This commit removes the part of that
which is a sleep; the larger part is timeout caps on waits that expire and are
swallowed by a catch, which is a separate defect and stays for now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
33a778891 added the first JavaScript unit test that reaches into a
Playwright helper. csp.js imports `expect` from @playwright/test at module
scope, so requiring it drags an npm package into a suite that is
contractually free of them: Makefile:837 calls the target "node:test, no
npm dependency", scripts/tests/unit/javascript.sh installs only node, and
node_modules/ is gitignored. On a fresh CI checkout the import fails and
the whole JavaScript suite dies, not just the new file. It passes here
only because this machine has node_modules, so neither the local run nor
`make quality-high` can see the break.
The suite now intercepts that single request through Module._load and
hands back a six-line `expect` implementing the one matcher these paths
use, restoring the hook immediately afterwards. Probed: the request never
reaches resolution, `@playwright/test` never enters require.cache.
Also corrects a test name. "an enabled injector without a base URL is not
asserted" claimed to pin `if (!host) continue`, and does not: with no base
URL the host is "" and `includes("")` is true, so the suite stays green
with the guard deleted - verified by removing it. 33a778891's message
overstates this in two places, saying the suite pins every guard in
assertCspInjections and that dropping either toLowerCase trips exactly one
test. Dropping `host.toLowerCase()` trips nothing either, because
`new URL(x).hostname` is already lowercased.
Keeping the unit test rather than dropping it is a deliberate call: the
mirrored-unit-test lint exempts `test-*` roles as circular, so this file is
voluntary. It has already earned its place by catching two live mutants,
the enforced-over-report-only precedence and the csp-side toLowerCase.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
readCspString re-fetched the current URL through the Playwright API
request context and, when that failed, waited actionTimeout (30s) for a
<meta http-equiv="Content-Security-Policy"> that no template in this
repo emits. On a healthy runner that path works and costs one extra
request; when name resolution degrades it collapses, because the API
request context resolves DNS separately from the browser: the re-fetch
burns its 10s cap and the meta wait then drains whatever is left of the
caller's test budget.
That is what killed web-svc-logout's guest persona in run 32597248629,
job Pretix#0, whose swarm manager had a single resolver on 127.0.0.1
with dnsmasq no longer answering. Every guest test on that host inflated
to ~40s and the logout one, loading two external assets on top, died on
its 60s timeout having asserted nothing. The same role in Bluesky#0 and
YOURLS#0 of the same run, same role set, finished in 0.85s and 0.82s -
so this is a resilience fix, not a repair of a broken assertion.
gotoOnion now installs a page.on("response") recorder, so the header is
taken from the document the browser already received. That also covers
the surface reached through an SSO round trip, where no navigation
Response is in scope at the assertion - only guest.js holds one, biber,
admin and the suitecrm saml spec discard theirs. The re-fetch stays as
the fallback for a spec that navigates without gotoOnion; the meta
fallback is deleted.
The suite pins the header precedence (enforced over report-only) and
every guard in assertCspInjections: a disabled injector, a gate that
throws, an enabled injector without a base URL, a missing gate, a second
enabled injector, and the case-insensitive host match. Inverting the
precedence or dropping toLowerCase each trips exactly one test.
Verified against a headless Chromium and a local server counting
requests: the header survives a 302, no extra request is issued, a
missing injector host still fails the assertion, and a header-less page
falls back to exactly one re-fetch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bookwyrm has tests for the same trusted-header SSO bridge and settings
patcher; postmarks and baserow ship theirs untested, and the composer
require-dev stripper is a pure transformation nobody exercised.
Each test was checked by breaking its source: the admin gate failing
open, the group matcher loosened to a prefix, require-dev left in place
and the SSO gate forced open all turn the new tests red.
Four parse checks join the `lint-*` family, each using the language's own
parser rather than a reimplementation: `php -l`, `ruby -c`, `sqlfluff parse`,
and stylelint with an empty rule set (it raises CssSyntaxError whatever the
rules say, so the check stays syntax-only and grows no style opinion). Three
unit runners join `test-unit`: node:test, PHPUnit, minitest -- the first and
third ship with their runtime, so only PHPUnit adds a dependency.
Provisioning is best effort and the lint script decides whether it can run. A
machine without PHP must not fail the gate, but it must not look like success
either: the skip names how many files went unchecked. The first version of
these scripts read `git ls-files`, which inside the compose container returns
nothing (git refuses the bind-mounted repo as "dubious ownership") and reported
"No SQL files found." over 52 files -- green, and a lie. They use `find` now,
as the surrounding lint scripts already did.
sqlfluff needed two corrections before it parsed anything real. `%(name)s` is a
psycopg parameter marker, not SQL, so the placeholder templater is configured
rather than the files exempted; and the dialect follows each role's declared
engine (`meta/services.yml`), which is postgres for six roles and mysql for
svc-db-mariadb and pixelfed.
Recipes stay in scripts, not in the Makefile: the targets delegate, matching
every other target in the file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
