The test timed out: the publish-button locator no longer matched the v8.2.1
manage-video wizard, and the inline 78-byte MP4 stub was structurally invalid,
so PeerTube rejected the upload as "Video file unreadable" and left the Save
button disabled.
- test-seaweedfs.js: target the v8.2.1 save control (.save-button > button),
upload PeerTube's real video_short.mp4 fixture, and assert a version-tolerant
saved marker.
- add fixture roles/web-app-peertube/files/playwright/fixtures/video_short.mp4.
- test-e2e-playwright: stage role-local files/playwright/fixtures/ next to the
tests (the *.js glob ships neither binaries nor subdirectories), so specs load
a fixture via path.join(__dirname, "fixtures", <name>).
- docs: document the fixtures/ folder and the per-scenario test-*.js split in the
playwright spec contract and the runner README.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
scripts/meta/wait/mirrors.sh embedded its logic as a `python - <<PY` heredoc
plus a bash skopeo/retry loop, and the heredoc imported iter_role_images from
utils.docker.image_discovery — a module that does not exist (it is
utils.docker.image.discovery). That raised ModuleNotFoundError, so the process
substitution yielded zero refs and the script logged "Nothing to wait for" and
exited 0, silently turning the fork-PR "wait for mirrored images" safeguard
into a no-op.
Move the whole thing into cli/contributing/mirror/wait/ (sibling to sync and
resolver): an argparse entrypoint that enumerates role images via
iter_role_images, computes each GHCR destination via GHCRProvider, and polls
GHCRProvider.tag_exists with --attempts/--sleep-seconds until all are present
or it times out. mirrors.sh becomes a thin env->args wrapper invoking
python -m cli.contributing.mirror.wait, matching scripts/meta/mirror/sync_missing.sh.
Behaviour is identical to the previous bash, minus the broken import.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The baserow (infinito_sso.py) and bookwyrm (header_auth.py) trusted-header bridges now read identity only from the X-Forwarded-* headers nginx overwrites from the oauth2-proxy auth response. The client-injectable X-Auth-Request-*/Remote-User/X-Forwarded-Name variants are dropped, closing an authenticated-user header-injection escalation (including a groups->superuser path) in the claim-less edge case where X-Forwarded-* would be empty.
Add SSO security e2e tests (test-oidc-security.js) to both roles: forged-header gate bounce, trusted-header bridge endpoint gating, SSO-disabled inertness, and injected-header re-identification of an authenticated session. Wire the new spec into baserow's playwright.spec.js.
Update the bookwyrm header_auth unit test: precedence now uses two listed headers, plus a regression guard asserting proxy-uncontrolled headers are ignored.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
OpenTalk's images (web-frontend, controller, recorder) live on
registry.opencode.de, which was absent from both _MIRRORABLE_REGISTRIES and
_ALL_REGISTRY_PREFIXES in utils/docker/image/discovery.py. So is_mirrorable_image
returned False, iter_role_images skipped them, the GHCR mirror never copied
them, and the deploy pulled direct from registry.opencode.de -> TLS handshake
timeout (deploy failed).
Add the registry to both sets so the single iter_role_images gate enrolls the
3 images: the mirror-sync copies them to GHCR and the resolver rewrites the
deploy to pull from the GHCR mirror. The prefix entry strips the host so the
mirror destination is well-formed (no doubled registry). Add a regression test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
shopware/core 6.7.8.2 locks symfony/messenger (and console/framework-bundle)
to ~7.4.0. The Dockerfile then ran `composer require symfony/redis-messenger:^6.4`,
which forces symfony/messenger ^6.x and conflicts with the locked 7.4 family,
so composer aborts with exit code 2 at the builder stage and the image fails
to build (deploy failed).
Bump the constraint to ^7.4. symfony/redis-messenger 7.4 requires
symfony/messenger ^7.3|^8.0, satisfied by the locked 7.4, so the require
resolves cleanly against the scaffolded 6.7.8.2 project.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The SSO-proxy sidecar block gated on the host-scoped set_fact
_sys_stk_backend_sso_proxy_enabled. That fact leaks: when an OIDC app
consumes the shared SeaweedFS object store, the re-entrant
sys-svc-objstore -> web-app-seaweedfs -> sys-stk-backend run overwrites
the fact to True (seaweedfs is oauth2-gated). On return application_id
reverts to the OIDC consumer but the fact stays True, so the block wrongly
renders an oauth2-proxy sidecar for a non-oauth2 app and the
web-app-keycloak sso_proxy assert fails (deploy failed=1).
Drop the leaked fact and re-resolve lookup('sso', application_id,
'is_proxy_gated') inline at the gate (and the debug line). A lookup is
evaluated against the correctly-restored application_id each time, so the
nested seaweedfs run can no longer pollute it. Matches the file's existing
lookup-in-when idiom (objstore block) and the adjacent
sso_proxy_application_id set_fact that already trusts application_id here.
Fixes the deploy failures for OIDC apps on shared SeaweedFS
(web-app-mastodon, web-app-pixelfed, web-app-fediwall).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Baserow's native SSO is Enterprise-only, so gate the community image with the shared Keycloak oauth2-proxy and patch urls.py (via a custom Dockerfile) with a trusted-header bridge. The bridge provisions/repairs a local Baserow user from the oauth2-proxy identity headers and mints native refresh/access/user_session tokens, mirroring Baserow's own SSO redirect contract (?token=&user_session=).
proxy.conf.j2 performs the nginx handoff: an authenticated browser without a jwt_token cookie is 302'd to /api/infinito/sso/login/, which redirects back to /?token=&user_session= for the frontend middleware to consume. bootstrap_admin marks the superuser email_verified + completed_onboarding to skip onboarding.
Replace the LDAP service/variant with proxy-gating and rewrite the seaweedfs + persona Playwright tests to drive Baserow through its API using the SSO-minted JWT.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump:
- web-app-seaweedfs: chrislusf/seaweedfs 3.80 -> 4.33; nginx proxy 1.29 -> 1.31-alpine
- web-app-peertube: chocobozzz/peertube v8.2.0 -> v8.2.1
- web-app-prometheus: prom/alertmanager v0.32.2 -> v0.33.0
Hold Nextcloud at 33-fpm-alpine: NC 34 breaks the OIDC+LDAP login flavor
because pulsejet/nextcloud-oidc-login (app id oidc_login) caps at NC
max-version 33 with no NC 34 release, leaving the hidden native login
form with no OIDC button -> lockout (root cause of the PR #312 deploy
failure, surfaced only as a Playwright login timeout).
Mark the OIDC login plugins (oidc_login, sociallogin, user_oidc)
mandatory: true and fail the deploy at install time when a mandatory,
enabled plugin is not runnable (incompatible/unavailable), instead of
silently provisioning a login-broken instance. This makes a future NC
bump that outpaces the OIDC app fail fast with a clear cause.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BookWyrm auth rides the Keycloak SSO-proxy sidecar (flavor: oauth2); the
trusted-header bridge is auth-source-agnostic, so the LDAP-specific role
config, variant, and test added no coverage. Remove them:
- meta/services.yml: drop the ldap service block
- meta/variants.yml: drop the LDAP-only variant (now OIDC default + disabled)
- vars/main.yml: drop BOOKWYRM_LDAP_ENABLED
- templates/playwright.env.j2: drop LDAP_SERVICE_ENABLED
- playwright: drop test-ldap-login.js and its aggregator require
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BookWyrm core has no OIDC/LDAP login, so behind the oauth2-proxy a gated visitor only reaches BookWyrm's own login form. Add a trusted-header SSO bridge (files/sso/header_auth.py: RemoteUser middleware + backend) that turns the oauth2-proxy identity (X-Forwarded-Preferred-Username, set by Keycloak after OIDC or LDAP-federated login) into a real BookWyrm session, auto-provisioning the local user. Active only when PROXY_HEADER_SSO is truthy (= sso is_proxy_gated, centralised in BOOKWYRM_SSO_PROXY_GATED).
Supporting fixes: USE_S3_FOR_EXPORTS=false (exports stay local; SeaweedFS has no per-bucket ACL so private exports must not share the public media bucket) fixes the fresh-build settings crash / preferences 502; proxy_pass_request_body off on the keycloak sso_proxy /oauth2/ block so the auth_request subrequest no longer hangs 60s on gated POST bodies (504->500); split playwright.spec.js into per-test files; add positive (oidc/ldap -> authenticated session) + negative (forged header -> gated) SSO e2e tests; fix the avatar test (form-scoped Save locator + valid PNG); remove dead BookWyrm OIDC_* env; unit tests for the bridge + the gated settings patch.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The sys-ctl-hlth-webserver checker HEAD-requests each domain. The ungated S3 api endpoint returns 405 (Method Not Allowed) for HEAD on the service root, and 403 for anonymous access — both mean a healthy, secured gateway. A 200 (anonymous ListBuckets) can never match a HEAD probe, so the prior expectation made the health service exit non-zero and abort deploys. Verified in-container: the checker's CA-verified HEAD returns 405, which is in [403, 405].
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add files/playwright/test-seaweedfs.js (+ SEAWEEDFS_* playwright.env vars, spec require wiring, and removal of the playwright-service-flag nocheck above the seaweedfs service) for baserow, bookwyrm, mattermost, openwebui, peertube, penpot, pixelfed. Each test reuses the role's real login flow and triggers an upload that writes to the app's S3 bucket (file-field, avatar, chat attachment, composer photo, video), verified via the shared runSeaweedfsStorageCheck (bucket grew through the Filer UI).
openwebui: also export dismissAllOpenModals from _shared.js (used by the new test) and match the uploaded file by basename (not the full filename) to avoid the extension-split false-negative.
Exempt gitea and opentalk via meta/services.yml nocheck markers (seaweedfs-playwright + playwright-service-flag): gitea has no automatable UI login in this stack (oauth2-proxy + OIDC account-link / native login both fail; personas blocked); opentalk only writes S3 for recordings/whiteboard exports, which need the recorder service + a live multi-party meeting.
Builds on 9b6c6415f (shared runSeaweedfsStorageCheck helper + consumer lint).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Render the S3 objectstore config (templates/config/objectstore.config.php.j2) into NC's config dir before install, gated on objstore.enabled — NC cannot switch an already-installed instance to objectstore primary storage, so it must be present at install time.
Add the SeaweedFS consumer storage Playwright test (files/playwright/test-seaweedfs.js, playwright.env.j2 SEAWEEDFS_* vars, spec wiring; drop the playwright-service-flag nocheck for the seaweedfs service) plus the shared runSeaweedfsStorageCheck helper (admin logs into the Filer once, counts bucket objects via the Filer JSON listing, asserts growth) and the seaweedfs-consumer integration lint.
web-app-seaweedfs: bind weed to 0.0.0.0 (-ip.bind) so the multi-homed gateway is reachable by consumers that resolve the service name to its IP on the shared network.
nextcloud_install: classify 'was not found on the appstore' as unavailable (skippable, non-fatal) so a plugin without a release for the pinned NC version does not abort the whole deploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The webserver health check expected 403 on api.<base>, but the ungated S3 gateway answers an unauthenticated GET / with 200 and an empty ListAllMyBucketsResult (anonymous sees no buckets). The wrong expectation failed hlth-webserver once the Playwright stage stopped masking it. Expect 200 to match the live S3 root response; filer/master keep [302, 403].
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three deploy-validated fixes for the oauth2-gated Filer/Master UIs: (1) make filer the primary canonical domain so the oauth2-proxy redirect_url/callback lands on a gated domain instead of the ungated S3 api host; (2) point the sso-proxy upstream at the unique container name seaweedfs-proxy (and give the nginx proxy that hostname) so it no longer resolves a foreign app's generic 'proxy' container over a shared network, which had round-robined admin traffic to nextcloud; (3) decode the playwright env via the shared decodeDotenvQuotedValue so the administrator password's escaped characters are read correctly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The oauth2-proxy sidecar set cookie_domains to the app's primary domain only. A role served under several sibling subdomains (e.g. SeaweedFS filer/master/api under one base) therefore could not read its session cookie on the non-primary hosts, looping the auth chain. Add the common_dns_suffix filter and use it so cookie_domains becomes the longest DNS suffix shared by all of the app's domains: a single-domain app keeps its exact host (unchanged), a multi-domain app collapses to the shared parent so one session cookie spans every subdomain.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the prose that an agent could read as 'no variant passed implies mode=reinstall' with an explicit situation-to-command table plus a FORBIDDEN list: every mode=update is variant-pinned, and a no-variant mode=reinstall is reserved for the initial baseline and the final gate only. A failing variant is retried with mode=update variant=<idx>, never by restarting the whole matrix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The auto-generated "Older Releases" index in CHANGELOG.md emitted dash
bullets while the rest of the file uses asterisks, failing markdownlint
MD004 (ul-style, consistent) with 49 errors. Switch the archive index
generator to asterisk bullets, convert the existing entries, and align
the unit test assertions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The proxy service injected resource limits via a bespoke filter/include block without the trailing newline emitter, so Jinja's indent filter dropped the trailing newline and trim_blocks merged 'pids_limit: 128' with the following 'volumes:' key, producing an invalid compose.yml (mapping values are not allowed here). Mirror the proven base.yml.j2 idiom (set service_name, indent(4), then an explicit newline) so the keys stay on separate lines.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The persona CSP helper whitelists the Matomo origin when a persona navigates the oauth2-gated filer/master UIs (served through sys-front-inj-all). Declaring DASHBOARD_SERVICE_ENABLED without MATOMO_BASE_URL made the persona-required-envs lint fail; render the canonical Matomo base URL like the sibling roles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a meta lint asserting every web-app role with postgres or mariadb declares a seaweedfs object-store service, exemptable via the seaweedfs-required nocheck marker. Mark the 33 database-backed roles that do not use object storage, and register the rule in the suppression catalog.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Switch penpot, nextcloud, gitea, mattermost, openwebui, funkwhale, baserow, openproject, pixelfed, mastodon, bookwyrm, listmonk, peertube and opentalk to the engine-agnostic objstore lookup: the minio service entries are pinned off, a seaweedfs service (gated on the provider being in group_names, public for the media apps) replaces them, and the matrix variants pin the new flag. OpenTalk keeps object storage mandatory through an embedded fallback and provisions its bucket via weed shell.
Add objstore lookup SPOT keys consumed by these roles: endpoint (host:port), public_domain, public_bucket (public S3 host plus bucket) and public_url (scheme plus public_bucket, scheme derived from the provider TLS state). This removes the repeated lookup('config', lookup('objstore', ...), 'server.domains.canonical.api') / host:port concatenations from the consumer templates. Listmonk settings render their JSON values via the to_json filter.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a database-style object-store abstraction: the objstore lookup resolves the active engine (seaweedfs|minio) per consumer and yields a uniform S3 payload, objstore_secret_key is auto-generated like database_password, sys-svc-objstore delegates per-consumer provisioning to the resolved provider's objstore_init entrypoint (lazy-loading the provider when it was not preloaded), and compose/network/volume plumbing supports both a shared central instance and an embedded storage-only instance per consumer.
Add the web-app-seaweedfs provider role: Filer and Master UIs behind the admin-only oauth2-proxy via a role-local nginx and proxy.conf.j2 (S3 API domain stays ungated, admin UIs deny with 403 when SSO is disabled), per-consumer S3 identities from s3.json, idempotent bucket provisioning via weed shell, a /24 on the central-provider subnet range, Playwright coverage for admin/biber/guest plus consumer bucket existence, and unit tests for the objstore lookup and the per-engine compose volumes.
Mark web-app-minio lifecycle: unsupported, move its mc client image into a dedicated services.client entry, align it with the objstore credential contract, and make its consumer provisioning idempotent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the generated role×role integration matrix (026-integration-matrix.md
+ generator) and expand 026 with: required-gated install-failure handling,
mechanism=bridge for network bridges, addon variant/matrix axis, and the
matrix-backed cross-role integration backlog. The matrix scans every role's
meta/services.yml for infinito-native wired edges (☑️) and overlays curated,
web-verified upstream plugin edges (✅/🪙): 511 wired + 96 upstream across 82
roles. Header repeats every 10 rows and columns for readability.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Promote per-app addon/plugin/extension/module/bridge declarations to a
single file-rooted meta/addons.yml contract: unified schema, explicit
cross-role bridge rule, required-gated install-failure handling (hard
fail vs warn+skip), addon state as a variants.yml matrix axis, network
bridges (Matrix mautrix) as mechanism=bridge addons, external drift
monitoring, and a Playwright test per user-facing addon. Audit expanded
to matrix, discourse, pretix, mattermost, desk-chromium/firefox/gnome;
mobilizon/bookwyrm DB extensions classified out of scope.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Define the unified MCP service, identity, proxy, and test contract for
server and client roles, with a repo-wide audit, bridge discovery via
roles_with_service, and a Playwright test per integrated role.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
No-SSO roles authenticate via their own username/password form. The admin persona only treated a completed Keycloak round-trip as proof of reaching an authenticated surface, so a role like Snipe-IT — whose only logout control sits inside a collapsed account dropdown the visible-marker poll never sees — failed the gate even after a successful native login.
Track native-login success symmetrically: a real password form was found and submitted (loginAttempted), the form is gone afterwards, and the page parked on the role's canonical domain. The loginAttempted guard prevents a form-less public landing on the canonical domain from passing the gate without authentication. Also strip the verbose mid-code comments from the persona helper.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the two registry-driven networks templates into pure-Python
rendering functions plus thin Ansible lookup plugins:
roles/sys-svc-compose/templates/networks.yml.j2 -> {{ lookup('compose_networks') }}
roles/sys-svc-container/templates/networks.yml.j2 -> {{ lookup('container_networks') | indent(4) }}
The rendering logic lives in utils/networks/render.py as pure-Python
functions that take a registry + closure-style config/database lookups,
making the logic directly unit-testable without a Jinja stub harness.
The lookup plugins are thin wrappers that resolve application_id,
DEPLOYMENT_MODE, and swarm.network.encryption from the templar, build
the service_registry, and forward to the render functions.
Migrate 233 include lines across 85 production templates. Convert
web-app-matrix's hand-coded MDAD networks block (default + conditional
postgres + conditional openldap) to the generic lookup; this adds an
openresty attachment when services.sso.enabled (harmless extra
interface, pinned in test_matrix_mdad_consumer_attaches_postgres_ldap_sso).
Mark local-DB / redis sidecars (sys-svc-rdbms/templates/services/,
svc-db-redis/templates/service.yml.j2) with `# nocheck: networks-literal`
because they intentionally attach to default-only.
Extend the lint to scan ALL roles/*/templates/**/*.yml.j2 (not only
compose.yml.j2) and to forbid the legacy include form; the marker
above is the documented escape hatch.
Coverage:
- tests/unit/utils/networks/test_render.py (30 cases incl. encrypted=false literal pin)
- tests/unit/plugins/lookup/test_compose_networks.py (5 cases incl. lookup_loader arg-schema)
- tests/unit/plugins/lookup/test_container_networks.py (5 cases incl. lookup_loader arg-schema)
- tests/integration/infrastructure/compose/test_networks_render.py (10 snapshot scenarios)
- tests/integration/infrastructure/compose/test_network_includes.py (3 lint rules)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the hand-coded if-chains for db/ldap/ollama/proxy shared overlays
with a single registry-driven loop in sys-svc-{compose,container}/networks.yml.j2.
Each provider declares its overlay metadata under meta/server.yml.networks.overlay;
discover_role_services passes the block through verbatim into the service_registry.
Schema (modes/topology/consumer/aliases/proxy_resolvable) with implicit defaults:
- consumer.kind defaults to services_flags
- consumer.key defaults to <provides or entity_name>
- consumer.flags defaults to [enabled, shared]
- aliases defaults to [entity_name] (shared_net) or [] (default_net)
Openresty becomes a generic alias-collecting proxy: web-app-keycloak declares
a proxy_resolvable beacon with its public auth.<domain> alias; openresty sweeps
the registry at render time and harvests all proxy_resolvable aliases onto its
default-net overlay. Openresty no longer hardcodes keycloak's domain.
Lock the new behaviour with 9 render-snapshot tests covering provider self-render,
consumer attachment, beacon sweep, default-derivation, and compose-mode fallback.
Replace the Ansible-only `_entry.get('canonical') is not defined` filter with
`'canonical' not in _entry` so the templates also render correctly under vanilla
Jinja2 (test harness).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
