computer-playbook

14,682 Commits 8 Branches 78 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
kevinveenbirkenbachandClaude Opus 5	acc9e25c0f	refactor(volumes): one canonical mount surface, context-wired lookups The mount pipeline was split across two layers: compose_volumes and container_volumes lived as filter modules that ansible's filter loader rejected on every start ("has no attribute 'FilterModule'", two warnings per run), while the same-named lookups wrapped them thinly. The filter bodies now live inside their lookups (the tor_socks precedent), the filter files are gone, and the render tests moved next to the lookup tests. As lookups they see the templating context, so the application_id term every call site passed back to its own role is gone too: both lookups read application_id from context and fail loudly when it is missing. 107 templates and nfs_prep.yml drop the term; the calls collapse to lookup('compose_volumes') and lookup('container_volumes', service_name). The container_volumes call is now canonical - four leading spaces, service_name in the service position, \| indent(4) - enforced by a new lint (test_container_volumes_call_shape). Normalizing the 85 literal call sites surfaced that erpnext keyed its meta/volumes.yml mounts by container name instead of compose service key, the only role to do so; its meta now uses the service keys. wordpress's column-0 call, which rendered its first line at top level, is indented like every other. Template logic that only restated meta/volumes.yml is deleted: mailu fed compose_volumes an extra_volumes dict whose eight names were looked up from the very meta file compose_volumes reads itself; zammad's init and railsserver extra mounts were the volumes.yml entries minus the two services, which the meta now lists (the when-gated OIDC binds included). Conditionality that meta cannot express stays in code, but in role-local lookup plugins instead of template loops: matrix_bridge_mounts renders the per-bridge registration mounts, nextcloud_extra_volumes the feature-gated whiteboard/recording volumes (type: volume entries are declared unconditionally by compose_volumes, so gating must happen before it). Parity is proven, not assumed: every mechanically rewritten template is byte-identical to HEAD after the transformation regexes (95/95), and the semantic roles render identically through the real functions against HEAD's meta - zammad across six sso/tls polarities, all seven erpnext services, mailu's top level. make test-lint passes complete. Also: suitecrm's compose template pins container_extra_hosts with host_alias (the SOCKS proxy URL host.docker.internal never resolved in compose+tor), and the five disabled mautrix bridge addons use the empty-default config lookup so templating the applications dict stops emitting 125 errors per batch in every CI job. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-18 23:28:16 +02:00
kevinveenbirkenbachandClaude Opus 5	2dbd6eb108	feat(socialhome): implement the role so it actually deploys The role was a scaffold: tasks/main.yml was a two-line comment, there was no compose.yml.j2, no domains.yml, no volumes.yml, no env.j2. It deployed nothing while carrying a Playwright spec and two matrix variants. It also described the wrong project. meta/main.yml and meta/info.yml pointed at jaywink/socialhome -- a Django ActivityPub hub. The target here is social-home-io/socialhome: created 2026-04-21, MPL-2.0, aiohttp + SQLite + Preact, federating peer-to-peer over Ed25519/WebRTC rather than ActivityPub. Different codebase, no fork relationship. It turns out to be the simplest web-app in the repo. One published multi-arch image (ghcr.io/social-home-io/socialhome:2026.6.16, pinned), no Postgres, no Redis, no queue, no worker, no collectstatic -- SQLite in one volume, one port, migrations applied in-process at startup. Two things the source forced that reading the README would not have: - [standalone].external_url is reachable only through the TOML file, never through SH_* env (config.py:238-261). Without it routes/pairing.py answers 422 NOT_CONFIGURED and federation pairing is dead, so the role renders a config and mounts it as type: config. - provision_admin() runs from SH_ADMIN_PASSWORD but never calls mark_complete(), so / keeps serving the setup wizard instead of the login form -- upstream's own docstring claims otherwise. 01_setup.yml closes it with an idempotent POST to /api/setup/standalone. No SSO sidecar, deliberately, against the pgadmin model this otherwise copies: oauth2-proxy would redirect POST /federation/inbox and /api/pairing/accept to Keycloak and break the one thing the software exists for. Verified against the real image rather than reasoned about. The http_status probe's exact argv ran inside the container and returned 0. /healthz answers with the byte-for-byte shape the spec asserts. The setup POST returns 201 then 409 and flips setup_required true -> false, which is the whole justification for 01_setup.yml existing. [standalone].external_url lands in the inbox_url of POST /api/pairing/initiate. The container runs as uid 10001, matching meta/users.yml. Both persona blocks are proven, not assumed: biber gets 401 from the token endpoint, and the 97 static files contain zero occurrences of logout / sign out / abmelden, so utils/logout.js has nothing to click. Injecting a control the way web-app-prometheus does was rejected -- that model points its button at OIDC.CLIENT.LOGOUT_URL, which does not exist for a role with no OIDC. swarm stays disabled for the first round. The orchestrator chains swarm -> compose -> host under mode_fail_fast, and sequencing measures serial at 198 jobs, so a red swarm job here would skip 92 unrelated jobs and leave 85 roles untested. Compose-only drops that to 10. Flip it once compose is green. Adversarial review caught five defects before this landed, three of them certain rather than likely: - SOCIALHOME_INTERNAL_URL hardcoded 127.0.0.1, which sys-svc-container rewrites to the bridge gateway whenever DOCKER_IN_CONTAINER is set -- from inside the Ansible container that address is the Ansible container, so both setup tasks would have burned five minutes of retries and failed the play. - SH_WEBRTC_STUN_URL sat outside the coturn guard while the TURN pair sat inside it. Upstream reads by env PRESENCE, so the empty value would have beaten the in-app default and left WebRTC with no STUN at all. - meta/csp.yml whitelisted blob: on worker-src. The bundle registers no service worker and constructs no Worker; the only blob of JavaScript is an audio worklet, and worklet modules load under script-src. Voice recording would have died with the app's own "Audio processing unavailable". - tasks/main.yml hand-rolled meta: flush_handlers instead of including sys-svc-compose's finalize.yml, which silently skipped utils/post_deploy.yml. - 01_setup.yml was gated on IS_STACK_HOST but not run_once, so a three-manager swarm would fire three concurrent setup POSTs at one ingress port. TODO.md carried a claim that steered the CSP design and was simply false -- that img-src is not a directive csp_filters.py emits. It appends "img-src * data: blob:" unconditionally at line 383. lifecycle: alpha puts the role in the CI deploy matrix before it has ever run anywhere, deliberately: the first CI round is its first execution. README and TODO say so rather than claiming a maturity it does not have. Still unverified: no browser has enforced the CSP, the swarm path was only rendered, TURN against a real coturn is untested, and arm64 has not been exercised. TODO.md lists them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-06 17:57:24 +02:00
kevinveenbirkenbachandClaude Opus 5	92faba5d2d	feat(playwright): make the env template mandatory so specs actually run test_has_spec.py forces every roles/web-app-* to ship a Playwright spec and calls a missing one a hard error, because the persona contract must never be silently absent from the deploy capstone. Nothing forced the other half. The runner discovers roles by rglob('templates/playwright.env.j2') (discover_playwright_roles.py:53), so a spec without that file is staged nowhere and executed never. 21 of 76 web-app roles were in exactly that state. Verified rather than inferred: running the discovery filter directly returns 61 roles and none of the 21 is among them, `make compose-playwright` refuses without a rendered .env from a prior deploy, and `git log --all` on those paths shows the file never existed on any branch -- the single hit is web-app-gitlab on a fork branch. Their specs have never run once. 15 of the 21 deploy in CI today, reporting green while their only functional verification never fires. The new lint mirrors test_has_spec and carries a per-role opt-out that is used nowhere: uniform beats exceptions. Every one of the 21 now has a template written against its own meta/services.yml rather than a shared skeleton -- espocrm splits ADMIN_PASSWORD from ADMIN_NATIVE_PASSWORD because OIDC_ALLOW_ADMIN_USER leaves both login paths open, the four permanently auth-less roles declare their personas blocked instead of shipping a knowingly-red spec, and web-app-mastodon gained a real test-seaweedfs.js because no honest suppression existed for a role that does route Paperclip to S3. web-app-magento moves pre-alpha -> alpha here: it now has a spec that runs, which is what the lifecycle gate is about. Six meta/services.yml files change only where adding a template falsified an existing suppression -- gitlab and shopware both claimed the role 'ships no templates/playwright.env.j2', which stopped being true in this commit. Not resolved here, and the adversarial review says so plainly: the specs are enabled, not proven. None has executed against a live stack. The review found three classes of defect that land in follow-up commits rather than being smuggled in silently -- persona blocks that are missing or should be tied to sso.enabled, eleven suppressions naming a sys-svc-mail test suite that does not exist, and eight claiming a CSP assertion that cannot fire because no consumer role renders CSS_BASE_URL. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-06 14:17:55 +02:00

Author

SHA1

Message

Date

kevinveenbirkenbachandClaude Opus 5

acc9e25c0f

refactor(volumes): one canonical mount surface, context-wired lookups

The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.

As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).

The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.

Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).

Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.

Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-18 23:28:16 +02:00

kevinveenbirkenbachandClaude Opus 5

2dbd6eb108

feat(socialhome): implement the role so it actually deploys

The role was a scaffold: tasks/main.yml was a two-line comment, there was no
compose.yml.j2, no domains.yml, no volumes.yml, no env.j2. It deployed nothing
while carrying a Playwright spec and two matrix variants.

It also described the wrong project. meta/main.yml and meta/info.yml pointed at
jaywink/socialhome -- a Django ActivityPub hub. The target here is
social-home-io/socialhome: created 2026-04-21, MPL-2.0, aiohttp + SQLite +
Preact, federating peer-to-peer over Ed25519/WebRTC rather than ActivityPub.
Different codebase, no fork relationship.

It turns out to be the simplest web-app in the repo. One published multi-arch
image (ghcr.io/social-home-io/socialhome:2026.6.16, pinned), no Postgres, no
Redis, no queue, no worker, no collectstatic -- SQLite in one volume, one port,
migrations applied in-process at startup.

Two things the source forced that reading the README would not have:

- [standalone].external_url is reachable only through the TOML file, never
  through SH_* env (config.py:238-261). Without it routes/pairing.py answers
  422 NOT_CONFIGURED and federation pairing is dead, so the role renders a
  config and mounts it as type: config.
- provision_admin() runs from SH_ADMIN_PASSWORD but never calls
  mark_complete(), so / keeps serving the setup wizard instead of the login
  form -- upstream's own docstring claims otherwise. 01_setup.yml closes it
  with an idempotent POST to /api/setup/standalone.

No SSO sidecar, deliberately, against the pgadmin model this otherwise copies:
oauth2-proxy would redirect POST /federation/inbox and /api/pairing/accept to
Keycloak and break the one thing the software exists for.

Verified against the real image rather than reasoned about. The http_status
probe's exact argv ran inside the container and returned 0. /healthz answers
with the byte-for-byte shape the spec asserts. The setup POST returns 201 then
409 and flips setup_required true -> false, which is the whole justification
for 01_setup.yml existing. [standalone].external_url lands in the inbox_url of
POST /api/pairing/initiate. The container runs as uid 10001, matching
meta/users.yml.

Both persona blocks are proven, not assumed: biber gets 401 from the token
endpoint, and the 97 static files contain zero occurrences of logout / sign out
/ abmelden, so utils/logout.js has nothing to click. Injecting a control the
way web-app-prometheus does was rejected -- that model points its button at
OIDC.CLIENT.LOGOUT_URL, which does not exist for a role with no OIDC.

swarm stays disabled for the first round. The orchestrator chains
swarm -> compose -> host under mode_fail_fast, and sequencing measures serial
at 198 jobs, so a red swarm job here would skip 92 unrelated jobs and leave 85
roles untested. Compose-only drops that to 10. Flip it once compose is green.

Adversarial review caught five defects before this landed, three of them
certain rather than likely:

- SOCIALHOME_INTERNAL_URL hardcoded 127.0.0.1, which sys-svc-container rewrites
  to the bridge gateway whenever DOCKER_IN_CONTAINER is set -- from inside the
  Ansible container that address is the Ansible container, so both setup tasks
  would have burned five minutes of retries and failed the play.
- SH_WEBRTC_STUN_URL sat outside the coturn guard while the TURN pair sat
  inside it. Upstream reads by env PRESENCE, so the empty value would have
  beaten the in-app default and left WebRTC with no STUN at all.
- meta/csp.yml whitelisted blob: on worker-src. The bundle registers no service
  worker and constructs no Worker; the only blob of JavaScript is an audio
  worklet, and worklet modules load under script-src. Voice recording would
  have died with the app's own "Audio processing unavailable".
- tasks/main.yml hand-rolled meta: flush_handlers instead of including
  sys-svc-compose's finalize.yml, which silently skipped utils/post_deploy.yml.
- 01_setup.yml was gated on IS_STACK_HOST but not run_once, so a three-manager
  swarm would fire three concurrent setup POSTs at one ingress port.

TODO.md carried a claim that steered the CSP design and was simply false --
that img-src is not a directive csp_filters.py emits. It appends
"img-src * data: blob:" unconditionally at line 383.

lifecycle: alpha puts the role in the CI deploy matrix before it has ever run
anywhere, deliberately: the first CI round is its first execution. README and
TODO say so rather than claiming a maturity it does not have.

Still unverified: no browser has enforced the CSP, the swarm path was only
rendered, TURN against a real coturn is untested, and arm64 has not been
exercised. TODO.md lists them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-06 17:57:24 +02:00

kevinveenbirkenbachandClaude Opus 5

92faba5d2d

feat(playwright): make the env template mandatory so specs actually run

test_has_spec.py forces every roles/web-app-* to ship a Playwright spec and
calls a missing one a hard error, because the persona contract must never be
silently absent from the deploy capstone. Nothing forced the other half. The
runner discovers roles by rglob('templates/playwright.env.j2')
(discover_playwright_roles.py:53), so a spec without that file is staged
nowhere and executed never.

21 of 76 web-app roles were in exactly that state. Verified rather than
inferred: running the discovery filter directly returns 61 roles and none of
the 21 is among them, `make compose-playwright` refuses without a rendered
.env from a prior deploy, and `git log --all` on those paths shows the file
never existed on any branch -- the single hit is web-app-gitlab on a fork
branch. Their specs have never run once. 15 of the 21 deploy in CI today,
reporting green while their only functional verification never fires.

The new lint mirrors test_has_spec and carries a per-role opt-out that is
used nowhere: uniform beats exceptions.

Every one of the 21 now has a template written against its own
meta/services.yml rather than a shared skeleton -- espocrm splits
ADMIN_PASSWORD from ADMIN_NATIVE_PASSWORD because OIDC_ALLOW_ADMIN_USER
leaves both login paths open, the four permanently auth-less roles declare
their personas blocked instead of shipping a knowingly-red spec, and
web-app-mastodon gained a real test-seaweedfs.js because no honest
suppression existed for a role that does route Paperclip to S3.

web-app-magento moves pre-alpha -> alpha here: it now has a spec that runs,
which is what the lifecycle gate is about.

Six meta/services.yml files change only where adding a template falsified an
existing suppression -- gitlab and shopware both claimed the role 'ships no
templates/playwright.env.j2', which stopped being true in this commit.

Not resolved here, and the adversarial review says so plainly: the specs are
enabled, not proven. None has executed against a live stack. The review found
three classes of defect that land in follow-up commits rather than being
smuggled in silently -- persona blocks that are missing or should be tied to
sso.enabled, eleven suppressions naming a sys-svc-mail test suite that does
not exist, and eight claiming a CSP assertion that cannot fire because no
consumer role renders CSS_BASE_URL.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-08-06 14:17:55 +02:00