Commit Graph
7 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 29015311b6 feat(lint): derive container paths from meta/volumes.yml everywhere
meta/volumes.yml says where a volume lands inside its container. Wherever a
second file spelled the same path out, the two drifted independently: the mount
moves, the consumer keeps pointing at the old place, and nothing fails until the
process looks for content that is not there. e031b7f61 fixed exactly this for
web-app-moodle; this generalises it and makes the rule hold.

The lookup gains what a caller needs. `targets` maps each mounting service to
its container path, and `target` is the one path an entry mounts at -- present
only when every mount agrees. An entry that lands at different paths per service
omits `target` rather than picking one, and the caller names the service:

    "{{ lookup('volume', application_id, 'prometheus_config').target }}"
    "{{ lookup('volume', 'web-app-prometheus', 'host_sys').targets['node-exporter'] }}"

66 literals across 48 role files now derive. Replacement only happened where the
path stood alone -- /srv/app inside /srv/app/data is a different path -- and
every rewritten file was re-parsed, Jinja and YAML alike. Three sites the tool
refused rather than guess were done by hand: zammad's path sat in a Jinja list
literal, seaweedfs' inside a quoted string feeding a filter, and openresty's
/var/run/docker.sock is not a mount target at all but the socket on the host,
which only happens to spell the same string. That one carries the single
nocheck.

tests/lint/repository/spot/test_volumes_path_single_spot.py keeps it that way.
roles/*/files/ is out of scope by construction: it is copied verbatim into an
image and executed there, so a Dockerfile or entrypoint has no Ansible to
evaluate a lookup with.

One defect this found, and it was in the tooling rather than the roles: deciding
ambiguity AFTER filtering short paths made web-app-prometheus' host_sys look
single-valued -- it mounts /sys for cadvisor and /host/sys for node-exporter,
and /sys is below the length filter. The template would have rendered an
undefined `.target`. Both the converter and the rule now decide ambiguity over
every target and filter afterwards, and that call reads targets['node-exporter'].

Verified by resolving rather than by reading: all 70 accesses in the diff were
run against the real registry and every one yields a non-empty path equal to the
literal it replaced. make test-lint: 328 passed.

Not verified: no Ansible render, so a call site whose surrounding context cannot
build the applications registry would still fail at deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:29 +02:00
kevinveenbirkenbachandClaude Opus 5 f713ba8350 refactor(credentials): address them at secrets.credentials, like every other meta file
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.

The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.

--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.

Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.

tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.

Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 16:51:56 +02:00
kevinveenbirkenbachandClaude Opus 5 2dbd6eb108 feat(socialhome): implement the role so it actually deploys
The role was a scaffold: tasks/main.yml was a two-line comment, there was no
compose.yml.j2, no domains.yml, no volumes.yml, no env.j2. It deployed nothing
while carrying a Playwright spec and two matrix variants.

It also described the wrong project. meta/main.yml and meta/info.yml pointed at
jaywink/socialhome -- a Django ActivityPub hub. The target here is
social-home-io/socialhome: created 2026-04-21, MPL-2.0, aiohttp + SQLite +
Preact, federating peer-to-peer over Ed25519/WebRTC rather than ActivityPub.
Different codebase, no fork relationship.

It turns out to be the simplest web-app in the repo. One published multi-arch
image (ghcr.io/social-home-io/socialhome:2026.6.16, pinned), no Postgres, no
Redis, no queue, no worker, no collectstatic -- SQLite in one volume, one port,
migrations applied in-process at startup.

Two things the source forced that reading the README would not have:

- [standalone].external_url is reachable only through the TOML file, never
  through SH_* env (config.py:238-261). Without it routes/pairing.py answers
  422 NOT_CONFIGURED and federation pairing is dead, so the role renders a
  config and mounts it as type: config.
- provision_admin() runs from SH_ADMIN_PASSWORD but never calls
  mark_complete(), so / keeps serving the setup wizard instead of the login
  form -- upstream's own docstring claims otherwise. 01_setup.yml closes it
  with an idempotent POST to /api/setup/standalone.

No SSO sidecar, deliberately, against the pgadmin model this otherwise copies:
oauth2-proxy would redirect POST /federation/inbox and /api/pairing/accept to
Keycloak and break the one thing the software exists for.

Verified against the real image rather than reasoned about. The http_status
probe's exact argv ran inside the container and returned 0. /healthz answers
with the byte-for-byte shape the spec asserts. The setup POST returns 201 then
409 and flips setup_required true -> false, which is the whole justification
for 01_setup.yml existing. [standalone].external_url lands in the inbox_url of
POST /api/pairing/initiate. The container runs as uid 10001, matching
meta/users.yml.

Both persona blocks are proven, not assumed: biber gets 401 from the token
endpoint, and the 97 static files contain zero occurrences of logout / sign out
/ abmelden, so utils/logout.js has nothing to click. Injecting a control the
way web-app-prometheus does was rejected -- that model points its button at
OIDC.CLIENT.LOGOUT_URL, which does not exist for a role with no OIDC.

swarm stays disabled for the first round. The orchestrator chains
swarm -> compose -> host under mode_fail_fast, and sequencing measures serial
at 198 jobs, so a red swarm job here would skip 92 unrelated jobs and leave 85
roles untested. Compose-only drops that to 10. Flip it once compose is green.

Adversarial review caught five defects before this landed, three of them
certain rather than likely:

- SOCIALHOME_INTERNAL_URL hardcoded 127.0.0.1, which sys-svc-container rewrites
  to the bridge gateway whenever DOCKER_IN_CONTAINER is set -- from inside the
  Ansible container that address is the Ansible container, so both setup tasks
  would have burned five minutes of retries and failed the play.
- SH_WEBRTC_STUN_URL sat outside the coturn guard while the TURN pair sat
  inside it. Upstream reads by env PRESENCE, so the empty value would have
  beaten the in-app default and left WebRTC with no STUN at all.
- meta/csp.yml whitelisted blob: on worker-src. The bundle registers no service
  worker and constructs no Worker; the only blob of JavaScript is an audio
  worklet, and worklet modules load under script-src. Voice recording would
  have died with the app's own "Audio processing unavailable".
- tasks/main.yml hand-rolled meta: flush_handlers instead of including
  sys-svc-compose's finalize.yml, which silently skipped utils/post_deploy.yml.
- 01_setup.yml was gated on IS_STACK_HOST but not run_once, so a three-manager
  swarm would fire three concurrent setup POSTs at one ingress port.

TODO.md carried a claim that steered the CSP design and was simply false --
that img-src is not a directive csp_filters.py emits. It appends
"img-src * data: blob:" unconditionally at line 383.

lifecycle: alpha puts the role in the CI deploy matrix before it has ever run
anywhere, deliberately: the first CI round is its first execution. README and
TODO say so rather than claiming a maturity it does not have.

Still unverified: no browser has enforced the CSP, the swarm path was only
rendered, TURN against a real coturn is untested, and arm64 has not been
exercised. TODO.md lists them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-06 17:57:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2a023ca6a1 chore(lint): autoformat strict-failure semantics + claude.sh rename
* scripts/lint/autoformat.sh:
  - rename run_sort_claude_settings worker to run_claude (matches the
    renamed scripts/lint/claude.sh).
  - drop the EXIT trap that wiped status_dir; cleanup happens explicitly
    after `wait` returns, so a parallel worker that interrupts the
    parent never races a vanished status dir.
  - each worker now captures its tool's exit code and writes one of
    OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
    (markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
    real failures count as FAIL.
  - all workers run to completion regardless of individual tool failures
    (so the user always sees the full picture); after the summary, the
    script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
  (history preserved via git mv); update test_claude_settings_sorted.py
  fix-hint accordingly.

Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 01:30:49 +02:00
kevinveenbirkenbach 8a81630391 Added newline at file and POSIX compatible 2026-02-04 19:49:56 +01:00
kevinveenbirkenbach aeeae776c7 Finished implementation of correct application id 2025-07-21 11:33:35 +02:00
kevinveenbirkenbach 563d5fd528 Huge role refactoring/cleanup. Other commits will propably follow. Because some bugs will exist. Still important for longrun and also for auto docs/help/slideshow generation 2025-07-08 23:43:13 +02:00