Commit Graph
8 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 c9e7f950d9 Merge branch 'feature/updater' into feature/store-i18n
68 conflicts, resolved per hunk; resolutions agreed with the Updater
session were applied as agreed:

- svc-db-postgres (superuser password script and core) and
  sys-svc-rdbms realign_dedicated_db: updater's argv/stdin redesign with
  expand_argument_vars: false, nested into store-i18n's block structure.
- svc-db-redis 01_init: store-i18n's ACL password on stdin.
- sys-svc-proxy forwarded.conf.j2: store-i18n's handover-aware file,
  without the forwarded_base include (Host and X-Forwarded-Proto would
  be sent twice); forwarded_base serves the inline consumers, e.g.
  web-app-seaweedfs nginx.conf.j2.
- basic.conf.j2 and web-app-openproject proxy.conf.j2: issuer redirect,
  then handover, else updater's location/sso_acl.conf.j2; the
  server-level webserver_extra_configuration hook stays.
- web-app-keycloak: updater's admin-REST settings step and renumbered
  update steps; update/01_client.yml removed; store-i18n's design step
  and LDAP block kept; a73435cc9d arrived through rename detection.
- web-app-nextcloud OpenProject provisioning: updater's rewrite, all
  credentials on the Ruby runner's stdin.
- web-app-semaphore: store-i18n's canonical web root (a1d7835863).
- Role READMEs: no generated Cosmos/Architecture/Quick Setup/Credits
  sections; desk-* paths resolved to the dsk-* renames.

Integration fixes for what merged cleanly but did not fit together:

- web-app-matrix ansible vars: drop updater's second
  matrix_client_element_configuration_extension_json, which silently
  overrode store-i18n's design-aware value.
- url.origin instead of a stripped url.base in 19 places store-i18n
  added since 44aaa68924, same lookup targets (the double-slash lint).
- Package batch (18ea75049e, cb4d340de9): installs the deploy-start
  batch covers are removed (caffeine, cursor, blender, npm in dsk-gnt-*,
  with shared references), the rest carry batch: false and a nocheck
  with the verified reason (PyPI fallback, before the batch, vendor
  repository, configured flavor only, addon gate); dev-npm references
  the root npm id.
- sys-token-store write_many.yml: store-i18n's change-gated block and
  IS_SWARM_MODE.
- web-app-openproject: IS_COMPOSE_MODE in 02_migrate/run.yml; the
  seeded-admin lock (ccbf8f168d) ported into 04_admin/provision.yml and
  covered by the admin-state digest.
- Bucket policy (3b4ac40df5 and follow-ups) moved to web-svc-seaweedfs,
  where store-i18n's object store lives; the S3 admin secret travels on
  stdin instead of a task-level environment.
- act workflow.sh: no Exception-labelled comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 02:28:16 +01:00
kevinveenbirkenbachandClaude Opus 5.5 5f85f382e5 perf(packages): skip the package manager when its database already shows the state
The apt module loads its whole cache before it finds that every name is installed, about 2.4 s per call; an update deploy spent ~28 s on twelve such no-op calls from roles outside the batch. package_install now asks dpkg-query, pacman --query or rpm --query under LC_ALL=C first and returns unchanged only when every plain name is proven to be in the requested state; anything it cannot read in full runs the module as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-29 01:28:46 +01:00
kevinveenbirkenbachandClaude Opus 5.5 3dc35655a3 style: apply make autoformat to files committed unformatted
ruff format reflows the package batch, registry, plan and invocation code and tests and the Stalwart and Nextcloud tests, ansible-lint rewraps long Jinja lines in folded scalars of sys-service, Nextcloud and SeaweedFS, mbake restores the Makefile's final newline, and the Nextcloud OAuth test merges its nested with statements (SIM117).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-29 01:17:22 +01:00
kevinveenbirkenbachandClaude Opus 5.5 18ea75049e perf(packages): install every role's packages per host in one run at deploy start
A constructor task after the update role and the swarm auto-placement installs,
in one package-manager call, every package id of every role the host's groups
may run: meta dependencies, service edges and every role invocation readable
from tasks and handlers, conditions ignored. Ids that must stay with their role
(a conditional install, a repository added first, a need before update) carry
batch: false in meta/packages.yml and keep their task; roles reference shared
root ids with shared: true. 42 per-role install tasks go, and a lint keeps the
split: no batched id in a role task, none in a role run ahead of the batch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 22:54:36 +01:00
kevinveenbirkenbachandClaude Opus 5 18e644e53d fix(packages): install the tool apt needs to read an xz control member
ansible.builtin.apt extracts a deb's control member through the xz
binary rather than through dpkg's own library, and Cursor's deb
compresses that member with xz. On a target without xz-utils the
install failed after the download and the checksum had already
passed, which is the most expensive place to discover a missing
tool.

The deb branch of the download plan now installs it first, the way
build_calls installs its declared depends. The rpm branch is
unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 15:01:37 +02:00
kevinveenbirkenbachandClaude Opus 5 66533e6b50 feat(packages): install a checksum-pinned vendor artefact where no repo exists
A vendor that ships a package file but runs no repository left the
registry nothing to resolve, so cursor declared an empty list on Debian
and RedHat and the role deployed while installing nothing.

The new 'download' source names the artefact directly and pins it by
digest: the URL alone would install whatever the vendor serves today.
get_url verifies before and after the transfer, the artefact is cached
root-owned so no other account can swap it in the window before the
install, and a closing package call fails loudly when the artefact
registered under a name other than the one declared.

Cursor's rpm carries no signature, which is why dnf runs with
disable_gpg_check; the digest is the integrity control. All 92 of its
requirements resolve against Fedora 44 and CentOS Stream 10 repodata.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 14:08:16 +02:00
kevinveenbirkenbachandClaude Opus 5 75a2726389 feat(packages): install from Flathub where no distro package exists
An empty distro entry made package_install return skipped, so the role
deployed and the binary was simply missing. Blender#0 showed the shape:
'blender declare nothing to install on centos', then 'blender
--background' with rc 2 and no output at all.

A flatpak source closes the largest group of those gaps in one
mechanism. It takes application ids, installs the runtime and the
Flathub remote first, and links flatpak.command onto PATH -- without
that link the export is named after the application id, so a role
calling the plain binary still finds nothing and the fix would look
applied while changing nothing.

Roughly fifty entries move across, every id verified against Flathub's
appstream API rather than guessed; four candidates were wrong and did
not survive that check. The external probe now asks Flathub for a
flatpak entry instead of the distribution index, so the same
verification runs on every sweep.

jrnl and msi-perkeyrgb have no package anywhere and install from PyPI
instead: the task registers package_install and falls back to
sys-pip-install only 'when: ... is skipped', which keeps the registry
the single source of the per-distro decision rather than copying it
into the role.

The entries that legitimately install nothing keep their nocheck and
lose the deferral: the myhostname module ships inside systemd, the
kernel server inside nfs-utils, wine carries its own mono and gecko. A
new rule holds that line -- a justification may explain why nothing is
installed, but may no longer park the install behind a TODO. Twelve
gaps still fail it and stay red until each has a mechanism.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 02:09:32 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00