Commit Graph
8 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 c9e7f950d9 Merge branch 'feature/updater' into feature/store-i18n
68 conflicts, resolved per hunk; resolutions agreed with the Updater
session were applied as agreed:

- svc-db-postgres (superuser password script and core) and
  sys-svc-rdbms realign_dedicated_db: updater's argv/stdin redesign with
  expand_argument_vars: false, nested into store-i18n's block structure.
- svc-db-redis 01_init: store-i18n's ACL password on stdin.
- sys-svc-proxy forwarded.conf.j2: store-i18n's handover-aware file,
  without the forwarded_base include (Host and X-Forwarded-Proto would
  be sent twice); forwarded_base serves the inline consumers, e.g.
  web-app-seaweedfs nginx.conf.j2.
- basic.conf.j2 and web-app-openproject proxy.conf.j2: issuer redirect,
  then handover, else updater's location/sso_acl.conf.j2; the
  server-level webserver_extra_configuration hook stays.
- web-app-keycloak: updater's admin-REST settings step and renumbered
  update steps; update/01_client.yml removed; store-i18n's design step
  and LDAP block kept; a73435cc9d arrived through rename detection.
- web-app-nextcloud OpenProject provisioning: updater's rewrite, all
  credentials on the Ruby runner's stdin.
- web-app-semaphore: store-i18n's canonical web root (a1d7835863).
- Role READMEs: no generated Cosmos/Architecture/Quick Setup/Credits
  sections; desk-* paths resolved to the dsk-* renames.

Integration fixes for what merged cleanly but did not fit together:

- web-app-matrix ansible vars: drop updater's second
  matrix_client_element_configuration_extension_json, which silently
  overrode store-i18n's design-aware value.
- url.origin instead of a stripped url.base in 19 places store-i18n
  added since 44aaa68924, same lookup targets (the double-slash lint).
- Package batch (18ea75049e, cb4d340de9): installs the deploy-start
  batch covers are removed (caffeine, cursor, blender, npm in dsk-gnt-*,
  with shared references), the rest carry batch: false and a nocheck
  with the verified reason (PyPI fallback, before the batch, vendor
  repository, configured flavor only, addon gate); dev-npm references
  the root npm id.
- sys-token-store write_many.yml: store-i18n's change-gated block and
  IS_SWARM_MODE.
- web-app-openproject: IS_COMPOSE_MODE in 02_migrate/run.yml; the
  seeded-admin lock (ccbf8f168d) ported into 04_admin/provision.yml and
  covered by the admin-state digest.
- Bucket policy (3b4ac40df5 and follow-ups) moved to web-svc-seaweedfs,
  where store-i18n's object store lives; the S3 admin secret travels on
  stdin instead of a task-level environment.
- act workflow.sh: no Exception-labelled comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 02:28:16 +01:00
kevinveenbirkenbachandClaude Opus 5.5 d92375922d feat(roles): pull custom roles from inventory-declared sources
An inventory group web-app-cst-<name> with applications.<id>.custom: true and
a pinned source {url, version, path} is pulled into roles/ before the play,
read raw (Jinja, Ansible tags and credential URLs refused), placed through a
staging directory under an exclusive lock, and owned by a marker that names
its own directory, so a pull or roles-clean never replaces or removes a
directory it did not place and a failed pull leaves roles/ as it was.
Directory inventories, host lists and non-UTF-8 files behave as Ansible
reads them; generated test deploys pass --skip-pull. make roles-pull,
roles-clean and roles list, INFINITO_EXTRA_MOUNTS for local sources, the
web-app-cst category, the reserved prefix lint and a central
repository_files() that also yields the gitignored custom roles let every
lint scan them like in-tree roles. Supersedes the uncommitted 045 External
Roles draft and reuses its fetch, sync and mount design.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-29 00:02:58 +01:00
kevinveenbirkenbachandClaude Opus 5.5 78678d220f fix(inventory/validate): keep the keys after a vaulted value
The validator cut every !vault block with a regex that ran up to the next line at column 0, so the keys after a vaulted password vanished, a whole user among them. utils.cache.yaml has parsed the !vault tag itself since c4556fe72d, so the loader now hands the file to it unchanged and a vaulted value loads as its ciphertext. An inventory-only user that brings its own username no longer warns as unknown.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-28 15:33:49 +01:00
kevinveenbirkenbachandClaude Opus 5.5 685ec790eb fix(inventory): count the disable= record as read by the deploy
0327114c44 made the provisioning CLI keep the keys of the last `disable=` run in host_vars under _INFINITO_SERVICES_DISABLED and read them back on the next run. The inventory validator from 285ce910d2 only collects keys that group_vars, role defaults or vars declare, that templates guard, or that plugins read by name, so a key read through a constant in cli/ counted as unread, and every local compose-deploy with disable= aborted with "Inventory validation failed" before the play (repro: make compose-deploy mode=reinstall apps=web-app-keycloak disable=matomo variant=0).

read_variables() now includes DISABLED_SERVICES_RECORD from the disabler module, so the name has one source. An inventory carrying the record validates; an unknown key next to it is still reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 15:18:24 +02:00
kevinveenbirkenbachandClaude Opus 5 acfc79db94 fix(merge): green the gate after the updater merge
Git conflicts only on lines both sides touched, so the feature/updater
merge dropped symbols and markers silently where one side alone had moved
them. Execution found them, not review.

- plugins/lookup/image.py bound get_entity_name while its test patches
  plugins.lookup.image.entity_name; an alias satisfies an importer but not
  mock.patch, which reads the module's own binding. The module now uses
  entity_name, the spelling 60+ call sites share. The alias stays for
  utils/update_check and plugins/filter/update, whose tests patch the old
  name.
- A download-source package resolves at its pinned URL, never in a
  distribution index, so the availability probe asked the wrong oracle and
  reported cursor absent. It now issues a HEAD against the declaration's
  URL. The CDN answers 403 rather than 404 for an object that is gone, so
  both count as absent.
- The validate suite still guarded the name "design", which moved into
  roles/web-svc-design/meta/services.yml in 86cefeff26.
- 193 meta/services.yml and meta/variants.yml declare why the guide edge
  is not a variant dimension of their role, and why docs rides each role's
  smallest round: the README replay is the costliest partner in the matrix.
  No new CI round comes out of it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 19:43:54 +02:00
kevinveenbirkenbachandClaude Opus 5.5 285ce910d2 feat(inventory): reject unread keys and a missing Cloudflare token before the deploy
The Keycloak stage on echoserver ran ten minutes before sys-svc-dns stopped
it with "API.cloudflare.api_token must be set": the inventory still kept the
token under the flat name CLOUDFLARE_API_TOKEN, which nothing reads any
more. The validator that runs before every deploy now catches that class.

- api.py reports a host that selects DNS_PROVIDER: cloudflare without
  API.cloudflare.api_token, with group_vars/all merged under the host_vars
- variables.py reports every top-level key of host_vars and group_vars that
  core does not read: not declared in group_vars or a role's defaults or
  vars, not tested with "is defined" or "| default(", not read by name in a
  plugin, and not a SYSTEM_EMAIL_<KEY> the email lookup builds; a flat
  credential name such as CLOUDFLARE_API_TOKEN names its API.<provider>.<key>
  path; ansible_* keys are exempt

A key that is read in a way the scan cannot see takes
"# nocheck: inventory-variable", documented in suppression.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-23 16:18:12 +02:00
kevinveenbirkenbachandClaude Opus 5.5 6e776a5780 refactor(python): clear the open CodeQL code-quality alerts
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
  is imported by its modules (163 packages), so the constant counts as
  exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
  imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
  re-exports included, instead of silencing them with `noqa: F401`; the
  list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
  rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
  BMP-only positive class; it matches exactly the same 2111 code points
  as the old negated class across U+0000..U+10FFFF.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 21:40:58 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00