Commit Graph
19 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 c7727a5996 fix(xwiki): let a rotated superadmin password reach the worker
The credential was baked into the image: files/Dockerfile appended xwiki.superadminpassword to WEB-INF/xwiki.cfg from a build arg. The swarm matrix rotates it between passes, so the manager rebuilds the image with the new value, but 'docker stack deploy --resolve-image never' leaves the mutable tag untouched and a worker that already holds it never re-pulls. The task keeps serving the pre-rotation password and answers 401 to every authenticated call the deploy makes next, which is where both XWiki jobs of run 33445831564 died: 03_administrator.yml:7 and 04_extensions.yml:25, after 31 attempts each.

The entrypoint writes it at start instead, from the environment the role already renders into templates/env.j2 and that sys-svc-container/templates/base.yml.j2 attaches as env_file. Both copies of xwiki.cfg are written, because the stock entrypoint restores the data volume over WEB-INF on every start but the first; writing only WEB-INF would survive one boot and be reverted by the next. The keys are deleted and re-appended rather than substituted, so a generated password never reaches a sed replacement.

Read from xwiki/xwiki-docker at master to settle three assumptions: other_starts() does restore xwiki.cfg, the stock entrypoint never touches the superadmin keys, and CONTEXT_PATH defaults to ROOT. The delete pattern was exercised against all four key spellings xwiki_replace accepts, and a password carrying sed metacharacters was verified to arrive byte for byte.

Dropping the build arg also keeps the credential out of the image history.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 22:58:04 +02:00
kevinveenbirkenbachandClaude Opus 5 acc9e25c0f refactor(volumes): one canonical mount surface, context-wired lookups
The mount pipeline was split across two layers: compose_volumes and
container_volumes lived as filter modules that ansible's filter loader
rejected on every start ("has no attribute 'FilterModule'", two warnings
per run), while the same-named lookups wrapped them thinly. The filter
bodies now live inside their lookups (the tor_socks precedent), the
filter files are gone, and the render tests moved next to the lookup
tests.

As lookups they see the templating context, so the application_id term
every call site passed back to its own role is gone too: both lookups
read application_id from context and fail loudly when it is missing.
107 templates and nfs_prep.yml drop the term; the calls collapse to
lookup('compose_volumes') and lookup('container_volumes', service_name).

The container_volumes call is now canonical - four leading spaces,
service_name in the service position, | indent(4) - enforced by a new
lint (test_container_volumes_call_shape). Normalizing the 85 literal
call sites surfaced that erpnext keyed its meta/volumes.yml mounts by
container name instead of compose service key, the only role to do so;
its meta now uses the service keys. wordpress's column-0 call, which
rendered its first line at top level, is indented like every other.

Template logic that only restated meta/volumes.yml is deleted: mailu fed
compose_volumes an extra_volumes dict whose eight names were looked up
from the very meta file compose_volumes reads itself; zammad's init and
railsserver extra mounts were the volumes.yml entries minus the two
services, which the meta now lists (the when-gated OIDC binds included).
Conditionality that meta cannot express stays in code, but in role-local
lookup plugins instead of template loops: matrix_bridge_mounts renders
the per-bridge registration mounts, nextcloud_extra_volumes the
feature-gated whiteboard/recording volumes (type: volume entries are
declared unconditionally by compose_volumes, so gating must happen
before it).

Parity is proven, not assumed: every mechanically rewritten template is
byte-identical to HEAD after the transformation regexes (95/95), and the
semantic roles render identically through the real functions against
HEAD's meta - zammad across six sso/tls polarities, all seven erpnext
services, mailu's top level. make test-lint passes complete.

Also: suitecrm's compose template pins container_extra_hosts with
host_alias (the SOCKS proxy URL host.docker.internal never resolved in
compose+tor), and the five disabled mautrix bridge addons use the
empty-default config lookup so templating the applications dict stops
emitting 125 errors per batch in every CI job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 23:28:16 +02:00
kevinveenbirkenbachandClaude Opus 5 e3e81e816f refactor(healthcheck): declare probes in services.yml, render them from one lookup
Seven include templates under sys-svc-container decided how a container was
probed, and they read their inputs from template-local `{% set %}` scope.
That scope leaks: a value set for one service silently reached every service
below it, which is how taiga's front and gateway ended up probing a port
declared for a different service, and how bluesky's frontend probed a PDS
endpoint.

Probes now live in utils/docker/healthcheck.py, one class per flavor behind a
PROBES registry, and each service declares its own:

    healthcheck:
      flavor: curl
      path: health/ready
      start_period: 20m

`port_key` picks another entry under ports.internal for the services that
expose their probe elsewhere (keycloak on management, nextcloud talk on
signaling). Without a flavor the entry must carry an explicit test argv, so a
missing declaration fails loudly instead of rendering a probe against nothing.

Rendering HEAD's templates against the new output confirms 49 of 56 call
sites are byte-identical; the remaining seven are covered by an Ansible run.
Three probes change on purpose: erpnext's backend gains a check it never had,
bluesky's frontend drops the leaked PDS path, and msmtp_curl moves from a
hardcoded localhost:80 to the service's own port on 127.0.0.1 like every
other flavor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-03 04:05:38 +02:00
kevinveenbirkenbachandClaude Fable 5 4a23e0cb09 feat(services): declare locally-built images via services.<key>.custom
The custom flag moves from 33 hardcoded template kwargs into the merged
service config: 'custom: true' names the shared <entity>_custom build,
a string names a second distinct '<string>_custom' image inside one app
(nextcloud whiteboard). The image lookup reads the declaration as a
fallback, so config consumers (e.g. backup image matching) resolve the
exact deployed reference from one declaration point. nextcloud cron now
runs the nextcloud_custom build like the app service instead of the
upstream image.

A new services lint couples the declaration to reality in both
directions: custom requires a role Dockerfile, and a role Dockerfile
requires a custom declaration. The eight legacy roles deploying local
builds under upstream tags or ad-hoc image= overrides are ratcheted via
'# nocheck: dockerfile-custom'.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 19:48:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d4ee9a85ba refactor(compose): align templates to the service_name skeleton
Every compose.yml.j2 now declares services as '{{ service_name }}:' with
its '{% set %}' knobs on two-space lines above the key, includes at four
spaces and if/for tags aligned to the content they wrap. The layout
renders correctly because stack_host_template enables lstrip_blocks for
compose.yml.j2; indent(4, True) in the container base and '{%+' guards
in whitespace-sensitive scalars keep the shared fragments identical
under both render modes. A new lint test enforces the skeleton.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 02:36:58 +02:00
kevinveenbirkenbachandClaude Opus 4.8 594109018a refactor(compose): publish every container port through the container_ports lookup
Take the host IP as an optional third element of each term
([service, protocol, host_ip]) instead of an ip= kwarg, and build the whole
ports: block from declared ports.local/public (host) + ports.internal
(container). Convert every compose template across 84 roles, folding the
remaining raw var/numeric/specific-ip mappings (including mailu's per-interface
mail ports bound to MAILU_IP4_PUBLIC). Multi-term calls render one term per line.

Add the compose-ports-must-use-container-ports lint: it forbids any raw
"<host>:<published>:<container>" publish item under a ports: block and requires
every term to declare ports.internal. Remove
tests/lint/roles/test_swarm_internal_ports.py, now subsumed.

Fix a container_depends_on rendering regression. The lookup self-indented
(default indent=4), so the added 4-space placement + | indent(4) triple-indented
the block. Flip the lookup default to column 0 (consistent with
container_networks), make every call use the 4-space + | indent(4) form, and add
a lint enforcing it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 03:30:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 25f286d2bf feat(swarm): per-service internal ports and entity-named services
Give each reverse-proxied app a swarm-resolvable upstream: add services.<entity>.ports.internal.http to the role metadata and rename the main compose service from the generic 'application' to the entity name, so its short DNS alias on the shared overlay is unique. Thread the service name through compose, vars (*_SERVICE_NAME), volume mounts and the oauth2 origin host. Consolidate the split image/ports blocks in bridgy-fed and postmarks into the entity block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 19:49:18 +02:00
kevinveenbirkenbachandClaude Opus 4.7 cb27ff3756 refactor(networks): replace Jinja includes with compose_networks/container_networks lookup plugins
Convert the two registry-driven networks templates into pure-Python
rendering functions plus thin Ansible lookup plugins:

  roles/sys-svc-compose/templates/networks.yml.j2  -> {{ lookup('compose_networks') }}
  roles/sys-svc-container/templates/networks.yml.j2 -> {{ lookup('container_networks') | indent(4) }}

The rendering logic lives in utils/networks/render.py as pure-Python
functions that take a registry + closure-style config/database lookups,
making the logic directly unit-testable without a Jinja stub harness.
The lookup plugins are thin wrappers that resolve application_id,
DEPLOYMENT_MODE, and swarm.network.encryption from the templar, build
the service_registry, and forward to the render functions.

Migrate 233 include lines across 85 production templates. Convert
web-app-matrix's hand-coded MDAD networks block (default + conditional
postgres + conditional openldap) to the generic lookup; this adds an
openresty attachment when services.sso.enabled (harmless extra
interface, pinned in test_matrix_mdad_consumer_attaches_postgres_ldap_sso).

Mark local-DB / redis sidecars (sys-svc-rdbms/templates/services/,
svc-db-redis/templates/service.yml.j2) with `# nocheck: networks-literal`
because they intentionally attach to default-only.

Extend the lint to scan ALL roles/*/templates/**/*.yml.j2 (not only
compose.yml.j2) and to forbid the legacy include form; the marker
above is the documented escape hatch.

Coverage:
  - tests/unit/utils/networks/test_render.py (30 cases incl. encrypted=false literal pin)
  - tests/unit/plugins/lookup/test_compose_networks.py (5 cases incl. lookup_loader arg-schema)
  - tests/unit/plugins/lookup/test_container_networks.py (5 cases incl. lookup_loader arg-schema)
  - tests/integration/infrastructure/compose/test_networks_render.py (10 snapshot scenarios)
  - tests/integration/infrastructure/compose/test_network_includes.py (3 lint rules)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 18:42:04 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0ffbb73d72 refactor(images): central container_image lookup + migrate all roles
Add plugins/lookup/container_image.py: returns the full `image: "<ref>"`
line so templates drop the redundant image: prefix and surrounding
quotes. Supports `custom=True` to auto-construct `<entity_name>_custom`
for locally-built images, plus explicit `image=`/`version=` overrides
for exotic cases (taiga back-auth/front-oidc variants, decidim infinito
suffix, etc.).

* 25 unit tests for the lookup cover compose vs swarm prefix, custom
  flag, overrides, tag_only=True, missing version warnings.
* New strict lint (tests/lint/.../test_no_raw_image_in_compose_templates)
  fails on any raw `image:` line in roles/*/templates/*.j2 unless
  nocheck-suppressed. Legacy `image: "{{ lookup(...) }}"` form no
  longer accepted.
* Migrate 159 image refs across ~80 roles to the standalone
  `{{ lookup('container_image', application_id, '<service_key>') }}`
  form. 26 roles use `custom=True` (mariadb, postgres, openldap,
  jenkins, jira, gitlab, nextcloud, mediawiki, shopware, wordpress,
  odoo, joomla, openproject, bookwyrm, espocrm, hugo, littlejs,
  fediwall, roulette-wheel, openresty, confluence and more).
* Fix svc-db-redis application_id leak: service.yml.j2 is included
  into consuming roles where application_id evaluates to the outer
  role, so the image lookup needs the hardcoded 'svc-db-redis'.
* Integration tests (test_templates_require_image,
  test_build_requires_image) recognise the standalone lookup as an
  implicit `image:` at the same indent.
* Mediawiki swarm tasks: add explicit `:{{ MEDIAWIKI_VERSION }}` to
  the registry-prefixed image string because the custom var is now
  bare.

Only 3 legitimate exotic nocheck-suppressed image lines remain:
matrix-mdad runner (nested services.matrix.runner.image path),
matrix mautrix bridges (per-iteration dynamic) and svc-registry-docker
(chicken-and-egg self-reference).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 00:17:11 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6e49fe74fb feat(volumes): canonical dict-of-dicts meta schema + lookup('volume')
Migrate every role's meta/volumes.yml from ad-hoc dict-of-named-volumes
to a canonical dict-of-dicts schema where the YAML key is the semantic
short name and the entry body carries type / name / source / mounts /
nfs / mode / read_only. The optional 'name:' field is the docker volume
name (defaults to the key); the previous 'docker_name:' field is gone.

What's in this commit (210 paths):

Infrastructure (23)
- New filter plugins/filter/{compose,container}_volumes.py — render the
  top-level volumes:/configs:/secrets: section and per-service mount
  blocks from the canonical meta. container_volumes uses the SEMANTIC
  name as per-service mount source so it resolves against compose_volumes'
  top-level dict key (docker_name as source produced 'undefined volume'
  on swarm).
- New lookup plugins/lookup/{compose,container,volume}_volumes.py.
  volume(application_id, name) returns a normalised single entry;
  volume(application_id) returns the whole canonical dict.
- utils/cache/applications.py keeps a sibling registry
  _CANONICAL_VOLUMES_BY_ROLE outside the merged applications payload so
  raw Jinja source: strings never enter the templar walk (root-cause of
  the 200x cache-miss + recursive get_merged_applications storm).
- utils/roles/applications/mounts.py — schema validator + normaliser;
  list-shape is rejected.
- plugins/lookup/nginx.py: _volume_path_by_name reads canonical directly.
- scripts/tests/deploy/swarm/_context.sh: NFS_VOLUMES extraction handles
  both legacy dict and canonical dict shapes.
- roles/svc-prx-openresty/handlers/restart_and_validate.yml: gate
  compose-only validate+reload behind a single when-block; swarm uses
  service-update instead (bind-mounts of a FILE pin source inode at
  task-create time, so atomic write-tmp+rename leaves the running task
  on a deleted inode).
- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch to
  lookup('volume', application_id) | dict2items so NFS subdir
  pre-creation works under the new cache split.

Tests (12)
- New strict lints flagging inline mount sections in compose templates,
  single-file bind mounts, and legacy lookup('config', ..., 'volumes.X.Y')
  call sites — all volume-mount surfaces MUST flow through container_volumes
  / compose_volumes / volume lookups.
- Integration: test_volumes_compose_chain + test_volume_constants_are_lookups
  accept both the legacy and the new lookup('volume', app, 'X').name forms
  during the migration window.
- Unit tests for the new lookup, filter, mounts helper, plus updated
  nginx lookup test.
- Prometheus role-wiring test reads canonical meta for the
  alert_rules.yml bind mount.

Role migrations (175)
- 62 meta/volumes.yml converted to dict-of-dicts canonical shape
- 63 templates/compose*.yml.j2 replace inline volumes:/configs:/secrets:
  with lookup('container_volumes', application_id, '<service>') |
  indent(N) per service + a single trailing lookup('compose_volumes',
  application_id) for the top-level block
- 50 vars/main.yml drop dead volume constants and refactor 23 remaining
  legacy lookup('config', ..., 'volumes.X.Y') to lookup('volume',
  application_id, 'X').Y across 9 roles

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1d322f4e10 refactor(compose): replace lookup('applications') | compose_volumes(...) pipe with lookup('compose_volumes', ...)
Single SPOT lookup `plugins/lookup/compose_volumes.py` absorbs the
applications-lookup pipe step and auto-wires `DEPLOYMENT_MODE` plus
the `storage` group var from the templating context. Callers only
pass what genuinely varies per service (`application_id` and an
optional `extra_volumes` dict).

Behavior change (bug fix): the filter was always invoked with the
default `deployment_mode='compose'` because no caller passed it
explicitly, which silently skipped the swarm-NFS driver-opts
injection path. The lookup reads `DEPLOYMENT_MODE` from globals,
so swarm-NFS volumes now render the `driver: local` / `driver_opts:
type: nfs` block correctly. Callers can still override via
`deployment_mode=` / `storage=` kwargs when a template needs
non-default rendering.

Other coordinated changes:
- 81 template call sites migrated from pipe form to lookup form
  (single-line and multi-line shapes).
- `FilterModule` registration in plugins/filter/compose_volumes.py
  removed so the deprecated pipe form fails fast at render time.
  The underlying `compose_volumes()` function stays exported and is
  imported by the new lookup plus the existing 21-test filter unit
  test module.
- 10 new lookup unit tests pin the contract (term shape, kwargs
  pass-through, auto-wiring, override precedence, error cases).
- Integration test `test_compose_volumes_call_required.py` updated:
  asserts the new lookup form is present in every `compose.yml.j2`
  and adds a second test forbidding the deprecated pipe form
  anywhere under `roles/**/templates/`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 02:24:51 +02:00
kevinveenbirkenbachandClaude Opus 4.7 587a3a1796 feat(swarm): consolidate container_name + pull_policy into generic compose_only lookup
Replace per-key `compose_container_name` lookup with a generic
`compose_only(key, value)` SPOT that emits `key: "<value>"` only in
compose mode and nothing in swarm. Same gating logic now serves
container_name (157 call sites) and pull_policy (17 call sites)
without duplicate plugin code.

Why this consolidation:
- The earlier `build-key` lint rule was misdirected. Swarm tolerates
  `build:` with a harmless warning when `image:` is pre-pushed (via
  the existing handlers/swarm_registry_push.py flow in
  roles/sys-svc-compose). The actual swarm-rejected key is
  `pull_policy:` ("Additional property pull_policy is not allowed"),
  and 10 templates were silently broken because they declared inline
  `pull_policy: never` outside the existing `{% if DEPLOYMENT_MODE
  \!= 'swarm' %}` gate in roles/sys-svc-container/templates/build.yml.j2.
- A single generic lookup is cleaner than N per-key wrappers when
  the gating logic is identical (emit-key-or-nothing).

Lint changes in tests/lint/ansible/jinja/test_mode_incompatible_keys.py:
- New `pull-policy-key` rule, regex `^\s*pull_policy\s*:`, remediation
  points at compose_only.
- Existing `container-name` rule keeps its raw-key regex; remediation
  now also points at compose_only.
- Broken-split / broken-embed regexes rekeyed on the `compose_only`
  lookup name (was `compose_container_name`).
- `build-key` rule and its 12 nocheck annotations across templates
  removed.

Coordinated changes:
- 86 templates rewritten (157 container_name calls migrated;
  17 inline pull_policy lines replaced).
- 1 redundant `{% if DEPLOYMENT_MODE \!= 'swarm' %}` guard dropped in
  build.yml.j2 (the lookup self-gates).
- 12 unit tests pin the compose_only contract.
- Catalog (suppression.md) updated: `pull-policy-key` row added,
  `container-name` row remediation updated.
- Old `plugins/lookup/compose_container_name.py` and its test module
  deleted.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 02:24:50 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a0f0e3ac03 feat(swarm): SPOT lookup gates container_name emission per deploy mode
container_name is illegal in swarm mode when replicas > 1 (docker rejects
with "services.deploy.replicas: can't set container_name and X as container
name must be unique"). All compose.yml.j2 templates now route through a
single lookup that suppresses the key in swarm mode and emits it in compose
mode.

SPOT: plugins/lookup/compose_container_name.py
  Reads DEPLOYMENT_MODE from the templating context, returns
  'container_name: "<name>"' under compose and an empty string under swarm.
  Call sites read:

      {{ lookup('compose_container_name', MY_CONTAINER) }}

  No explicit DEPLOYMENT_MODE pass-through needed.

Sweep: 82 templates rewritten across roles/web-app-*, roles/web-svc-*,
roles/svc-*, plus 4 partial includes (sso_proxy, redis, mariadb, postgres).
For container names assembled from multiple Jinja fragments the args use
tilde concat (X ~ '-' ~ Y) so the lookup receives a single resolved string
and the call stays a single well-formed Jinja expression.

Lint: tests/lint/ansible/jinja/test_no_raw_container_name.py blocks
  (a) raw 'container_name:' YAML keys in any roles/*/templates/**/*.j2,
  (b) split-Jinja regression where the outer braces close mid-call:
      {{ lookup('compose_container_name', A }}-{{ B) }},
  (c) embedded-Jinja regression inside a string argument:
      lookup(..., "{{ X }}_suffix") (removed in ansible-core 2.23).
  The (b) and (c) regexes accept either single- or double-quoted lookup
  name and scan the full file so multi-line shapes also surface. Suppress
  per-site with '# nocheck: container-name' on the same or preceding line.

Tests: tests/unit/plugins/lookup/test_compose_container_name.py pins the
lookup contract: compose vs swarm output, whitespace tolerance, default-to
compose when DEPLOYMENT_MODE is absent, AnsibleError on missing or excess
terms, non-string name coercion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 02:24:50 +02:00
kevinveenbirkenbachandClaude Opus 4.7 222cc4015b refactor(web-app-xwiki): extract inline XML payloads + SPOT admin-group page
Six ``uri_retry`` tasks under ``roles/web-app-xwiki/tasks/`` carried
their XML payload as inline ``body: |`` blocks (CheckExtension Groovy
page, InstallExtensions Groovy page, the administrator user-page
PUT/POST pair, and the post-Fix-#8 XWikiAdminGroup PUT/POST pair).
Inline XML mixes Ansible YAML indentation with Jinja control flow
(``{% raw %}``-escaping for ``{{groovy}}`` delimiters), defeats
editor XML support, and silently rots when a future contributor
copy/edits the snippet without realising the surrounding ``body: |``
context.

Split by concern:

- ``templates/xml/*.xml.j2`` — six XML files that embed Jinja:
  ``user_page``, ``user_xwikiusers_object``,
  ``xwiki_admin_group_page``, ``xwiki_admin_group_member_object``,
  ``install_extensions_page``, ``main_webhome_page`` (the last one
  was already a template; moved here for symmetry with the new five).
- ``files/xml/check_extension_page.xml`` — fully static; rewritten
  without ``{% raw %}{{groovy}}{% endraw %}`` because the file is
  served via ``lookup('file', …)`` (no Jinja pass), so ``{{groovy}}``
  reaches XWiki literally.

Task references migrate to ``body: "{{ lookup('template',
'xml/<name>.xml.j2') }}"`` (or ``lookup('file', 'xml/<name>.xml')``
for the static case), normalising the previously bespoke long-form
``roles/web-app-xwiki/templates/main_webhome_page.xml.j2`` lookup in
the bargain.

SPOT-extract the magic string ``XWikiAdminGroup``:

- ``XWIKI_ADMIN_GROUP_PAGE: "XWikiAdminGroup"`` in vars/main.yml,
  documented as the page-name part used by REST URLs, the
  XAR-shipped page, the LDAP ``group_mapping`` argument, and the
  OIDC ``groups.mapping`` argument.
- Five call sites updated to ``{{ XWIKI_ADMIN_GROUP_PAGE }}`` or
  bare ``XWIKI_ADMIN_GROUP_PAGE`` inside list contexts:
  ``tasks/03_administrator.yml`` (×4 — two task names, the PUT URL,
  the POST URL), ``templates/xml/xwiki_admin_group_page.xml.j2``,
  ``templates/compose.yml.j2`` (LDAP group_mapping), and
  ``templates/xwiki.properties.j2`` (OIDC groups.mapping, in a
  commented example).

Lint guard: new ``tests/lint/ansible/tasks/test_no_inline_xml_body.py``
scans every ``roles/`` and ``tasks/`` YAML for ``body: |`` (or
``body: >``) folded scalars whose first content line starts with
``<`` and fails with a pointer at the matching ``templates/xml/...``
or ``files/xml/...`` alternative. JSON / shell / SQL bodies stay
allowed. Per-task opt-out: ``# nocheck: inline-xml-body`` on the
``body:`` line or the line above. Five unit tests cover the
scanner: matches XML body, ignores JSON body, honours same-line
nocheck, honours line-above nocheck, repo-wide sweep clean after
this refactor.

XWiki v17.10.8 distribution-merge behaviour was cross-checked
against upstream (``AbstractXarEntryType.upgradeType =
THREEWAYS``); the empty ``<member/>`` placeholder in the
XAR-shipped ``XWikiAdminGroup`` cannot overwrite our explicitly-added
member (different GUID / number).

``make test`` green across all 5 targets.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 13:25:15 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c54895dba2 refactor(depends_on): replace dmbs_*/dbms_base templates with container_depends_on lookup
Eliminate include-style depends_on assembly. The lookup resolves DB/redis
inductively from applications config and returns indented YAML (or "" when
no entries), removing the empty `depends_on:` block bug in libretranslate.

Migrates 43 compose templates from `{% include … dmbs_*.yml.j2 %}` to
`{{ lookup('container_depends_on', application_id) }}`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 19:43:19 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c0ef883fc1 refactor(compose_volumes): auto-resolve database volume, drop manual override
Filter no longer takes a `database_volume=` parameter. The volume name is
derived from the central provider's `services.<dbtype>.name` (shared) or
`<entity>_database` (dedicated) — mirroring `plugins/lookup/database.py`'s
`volume_prefix + host` construction. Simultaneous postgres + mariadb on
the same role now raises an AnsibleFilterError with a clear message;
support is deferred per the docstring TODO.

The integration test (renamed from `test_shared_service_volumes.py` to
`test_compose_volumes_call_required.py`) now requires every
`roles/*/templates/compose.yml.j2` to call `compose_volumes`, so the
top-level `volumes:` block is always rendered. Without it, flipping a
shared service to dedicated mode adds a named-volume reference that
Compose rejects with `service "<name>" refers to undefined volume`.

Adds the call to 14 templates that were missing it (dashboard, hugo,
sphinx, bridgy-fed, fediwall, littlejs, mig, mini-qr, navigator,
roulette-wheel, simpleicons, collabora, logout, openresty) and strips
the `database_volume=...` argument from every other role plus Taiga's
now-unused `TAIGA_DATABASE_VOLUME` var.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-11 11:22:38 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 f5a81736cf refactor(roles): convert Dockerfile.j2 templates to plain files/Dockerfile
Replace Jinja2 Dockerfile templates with static Dockerfiles using Docker
ARG statements. Variables are now passed as build args in compose.yml.j2,
following the existing pattern from web-app-bridgy-fed. Affects 12 roles:

- svc-db-openldap
- web-app-bookwyrm
- web-app-confluence
- web-app-espocrm
- web-app-jira
- web-app-joomla
- web-app-mediawiki
- web-app-pretix
- web-app-shopware
- web-app-suitecrm
- web-app-xwiki
- web-svc-simpleicons

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 00:28:37 +02:00
kevinveenbirkenbach 53e817fa22 refactor(compose): migrate from docker-compose.yml to compose.yml everywhere
- Rename all docker-compose.yml templates to compose.yml
- Update override and CA override filenames accordingly
- Adjust compose wrapper, lookup plugins, filters and path helpers
- Update roles, tasks, handlers and scripts to use compose CLI
- Rewrite documentation and admin guides to reference compose
- Update unit and integration tests to expect compose.yml
- Keep internal variable keys for backward compatibility where needed

https://chatgpt.com/share/69870272-b168-800f-84c3-6417398e441f
2026-02-07 10:14:21 +01:00