Commit Graph
34 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 29015311b6 feat(lint): derive container paths from meta/volumes.yml everywhere
meta/volumes.yml says where a volume lands inside its container. Wherever a
second file spelled the same path out, the two drifted independently: the mount
moves, the consumer keeps pointing at the old place, and nothing fails until the
process looks for content that is not there. e031b7f61 fixed exactly this for
web-app-moodle; this generalises it and makes the rule hold.

The lookup gains what a caller needs. `targets` maps each mounting service to
its container path, and `target` is the one path an entry mounts at -- present
only when every mount agrees. An entry that lands at different paths per service
omits `target` rather than picking one, and the caller names the service:

    "{{ lookup('volume', application_id, 'prometheus_config').target }}"
    "{{ lookup('volume', 'web-app-prometheus', 'host_sys').targets['node-exporter'] }}"

66 literals across 48 role files now derive. Replacement only happened where the
path stood alone -- /srv/app inside /srv/app/data is a different path -- and
every rewritten file was re-parsed, Jinja and YAML alike. Three sites the tool
refused rather than guess were done by hand: zammad's path sat in a Jinja list
literal, seaweedfs' inside a quoted string feeding a filter, and openresty's
/var/run/docker.sock is not a mount target at all but the socket on the host,
which only happens to spell the same string. That one carries the single
nocheck.

tests/lint/repository/spot/test_volumes_path_single_spot.py keeps it that way.
roles/*/files/ is out of scope by construction: it is copied verbatim into an
image and executed there, so a Dockerfile or entrypoint has no Ansible to
evaluate a lookup with.

One defect this found, and it was in the tooling rather than the roles: deciding
ambiguity AFTER filtering short paths made web-app-prometheus' host_sys look
single-valued -- it mounts /sys for cadvisor and /host/sys for node-exporter,
and /sys is below the length filter. The template would have rendered an
undefined `.target`. Both the converter and the rule now decide ambiguity over
every target and filter afterwards, and that call reads targets['node-exporter'].

Verified by resolving rather than by reading: all 70 accesses in the diff were
run against the real registry and every one yields a non-empty path equal to the
literal it replaced. make test-lint: 328 passed.

Not verified: no Ansible render, so a call site whose surrounding context cannot
build the applications registry would still fail at deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 97f6679243 fix(friendica): serialise first-run init through a task-slot gate
The upstream entrypoint serialises its first-run init with flock on friendica-init-sync.lock inside the shared /var/www/html, which sits on NFSv4 on this branch; local_lock=flock is silently ignored on vers=4 mounts, the lock contends across replicas over server-side locking and the set -eu entrypoint dies with exit 65 on every worker, pushing all replicas onto the manager. Build a custom image whose entrypoint gate lets only task slot 1 run the init while other slots wait until the volume VERSION matches the image VERSION - the same pattern nextcloud already uses.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-22 20:50:17 +02:00
kevinveenbirkenbachandClaude Fable 5 9c84ae2240 feat(engine): shared-aware container field + SPOT container names via services.<x>.name
Container-name vars derived their value inconsistently (hard-coded literals, entity_name composites) instead of the role-owned SPOT. Add a lint requiring every *_CONTAINER var to resolve through lookup('config', app, 'services.<x>.name'), the bare entity_name form, or the engine lookup; give each multi-container worker a name: in meta/services.yml (with realistic resource limits) and switch its var to the lookup. Extend the engine lookup with a shared-aware 'container' field (central provider container when shared, embedded <entity>-<engine> otherwise) and gate zammad's embedded memcached sidecar on not-shared so its container_name never collides with the central memcached.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:00:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 72030f47d8 fix(swarm): route container-exec through the hosting node (resolve_host_cid)
In swarm an app task can be scheduled on a worker, but the role exec'd
`container exec $(resolve-container-id ...)` on the manager -> "No such
container". Resolve the hosting node + node-local container id and delegate
each exec to that node.

- database.py: add a `service_name` want; resolve_host_cid/wait_converged
  gain `*_service_name` for cross-stack services (mailu's shared mariadb)
- mailu wait-for-mariadb (DB probe + admin), mastodon db:migrate,
  gitea migrate/admin-create, friendica manager-ops/patch/addons/bootstrap
  (~13 exec sites) switch to resolve_host_cid + delegate_to

Compose-safe: resolve_host_cid degrades to HOST_NODE=inventory_hostname
(delegate is a no-op) + LOCAL_CID=bare compose container name, matching the
8 roles already using this pattern.

Note: friendica also needs service_replicas=1 (next bundle) so its resolved
container id does not churn across replicas.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 22:55:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 36ebf34b64 Merge branch 'main' into feature/swarm-nfs
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 17:13:55 +02:00
kevinveenbirkenbachandClaude Opus 4.8 07da73d88c feat(friendica): migrate ldapauth to meta/addons (req 026)
ldapauth addon bridges [ldap]; ldap+sso service blocks stay in meta/services.yml; tasks/04_addons.yml reads the new addons path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:30:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0ffbb73d72 refactor(images): central container_image lookup + migrate all roles
Add plugins/lookup/container_image.py: returns the full `image: "<ref>"`
line so templates drop the redundant image: prefix and surrounding
quotes. Supports `custom=True` to auto-construct `<entity_name>_custom`
for locally-built images, plus explicit `image=`/`version=` overrides
for exotic cases (taiga back-auth/front-oidc variants, decidim infinito
suffix, etc.).

* 25 unit tests for the lookup cover compose vs swarm prefix, custom
  flag, overrides, tag_only=True, missing version warnings.
* New strict lint (tests/lint/.../test_no_raw_image_in_compose_templates)
  fails on any raw `image:` line in roles/*/templates/*.j2 unless
  nocheck-suppressed. Legacy `image: "{{ lookup(...) }}"` form no
  longer accepted.
* Migrate 159 image refs across ~80 roles to the standalone
  `{{ lookup('container_image', application_id, '<service_key>') }}`
  form. 26 roles use `custom=True` (mariadb, postgres, openldap,
  jenkins, jira, gitlab, nextcloud, mediawiki, shopware, wordpress,
  odoo, joomla, openproject, bookwyrm, espocrm, hugo, littlejs,
  fediwall, roulette-wheel, openresty, confluence and more).
* Fix svc-db-redis application_id leak: service.yml.j2 is included
  into consuming roles where application_id evaluates to the outer
  role, so the image lookup needs the hardcoded 'svc-db-redis'.
* Integration tests (test_templates_require_image,
  test_build_requires_image) recognise the standalone lookup as an
  implicit `image:` at the same indent.
* Mediawiki swarm tasks: add explicit `:{{ MEDIAWIKI_VERSION }}` to
  the registry-prefixed image string because the custom var is now
  bare.

Only 3 legitimate exotic nocheck-suppressed image lines remain:
matrix-mdad runner (nested services.matrix.runner.image path),
matrix mautrix bridges (per-iteration dynamic) and svc-registry-docker
(chicken-and-egg self-reference).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 00:17:11 +02:00
kevinveenbirkenbach 29519603ef Merge remote-tracking branch 'origin/main' into feature/swarm-nfs 2026-06-14 20:32:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6e49fe74fb feat(volumes): canonical dict-of-dicts meta schema + lookup('volume')
Migrate every role's meta/volumes.yml from ad-hoc dict-of-named-volumes
to a canonical dict-of-dicts schema where the YAML key is the semantic
short name and the entry body carries type / name / source / mounts /
nfs / mode / read_only. The optional 'name:' field is the docker volume
name (defaults to the key); the previous 'docker_name:' field is gone.

What's in this commit (210 paths):

Infrastructure (23)
- New filter plugins/filter/{compose,container}_volumes.py — render the
  top-level volumes:/configs:/secrets: section and per-service mount
  blocks from the canonical meta. container_volumes uses the SEMANTIC
  name as per-service mount source so it resolves against compose_volumes'
  top-level dict key (docker_name as source produced 'undefined volume'
  on swarm).
- New lookup plugins/lookup/{compose,container,volume}_volumes.py.
  volume(application_id, name) returns a normalised single entry;
  volume(application_id) returns the whole canonical dict.
- utils/cache/applications.py keeps a sibling registry
  _CANONICAL_VOLUMES_BY_ROLE outside the merged applications payload so
  raw Jinja source: strings never enter the templar walk (root-cause of
  the 200x cache-miss + recursive get_merged_applications storm).
- utils/roles/applications/mounts.py — schema validator + normaliser;
  list-shape is rejected.
- plugins/lookup/nginx.py: _volume_path_by_name reads canonical directly.
- scripts/tests/deploy/swarm/_context.sh: NFS_VOLUMES extraction handles
  both legacy dict and canonical dict shapes.
- roles/svc-prx-openresty/handlers/restart_and_validate.yml: gate
  compose-only validate+reload behind a single when-block; swarm uses
  service-update instead (bind-mounts of a FILE pin source inode at
  task-create time, so atomic write-tmp+rename leaves the running task
  on a deleted inode).
- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch to
  lookup('volume', application_id) | dict2items so NFS subdir
  pre-creation works under the new cache split.

Tests (12)
- New strict lints flagging inline mount sections in compose templates,
  single-file bind mounts, and legacy lookup('config', ..., 'volumes.X.Y')
  call sites — all volume-mount surfaces MUST flow through container_volumes
  / compose_volumes / volume lookups.
- Integration: test_volumes_compose_chain + test_volume_constants_are_lookups
  accept both the legacy and the new lookup('volume', app, 'X').name forms
  during the migration window.
- Unit tests for the new lookup, filter, mounts helper, plus updated
  nginx lookup test.
- Prometheus role-wiring test reads canonical meta for the
  alert_rules.yml bind mount.

Role migrations (175)
- 62 meta/volumes.yml converted to dict-of-dicts canonical shape
- 63 templates/compose*.yml.j2 replace inline volumes:/configs:/secrets:
  with lookup('container_volumes', application_id, '<service>') |
  indent(N) per service + a single trailing lookup('compose_volumes',
  application_id) for the top-level block
- 50 vars/main.yml drop dead volume constants and refactor 23 remaining
  legacy lookup('config', ..., 'volumes.X.Y') to lookup('volume',
  application_id, 'X').Y across 9 roles

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a4cb5b89e1 feat(swarm): swarm-aware container exec via container_address lookup
Adds a mode-aware `container_address` lookup plugin and a
`resolve-container-id` helper script that resolve swarm task IDs at
exec time (compose mode returns bare names unchanged). Migrates ~25
roles + ~28 task files from `command:` to `shell:` so the lookup's
`$(...)` subshell evaluates correctly, and extends `database` lookup
with an `address` key for cross-role DB exec.

Three new lint tests enforce the contract: every container exec
target goes through the resolver, lives behind an IS_STACK_HOST gate
(walks the include graph one hop deep), and uses `shell:` rather than
`command:` when the target is a Jinja reference. Existing nocheck
suppressions on bare-name DB exec sites (nextcloud redis, listmonk,
moodle OIDC, mailu) are replaced with the lookup-based form.

Fixes the systemd Environment= injection in sys-ctl-cln-disc-space
(it never expanded $(...)) by passing STACK / SERVICE_KEY parts and
resolving inside the script. Extracts shopware admin and mediawiki
extension install bodies into dedicated .sh files invoked via env.
Rewrites password-bearing shell tasks to either `| quote` directly or
the `-e VAR=... | quote` + `sh -c '... ${VAR} ...'` env-bridge pattern.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-10 03:46:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 dac52a7c42 fix(web-app-friendica): strip trailing slash from base URL
lookup('tls', ...).url.base keeps a trailing slash, so Friendica generated double-slash profile URLs (host//profile/<nick>) that broke local-follow contact resolution (no confirmation form rendered). Strip it via a shared FRIENDICA_BASE_URL var reused by env.j2 and the local.config.php patch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 20:06:35 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5adf3011fe refactor(swarm-nfs): generalize swarm+NFS chain across all roles
Migrate the mediawiki-specific swarm patterns into shared sys-svc-compose
helpers so every role inherits swarm + NFS portability:

- meta/volumes.yml: dict schema with mandatory name|path + optional nfs
  block (uid/gid/mode); 47 roles migrated, 48 lookup consumers updated to
  volumes.<key>.{name,path}
- sys-svc-compose helpers: finalize.yml (flush + post_deploy),
  post_deploy.yml (swarm-resolve + DB-healthy-wait), nfs_prep.yml
  (pre-create NFS subdirs), swarm/resolve_self.yml (canonical
  <ENTITY>_CONTAINER fact), render_replicated_templates.yml +
  _render_one_template.yml (template render + swarm-peer replication,
  per-entry notify, fileglob support)
- 15 roles migrated to render_replicated_templates.yml helper
  (mediawiki, prometheus, kix, friendica, dashboard, nextcloud,
  suitecrm, matrix, flowise, openldap)
- 5 new lint tests pin the architecture: volumes schema, compose chain,
  volume-constants-are-lookups, template-render-centralization,
  meta-no-role-var-refs
- ROLE_FILE_TEMPL_COMPOSE SPOT constant + 7 tests migrated to use it
- Handler test scanner extended to detect nested notify in
  vars: role_templates: lists

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 18:57:35 +02:00
kevinveenbirkenbachandClaude Opus 4.7 24d4ab8e4a refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:

* Schema: every role's meta/services.yml replaces the parallel
  `oauth2:` / `oidc:` top-level blocks with a single `sso:` block
  carrying `enabled`, `shared`, and a `flavor` discriminator
  (`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
  sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
  allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
  entirety; its 5 sidecar templates move under
  roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
  rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
  web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
  centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
  `is_oidc_native`) and the oauth2 sub-values
  (`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
  Three lookup plugins consume the resolver: `sso` (single-app
  predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
  consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
  plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
  and plugins/filter/compose_volumes.py all gate on
  `lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
  `enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
  → `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
  `OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
  `oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
  `OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
  `SSO_SERVICE_ENABLED`; Playwright gating helpers
  (isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
  safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
  is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
  `flavor: oauth2` with their existing `oauth2.acl.blacklist`
  (/user/login) and `oauth2.acl.whitelist` (federation + discovery
  endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
  (tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
  test_acl_mutual_exclusive.py) are deleted per Decision #10;
  test_oauth2_contract.py is renamed and rewritten to
  test_sso_contract.py, asserting that `flavor: oauth2` roles carry
  `oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
  (web-app integration) now requires the unified `sso:` block with an
  explicit flavor or a `# nocheck: sso` opt-out; new
  tests/lint/repository/no_legacy_sso_paths/ guards the source tree
  against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
  009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
  docs/contributing/actions/testing/suppression.md are updated to the
  unified SSO naming.

Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).

BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 12:59:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2a023ca6a1 chore(lint): autoformat strict-failure semantics + claude.sh rename
* scripts/lint/autoformat.sh:
  - rename run_sort_claude_settings worker to run_claude (matches the
    renamed scripts/lint/claude.sh).
  - drop the EXIT trap that wiped status_dir; cleanup happens explicitly
    after `wait` returns, so a parallel worker that interrupts the
    parent never races a vanished status dir.
  - each worker now captures its tool's exit code and writes one of
    OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
    (markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
    real failures count as FAIL.
  - all workers run to completion regardless of individual tool failures
    (so the user always sees the full picture); after the summary, the
    script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
  (history preserved via git mv); update test_claude_settings_sorted.py
  fix-hint accordingly.

Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 01:30:49 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00
kevinveenbirkenbach a55c99ef62 refactor(compose): replace docker lookup with container and unify compose paths
- Rename lookup plugin from `docker` to `container`
- Switch all lookups from `lookup('docker', ...)` to `lookup('container', ...)`
- Normalize compose file keys:
  - docker_compose → compose
  - docker_compose_override → compose_override
  - docker_compose_ca_override → compose_ca_override
- Centralize compose path resolution via docker_paths_utils
- Update compose_file_args to always derive paths from DIR_COMPOSITIONS
- Align all roles, templates, and vars with container/compose naming
- Update and rename unit tests accordingly

This prepares the codebase for engine-agnostic container backends
(e.g. Docker ↔ Podman) and removes legacy docker-specific assumptions.

https://chatgpt.com/share/6987a7cc-d374-800f-99ad-bb4c8f92c59e
2026-02-07 21:59:39 +01:00
kevinveenbirkenbach b78ab29218 Renamed config entry from docker to compose
https://chatgpt.com/share/69870f54-d708-800f-9870-e9ae5a4bff44
2026-02-07 11:09:36 +01:00
kevinveenbirkenbach 3fa7887dc8 Added whitespaces for better readability 2026-02-06 16:21:10 +01:00
kevinveenbirkenbach 5100c2c1ce refactor(config): replace get_app_conf filter with config lookup
Replace usages of the get_app_conf filter with the config lookup plugin
across roles and templates. Adjust quoting where required to keep YAML
and Jinja syntax valid. No functional changes intended.

https://chatgpt.com/share/69860678-cef8-800f-8402-7fe6231fdc2d
2026-02-06 16:19:14 +01:00
kevinveenbirkenbach 26a5b159e0 refactor(docker-lookup): replace want= syntax with path-based access
- Replace lookup('docker', ..., want='...') with lookup('docker', ..., 'path')
- Allow both top-level keys (directories/files) and dotted paths
- Simplify docker lookup API and reduce lazy dict propagation
- Update unit tests to reflect new lookup semantics

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:45:43 +01:00
kevinveenbirkenbach 56aea31667 refactor(docker): replace docker_compose vars with docker lookup across roles
Remove implicit docker_compose variable usage and resolve all docker paths
(files, directories, env) via lookup('docker', application_id).
This prevents scope leakage, centralizes path resolution, and makes docker
context deterministic across roles and handlers.

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:15:10 +01:00
kevinveenbirkenbach 5d9e161d55 refactor(friendica): align service/image config and unify FRIENDICA_* variables
- Move Friendica image/version into docker.services.friendica
- Replace legacy friendica_* vars with FRIENDICA_* namespace
- Make compose service name configurable via FRIENDICA_SERVICE
- Unify addon handling and IAM flags (OIDC / LDAP)
- Fix env boolean rendering for FRIENDICA_NO_VALIDATION

https://chatgpt.com/share/6980de90-2f5c-800f-8387-10e0f8612782
2026-02-02 18:27:40 +01:00
kevinveenbirkenbach 04c9490745 Solved unit and integration tests 2026-01-10 13:07:12 +01:00
kevinveenbirkenbach e2bf636ba9 Changed features structure 2026-01-10 11:13:29 +01:00
kevinveenbirkenbach 716ebef33b Refactor task includes and update variable handling for Ansible 2.20 migration
This commit updates multiple roles to ensure compatibility with Ansible 2.20.
Several include paths and task-loading mechanisms required adjustments,
as Ansible 2.20 applies stricter evaluation rules for complex Jinja expressions
and no longer resolves certain relative include paths the way Ansible 2.18 did.

Key changes:
- Replaced legacy once_finalize.yml and once_flag.yml with the new structure
  under tasks/utils/once/finalize.yml and tasks/utils/once/flag.yml.
- Updated all include_tasks statements to use 'path_join' with playbook_dir,
  ensuring deterministic and absolute file resolution across roles.
- Fixed all network helper includes by converting direct relative paths such as
  'roles/docker-compose/tasks/utils/network.yml' to proper Jinja-evaluated paths.
- Normalized MATOMO_* variable names for consistency with the updated variable
  scope behavior in Ansible 2.20.
- Removed deprecated patterns that were implicitly supported in Ansible 2.18
  but break under the more strict variable and path resolution model in 2.20.

These changes are part of the full migration step required to ensure the
infinito-nexus roles remain stable, deterministic, and forward-compatible with
Ansible 2.20.

Details of the discussion and reasoning can be found in this conversation:
https://chatgpt.com/share/69300a8d-24d4-800f-bec0-e895a695618a
2025-12-03 11:02:34 +01:00
kevinveenbirkenbach c67ccc1df6 Used path_join @ web-app-friendica 2025-10-26 15:48:28 +01:00
kevinveenbirkenbach 94da112736 perf(friendica): single-pass patch for DB creds + system.url; align env URL; tidy vars
- Patch local.config.php in one sed exec:
  * hostname, database, username, password
  * system.url via '#' delimiter to avoid URL slash escaping
  * Single notify: docker compose up
- env.j2:
  * FRIENDICA_URL now uses domains|get_url(application_id, WEB_PROTOCOL)
  * Simplify FRIENDICA_DEBUGGING with |lower
  * Normalize spacing for readability
- vars/main.yml:
  * Minor cleanups (comment header, spacing)
  * Consistent friendica_docker_ldap_config path construction

Why: fewer container execs ⇒ faster runs; idempotent key updates; consistent URL configuration across env and PHP config.
Risk: requires WEB_PROTOCOL and domains|get_url to be defined in inventory/vars as elsewhere in the project.

https://chatgpt.com/share/689b92af-b184-800f-9664-2450e00b29d6
2025-08-12 21:15:33 +02:00
kevinveenbirkenbach fe76fe1e62 Added correct flush parameters for docker compose 2025-08-11 10:33:48 +02:00
kevinveenbirkenbach eedfe83ece Solved missing redirect bug 2025-08-08 11:03:43 +02:00
kevinveenbirkenbach 2699edd197 Optimized friendica database patch 2025-07-20 16:13:48 +02:00
kevinveenbirkenbach 45d9da3125 Implemented friendica database credentials update (untested) 2025-07-19 16:45:04 +02:00
kevinveenbirkenbach 756597668c Semi bsr for applications[] to prevent heavy to debug bugs in j2 - part 1 2025-07-13 15:11:38 +02:00
kevinveenbirkenbach 22b4342300 Implemented schema/main.yml und config/main.yml file 2025-07-09 02:03:32 +02:00
kevinveenbirkenbach 563d5fd528 Huge role refactoring/cleanup. Other commits will propably follow. Because some bugs will exist. Still important for longrun and also for auto docs/help/slideshow generation 2025-07-08 23:43:13 +02:00