Commit Graph
32 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 e203d6f7ab feat(plugins): size memory parameters from the container limit with one lookup
lookup('service_memory_mb', application_id, service_name, share) returns
a share of a service's mem_limit in whole MiB. It resolves the limit the
way sys-svc-container resolves the container's own: the named service,
then the role's primary entity, then RESOURCE_MEM_LIMIT.

It replaces the three filters of memory_filters, each with its own unit
parser and its own idea of the limit: Redis maxmemory at 0.8 of the
limit, and the Confluence and Jira heap at 0.7 and 0.35.

The central Redis (409 MiB), Confluence (5734/2867) and Jira (4300/2150)
keep their values. A Redis sidecar without a mem_limit of its own does
not: the filter fell back to a fixed 256m, which gave 204 MiB, while the
lookup takes the limit that container really runs with, the primary
entity's. That moves 25 roles to between 409 and 3276 MiB.

The heap clamps of the old filters are gone: the 1024 MiB floor, limit
minus 1024, the 12288 cap and Xms bounded by mem_reservation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 23:05:26 +02:00
kevinveenbirkenbachandClaude Opus 5 29015311b6 feat(lint): derive container paths from meta/volumes.yml everywhere
meta/volumes.yml says where a volume lands inside its container. Wherever a
second file spelled the same path out, the two drifted independently: the mount
moves, the consumer keeps pointing at the old place, and nothing fails until the
process looks for content that is not there. e031b7f61 fixed exactly this for
web-app-moodle; this generalises it and makes the rule hold.

The lookup gains what a caller needs. `targets` maps each mounting service to
its container path, and `target` is the one path an entry mounts at -- present
only when every mount agrees. An entry that lands at different paths per service
omits `target` rather than picking one, and the caller names the service:

    "{{ lookup('volume', application_id, 'prometheus_config').target }}"
    "{{ lookup('volume', 'web-app-prometheus', 'host_sys').targets['node-exporter'] }}"

66 literals across 48 role files now derive. Replacement only happened where the
path stood alone -- /srv/app inside /srv/app/data is a different path -- and
every rewritten file was re-parsed, Jinja and YAML alike. Three sites the tool
refused rather than guess were done by hand: zammad's path sat in a Jinja list
literal, seaweedfs' inside a quoted string feeding a filter, and openresty's
/var/run/docker.sock is not a mount target at all but the socket on the host,
which only happens to spell the same string. That one carries the single
nocheck.

tests/lint/repository/spot/test_volumes_path_single_spot.py keeps it that way.
roles/*/files/ is out of scope by construction: it is copied verbatim into an
image and executed there, so a Dockerfile or entrypoint has no Ansible to
evaluate a lookup with.

One defect this found, and it was in the tooling rather than the roles: deciding
ambiguity AFTER filtering short paths made web-app-prometheus' host_sys look
single-valued -- it mounts /sys for cadvisor and /host/sys for node-exporter,
and /sys is below the length filter. The template would have rendered an
undefined `.target`. Both the converter and the rule now decide ambiguity over
every target and filter afterwards, and that call reads targets['node-exporter'].

Verified by resolving rather than by reading: all 70 accesses in the diff were
run against the real registry and every one yields a non-empty path equal to the
literal it replaced. make test-lint: 328 passed.

Not verified: no Ansible render, so a call site whose surrounding context cannot
build the applications registry would still fail at deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:29 +02:00
kevinveenbirkenbachandClaude Fable 5 d88f272f9f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 15:31:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c8a4320478 feat(lookup): container_hostname — kernel-safe container hostname
`lookup('container_hostname', app)` returns the app's canonical primary
domain when it fits sethostname(2)'s 63-byte cap, else the short entity
name. Onion subdomains (sub.<56-char>.onion) overflow that limit and make
`docker run` fail with "sethostname: invalid argument", killing container
init on onion nodes. Replaces the seven roles that hostname'd their
container on the raw domain (bookwyrm, bridgy-fed, confluence, jira,
postmarks, pretix, xwiki) with the encapsulated lookup.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 02:34:00 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f5412682b0 refactor(container-port): derive healthcheck port, drop redundant defs
The sys-svc-container healthcheck templates (curl/wget/nc/tcp/http) now
derive container_port from services.<service_name>.ports.internal.http,
so a role no longer needs to set container_port only to feed the shared
healthcheck.

Add the container-port-set-unused lint that fails a role setting
container_port without using it elsewhere, then resolve every flagged
role: drop the redundant assignment where the derivation reproduces the
value, or mark it with nocheck where the port is non-http, a constant, or
keyed on a service the derivation cannot reach.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f2af680a2f test(lint): require container_port to read the internal-port config lookup
Add a lint that rejects hardcoded container_port literals and the
host-published ports.local port; container_port is the port the container
listens on, so it must resolve from
lookup('config', application_id, 'services.<svc>.ports.internal.<name>')
or a constant built that way. Bring 42 roles into compliance: move each
literal into services.<svc>.ports.internal.<name> and reference it via the
lookup, using entity_name/service_name where they equal the service key.
Values are preserved (mattermost/prometheus pointed at ports.local with the
same value as internal). Drop the dead container_port from fider and
prometheus, and the narrative comments the edits pulled into the comment
lint's changed-file scope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 00:20:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0ffbb73d72 refactor(images): central container_image lookup + migrate all roles
Add plugins/lookup/container_image.py: returns the full `image: "<ref>"`
line so templates drop the redundant image: prefix and surrounding
quotes. Supports `custom=True` to auto-construct `<entity_name>_custom`
for locally-built images, plus explicit `image=`/`version=` overrides
for exotic cases (taiga back-auth/front-oidc variants, decidim infinito
suffix, etc.).

* 25 unit tests for the lookup cover compose vs swarm prefix, custom
  flag, overrides, tag_only=True, missing version warnings.
* New strict lint (tests/lint/.../test_no_raw_image_in_compose_templates)
  fails on any raw `image:` line in roles/*/templates/*.j2 unless
  nocheck-suppressed. Legacy `image: "{{ lookup(...) }}"` form no
  longer accepted.
* Migrate 159 image refs across ~80 roles to the standalone
  `{{ lookup('container_image', application_id, '<service_key>') }}`
  form. 26 roles use `custom=True` (mariadb, postgres, openldap,
  jenkins, jira, gitlab, nextcloud, mediawiki, shopware, wordpress,
  odoo, joomla, openproject, bookwyrm, espocrm, hugo, littlejs,
  fediwall, roulette-wheel, openresty, confluence and more).
* Fix svc-db-redis application_id leak: service.yml.j2 is included
  into consuming roles where application_id evaluates to the outer
  role, so the image lookup needs the hardcoded 'svc-db-redis'.
* Integration tests (test_templates_require_image,
  test_build_requires_image) recognise the standalone lookup as an
  implicit `image:` at the same indent.
* Mediawiki swarm tasks: add explicit `:{{ MEDIAWIKI_VERSION }}` to
  the registry-prefixed image string because the custom var is now
  bare.

Only 3 legitimate exotic nocheck-suppressed image lines remain:
matrix-mdad runner (nested services.matrix.runner.image path),
matrix mautrix bridges (per-iteration dynamic) and svc-registry-docker
(chicken-and-egg self-reference).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 00:17:11 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6e49fe74fb feat(volumes): canonical dict-of-dicts meta schema + lookup('volume')
Migrate every role's meta/volumes.yml from ad-hoc dict-of-named-volumes
to a canonical dict-of-dicts schema where the YAML key is the semantic
short name and the entry body carries type / name / source / mounts /
nfs / mode / read_only. The optional 'name:' field is the docker volume
name (defaults to the key); the previous 'docker_name:' field is gone.

What's in this commit (210 paths):

Infrastructure (23)
- New filter plugins/filter/{compose,container}_volumes.py — render the
  top-level volumes:/configs:/secrets: section and per-service mount
  blocks from the canonical meta. container_volumes uses the SEMANTIC
  name as per-service mount source so it resolves against compose_volumes'
  top-level dict key (docker_name as source produced 'undefined volume'
  on swarm).
- New lookup plugins/lookup/{compose,container,volume}_volumes.py.
  volume(application_id, name) returns a normalised single entry;
  volume(application_id) returns the whole canonical dict.
- utils/cache/applications.py keeps a sibling registry
  _CANONICAL_VOLUMES_BY_ROLE outside the merged applications payload so
  raw Jinja source: strings never enter the templar walk (root-cause of
  the 200x cache-miss + recursive get_merged_applications storm).
- utils/roles/applications/mounts.py — schema validator + normaliser;
  list-shape is rejected.
- plugins/lookup/nginx.py: _volume_path_by_name reads canonical directly.
- scripts/tests/deploy/swarm/_context.sh: NFS_VOLUMES extraction handles
  both legacy dict and canonical dict shapes.
- roles/svc-prx-openresty/handlers/restart_and_validate.yml: gate
  compose-only validate+reload behind a single when-block; swarm uses
  service-update instead (bind-mounts of a FILE pin source inode at
  task-create time, so atomic write-tmp+rename leaves the running task
  on a deleted inode).
- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch to
  lookup('volume', application_id) | dict2items so NFS subdir
  pre-creation works under the new cache split.

Tests (12)
- New strict lints flagging inline mount sections in compose templates,
  single-file bind mounts, and legacy lookup('config', ..., 'volumes.X.Y')
  call sites — all volume-mount surfaces MUST flow through container_volumes
  / compose_volumes / volume lookups.
- Integration: test_volumes_compose_chain + test_volume_constants_are_lookups
  accept both the legacy and the new lookup('volume', app, 'X').name forms
  during the migration window.
- Unit tests for the new lookup, filter, mounts helper, plus updated
  nginx lookup test.
- Prometheus role-wiring test reads canonical meta for the
  alert_rules.yml bind mount.

Role migrations (175)
- 62 meta/volumes.yml converted to dict-of-dicts canonical shape
- 63 templates/compose*.yml.j2 replace inline volumes:/configs:/secrets:
  with lookup('container_volumes', application_id, '<service>') |
  indent(N) per service + a single trailing lookup('compose_volumes',
  application_id) for the top-level block
- 50 vars/main.yml drop dead volume constants and refactor 23 remaining
  legacy lookup('config', ..., 'volumes.X.Y') to lookup('volume',
  application_id, 'X').Y across 9 roles

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5adf3011fe refactor(swarm-nfs): generalize swarm+NFS chain across all roles
Migrate the mediawiki-specific swarm patterns into shared sys-svc-compose
helpers so every role inherits swarm + NFS portability:

- meta/volumes.yml: dict schema with mandatory name|path + optional nfs
  block (uid/gid/mode); 47 roles migrated, 48 lookup consumers updated to
  volumes.<key>.{name,path}
- sys-svc-compose helpers: finalize.yml (flush + post_deploy),
  post_deploy.yml (swarm-resolve + DB-healthy-wait), nfs_prep.yml
  (pre-create NFS subdirs), swarm/resolve_self.yml (canonical
  <ENTITY>_CONTAINER fact), render_replicated_templates.yml +
  _render_one_template.yml (template render + swarm-peer replication,
  per-entry notify, fileglob support)
- 15 roles migrated to render_replicated_templates.yml helper
  (mediawiki, prometheus, kix, friendica, dashboard, nextcloud,
  suitecrm, matrix, flowise, openldap)
- 5 new lint tests pin the architecture: volumes schema, compose chain,
  volume-constants-are-lookups, template-render-centralization,
  meta-no-role-var-refs
- ROLE_FILE_TEMPL_COMPOSE SPOT constant + 7 tests migrated to use it
- Handler test scanner extended to detect nested notify in
  vars: role_templates: lists

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 18:57:35 +02:00
kevinveenbirkenbachandClaude Opus 4.7 24d4ab8e4a refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:

* Schema: every role's meta/services.yml replaces the parallel
  `oauth2:` / `oidc:` top-level blocks with a single `sso:` block
  carrying `enabled`, `shared`, and a `flavor` discriminator
  (`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
  sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
  allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
  entirety; its 5 sidecar templates move under
  roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
  rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
  web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
  centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
  `is_oidc_native`) and the oauth2 sub-values
  (`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
  Three lookup plugins consume the resolver: `sso` (single-app
  predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
  consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
  plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
  and plugins/filter/compose_volumes.py all gate on
  `lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
  `enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
  → `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
  `OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
  `oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
  `OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
  `SSO_SERVICE_ENABLED`; Playwright gating helpers
  (isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
  safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
  is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
  `flavor: oauth2` with their existing `oauth2.acl.blacklist`
  (/user/login) and `oauth2.acl.whitelist` (federation + discovery
  endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
  (tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
  test_acl_mutual_exclusive.py) are deleted per Decision #10;
  test_oauth2_contract.py is renamed and rewritten to
  test_sso_contract.py, asserting that `flavor: oauth2` roles carry
  `oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
  (web-app integration) now requires the unified `sso:` block with an
  explicit flavor or a `# nocheck: sso` opt-out; new
  tests/lint/repository/no_legacy_sso_paths/ guards the source tree
  against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
  009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
  docs/contributing/actions/testing/suppression.md are updated to the
  unified SSO naming.

Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).

BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 12:59:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 951f6d36d2 chore(lint): unify autoformat/galaxy parallelism into one PARALLEL env var
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
  declare it at file start with default `true`; still env-overridable.
  Reword the worker comment so the literal `# shellcheck` token no
  longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
  GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
  is always fatal. NAMESPACE is an internal constant. Add the same
  PARALLEL env var (default `true`, env-overridable); falsy demotes the
  xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
  AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
  consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
  add the lint-galaxy parallelism note.

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:59:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbach b78ab29218 Renamed config entry from docker to compose
https://chatgpt.com/share/69870f54-d708-800f-9870-e9ae5a4bff44
2026-02-07 11:09:36 +01:00
kevinveenbirkenbach 3fa7887dc8 Added whitespaces for better readability 2026-02-06 16:21:10 +01:00
kevinveenbirkenbach 5100c2c1ce refactor(config): replace get_app_conf filter with config lookup
Replace usages of the get_app_conf filter with the config lookup plugin
across roles and templates. Adjust quoting where required to keep YAML
and Jinja syntax valid. No functional changes intended.

https://chatgpt.com/share/69860678-cef8-800f-8402-7fe6231fdc2d
2026-02-06 16:19:14 +01:00
kevinveenbirkenbach 2d344fb129 refactor(domains): switch to domain() lookup across config and templates
Replace `domains | get_domain(...)` usage with `lookup('domain', ...)` throughout
group_vars, roles, and templates. Fix load_app domain resolution to use the
current application_id.

https://chatgpt.com/share/6985ef1e-11b8-800f-b231-52523055060b
2026-02-06 14:46:29 +01:00
kevinveenbirkenbach 3bb8c0ce30 refactor(tls): rename tls_resolve lookup to tls
The TLS lookup has been renamed from `tls_resolve` to `tls` to better
reflect its responsibility as a strict TLS state resolver.

This refactor updates all usages across the codebase, including:
- lookup plugins and internal helpers
- roles, vars, tasks, and templates
- tests, stubs, and error messages
- documentation and changelog references

Behavior, lookup API, and semantics remain unchanged.

https://chatgpt.com/share/6985e635-5a94-800f-b794-937e566ee308
2026-02-06 14:01:32 +01:00
kevinveenbirkenbach 8fe6b92e63 refactor(lookups): migrate nginx_paths, tls_resolve and cert_plan to strict positional want-path API
- Replace legacy want= keyword usage with mandatory positional want-path
- Align all nginx_paths, tls_resolve and cert_plan lookups to strict API
- Keep behavior unchanged while enforcing consistent lookup semantics

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:06:09 +01:00
kevinveenbirkenbach 8a81630391 Added newline at file and POSIX compatible 2026-02-04 19:49:56 +01:00
kevinveenbirkenbach fe9cdfb855 feat(images): standardize custom image tagging across roles
Normalize `*_CUSTOM_IMAGE` variables to the consistent pattern `{{ entity_name }}_custom:<version-or-ref>` (including SuiteCRM app version) and update MediaWiki to build a tool-preloaded custom image, removing runtime apt/composer provisioning.

https://chatgpt.com/share/697fa459-abbc-800f-954d-b18848890ff6
2026-02-01 20:06:52 +01:00
kevinveenbirkenbach 0b9f71463c refactor(tls): replace legacy tls filters with strict tls_resolve lookup
- Remove deprecated filter_plugins/tls.py and all dependent filters
- Introduce strict lookup plugin lookup_plugins/tls_resolve.py as single TLS authority
- Migrate all roles, templates, vars, and tasks to use tls_resolve (protocols, ports, URLs, cert paths)
- Enforce explicit TLS context (domains, applications, TLS_ENABLED, TLS_MODE)
- Support letsencrypt and self_signed with SAN handling and per-app overrides
- Update TLS provider roles to assert resolver state instead of setting facts
- Add comprehensive unit tests for tls_resolve lookup

https://chatgpt.com/share/6972b49b-b074-800f-a323-dd9fe69d02eb
2026-01-23 00:36:46 +01:00
kevinveenbirkenbach 9fdb0cddd0 Replace global WEB_PROTOCOL / WEB_PORT / WEBSOCKET_PROTOCOL logic
with explicit per-application TLS resolution.

Key changes:
- Introduce sys-front-tls-layer as the single TLS abstraction
- Centralize certificate paths via TLS_CERT_FILE / TLS_KEY_FILE / TLS_CA_FILE
- Replace sys-svc-letsencrypt SSL templates with generic TLS layer includes
- Add tls_* Jinja filters (enabled, mode, protocol, port, websocket)
- Resolve TLS mode per application (letsencrypt | self_signed | off)
- Remove implicit defaults and WEB_* globals to fail fast on misconfiguration
- Support self-signed certificates with SAN awareness
- Make TLS behavior explicit and consistent across all roles

This decouples application logic from certificate providers and
allows switching TLS flavor without touching app roles.

https://chatgpt.com/share/69727c61-8e68-800f-a267-473a7e627978
2026-01-22 20:36:49 +01:00
kevinveenbirkenbach 04c9490745 Solved unit and integration tests 2026-01-10 13:07:12 +01:00
kevinveenbirkenbach e2bf636ba9 Changed features structure 2026-01-10 11:13:29 +01:00
kevinveenbirkenbach 7d9cb5820f feat(jvm): add robust JVM sizing filters and apply across Confluence/Jira
Introduce filter_plugins/jvm_filters.py with jvm_max_mb/jvm_min_mb. Derive Xmx/Xms from docker mem_limit/mem_reservation using safe rules: Xmx=min(70% limit, limit-1024MB, 12288MB), floored at 1024MB; Xms=min(Xmx/2, reservation, Xmx), floored at 512MB. Parse human-readable sizes (k/m/g/t) with binary units.

Wire filters into roles: set JVM_MINIMUM_MEMORY/JVM_MAXIMUM_MEMORY via filters; stop relying on host RAM. Keep env templates simple and stable.

Add unit tests under tests/unit/filter_plugins/test_jvm_filters.py covering typical sizes, floors, caps, invalid inputs, and entity-name derivation.

Ref: https://chatgpt.com/share/68d3b9f6-8d18-800f-aa8d-8a743ddf164d
2025-09-24 11:29:40 +02:00
kevinveenbirkenbach 426ba32c11 feat(services): add CPU/RAM/PIDs defaults for heavy roles and align service names
Add per-service resource overrides (cpus, mem_reservation, mem_limit, pids_limit) for ollama, mariadb, postgres, confluence, gitlab, jira, keycloak, nextcloud; light formatting fixes in wordpress.

Rename service keys from generic 'application/web' to concrete names (jira, confluence, gitlab, keycloak) and update compose templates accordingly.

Jira: introduce JIRA_STORAGE_PATH and switch mounts/README accordingly.

https://chatgpt.com/share/68d2d96c-9bf4-800f-bbec-d4f2c0051c06
2025-09-23 21:43:50 +02:00
kevinveenbirkenbach 1a5ce4a7fa web-app-bookwyrm, web-app-confluence:
- Fix BookWyrm email SSL/TLS handling (use ternary without 'not' for clarity)
- Add truststore_enabled flag in Confluence config and vars
- Wire JVM_SUPPORT_RECOMMENDED_ARGS to disable UPM signature check if truststore is disabled
- Add placeholder style.css.j2 for Confluence

See conversation: https://chatgpt.com/share/68b80024-7100-800f-a2fe-ba8b9f5cec05
2025-09-03 10:45:41 +02:00
kevinveenbirkenbach 71ceb339fc Fix Confluence & BookWyrm setup:
- Add docker compose build trigger in docker-compose tasks
- Cleanup svc-prx-openresty vars
- Enable unsafe-inline CSP flags for BookWyrm, Confluence, Jira to allow Atlassian inline scripts
- Generalize CONFLUENCE_HOME usage in vars, env and docker-compose
- Ensure confluence-init.properties written with correct home
- Add JVM_SUPPORT_RECOMMENDED_ARGS to pass atlassian.home
- Update README to reference {{ CONFLUENCE_HOME }}

See: https://chatgpt.com/share/68b7582a-aeb8-800f-a14f-e98c5b4e6c70
2025-09-02 22:49:02 +02:00
kevinveenbirkenbach d760c042c2 Atlassian JVM sizing: cast memory vars to int before floor-division
Apply |int to TOTAL_MB and dependent values to prevent 'unsupported operand type(s) for //' during templating in Confluence and Jira roles.

Context: discussion on 2025-09-02 — https://chatgpt.com/share/68b6d386-4490-800f-9bad-aa7be1571ebe
2025-09-02 13:22:59 +02:00
kevinveenbirkenbach 188b098503 Confluence/Jira roles: add READMEs, switch to custom images, proxy/JVM envs, and integer-safe heap sizing
Confluence: README added; demo disables OIDC/LDAP; Dockerfile overlay; docker-compose now uses CONFLUENCE_CUSTOM_IMAGE and DB depends include; env.j2 adds ATL_* and JVM_*; vars use integer math (//) for Xmx/Xms and expose CUSTOM_IMAGE.

Jira: initial role skeleton with README, config/meta/tasks; Dockerfile overlay; docker-compose using JIRA_CUSTOM_IMAGE and DB depends include; env.j2 with proxy + JVM envs; vars with integer-safe memory sizing.

Context: https://chatgpt.com/share/68b6b592-2250-800f-b68e-b37ae98dbe70
2025-09-02 12:07:34 +02:00
kevinveenbirkenbach c23fbd8ec4 ✨ Add new role web-app-confluence
Introduced a new Ansible role for deploying Atlassian Confluence within the Infinito.Nexus ecosystem.
The role follows the same structure as web-app-pretix and includes:

- : Core variables, database config, OIDC integration.
- : Docker service definitions, features (Matomo, CSS, OIDC, logout, central DB).
- : Loads docker, db and proxy stack.
- : Placeholder for schema definitions.
- :
  -  (base for OIDC plugins/extensions),
  -  (service orchestration),
  -  (environment configuration).
- : Metadata, license, company, logo (Font Awesome book-open icon).

Canonical domain is set to `confluence.{{ PRIMARY_DOMAIN }}`.
This role ensures Confluence integrates seamlessly with Keycloak OIDC and the Infinito.Nexus service stack.

Conversation: https://chatgpt.com/share/68b1d006-bbd4-800f-9d2e-9c8a8af2c00f
2025-08-29 18:07:01 +02:00