Commit Graph
194 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.8 9aa975d665 fix(web-app-nextcloud): JSON-encode list/dict addon configvalues in plugin loader
occ config:app:set serialised non-mapping sequences via | string (Python repr), so list/dict configvalues like xwiki instances persisted as invalid JSON and the consumer fell back to empty. Encode any non-string sequence as JSON too.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 06:03:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4e8940caa6 refactor(web-app-nextcloud): migrate addons to unified addon syntax (req 026)
Schema-migrate every nextcloud addon to the unified per-file addon syntax (mechanism/source/required/bridges/config); add the non-proprietary integration addons (matrix, mattermost, moodle, suitecrm, zammad) and the local gitlab bridge; wire the bridged service blocks into meta/services.yml and pin every new (service,enabled)/(service,shared) pair in meta/variants.yml for variant coverage; relocate the per-addon user_ldap hook from tasks/plugin/hooks/ to tasks/addons/ for a uniform tasks/addons/ layout, dropping the obsolete vars/plugins README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 00:16:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 51c3d41ede Merge origin/main into feature/swarm-nfs
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.

objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.

deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.

Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 21:08:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b2fddf70a6 feat(nextcloud): migrate plugins to meta/addons (req 026)
Each Nextcloud app declared as a per-file addon; sociallogin bridges sso, user_ldap bridges ldap, bbb/onlyoffice/richdocuments/spreed/whiteboard/xwiki bridge their partner services. Per-plugin runtime config absorbed from vars/plugins/* (now deleted); install/configure tasks read the addon config via include_vars for lazy var rendering.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 11:30:45 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fb1414bb56 chore(web-app): bump image versions; gate Nextcloud OIDC plugins as mandatory
Bump:
- web-app-seaweedfs: chrislusf/seaweedfs 3.80 -> 4.33; nginx proxy 1.29 -> 1.31-alpine
- web-app-peertube: chocobozzz/peertube v8.2.0 -> v8.2.1
- web-app-prometheus: prom/alertmanager v0.32.2 -> v0.33.0

Hold Nextcloud at 33-fpm-alpine: NC 34 breaks the OIDC+LDAP login flavor
because pulsejet/nextcloud-oidc-login (app id oidc_login) caps at NC
max-version 33 with no NC 34 release, leaving the hidden native login
form with no OIDC button -> lockout (root cause of the PR #312 deploy
failure, surfaced only as a Playwright login timeout).

Mark the OIDC login plugins (oidc_login, sociallogin, user_oidc)
mandatory: true and fail the deploy at install time when a mandatory,
enabled plugin is not runnable (incompatible/unavailable), instead of
silently provisioning a login-broken instance. This makes a future NC
bump that outpaces the OIDC app fail fast with a clear cause.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:30:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1fa5161dd2 feat(web-app-nextcloud): provision on SeaweedFS S3 object store + consumer e2e test
Render the S3 objectstore config (templates/config/objectstore.config.php.j2) into NC's config dir before install, gated on objstore.enabled — NC cannot switch an already-installed instance to objectstore primary storage, so it must be present at install time.

Add the SeaweedFS consumer storage Playwright test (files/playwright/test-seaweedfs.js, playwright.env.j2 SEAWEEDFS_* vars, spec wiring; drop the playwright-service-flag nocheck for the seaweedfs service) plus the shared runSeaweedfsStorageCheck helper (admin logs into the Filer once, counts bucket objects via the Filer JSON listing, asserts growth) and the seaweedfs-consumer integration lint.

web-app-seaweedfs: bind weed to 0.0.0.0 (-ip.bind) so the multi-homed gateway is reachable by consumers that resolve the service name to its IP on the shared network.

nextcloud_install: classify 'was not found on the appstore' as unavailable (skippable, non-fatal) so a plugin without a release for the pinned NC version does not abort the whole deploy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 11:30:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e261dc2dc6 chore(web-app): bump image versions; gate Nextcloud OIDC plugins as mandatory
Bump:
- web-app-seaweedfs: chrislusf/seaweedfs 3.80 -> 4.33; nginx proxy 1.29 -> 1.31-alpine
- web-app-peertube: chocobozzz/peertube v8.2.0 -> v8.2.1
- web-app-prometheus: prom/alertmanager v0.32.2 -> v0.33.0

Hold Nextcloud at 33-fpm-alpine: NC 34 breaks the OIDC+LDAP login flavor
because pulsejet/nextcloud-oidc-login (app id oidc_login) caps at NC
max-version 33 with no NC 34 release, leaving the hidden native login
form with no OIDC button -> lockout (root cause of the PR #312 deploy
failure, surfaced only as a Playwright login timeout).

Mark the OIDC login plugins (oidc_login, sociallogin, user_oidc)
mandatory: true and fail the deploy at install time when a mandatory,
enabled plugin is not runnable (incompatible/unavailable), instead of
silently provisioning a login-broken instance. This makes a future NC
bump that outpaces the OIDC app fail fast with a clear cause.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 11:58:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c86ab1e37b feat(web-app-nextcloud): provision on SeaweedFS S3 object store + consumer e2e test
Render the S3 objectstore config (templates/config/objectstore.config.php.j2) into NC's config dir before install, gated on objstore.enabled — NC cannot switch an already-installed instance to objectstore primary storage, so it must be present at install time.

Add the SeaweedFS consumer storage Playwright test (files/playwright/test-seaweedfs.js, playwright.env.j2 SEAWEEDFS_* vars, spec wiring; drop the playwright-service-flag nocheck for the seaweedfs service) plus the shared runSeaweedfsStorageCheck helper (admin logs into the Filer once, counts bucket objects via the Filer JSON listing, asserts growth) and the seaweedfs-consumer integration lint.

web-app-seaweedfs: bind weed to 0.0.0.0 (-ip.bind) so the multi-homed gateway is reachable by consumers that resolve the service name to its IP on the shared network.

nextcloud_install: classify 'was not found on the appstore' as unavailable (skippable, non-fatal) so a plugin without a release for the pinned NC version does not abort the whole deploy.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 20:17:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a4cb5b89e1 feat(swarm): swarm-aware container exec via container_address lookup
Adds a mode-aware `container_address` lookup plugin and a
`resolve-container-id` helper script that resolve swarm task IDs at
exec time (compose mode returns bare names unchanged). Migrates ~25
roles + ~28 task files from `command:` to `shell:` so the lookup's
`$(...)` subshell evaluates correctly, and extends `database` lookup
with an `address` key for cross-role DB exec.

Three new lint tests enforce the contract: every container exec
target goes through the resolver, lives behind an IS_STACK_HOST gate
(walks the include graph one hop deep), and uses `shell:` rather than
`command:` when the target is a Jinja reference. Existing nocheck
suppressions on bare-name DB exec sites (nextcloud redis, listmonk,
moodle OIDC, mailu) are replaced with the lookup-based form.

Fixes the systemd Environment= injection in sys-ctl-cln-disc-space
(it never expanded $(...)) by passing STACK / SERVICE_KEY parts and
resolving inside the script. Extracts shopware admin and mediawiki
extension install bodies into dedicated .sh files invoked via env.
Rewrites password-bearing shell tasks to either `| quote` directly or
the `-e VAR=... | quote` + `sh -c '... ${VAR} ...'` env-bridge pattern.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-10 03:46:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 05f6a97bd8 refactor(stack_host_template): generalise IS_STACK_HOST template wrapper across the codebase
Renames the local action plugin from `conf_template` to
`stack_host_template` so the name carries the actual semantics
(IS_STACK_HOST gate, not nginx config), broadens the lint guard to
every bare `template:` task, and migrates every consumer found in the
audit.

Plugin / tests / lint:
  * plugins/action/conf_template.py -> plugins/action/stack_host_template.py
    (drop nginx-specific docstring; trust_as_template gate unchanged)
  * test_conf_template.py            -> test_stack_host_template.py
  * test_conf_template_migration.py  -> test_stack_host_template_migration.py
    Scope expanded from `.conf.j2` to every `template:` task; marker
    renamed to `nocheck: stack-host-template`.

Migrations (~50 tasks now using stack_host_template:):
  * sys-svc-compose/04_files.yml: compose.yml, .env, Dockerfile,
    compose.override (every per-app compose-tree file)
  * web-app-keycloak: sso_proxy upstream-config (fixes the friendica
    CI failure that prompted this refactor) + realm-import
  * web-app-taiga, web-app-matrix, web-app-opentalk, web-app-pgadmin,
    web-app-moodle, web-app-listmonk, web-app-littlejs, web-app-fediwall,
    web-app-wordpress, web-app-discourse, web-app-bigbluebutton,
    web-app-jenkins, web-app-jitsi, web-app-mobilizon, web-app-suitecrm,
    web-app-decidim, web-app-dashboard, web-app-nextcloud, web-app-xwiki,
    web-app-mailu, web-app-magento, web-app-odoo, web-app-prometheus,
    web-app-matrix, web-app-peertube, web-app-mailu (+ existing `conf_template:`
    callers updated to the new name)
  * sys-front-tls, svc-db-openldap, sys-svc-webserver-core,
    sys-svc-rdbms (dedicated DB env + initdb), sys-front-inj-css,
    sys-front-inj-dashboard, sys-front-inj-all, sys-stk-front-proxy,
    web-svc-cdn-css, web-svc-xmpp, web-svc-legal, web-svc-simpleicons,
    web-opt-rdr-domains

Nochecks (per-host or already-structurally-stack-host):
  * sys-svc-mail-smtp/postfix, sys-svc-sshd, sys-svc-container/daemon.json
  * sys-timer, sys-service, sys-svc-mail-msmtp, sys-svc-journalctl,
    sys-daemon, sys-svc-compose/utils/_render_one_template (delegate_to
    stack host with peer slurp dance)
  * dev-locales, dev-nix, desk-*, user, user-backup, user-workstation
  * svc-storage-nfs-server/exports, test-e2e-playwright stage dirs

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-06 15:45:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5adf3011fe refactor(swarm-nfs): generalize swarm+NFS chain across all roles
Migrate the mediawiki-specific swarm patterns into shared sys-svc-compose
helpers so every role inherits swarm + NFS portability:

- meta/volumes.yml: dict schema with mandatory name|path + optional nfs
  block (uid/gid/mode); 47 roles migrated, 48 lookup consumers updated to
  volumes.<key>.{name,path}
- sys-svc-compose helpers: finalize.yml (flush + post_deploy),
  post_deploy.yml (swarm-resolve + DB-healthy-wait), nfs_prep.yml
  (pre-create NFS subdirs), swarm/resolve_self.yml (canonical
  <ENTITY>_CONTAINER fact), render_replicated_templates.yml +
  _render_one_template.yml (template render + swarm-peer replication,
  per-entry notify, fileglob support)
- 15 roles migrated to render_replicated_templates.yml helper
  (mediawiki, prometheus, kix, friendica, dashboard, nextcloud,
  suitecrm, matrix, flowise, openldap)
- 5 new lint tests pin the architecture: volumes schema, compose chain,
  volume-constants-are-lookups, template-render-centralization,
  meta-no-role-var-refs
- ROLE_FILE_TEMPL_COMPOSE SPOT constant + 7 tests migrated to use it
- Handler test scanner extended to detect nested notify in
  vars: role_templates: lists

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 18:57:35 +02:00
kevinveenbirkenbach eab16c0568 refactor(web-app-nextcloud): reshape plugin pipeline into install/enable/configure stages
Split the previous two-file plugin pipeline into three single-responsibility
stages and group them under tasks/plugin/, with per-plugin hooks now sitting
under tasks/plugin/hooks/. Each stage owns exactly one OCC call (app:install,
app:enable, config:app:set) and one continuation decision driven by the
nextcloud_install_status filter.

Layout:

  tasks/
  ├── 06_plugin.yml                 (entry; includes plugin/01_install.yml)
  └── plugin/
      ├── 01_install.yml            (occ app:install + classify)
      ├── 02_enable.yml             (occ app:enable + classify)
      ├── 03_configure.yml          (occ config:app:set + run hook)
      ├── hooks/                    (per-plugin extension tasks)
      │   └── user_ldap.yml
      └── README.md                 (pipeline contract + extension guide)

The new 02_enable.yml mirrors the version-compatibility tolerance that
01_install.yml already had: when occ rejects the enable with the
"is not compatible with this version of the server" marker, the
failed_when clause downgrades the task to a non-fatal classification,
emits a debug warning, and skips the entire 03_configure.yml include via
a single set_fact-driven flag instead of repeating the filter call on
every downstream task. This unblocks deploys against a Nextcloud release
whose appstore lags briefly while the plugin's compatibility metadata
catches up (e.g. metadata, drawio, files_bpm in the NC 33 window).

NEXTCLOUD_CNODE_PLUGIN_TASKS_PATH in vars/main.yml is rebased to the new
tasks/plugin/hooks/ directory so plugin hooks resolve at the new location
without touching call sites.

README.md follows the role + docs README contracts: one sentence per line,
no em dashes, RFC 2119 keywords, no historical narrative, cross-links to
the filter plugin and the role vars file.
2026-05-19 23:01:18 +02:00
kevinveenbirkenbach 7dfe18010c feat(web-app-nextcloud): tolerate version-incompatible plugins via filter
Re-enable bbb, files_bpm, and metadata plugins. Their previous opt-out
existed only because the upstream Nextcloud appstore release lagged the
pinned server major, which made `occ app:install` fail and abort the
deploy.

Introduce a role-local Jinja filter `nextcloud_install_status` that
classifies the registered `occ app:install` result into a status dict
(ok/changed/runnable/already/incompatible). The install task now uses
this filter in `until`/`failed_when`/`changed_when`, so a plugin
emitting "is not compatible with this version of the server" terminates
the retry loop immediately, downgrades to a debug warning, and skips
the subsequent enable/configure path instead of failing the play.

Net effect: operators no longer have to toggle services.yml when an
upstream plugin temporarily lags the pinned Nextcloud version; the
deploy proceeds and the plugin auto-recovers on the next run once a
compatible release ships.
2026-05-19 17:00:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 19254a23b4 refactor(sys-svc-compose): rename docker_compose_flush_handlers → compose_handlers_flush
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.

Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).

Breaking change for any external override that still sets the old
name — no alias is provided.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2b72d8b550 test(lint): forbid redundant default(...) on module IO in same-task conditionals
Adds tests/lint/ansible/test_no_redundant_default_on_module_io.py which
flags `<reg>.{stdout,stderr,rc} | default(...)` inside `changed_when:`,
`failed_when:`, or `until:` lines. Those conditionals only fire after
the module has actually executed, so the fallback can never apply --
keeping it just hides bugs and noise.

Cleans up the 21 existing offenders across 14 role files. The taiga
upsert-admin task additionally gains a `not ASYNC_ENABLED and ...`
short-circuit so its async-poll-0 path no longer leans on a silent
empty fallback for stdout/stderr; with the short-circuit, the
`default('')` is genuinely dead code.

Cross-task references (a later task reading a registered result of an
earlier task that may have been skipped via `when:`) stay out of scope
and remain a legitimate use of `| default(...)`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:06:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2a023ca6a1 chore(lint): autoformat strict-failure semantics + claude.sh rename
* scripts/lint/autoformat.sh:
  - rename run_sort_claude_settings worker to run_claude (matches the
    renamed scripts/lint/claude.sh).
  - drop the EXIT trap that wiped status_dir; cleanup happens explicitly
    after `wait` returns, so a parallel worker that interrupts the
    parent never races a vanished status dir.
  - each worker now captures its tool's exit code and writes one of
    OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
    (markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
    real failures count as FAIL.
  - all workers run to completion regardless of individual tool failures
    (so the user always sees the full picture); after the summary, the
    script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
  (history preserved via git mv); update test_claude_settings_sorted.py
  fix-hint accordingly.

Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 01:30:49 +02:00
kevinveenbirkenbachandClaude Opus 4.7 951f6d36d2 chore(lint): unify autoformat/galaxy parallelism into one PARALLEL env var
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
  declare it at file start with default `true`; still env-overridable.
  Reword the worker comment so the literal `# shellcheck` token no
  longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
  GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
  is always fatal. NAMESPACE is an internal constant. Add the same
  PARALLEL env var (default `true`, env-overridable); falsy demotes the
  xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
  AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
  consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
  add the lint-galaxy parallelism note.

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:59:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00
kevinveenbirkenbachandClaude Opus 4.7 cc6e77b33f fix(nextcloud): clear appstore-fetcher lastFailure before each install
The appstore-fetcher caches a 300s failure cooldown on any fetch
error. With retries:6/delay:30 the whole 180s retry window stayed
inside one cached-empty catalog, so plugins that transiently 404
('not found on the appstore') never recovered. Clearing
settings/appstore-fetcher-lastFailure before each attempt forces a
fresh catalog fetch on retry.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 09:26:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ada7c4b5ae fix(nextcloud): fail loudly on unsupported / unavailable plugins
Remove silent tolerance for `occ app:install` errors — both
"not compatible with this version of the server" and
"not found on the appstore" now propagate as task failures.

Rationale: these are config or upstream issues (typo, removed app,
NC-version mismatch), not transients. 6× 30s retries still absorb
real flakes; residual failures deserve to surface, not to ship a
"deployment" that silently lacks features the config lists.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 00:48:52 +02:00
Environment Test 83cd881dee feat(nextcloud): wire standalone coturn TURN + admin identity tasks
- Add 07_admin_identities.yml: resolves LDAP-backed admin identities
  for OCC calls that require the actual NC username, not the LDAP DN
- vars: add NEXTCLOUD_HPB_TURN/STUN_ACTIVE_CONFIG selecting standalone
  or onboard TURN based on enabled flags; add TALK_ADMIN_URL
- config: talk container gated on spreed plugin flag
- env.j2 + playwright.env.j2: expose active TURN/STUN config and
  Talk admin URL for container env and E2E tests
- playwright.spec.js: extend with Talk TURN/STUN E2E assertions
- schema: add spreed.enabled validation
- coturn compose: move relay port range before TLS block so ports are
  always set regardless of TLS mode
- docs/requirements: document TURN relay requirement
- test_resolver: cover new resolution paths
2026-04-14 02:29:29 +02:00
Environment TestandClaude Sonnet 4.6 750c110f45 fix(docs): apply documentation rules, fix URLs, and remove orphaned references
- Apply documentation.md rules across staged files (emojis after headlines, RFC 2119 keywords, correct link text, Credits format)
- Remove expired ChatGPT share URLs from 39+ role READMEs and templates
- Remove orphaned text after link deletion: bare # @see, {# @see  #}, Discussion: #}, empty Chat Conversation sections, dangling "Conversation is here available.", "Refer to the conversation with ChatGPT ()", "See ChatGPT Conversation", "ChatGPT Session Transcript" references
- Remove empty ## Further Resources and ## Credits sections where no links remain
- Fix URL reachability test: treat 401/403/405 as ok, 429/500/503 as warn, Timeout as warn; add # nocheck: url suppression (inline and line-above)
- Add lint test for broken relative and repo-root-absolute markdown links
- Add # nocheck: url to AUR git clone URLs, CSP allowlist entries, and other non-browser URLs
- Restore docs.infinito.nexus/setup in documentation.md MUST NOT example
- Fix Mobilizon geospatial endpoints (Addok, Photon) reverted to original values with # nocheck: url
- Add integration test rule: MUST NOT write unit tests for integration test logic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-11 00:38:42 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 96649d8c9b refactor(services): centralize frontend service loading with generic lookup plugin
Introduces a SPOT for all frontend service loading in sys-front-inj-all/tasks/01_services.yml,
driven by a new generic `service` lookup plugin that resolves services by key or role name
and returns {id, role, enabled, shared, needed}. Needed = enabled AND shared (direct or
transitive). The canonical key→role mapping moves to group_vars/all/20_services.yml
(previously scattered hardcoded rules). Removes per-role service loads from bigbluebutton,
nextcloud, and dashboard. Renames 01_dependencies.yml→01_services.yml and
08_services.yml→08_daemons.yml. Adds flush_handlers after each service load.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 08:27:21 +02:00
Alejandro Roman 5018ecf145 fix all windows issues with the app + more robust for all 2026-03-21 16:08:28 +01:00
kevinveenbirkenbach 74a70e2327 nextcloud: make client_max_body_size configurable and increase limit to support Talk recordings 2026-03-13 14:31:41 +01:00
kevinveenbirkenbach d44eee1048 Fix Ansible fact deprecations and loop variable collisions 2026-02-26 17:54:14 +01:00
kevinveenbirkenbach fb9bdaedb3 fix(nextcloud): harden LDAP config id handling and centralize prefix 2026-02-21 15:35:39 +01:00
kevinveenbirkenbach 522ab9e792 Fix Nextcloud app installation failures caused by transient GitHub 502 errors
- Add retry logic with delay to Nextcloud app installation via occ
- Retry installation up to 6 times with 30 second intervals
- Treat "already installed" and "not compatible" as non-fatal states
- Prevent CI and deployments from failing due to temporary upstream outages

https://chatgpt.com/share/6988a766-1bfc-800f-b2e8-374b5514ec65
2026-02-08 16:10:08 +01:00
kevinveenbirkenbach 5ad0ecfd99 refactor(compose): switch build calls to compose wrapper
Replace all `docker compose build` invocations with the `compose` wrapper
to ensure automatic env-file detection and override handling. Update
handlers, notify targets, direct commands, and log labels consistently
across roles.

https://chatgpt.com/share/69866b76-ce6c-800f-ac9e-dba6bba27008
2026-02-06 23:30:04 +01:00
kevinveenbirkenbach 0a7b4efa22 Replaced docker compose pull by compose pull 2026-02-06 23:26:20 +01:00
kevinveenbirkenbach c1d50d78da refactor(container): replace BIN_CONTAINER usages with system-wide container wrapper
https://chatgpt.com/share/698660b7-4c78-800f-b25f-af5408cff034
2026-02-06 23:00:21 +01:00
kevinveenbirkenbach 46238d4c89 refactor: use BIN_CONTAINER wrapper for container exec commands
https://chatgpt.com/share/698657ee-e708-800f-ba5d-4c7c5198a5fe
2026-02-06 22:06:42 +01:00
kevinveenbirkenbach 4ca9bc61e0 Removed MODE_UPDATE conditions to make behaviour more deterministic 2026-02-05 11:39:09 +01:00
kevinveenbirkenbach 8a81630391 Added newline at file and POSIX compatible 2026-02-04 19:49:56 +01:00
kevinveenbirkenbach c5f4146910 fix(nextcloud,ldap): warm LDAP cache after bind config to enable immediate logins
- Set LDAP bind password synchronously
- Warm LDAP cache via occ ldap:search to avoid first-login failures
- Normalize env formatting for admin credentials

https://chatgpt.com/share/697f95d1-f608-800f-a25c-5f16e7c29bf2
2026-02-01 19:04:56 +01:00
kevinveenbirkenbach caef37df92 fix(nextcloud): switch includes.php to volume mount, harden config loader, and move proxy settings to system config
- Mount includes.php read-only instead of docker cp
- Ensure include loader starts with proper PHP header
- Add overwriteprotocol, overwritehost, and trusted_proxies via system config
- Simplify env handling for protocol and trusted proxies
- Remove redundant permission and copy tasks

https://chatgpt.com/share/697f7aff-859c-800f-bfaa-0d4c2378fd85
2026-02-01 17:10:28 +01:00
kevinveenbirkenbach c0fdfaa759 fix(nextcloud): stabilize config permissions, loader injection and bootstrap flow
- ensure host-mounted infinito config directory ownership and permissions (recursive)
- ensure includes.php ownership on host before container start
- inject /tmp/includes.php loader into config.php idempotently using printf (no heredoc)
- prepare config directory with correct permissions in Dockerfile to avoid root-owned bootstrap
- add read_only.config.php template (default disabled for installer phase)

This fixes broken config.php injection, prevents root-owned config dir on fresh volumes,
and stabilizes Nextcloud auto-installer and OCC plugin routines.

https://chatgpt.com/share/696ed102-9e74-800f-ad52-f09ad0a9da80
2026-01-20 01:48:46 +01:00
kevinveenbirkenbach 772c2f44ee Solved tab error 2026-01-19 00:53:31 +01:00
kevinveenbirkenbach fab38c18a0 fix(nextcloud): remove unsafe config chown and align host file permissions
- pass application_id to sys-stk-full for proper app context
- remove recursive chown on /var/www/html/config (breaks with read-only infinito mount)
- set correct ownership and permissions for host config and include files
- normalize file modes for generated Nextcloud config snippets

https://chatgpt.com/share/696d5a7c-f248-800f-b228-b26afd8c24bf
2026-01-18 23:10:59 +01:00
kevinveenbirkenbach 88aa83ae9c fix(nextcloud): ensure config directory writable and harden install flow
Ensure /var/www/html/config is created and owned by www-data before running occ
and appending dynamic configuration. This prevents install failures caused by
root-owned Docker volumes.

Also:
- fail fast if occ status does not report a successful installation
- align nginx root with NEXTCLOUD_DOCKER_WORK_DIRECTORY

Fixes: "Cannot write into config directory" and missing config.php on fresh volumes

https://chatgpt.com/share/6968e0a3-05c8-800f-aa3e-8c0d3c155552
2026-01-15 13:42:04 +01:00
kevinveenbirkenbach 7313533b54 Removed docker compose restart and replaced by up due to interference during setup 2026-01-15 06:53:38 +01:00
kevinveenbirkenbach 989c0c0fa2 refactor(stacks): merge stateful/stateless backend+full stacks into sys-stk-backend/sys-stk-full
- Replace sys-stk-back-stateful + sys-stk-back-stateless with unified sys-stk-backend role
- Replace sys-stk-full-stateful + sys-stk-full-stateless with unified sys-stk-full role
- Add reusable task helpers: _load_service.yml (optional deps) and 01_database.yml (DB bootstrap)
- Switch optional dependency loading (OpenLDAP, Ollama) to service_load_enabled (enabled && shared) + run_once guard
- Remove legacy OLLAMA_LOCAL_ENABLED flag; rely on docker.services.ollama.enabled/shared config
- Update all affected app/service roles and web-app template to use new stack role names

https://chatgpt.com/share/69685fb2-cb1c-800f-a9e8-2f5f765a1a53
2026-01-15 04:31:54 +01:00
kevinveenbirkenbach dce01e4540 wait for nextcloud installation before modifying config.php
Ensure Nextcloud is fully installed before appending generated config:
- wait for successful `occ status` with `installed: true`
- add retry loop with timeout
- prevent premature access to config.php during initial setup

This avoids race conditions where config.php does not yet exist.

https://chatgpt.com/share/69680205-d6e0-800f-b47a-f03da156868f
2026-01-14 21:52:13 +01:00
kevinveenbirkenbach 1104d4710d Normalize docker service reachability via DOCKER_REACH_HOST
Introduce DOCKER_REACH_HOST as a single source of truth for service
reachability checks (wait_for), decoupled from DOCKER_BIND_HOST.

- Default reach host is 127.0.0.1 for host-based execution
- Automatically override to Docker bridge gateway when running
  inside a container (Docker socket mounted)
- Add custom docker_bridge_gateway module for reliable detection
- Update all wait_for tasks to use DOCKER_REACH_HOST
- Remove ad-hoc debug TCP probing logic

This ensures consistent behavior across host runs and
container-based control nodes without breaking existing setups.

https://chatgpt.com/share/694aaf8b-1fe4-800f-88ac-b5f6e508a2e3
2025-12-23 16:04:36 +01:00
kevinveenbirkenbach d0a244ef3a Corrected variable name 2025-12-22 17:49:13 +01:00
kevinveenbirkenbach 2b5ff60a6f infra(docker): drop host networking and normalize port binding logic
This change removes --network=host usage, introduces DOCKER_IN_CONTAINER
for reliable environment detection, and replaces hardcoded localhost
bindings with an explicit DOCKER_BIND_HOST abstraction to avoid nftables
issues in CI/DiD environments.

https://chatgpt.com/share/69497616-7b00-800f-8fa3-ddc6eb9c5e98
2025-12-22 17:46:08 +01:00
kevinveenbirkenbach d0efc77b6c Optimized variable conditions for ansible update 2.18 -> 2.20 2025-12-04 12:24:01 +01:00
kevinveenbirkenbach 8fad77f17f Solved CDN and Logout deployment bugs existing due to 2.18->2.20 update 2025-12-03 22:45:48 +01:00
kevinveenbirkenbach 031775c096 Removed redundancies and simplified run_once flag 2025-12-03 18:12:09 +01:00
kevinveenbirkenbach 116c20d61d Refactor Nextcloud fullstack and config tasks:
- Load proxy in fullstack
- Simplify config merging logic
- Improve handler flushing and container config update

Context: https://chatgpt.com/share/692dc9d4-a884-800f-85e5-e92471c7b52b
2025-12-01 18:01:25 +01:00