Commit Graph
1042 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.8 0837630ac9 docs(requirements): emit markdownlint-clean Notes block in the matrix generator
Add the blank line after `Notes:` (markdownlint MD032) in the generator HEADER so
a fresh `027-integration-matrix.gen.py` run stays lint-clean and idempotent with
autoformat. Regenerate the matrix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:53:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a0a0eac61e chore(requirements): mark integration-matrix generator executable (ruff EXE001)
The generator carries a `#\!/usr/bin/env python3` shebang; make the file
executable so ruff's flake8-executable check passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:53:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 48f8000c6e docs(requirements): renumber MCP matrix + addon-syntax (026 -> 027/028) after merge
feature/mcp's 026-integration-matrix and 026-unified-addon-syntax collided with
main's numbering on merge. Renumber both and fix every cross-reference:
- 026-integration-matrix.{md,gen.py} -> 027-integration-matrix.{md,gen.py}
- 026-unified-addon-syntax.md        -> 028-unified-addon-syntax.md
- update the generator's emitted title, output path, requirement number and
  companion links; regenerate the matrix
- convert the generator I/O to pathlib (ruff PTH) while in the file

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:53:07 +02:00
kevinveenbirkenbach 6332071668 Merge branch 'feature/mcp' into fix/ci-deploy-server-failures
# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.
2026-06-13 15:29:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6c38fcf5c8 docs(requirements): add 026 cross-role integration matrix
Add the generated role×role integration matrix (026-integration-matrix.md
+ generator) and expand 026 with: required-gated install-failure handling,
mechanism=bridge for network bridges, addon variant/matrix axis, and the
matrix-backed cross-role integration backlog. The matrix scans every role's
meta/services.yml for infinito-native wired edges (☑️) and overlays curated,
web-verified upstream plugin edges (✅/🪙): 511 wired + 96 upstream across 82
roles. Header repeats every 10 rows and columns for readability.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:29:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8772d2fac7 docs(requirements): add 026 unified addon syntax
Promote per-app addon/plugin/extension/module/bridge declarations to a
single file-rooted meta/addons.yml contract: unified schema, explicit
cross-role bridge rule, required-gated install-failure handling (hard
fail vs warn+skip), addon state as a variants.yml matrix axis, network
bridges (Matrix mautrix) as mechanism=bridge addons, external drift
monitoring, and a Playwright test per user-facing addon. Audit expanded
to matrix, discourse, pretix, mattermost, desk-chromium/firefox/gnome;
mobilizon/bookwyrm DB extensions classified out of scope.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:29:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 dc84016f9d docs(requirements): add 025 MCP role integration
Define the unified MCP service, identity, proxy, and test contract for
server and client roles, with a repo-wide audit, bridge discovery via
roles_with_service, and a Playwright test per integrated role.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 15:29:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 70ad33bc13 fix(web-app-peertube): repair the SeaweedFS object-store e2e test for PeerTube v8.2.1
The test timed out: the publish-button locator no longer matched the v8.2.1
manage-video wizard, and the inline 78-byte MP4 stub was structurally invalid,
so PeerTube rejected the upload as "Video file unreadable" and left the Save
button disabled.

- test-seaweedfs.js: target the v8.2.1 save control (.save-button > button),
  upload PeerTube's real video_short.mp4 fixture, and assert a version-tolerant
  saved marker.
- add fixture roles/web-app-peertube/files/playwright/fixtures/video_short.mp4.
- test-e2e-playwright: stage role-local files/playwright/fixtures/ next to the
  tests (the *.js glob ships neither binaries nor subdirectories), so specs load
  a fixture via path.join(__dirname, "fixtures", <name>).
- docs: document the fixtures/ folder and the per-scenario test-*.js split in the
  playwright spec contract and the runner README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-13 14:07:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7c84617758 docs(role-loop): make matrix-variant command selection unambiguous
Replace the prose that an agent could read as 'no variant passed implies mode=reinstall' with an explicit situation-to-command table plus a FORBIDDEN list: every mode=update is variant-pinned, and a no-variant mode=reinstall is reserved for the initial baseline and the final gate only. A failing variant is retried with mode=update variant=<idx>, never by restarting the whole matrix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 22:38:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 bc075906d7 test(objstore): require seaweedfs for database-backed web-app roles
Add a meta lint asserting every web-app role with postgres or mariadb declares a seaweedfs object-store service, exemptable via the seaweedfs-required nocheck marker. Mark the 33 database-backed roles that do not use object storage, and register the rule in the suppression catalog.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 21:00:13 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3bdc8180c7 chore(changelog): archive releases past active window
Trim CHANGELOG.md to the active window and move older entries (4.0.3 through 8.0.4) into docs/changelog/; regenerate the Debian and Fedora package changelogs from the kept entries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 10:46:05 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5f44943d10 refactor(svc-swarm): categorise as svc.swarm + split into svc-swarm-node/svc-swarm-manager
Adds a `swarm:` subcategory under `svc:` in roles/categories.yml so the
shared-cluster role is properly invokable, and renames the existing
`svc-swarm` role + inventory group to `svc-swarm-node` (with
`get_entity_name`-aligned primary key `node:` in meta/services.yml).
`svc-swarm-manager` stays as the manager-marker role, becoming invokable
via the same new category and re-keyed to `manager:`.

Consumers of the cluster-orchestration service (svc-registry-cache,
svc-registry-docker, svc-swarm-manager) now reference the service-key
`node:` (was `swarm:`), matching the renamed primary entry. The planner
empirically resolves `service_registry['node']` -> `svc-swarm-node` and
emits it into --include for stack-host deploys, so the CI failure on
`universal-svc-registry-docker` (`svc-swarm: not present in inventory`)
no longer reaches `ValidDeployId`.

Breaking change for operators: inventory groups named `svc-swarm:` must
be renamed to `svc-swarm-node:`. `DEPLOYMENT_MODE` reads
`groups['svc-swarm-node']`, so a stale `svc-swarm:` group silently falls
back to compose mode.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-10 18:53:08 +02:00
evangelostsak 72a56b8aa8 fix(web-app-penpot): derive native_login/registration from sso flag (variant-aware)
The all-off matrix variant pins services.sso.enabled=false while
web-app-keycloak stays in group_names, so the previous group_names-based
defaults yielded disable-login-with-password AND no OIDC — leaving no usable
login method, and the native Playwright test failed (no password form).

Derive both toggles from PENPOT_SSO_ENABLED (the variant-aware sso flag),
matching how the oidc/ldap flags are already computed:
- native_login = not sso  (off under OIDC, on otherwise)
- registration = sso       (on under OIDC for JIT provisioning)

Drops the services.penpot.native_login / registration_enabled meta keys
(a group_names default can't see variant overrides). README + req-023 updated.
2026-06-10 18:03:04 +02:00
evangelostsak e7a799f542 feat(web-app-penpot): disable native login under OIDC (Kevin review)
- meta/services.yml: new `penpot.native_login` toggle, default off when
  web-app-keycloak is present (forces SSO), on otherwise.
- vars: PENPOT_FLAGS renders enable-/disable-login-with-password from it.
- tasks: administrator local-password bootstrap gated on native_login.
- playwright: test-login-native.js skips when `sso` is enabled.
- meta/main.yml: drop the Figma parenthetical from the description.
- README + req-023 updated.
2026-06-10 15:35:03 +02:00
evangelostsak e6afb12078 Merge upstream/main into feature/web-app-penpot
# Conflicts:
#	docs/requirements/README.md
2026-06-10 14:07:51 +02:00
evangelostsak 4c051d9939 fix(web-app-penpot): registration on for OIDC JIT + split login specs per persona
Biber OIDC login failed the deploy with `?error=registration-disabled`:
Penpot routes first-time OIDC logins through the registration path, so a
disabled flag blocks JIT-provisioning of not-yet-seen federated users
(admin already existed, biber did not). LDAP self-provisions without it.

- meta/services.yml: registration_enabled now follows OIDC (on when
  web-app-keycloak is present), off otherwise; native admin is bootstrapped
  and LDAP needs no registration. vars/README/req-023 updated to match.
- playwright: split the combined OIDC and LDAP login specs into one file
  per persona — test-login-{oidc,ldap}-{admin,biber}.js — wired through the
  orchestrator; README Testing section updated.
2026-06-10 01:02:55 +02:00
evangelostsakandClaude Opus 4.8 532c2893d6 feat(web-app-penpot): native local-password login + docs
Bootstrap a local password for the administrator profile via the backend
PREPL (enable-prepl-server) so native email/password login works alongside
OIDC/LDAP. Adds enable-login-with-password and a registration toggle driven
by services.penpot.registration_enabled (off once OIDC is the login path).

- vars: PENPOT_FLAGS gains password login + prepl + conditional registration;
  PENPOT_REGISTRATION_ENABLED and PENPOT_ADMIN_{EMAIL,PASSWORD,FULLNAME}.
- tasks: idempotent create-profile / update-profile bootstrap (no_log).
- playwright: test-login-native.js drives the local password Login button.
- README + requirement 023: native login, registration toggle, canonical
  domain penpot.design, CA-trust gating, spec split; tick project/asset
  coverage; fix header 235 -> 023.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 23:21:20 +02:00
evangelostsak ae3eb76e6a add trailing new_line no meet CI tests 2026-06-06 17:33:51 +02:00
evangelostsak 771c449c8f feat(web-app-penpot) add missing requirement file used by the agent and its necessary documentation 2026-06-06 17:24:05 +02:00
evangelostsak 83ab973dfa feat(web-app-penpot): live-verify OIDC/LDAP + project/asset; JVM CA-trust fix; archive req 235
Deployed Penpot + Keycloak + OpenLDAP end to end and drove the per-role
Playwright suite green against the live stack (8 passed, 2 skipped):
TLS baseline, OIDC (administrator + biber), LDAP (administrator + biber),
project creation, and image asset upload. The two skips are the generic
biber/administrator persona scenarios, declared blocked via
PERSONA_*_BLOCKED (mirroring web-app-taiga): Penpot's in-app "OpenID"
login entry and SPA user-menu logout are not driveable by the generic
persona helper, so both users' real auth is exercised by the dedicated
OIDC + LDAP scenarios.

Fixes found during bring-up:
- OIDC JVM CA trust: the backend command imports the internal CA into a
  writable cacerts copy and points the JVM at it via JAVA_TOOL_OPTIONS.
  with-ca-trust.sh covers OS/NSS/env trust but not the JVM truststore, so
  server-side token/userinfo calls to Keycloak failed with "PKIX path
  building failed". CA path hardcoded (the CA-override double-escapes a $).
- disable-onboarding added to PENPOT_FLAGS so users land on the dashboard.
- Playwright env: ADMIN_EMAIL/BIBER_EMAIL for the LDAP form; alphabetised.

Requirement 235 is fully checked off and archived out of docs/requirements/.
2026-06-06 08:22:13 +02:00
evangelostsak edef635fbb feat(web-app-penpot): add Penpot design platform role (req 235)
Add a new `web-app-penpot` role deploying Penpot's frontend + backend +
exporter containers behind the standard reverse proxy, with shared
svc-db-postgres and a svc-db-redis sidecar. Wires native OIDC (Keycloak)
and LDAP (OpenLDAP) login via PENPOT_FLAGS, CSP/vhost via sys-stk-full,
backup-ready asset volume, marketplace tags, matrix variants
(all-on/all-off/LDAP-only), and a service-gated Playwright spec with
guest/biber/administrator personas plus separate OIDC and LDAP scenarios.

Implements docs/requirements/235-setup-penpot.md. Live-deploy DoD items
(fresh-host deploy, domain reachability, full in-container `make test`)
remain unverified in the sandbox and are left unchecked.
2026-06-06 08:22:13 +02:00
evangelostsak 4757a639b0 revert: clean branch to upstream main state
Remove the entire web-app-penpot role, all associated docs, and all other
changes accumulated on this branch. Working tree now matches origin/main
exactly. Commit history preserved for reference.
2026-06-06 08:22:13 +02:00
evangelostsak d57c0aca2d feat(web-app-penpot): complete role setup with playwright e2e tests
- Move playwright spec to roles/web-app-penpot/files/playwright/ (standard location)
- Fix APP_BASE_URL: use normalizeUrl() to strip dotenv_quote double-quotes
- Gate biber/admin tests with safeSkipUnlessEnabled('oidc') (penpot requires OIDC)
- Add meta/variants.yml with all-true, all-false, LDAP-only variants
- Fix templates: use container_depends_on lookup, correct LDAP group vars
- Align services.yml: javascript service static false with nocheck comment
- Update playwright.env.j2: OIDC_SERVICE_ENABLED env var for service gate
- All make test targets pass (lint, test-unit, test-integration, test-lint, test-external)
2026-06-06 08:22:13 +02:00
evangelostsak 07344e2ca7 feat(penpot): add setup documentation for Penpot integration in Infinito.Nexus 2026-06-06 08:22:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 23e4d5a0b2 chore(workflows): replace whale emoji with picture frame in CI image build
Match the workflow's actual purpose (building OCI images) more
literally than the Docker mascot, freeing 🐳 for compose-specific
contexts (see test-deploy-compose.yml).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 20:57:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 16a4298dbb refactor(ci): rename test-deploy-server to test-deploy-compose
Use whale (🐳) for the per-app matrix job to mirror the swarm
workflow's bee (🐝) pattern, and magnifying glass (🔍) for the
discover step. Keep the workflow header on the existing 📦 prefix to
stay consistent with the other test-deploy-* siblings.

Rename the workflow file and align all references:
  - ci-orchestrator.yml job key + uses: path
  - artifact names: inventory-server-* -> inventory-compose-*,
                    playwright-server-* -> playwright-compose-*
  - scripts/github/inventory/zip_compose_for_upload.sh output path
  - 3 docs files

INFINITO_DEPLOY_TYPE: server stays - that key is wired to the
invokable-rule registry in utils/roles/validation/invokable.py and is
a separate refactor concern.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 18:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 23ec7a99fb refactor(roles): rename svc-{cache,docker}-* roles for prefix consistency
svc-cache-registry       -> svc-registry-cache
  svc-docker-registry      -> svc-registry-docker
  svc-docker-swarm         -> svc-swarm
  svc-docker-swarm-manager -> svc-swarm-manager

categories.yml: drop svc.docker, add svc.registry. Deliberately no svc.swarm
category so get_entity_name('svc-swarm') stays 'swarm' instead of collapsing
to '' (which would break discover_role_services).

Entity keys updated to match the new svc.registry prefix:
  services.cache-registry.* -> services.cache.*
  services.registry.*       -> services.docker.*

01_constructor: replace svc-docker bootstrap-loop entry with explicit
include_role of svc-swarm + bootstrap loop over [svc-registry, svc-storage].

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 15:18:19 +02:00
kevinveenbirkenbachandClaude Opus 4.7 70c1a03f13 feat(web-app-matrix): add ansible flavor via unified compose.yml.j2
Implements requirement 025 (docs/requirements/025-web-app-matrix-ansible-flavor.md).

Architecture:
- services.matrix.flavor: ansible | compose
- Single templates/compose.yml.j2 branches on matrix_flavor
- ansible flavor: defines matrix-mdad-runner one-shot compose service that
  DiD-executes upstream matrix-docker-ansible-deploy (pinned ref + migration
  validated version). MDAD writes /etc/systemd/system/matrix-*.service units
  on the host; post-up Ansible task container-waits the runner, then probes
  loopback ports + wires sys-stk-front-proxy.
- compose flavor: legacy in-repo stack (synapse/element/coturn/bridges) kept
  for regression coverage, marked deprecated.

Variant-switch ansible -> compose:
- 00_cleanup_mdad_residue.yml tears down MDAD-installed systemd units +
  leftover matrix-* containers before compose-flavor brings up its own
  matrix-synapse (else: name conflict, compose down doesn't touch
  systemd-managed state).

Reorganization:
- roles/web-app-matrix/{tasks,templates,files}/flavor/{ansible,compose}/
- Renamed legacy compose-flavor task files into flavor/compose/
- New runner Dockerfile (multi-stage, ARG-parameterized FROM via build-args)

Verified: 3 variants x 2 passes (sync+async) full_cycle green, all 6
PLAY RECAPs failed=0. V1 (ansible all-on), V2 (ansible no-auth), V3
(compose deprecated all-on). Playwright matrix tests pass per variant.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 10:21:41 +02:00
kevinveenbirkenbachandClaude Opus 4.7 587a3a1796 feat(swarm): consolidate container_name + pull_policy into generic compose_only lookup
Replace per-key `compose_container_name` lookup with a generic
`compose_only(key, value)` SPOT that emits `key: "<value>"` only in
compose mode and nothing in swarm. Same gating logic now serves
container_name (157 call sites) and pull_policy (17 call sites)
without duplicate plugin code.

Why this consolidation:
- The earlier `build-key` lint rule was misdirected. Swarm tolerates
  `build:` with a harmless warning when `image:` is pre-pushed (via
  the existing handlers/swarm_registry_push.py flow in
  roles/sys-svc-compose). The actual swarm-rejected key is
  `pull_policy:` ("Additional property pull_policy is not allowed"),
  and 10 templates were silently broken because they declared inline
  `pull_policy: never` outside the existing `{% if DEPLOYMENT_MODE
  \!= 'swarm' %}` gate in roles/sys-svc-container/templates/build.yml.j2.
- A single generic lookup is cleaner than N per-key wrappers when
  the gating logic is identical (emit-key-or-nothing).

Lint changes in tests/lint/ansible/jinja/test_mode_incompatible_keys.py:
- New `pull-policy-key` rule, regex `^\s*pull_policy\s*:`, remediation
  points at compose_only.
- Existing `container-name` rule keeps its raw-key regex; remediation
  now also points at compose_only.
- Broken-split / broken-embed regexes rekeyed on the `compose_only`
  lookup name (was `compose_container_name`).
- `build-key` rule and its 12 nocheck annotations across templates
  removed.

Coordinated changes:
- 86 templates rewritten (157 container_name calls migrated;
  17 inline pull_policy lines replaced).
- 1 redundant `{% if DEPLOYMENT_MODE \!= 'swarm' %}` guard dropped in
  build.yml.j2 (the lookup self-gates).
- 12 unit tests pin the compose_only contract.
- Catalog (suppression.md) updated: `pull-policy-key` row added,
  `container-name` row remediation updated.
- Old `plugins/lookup/compose_container_name.py` and its test module
  deleted.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 02:24:50 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3816e69638 feat(swarm): compose_restart lookup gates restart key per deploy mode
Mirror compose_container_name pattern: plugins/lookup/compose_restart.py
emits `restart: <policy>` only in compose mode. In swarm the key is
silently ignored (replaced by deploy.restart_policy) and produces
"Ignoring unsupported options" warnings at docker stack deploy time.

Sweep 19 raw restart: sites across 10 templates to the no-arg form
`{{ lookup('compose_restart') }}` (the lookup falls back to
DOCKER_RESTART_POLICY from ansible-vars).

base.yml.j2 keeps the explicit-arg form
`{{ lookup('compose_restart', docker_restart_policy | default(DOCKER_RESTART_POLICY)) }}`
so the 5 templates that override the policy via Jinja-scope
`{% set docker_restart_policy = ... %}` (erpnext, zammad, shopware,
matomo, openproject) retain their override path. Jinja-scope bindings
are visible only in the rendering template, never in the Ansible
variable context that the lookup sees, so the caller must resolve them.

Generalize the container-name lint into a data-driven multi-rule
scanner at tests/lint/ansible/jinja/test_mode_incompatible_keys.py
covering both container-name and restart-key. Adding a new rule is one
ModeRule row sharing the file-walk / suppression / line-grouping
scaffolding.

Also gate roles/svc-db-redis/templates/service.yml.j2's
resource.yml.j2 include with DEPLOYMENT_MODE \!= 'swarm'. Mirrors
base.yml.j2's existing pattern; prevents cpus / mem_limit /
mem_reservation / pids_limit from leaking into swarm output where
deploy.resources.* in deploy.yml.j2 already handles them.

11 unit tests pin the compose_restart lookup contract. Both rules
registered in docs/contributing/actions/testing/suppression.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 02:24:50 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9cacfbf959 refactor(swarm): inventory-driven topology replaces per-role swarm flags
DEPLOYMENT_MODE is now derived per role from group membership:
swarm iff len(groups[application_id]) > 1, else compose. The cluster
infrastructure flag SWARM_CLUSTER_AVAILABLE is true whenever any host
sits in svc-docker-swarm, so DB and proxy services render attachable
overlay networks even when their own group lists only one host.

All service_is_stateful flags are removed across svc-db-mariadb,
svc-db-postgres, svc-db-openldap, svc-prx-openresty, svc-docker-registry
and svc-db-mariadb's compose template; the inventory shape now expresses
the same intent. Cluster-wide handlers (swarm build/push/validate/deploy,
external network creation, DB healthcheck) gain run_once + delegate_to
the manager so they execute exactly once regardless of how many hosts
the role's play targets. CA-injection and compose validate are gated to
compose-mode only; their compose-config strictness breaks when the
deploy block coexists with container_name under swarm.

Dead per-host swarm-resolve infrastructure is removed: swarm_resolve_
container.sh, resolve_container.yml, resolve_self.yml, the MariaDB-
specific swarm_mariadb_cid.sh, the cross-host template replication
step, and the _swarm_retries / _swarm_delay defaults. With run_once +
delegate_to: manager replacing the old "render every template on every
swarm peer" pattern, every consumer becomes a noop on workers and
collapses to the manager's one-shot path.

The web-app-nextcloud memcache override mapping flips from the legacy
"single" sentinel to "compose" (semantically equivalent under the new
model). Requirement 023 is updated to reflect the new model and drops
the now-irrelevant per-role opt-out / placement-constraint clauses.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-04 17:11:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a71c5a2de6 docs(requirements): add 025 web-app-matrix ansible flavor
New requirement for adding a second deployment flavor to web-app-matrix
that delegates to the upstream matrix-docker-ansible-deploy (MDAD)
playbook. The existing role-internal compose stack stays selectable as
the `compose` flavor for one deprecation window.

Captures 12 proposed design decisions for operator review before
implementation: flavor dispatch in services.matrix.flavor (ansible
default; compose deprecated), MDAD pulled in as a pinned git checkout at
deploy time (not vendored), MDAD inventory templated from
lookup('config', ...), every supported bridge + Jitsi + Element Call
enabled by default when no operator-issued third-party token is needed,
central svc-db-postgres + svc-db-redis + Mailu reuse, sys-svc-proxy
fronts MDAD's services (MDAD's bundled traefik disabled), four variants
(V1 ansible all-on, V2 ansible all-off, V3 ansible ldap-only, Vcompose-1
regression), two new Playwright specs (element-call, bridge-roster),
lifecycle: alpha on the new flavor.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-03 12:01:19 +02:00
kevinveenbirkenbachandClaude Opus 4.7 fa4eea5a86 refactor(deployment-mode): SPOT-define DEPLOYMENT_MODE in group_vars
Replace the host-scoped set_fact resolvers in sys-svc-compose/tasks/01_core.yml
and tasks/stages/01_constructor.yml with a single group_vars definition driven
by group_names. Uppercase the variable to match the SPOT convention and drop
every `| default('compose')` fallback now that the resolution is unconditional.

Role-var opt-out on a swarm host still works via Ansible's role-vars > group_vars
precedence.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-03 10:06:20 +02:00
kevinveenbirkenbach cd14ded859 Merge branch 'main' into feature/swarm-nfs 2026-06-03 09:53:56 +02:00
kevinveenbirkenbachandClaude Opus 4.7 39c990a7ad feat(web-app-erpnext): ship ERPNext role with OIDC SSO, LDAP, outbound mail
Implements requirement docs/requirements/024-web-app-erpnext.md.

Stack (per frappe_docker v15 single-image multi-role pattern):
- configurator (one-shot bench set-config: db_host/port, three redis URLs
  split by DB number 0/1/2 for cache/queue/socketio, socketio_port, host_name
  HTTPS)
- backend (gunicorn), frontend (nginx-entrypoint.sh), websocket (socketio),
  scheduler, queue-short (--queue short,default), queue-long
  (--queue long,default,short)

Central-service consumer pattern (Decision #7):
- ERPNEXT_DB_ROOT_PASSWORD branches on services.mariadb.shared: central
  svc-db-mariadb root pw when true, per-role MariaDB consumer pw when false
- Redis via in-compose `redis` alias, 3-DB-number split for Frappe's logical
  buses
- Explicit erpnext_assets volume mount on every container (image's
  VOLUME ['sites/assets'] otherwise creates anonymous volumes that fail the
  hlth-volumes check)

Bench bootstrap (Decision #9):
- bench new-site --install-app erpnext with 30-min hard timeout
- Post-install restart of all long-lived Frappe containers (otherwise the
  cached app set serves HTTP 500 on /login)
- TCP-port readiness probe (HTTP would 404 pre-site)
- API-bot user seed + setup_complete=1 via apply_api_bot.py

OIDC (Decisions #4 + #5):
- redirect_uris filter auto-includes ERPNext in the shared Keycloak client
- Social Login Key wired via apply_oidc_settings.py with
  social_login_provider=Keycloak (delete+recreate due to set_only_once),
  custom_base_url=1, relative authorize/token/userinfo paths,
  auth_url_data={response_type:code,scope:openid}, login_via_keycloak
  redirect path
- Group-to-Frappe-role mapping persisted as a frappe.db default; per-login
  reconciliation hook deferred to follow-up (documented in README)

LDAP (V3): LDAP Settings doctype applied via apply_ldap_source.py. End-to-end
LDAP login (auto-create Frappe user from LDAP bind) deferred to follow-up;
LDAP-login Playwright specs intentionally absent in v1 (documented).

Email (Decision #8, outbound only): outbound Email Account applied via
apply_email_account.py using db_insert/db_update to bypass Frappe's
deploy-time SMTP socket validation. IMAP inbound deferred to follow-up.

Variant matrix (Decision #11): V1 sso+ldap, V2 all-false, V3 ldap-only.
Three variants verified end-to-end via
`make compose-deploy mode=reinstall apps=web-app-erpnext full_cycle=true
purge=true` — 6 PLAY RECAPs, all failed=0, all Playwright runs
passed-or-skipped with no failures.

Playwright specs: test-landing, test-csp-headers, test-login-administrator
(OIDC), test-login-biber (OIDC), test-login-native-administrator
(break-glass local login, always runs), test-guest-persona. _shared.js
exposes signInViaErpnextOidc + signInViaErpnextLocal.

Repo-wide lint addition: tests/lint/ansible/modules/test_task_name_length.py
caps Ansible task `name:` strings at 120 chars (opt-out via
`# nocheck: name-length`). Fixed two pre-existing violations in
web-app-keycloak/tasks/update/ldap/per_app_mapper_one.yml.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-03 09:53:56 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b895322a63 docs(requirements): add 024 web-app-erpnext with OIDC SSO
New requirement for a `web-app-erpnext` role that integrates ERPNext
(Frappe Framework) into Infinito.Nexus with OIDC-direct SSO against
Keycloak, reusing central `svc-db-mariadb` and `svc-db-redis` (three
Redis logical roles split by DB number 0/1/2 for cache/queue/socketio).

Captures 13 proposed design decisions for operator review before
implementation: upstream `frappe/erpnext` image pinning, central-service
reuse (Decision #7), wizard-bypass `bench` bootstrap (Decision #9),
Keycloak-group → Frappe-role reconciliation (Decision #5), Mailu IMAP
inbound + SMTP outbound (Decision #8), and the standard
V1/V2/V3 variant matrix (Decision #11) mirroring web-app-kix /
web-app-zammad. Out of scope for v1: multi-site / multi-tenant Frappe
(Decision #12).

Modeled on the layout precedents from web-app-odoo (ERP shape,
dual HTTP + WebSocket vhost) and web-app-zammad (central-service
reuse pattern, three-variant matrix).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-03 09:53:55 +02:00
kevinveenbirkenbachandClaude Opus 4.7 014fa54ad3 feat(web-app-erpnext): ship ERPNext role with OIDC SSO, LDAP, outbound mail
Implements requirement docs/requirements/024-web-app-erpnext.md.

Stack (per frappe_docker v15 single-image multi-role pattern):
- configurator (one-shot bench set-config: db_host/port, three redis URLs
  split by DB number 0/1/2 for cache/queue/socketio, socketio_port, host_name
  HTTPS)
- backend (gunicorn), frontend (nginx-entrypoint.sh), websocket (socketio),
  scheduler, queue-short (--queue short,default), queue-long
  (--queue long,default,short)

Central-service consumer pattern (Decision #7):
- ERPNEXT_DB_ROOT_PASSWORD branches on services.mariadb.shared: central
  svc-db-mariadb root pw when true, per-role MariaDB consumer pw when false
- Redis via in-compose `redis` alias, 3-DB-number split for Frappe's logical
  buses
- Explicit erpnext_assets volume mount on every container (image's
  VOLUME ['sites/assets'] otherwise creates anonymous volumes that fail the
  hlth-volumes check)

Bench bootstrap (Decision #9):
- bench new-site --install-app erpnext with 30-min hard timeout
- Post-install restart of all long-lived Frappe containers (otherwise the
  cached app set serves HTTP 500 on /login)
- TCP-port readiness probe (HTTP would 404 pre-site)
- API-bot user seed + setup_complete=1 via apply_api_bot.py

OIDC (Decisions #4 + #5):
- redirect_uris filter auto-includes ERPNext in the shared Keycloak client
- Social Login Key wired via apply_oidc_settings.py with
  social_login_provider=Keycloak (delete+recreate due to set_only_once),
  custom_base_url=1, relative authorize/token/userinfo paths,
  auth_url_data={response_type:code,scope:openid}, login_via_keycloak
  redirect path
- Group-to-Frappe-role mapping persisted as a frappe.db default; per-login
  reconciliation hook deferred to follow-up (documented in README)

LDAP (V3): LDAP Settings doctype applied via apply_ldap_source.py. End-to-end
LDAP login (auto-create Frappe user from LDAP bind) deferred to follow-up;
LDAP-login Playwright specs intentionally absent in v1 (documented).

Email (Decision #8, outbound only): outbound Email Account applied via
apply_email_account.py using db_insert/db_update to bypass Frappe's
deploy-time SMTP socket validation. IMAP inbound deferred to follow-up.

Variant matrix (Decision #11): V1 sso+ldap, V2 all-false, V3 ldap-only.
Three variants verified end-to-end via
`make compose-deploy mode=reinstall apps=web-app-erpnext full_cycle=true
purge=true` — 6 PLAY RECAPs, all failed=0, all Playwright runs
passed-or-skipped with no failures.

Playwright specs: test-landing, test-csp-headers, test-login-administrator
(OIDC), test-login-biber (OIDC), test-login-native-administrator
(break-glass local login, always runs), test-guest-persona. _shared.js
exposes signInViaErpnextOidc + signInViaErpnextLocal.

Repo-wide lint addition: tests/lint/ansible/modules/test_task_name_length.py
caps Ansible task `name:` strings at 120 chars (opt-out via
`# nocheck: name-length`). Fixed two pre-existing violations in
web-app-keycloak/tasks/update/ldap/per_app_mapper_one.yml.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-02 21:01:38 +02:00
kevinveenbirkenbachandClaude Opus 4.7 e8a120f898 docs(requirements): add 024 web-app-erpnext with OIDC SSO
New requirement for a `web-app-erpnext` role that integrates ERPNext
(Frappe Framework) into Infinito.Nexus with OIDC-direct SSO against
Keycloak, reusing central `svc-db-mariadb` and `svc-db-redis` (three
Redis logical roles split by DB number 0/1/2 for cache/queue/socketio).

Captures 13 proposed design decisions for operator review before
implementation: upstream `frappe/erpnext` image pinning, central-service
reuse (Decision #7), wizard-bypass `bench` bootstrap (Decision #9),
Keycloak-group → Frappe-role reconciliation (Decision #5), Mailu IMAP
inbound + SMTP outbound (Decision #8), and the standard
V1/V2/V3 variant matrix (Decision #11) mirroring web-app-kix /
web-app-zammad. Out of scope for v1: multi-site / multi-tenant Frappe
(Decision #12).

Modeled on the layout precedents from web-app-odoo (ERP shape,
dual HTTP + WebSocket vhost) and web-app-zammad (central-service
reuse pattern, three-variant matrix).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-02 21:01:37 +02:00
kevinveenbirkenbach 43b614ec26 Merge branch 'main' into feature/swarm-nfs
# Conflicts:
#	roles/dev-nix/tasks/02_non_arch_installer.yml
2026-06-02 19:30:12 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3c07721303 feat(swarm-nfs): bootstrap Docker Swarm + NFS-backed shared volumes (#023)
Adds the svc-docker-swarm, svc-docker-swarm-manager, svc-storage-nfs-server
and svc-storage-nfs-client roles, the compose_volumes filter NFS support,
the test-deploy-swarm-nfs CI workflow with its swarm-nfs-pilot script set,
and the deploy_mode-aware compose/swarm wiring across mediawiki, mariadb
and sys-svc-* helpers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 18:57:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b3541d873a docs(requirements): add 023 - Docker Swarm with NFS-backed shared volumes
Defines the v1 scope for multi-node Docker Swarm support: group-based
trigger (svc-docker-swarm + svc-docker-swarm-manager), three new roles
(svc-docker-swarm, svc-storage-nfs-server, svc-storage-nfs-client),
per-volume NFS opt-in (DB volumes stay local + pinned), overlay
networks with default encryption, and openresty kept as the swarm-mode
edge (pinned to manager, L7 LB via tasks.<service> upstream blocks) to
preserve the sys-front-inj-* injection pipeline unchanged. Pilot role
is web-app-mediawiki, validated by a 3-node DinD CI workflow whose
inventory is bootstrapped via the canonical inventory provision CLI.
Procedure block follows the autonomous-run pattern from 014.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 15:33:16 +02:00
kevinveenbirkenbachandClaude Opus 4.7 278d5a1b48 refactor(web-app-zammad): sweep to unified services.sso schema
Migrates the role from the pre-v9.0.0 legacy oidc/oauth2 top-level
service blocks to the unified services.sso shape with flavor: oidc.
Drops the playwright.spec.js require for the test-alias-domain module
that was deleted in 82491791e, and removes the README link to the
TODO.md file dropped in c48386eee. Archives the in-flight 022 zammad
requirement doc into the README index since the role itself is now
landed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 13:12:05 +02:00
kevinveenbirkenbach 85c1ed1a6e Merge branch 'main' into feature/web-app-zammad
# Conflicts:
#	CHANGELOG.md
#	packaging/arch/PKGBUILD
#	packaging/debian/changelog
#	packaging/fedora/infinito-nexus.spec
#	pyproject.toml
#	roles/sys-svc-compose/README.md
#	scripts/docker/entry.sh
#	scripts/github/update/image_versions.sh
#	scripts/github/update/repository_refs.sh
2026-05-29 11:18:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7e38fca6d7 docs: fix broken links to archived requirement files
Drops markdown links in 017/018 and role TODOs that pointed at the
016 and 019 requirement files archived in 027ba06ee. The text
references stay; only the now-broken link targets are removed so the
markdown-links lint passes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 21:06:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ce078d1d78 docs(requirements): archive 7 completed requirements
Moves 008/009/010/011/016/019/020 to the archive list in
docs/requirements/README.md and removes the per-requirement files;
their acceptance criteria are considered closed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 21:06:33 +02:00
kevinveenbirkenbach 84bd56e72c Merge branch 'main' into feature/req-21-sso-flavor-migration 2026-05-28 18:07:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8e293bd921 fix(compose-deploy): stop persisting runtime routing knobs into .env
Root cause of step-08 deploy failures across all 4 test-development
distros in run 26572559126: `make compose-deploy mode=reinstall
apps=web-app-dashboard INFINITO_SERVICES_DISABLED=matomo` (step 07)
made make export the cmdline var into the recipe's shell env, which
reached `runner.py` → `make setup` → `make dotenv`. The dotenv
generator's `setdefault` then baked the runtime-only value into the
on-disk `.env`. Step 08 sourced that .env, filtered `web-app-matomo`
out of its primary apps, and exited with "All primary apps disabled".

Fix:

- Drop the 7 routing knobs (apps, mode, purge, bundles, disable,
  full_cycle, variant) from `passthrough.STATIC_KEYS` and from
  `default.env`. They are per-invocation; .env is the persistent SPOT
  and must not capture them.
- Switch every consumer from `os.environ["KEY"]` strict access to
  `os.environ.get("KEY", "")` so the missing-key path stops at "no
  filter" instead of a KeyError.
- Rename the env-var surface to match the make-alias surface 1:1
  (lowercase): INFINITO_APPS → apps, INFINITO_SERVICES_DISABLED →
  disable, INFINITO_BUNDLES → bundles, INFINITO_FULL_CYCLE →
  full_cycle, INFINITO_VARIANT → variant, INFINITO_DEPLOY_MODE → mode,
  INFINITO_PURGE_ENTITIES → purge. The make alias was the user-facing
  name in every doc; routing-knob env vars now use the same name so
  the cmdline form, the recipe env-prefix, and the consumer reads
  all line up.
- Collapse the Makefile `compose-deploy` recipe to just the two
  knobs whose alias differs from the env name (`type` →
  INFINITO_DEPLOY_TYPE, `debug` → INFINITO_DEBUG). GNU Make
  auto-exports cmdline variables into the recipe's shell env, so the
  identical-name `$(if $(apps),apps="$(apps)")` lines were pure
  noise.
- Drop `disable: ${disable}` from compose.yml. The dev CLI builds
  `extra_env` per `docker exec` call and forwards `disable` via
  `-e`, so the container env never needs the value frozen at
  compose-up time.
- Update the 3 actual leak call sites
  (07/08/09_deploy_*.sh) to the make-alias form
  (`disable=matomo`, `full_cycle=true`, `variant=1`).
- Add `SIM112` to ruff's ignore set: the lowercase env names are
  the deliberate make/env parity, not a typo.

Repro before fix: `INFINITO_SERVICES_DISABLED=matomo make dotenv`
wrote `INFINITO_SERVICES_DISABLED=matomo` to `.env`. After fix:
`disable=matomo make dotenv` leaves `.env` untouched.

Tests: `make test` and `make autoformat` green (2711 tests, 8 lint
sub-targets).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 17:47:03 +02:00
kevinveenbirkenbach 6b63c12573 Merge branch 'main' into feature/req-21-sso-flavor-migration 2026-05-28 13:01:55 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f473b12888 chore(workflows): migrate create-github-app-token from app-id to client-id
actions/create-github-app-token@v3 emits "Input 'app-id' has been
deprecated with message: Use 'client-id' instead." on every run.
Swap the input + rename the corresponding repository secret from
BOT_APP_ID to BOT_APP_CLIENT_ID so the value reflects what the
action actually expects: the OAuth-style Client ID (`Iv…` format
shown on the App's General page), not the numeric App ID.

Requires the operator to create a fresh `BOT_APP_CLIENT_ID`
repository secret on infinito-nexus/core before merging; the old
`BOT_APP_ID` secret becomes orphaned and SHOULD be deleted in the
same settings pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 13:01:55 +02:00
kevinveenbirkenbachandClaude Opus 4.7 29a01ff538 chore(workflows): migrate create-github-app-token from app-id to client-id
actions/create-github-app-token@v3 emits "Input 'app-id' has been
deprecated with message: Use 'client-id' instead." on every run.
Swap the input + rename the corresponding repository secret from
BOT_APP_ID to BOT_APP_CLIENT_ID so the value reflects what the
action actually expects: the OAuth-style Client ID (`Iv…` format
shown on the App's General page), not the numeric App ID.

Requires the operator to create a fresh `BOT_APP_CLIENT_ID`
repository secret on infinito-nexus/core before merging; the old
`BOT_APP_ID` secret becomes orphaned and SHOULD be deleted in the
same settings pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 12:47:59 +02:00