Commit Graph
752 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.8 d0ae004eb0 feat(env): fork-identity vars + CI-redeploy aliases
INFINITO_FORK_REPOSITORY_NAME (default.env, static) plus two handlers: INFINITO_FORK_ACCOUNT defaults to the local OS username (getpass.getuser()), and INFINITO_FORK_REPOSITORY_URL composes https://github.com/<account>/<name>. Registered in ORDERED_HANDLERS with the account handler before the URL handler; documented in docs/contributing/environment/variables.md.

Two operator aliases build on it: i8cire (make quality + git-sign-push + retrigger the failed swarm CI jobs of a run) and i8puci (the same without the quality gate). Both take a run id or full URL: a bare id is expanded against INFINITO_FORK_REPOSITORY_URL, a URL is passed through, and a missing INFINITO_FORK_REPOSITORY_URL fails fast.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-16 03:44:16 +02:00
kevinveenbirkenbachandClaude Fable 5 40e553069d refactor(structure): nest prefix-grouped sibling files into subfolders
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.

Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fc0799883f refactor(structure): move prefix-grouped files into named subfolders
Resolve every 3+-file same-prefix group flagged by the new prefix lint
by moving the files into a subfolder named after the shared prefix and
dropping the prefix from the file names, updating all references:

- utils/env/handlers/infinito_*.py -> infinito/ (dir/, package_cache/
  nested a second level); registry imports and the four handler-dir
  globs switched to iter_project_files
- utils/install/lint/ansible_*.py -> ansible/ (aliased imports)
- roles/sys-svc-compose/files/swarm_*.{py,sh} -> files/swarm/
- web-app-erpnext/web-app-zammad files/scripts/apply_*.{py,rb} -> apply/
- web-app-odoo files/sql/{select,update}-* and sql/oidc/{grant,insert,
  select,update}-* -> matching subfolders
- web-app-keycloak tasks/update/05_ldap*.yml -> 05_ldap/ (internal
  includes made role_path-absolute)
- plugins/filter/get_*.py -> plugins/filter/get/ (filter names are the
  filters() dict keys, so Jinja usage is unaffected; ansible discovers
  the one-level-deep subdir)

Ansible lookup dirs, addon dirs and .github/workflows keep their flat
layout (the filename is the public id / GitHub only discovers flat).
Unit tests mirror the new source layout.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Fable 5 715b207990 docs(backup): deploy-time trigger flow diagram
Second diagram in the backup architecture doc: when the units run
during a deploy, keyed on MODE_BACKUP and install vs update, including
the preload instant snapshots, the end-of-play remote pull and the
abort-on-failed-snapshot semantics.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 01:47:52 +02:00
kevinveenbirkenbachandClaude Fable 5 7b6ca22d75 feat(backup): recover.py framework for the svc-bkp-* role family
Add utils/recovery/base.py DirectoryRecovery: a service-backed pre-recover
(start the role's backup unit first) + rsync -a --delete mirror. Each
svc-bkp-* role gets a files/recover.py subclass (volume resolves the docker
mountpoint, nfs mirrors the export, local-2-device opens the newest LUKS
snapshot across machine hashes) or a documented one-way opt-out
(remote-2-local recover.py.nocheck). A lint guard enforces every svc-bkp-*
role ships a recover.py and a ## Recover README section; docs/architecture/
backup.md documents the end-to-end schema. Unit tests cover the base flow,
the per-role class logic and every recover.py main() CLI wiring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 03:15:52 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f76bfcf531 refactor(svc-bkp-volume-2-local): rename role from svc-bkp-container-2-local
Rename the role, its test tree, the primary entity key
(volume-2-local), the derived vars (BKP_VOLUME_2_LOCAL_*,
SYS_SERVICE_BACKUP_VOLUME_2_LOCAL), the run-once fact, and every
group_names consumer flag, include path, doc, and lint reference. The
container_backup service key and the on-disk backup-docker-to-local
tree keep their names: both are contracts with existing consumer
entries, snapshot trees, and the baudolo pull tooling.

Operators must rename the svc-bkp-container-2-local inventory group and
disable the previous versioned units on already-provisioned hosts; the
pre-deploy snapshot glob derives its base from the new role name and
will not find them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:52:02 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f08d71534a feat(tooling): operator alias, shortcut, skill, cheat and onboard system
Add the i8* terminal aliases (make install-alias) and ai8* agent
conversation shortcuts (docs/.../agents/alias.md), move skill
distribution to the external INFINITO_SKILLS_REPOSITORY (make
install-skills), and add make cheat, make alias and make onboard. Introduce
custom.env operator overrides (win over defaults and handlers) and pin
INFINITO_ALIAS_MD / INFINITO_ALIAS_REPOSITORY / INFINITO_SKILLS_REPOSITORY
in default.env. Drop the in-repo skills-lock.json and its update-skills CI
job. AGENTS.md instructs agents to suggest the matching cli/prompt alias
when an operator writes something an alias covers. Lints enforce alias
sorting, i8/ai8 prefixes, no consonant clusters, full make-target
coverage, and terminal/agent shortcut non-overlap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:43:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c057b1a1c8 fix(roles): never quote container_address swarm resolver fragments
In swarm mode the container_address lookup returns a $(/usr/bin/resolve-container-id ...) fragment that must expand at exec time; piping it through Jinja quote hands docker the literal string and every exec fails with 'No such container' (broke all three snipe-it CI variants). Remove the quote at the seven snipe-it sites and the live matrix/openproject sites, convert openproject's argv exec to shell so the fragment can expand, split the matrix bridge restart per deployment mode with a manager gate, and mark the compose-only or eval-contract sites with nocheck. A new lint (container-address-quote) bans the pattern repo-wide.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 17:30:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 09a131d007 refactor(placement): rename meta key default_placement to placement
Rename the meta/services.yml key across all roles, the roles_with_default_placement lookup (now roles_with_placement), the get_role_default_placement / iter_roles_with_default_placement helpers, and every consumer (swarm closure, compose_volumes filter, nfs_prep, container deploy template, constructor stage, extend_inventory) plus their tests and docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 02:19:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 af2ea998c2 docs(cheatsheet): extend the role-iteration prompt to compose AND swarm
The web-app role-iteration prompt only covered the compose loop; update it to iterate both deploy modes (compose then swarm) per the swarm and roundtrip loop docs, including the swarm-exec/shell/down/clean live-cluster steps and the compose-AND-swarm green gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 00:03:34 +02:00
kevinveenbirkenbachandClaude Fable 5 15f95f0725 fix(curl): timeout-hardened curl SPOT filter, lint, and fleet-wide rollout
A curl against an accepted-but-silent peer hangs forever: task retries
never fire because the first attempt never returns, and the CI job dies
at the 6h runner cut (proven by the mailu admin-API hang that took the
suitecrm/peertube swarm legs down). The new `curl` filter renders the
hardened base command ("{{ 30 | curl }}" -> curl -s --connect-timeout 5
--max-time 30); scaling composes with lookup('timeout').

test_curl_via_filter enforces it: bare curl invocations in role tasks,
templates and shipped scripts fail the lint with a pointer to the
filter. Plain .sh files, which cannot render Jinja, pass by pinning
--max-time inline; Docker healthchecks are exempt (their timeout field
caps them); `# nocheck: curl-timeout` opts out (catalogued in
suppression.md).

Convert all 34 existing invocations: mailu (replacing the short-lived
MAILU_CURL var), elasticsearch, checkmk, yourls (300s for the
schema-creating installer POST), the telegram alert script, and the
plain scripts of jellyfin, svc-runner, apt_repo, moodle (600s tarball)
and the mattermost provisioner. Mailu's user/token tasks additionally
carry task-level timeouts so a wedged container exec cannot stall the
play.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 02:37:28 +02:00
kevinveenbirkenbachandClaude Fable 5 3f32176241 docs(cheatsheet): tighten pipeline-debugging prompt
Rewrite the operator-drafted additions in machine-coherent MUST language:
15-minute polling of newly failed jobs, 99% root-cause certainty before
fixing, no commit/push/ask commands, and a verified-fix completion gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 17:26:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 945fbccc1e refactor(swarm): move service_replicas into services.yml
Replace the per-role service_replicas Jinja variable with a per-service
replicas: field in meta/services.yml. deploy.yml.j2 resolves it via
resource_filter (with entity fallback), the same path already used for
cpus/mem, so services.yml is the single source of truth for replica
overrides.

- 13 web roles migrated to per-service (or entity-level) replicas: 1
- matomo bootstrap -> replicas: 0, mastodon migrate -> replicas: 1
- minio/seaweedfs provider entries -> replicas: 1, which also pins the
  central single-writer object-store providers instead of relying on the
  topology default
- service_replicas removed entirely; dropped its dead reference from the
  compose-structure lint allowlist and updated requirement 023 wording

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:11 +02:00
kevinveenbirkenbach 9ee4cd0fb9 Merge branch 'main' of github.com:infinito-nexus/core into feature/swarm-nfs
# Conflicts:
#	docs/agents/action/iteration/compose.md
#	roles/svc-db-openldap/vars/main.yml
#	roles/svc-runner/defaults/main.yml
#	roles/svc-runner/tasks/02_cleanup_instance.yml
#	roles/web-app-discourse/meta/server.yml
#	roles/web-app-mailu/tasks/01_core.yml
#	roles/web-app-odoo/tasks/main.yml
2026-07-04 18:16:12 +02:00
kevinveenbirkenbach 3b11c0b259 Merge pull request #358 from kevinveenbirkenbach/feature/web-app-checkmk
feat(web-app-checkmk): add Checkmk role with oauth2-proxy SSO + LDAP
2026-07-04 16:38:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fc55e325ae docs(agents/role-loop): add hard pre-redeploy verification gate
Before every redeploy that follows a failed deploy, require one explicit
line: either verified-in-container or a named post-deploy-state-exception.
Forbid treating the redeploy itself as the verification, and reject
state-reset friction, logical inference from a prior deploy, and
"looks correct" as reasons to skip. Playwright-spec failures must reset
any polluted state and be green via make compose-playwright first.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 11:35:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4bb432db70 feat(web-app-jellyfin): add Jellyfin role with LDAP + OIDC SSO plugins
Implements requirement 030: web-app-jellyfin deploying jellyfin/jellyfin
10.11.11 behind sys-svc-proxy. No external DB (internal SQLite); /config,
/cache, /media named volumes; CPU transcoding. Jellyfin has no native SSO, so
auth is plugin-based (operator chose BOTH): jellyfin-plugin-ldapauth (central
OpenLDAP, all clients incl. native apps) + 9p4/jellyfin-plugin-sso (Keycloak
OIDC, web one-click). files/configure-auth.sh completes the first-run /Startup
wizard (seeds break-glass admin), installs both plugins via Jellyfin's own
/Packages installer, writes the plugin configs, and restarts.

Schemas source-verified against the pinned upstream: Jellyfin v10.11.11, LDAP
plugin v23 (GUID 958aad66-..., flat XmlSerializer config), SSO plugin v4.0.0.3
(OidConfig fields). Honest caveats: the SSO OidConfigs is a SerializableDictionary
(XML shape best-effort), plus plugin ABI compat + the SPA Playwright selectors
are live-deploy-validation items (env-blocked). Each config file is XML-well-
formedness checked before reload.

Validation: make autoformat clean; make test green (lint, test-lint, test-unit,
test-integration 190, test-external).

Refs: docs/requirements/030-web-app-jellyfin.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-04 11:35:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 2f9a54cea7 fix(deploy): honor variants in include resolution
Inventory include sets ignored the active variant: swarm
derive_includes walked the variant-free base config (dynamic service
flags count as enabled) and the compose CombinedResolver followed
run_after as an include edge. Providers a variant pins to false
(e.g. mobilizon variant 1 sso) were still deployed.

- derive_includes: swap every app listed in INFINITO_APP_VARIANTS to
  its variant config, index 0 included (nextcloud variant 0 disables
  11 partner services)
- CombinedResolver: follow_run_after flag; include resolution passes
  False so run_after stays a pure ordering hint
- drop dead deps.py chain (resolve_deploy_ids_for_*) and unused
  resolve_with_edges; rewiring them would resurrect variant-blind
  run_after-following includes
- inventory.md: document the real include pipeline and
  is_explicit_truth semantics

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 03:21:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d4ee9a85ba refactor(compose): align templates to the service_name skeleton
Every compose.yml.j2 now declares services as '{{ service_name }}:' with
its '{% set %}' knobs on two-space lines above the key, includes at four
spaces and if/for tags aligned to the content they wrap. The layout
renders correctly because stack_host_template enables lstrip_blocks for
compose.yml.j2; indent(4, True) in the container base and '{%+' guards
in whitespace-sensitive scalars keep the shared fragments identical
under both render modes. A new lint test enforces the skeleton.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 02:36:58 +02:00
kevinveenbirkenbach 5ca10de67d Merge branch 'main' into feature/web-app-checkmk 2026-07-03 01:03:37 +02:00
kevinveenbirkenbachandClaude Opus 4.8 674d26de3a feat(lint/role-readme): derive Credits author from meta author SPOT
Role README Credits now name galaxy_info.author (plain bold or wrapped
in a Markdown link), matched byte-for-byte against the role's own
meta/main.yml, instead of a hard-coded maintainer name. Regenerate all
245 role README Credits blocks to the new 'Implemented by ...' format.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 16:47:34 +02:00
kevinveenbirkenbach 506da992d7 Merge branch 'main' into feature/web-app-semaphore 2026-07-02 13:10:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ccca758051 test(lint): ban templated conditionals in when/failed_when/changed_when/until
Catch the ansible-core 'conditionals surrounded by templating delimiters' deprecation at lint time. Excludes meta/{volumes,networks,services}.yml filter-when fields and templates; suppress with # nocheck: templated-conditional.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 14:28:42 +02:00
kevinveenbirkenbachandClaude Opus 4.8 001d880a72 fix(test/swarm): pin roundtrip swarm to one variant per app
A multi-variant app expanded into N swarm matrix entries, and
swarm-zombie ran them all on one host under identical SWARM_NAME
cluster names, so every deploy after the first aborted with
'network ... already exists'. Pin the act matrix to variant 0
(default, overridable via variant=) so a multi-variant app validates
a single cluster, in parity with the compose step's variant=0.
workflow.sh now splits ACT_MATRIX on ';' into multiple --matrix
filters so apps and variant can be pinned together.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 14:28:42 +02:00
kevinveenbirkenbach 9b2f870777 Merge branch 'main' into feature/swarm-nfs 2026-07-01 05:37:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 87c78275b8 feat(meta/complexity): multi-key sort, filter DSL, CI columns
Extend the 'meta roles applications complexity' report:

- --sort takes comma-separated multi-key clauses with per-key direction (e.g. 'desc embeds, asc weight'); replaces --order.
- --variant lists one row per meta/variants.yml variant; --format adds yaml.
- New columns: weight (was total), random, id, row, variants, bundles, jobs, lifecycle, covered_by, compose, swarm.
- covered_by marks the first green (1-based id) different-role predecessor that embeds the row; variant>0 never covered.
- bundles is the compose bundle/job count (whole-role) or 1 (per variant); jobs is the running sum.
- compose/swarm reflect the CI test-deploy skip logic (invokable + tested lifecycle + per-mode skip + swarm deployable variant).
- --filter is a boolean expression DSL (%% == \!= < > <= >=, and/or/xor/not, parens, set literals) over row fields.

Add shared SPOTs: variant_bundles.compose_bundle_counts and variant_status.deployable_variant_indices, reused by the swarm/compose matrix. Update roundtrip.sh/md to --sort "desc weight".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 20:33:01 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c1fb96584b refactor(tests): rename swarm make targets to swarm-* and reorganize env steps
Rename the scripts/tests/environment deploy_* steps (07/08/09) to compose_*; add 12_swarm.sh (svc-db-mariadb via the swarm-* targets) and 13_roundtrip.sh (svc-db-postgres via make roundtrip), with MARIADB_APP/POSTGRES_APP in common.sh. Rename every make act-swarm-* target to make swarm-* (drop the act- prefix) across the Makefile (moved to the alphabetical slot), the deploy/act, deploy/swarm and e2e helper scripts, and the agent iteration docs. Strip mid-code narration comments from the touched scripts per the comment policy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8390cc4ce4 test(comments): lint .j2 templates and add file-level nocheck opt-out
Drop trap/warning from the exception markers; lint .j2 by stripping the suffix to the underlying language plus Jinja {# #} comments; add a head-level '# nocheck: comments-valid' opt-out via is_suppressed_in_head, documented in suppression.md; strip narration comments from friendica/nextcloud templates and opt the vendored discourse app.yml out with the new marker.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 12:24:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c28ea68326 fix(web-app-semaphore): pin published image tag v2.18.12
v2.18.13 is not published on Docker Hub, so 'compose pull' failed with 'manifest unknown' and aborted the deploy. Pin v2.18.12 (latest stable 2.x, verified pullable in the nested daemon) and align the README bump-policy note and the 028 requirement spec.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 01:38:41 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b6e79878fd fix(ci): make 🧩 Subset resolver comply with repo lint rules
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.

- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 00:59:49 +02:00
kevinveenbirkenbach 19b344d1dc Merge branch 'main' into feature/swarm-nfs 2026-06-30 00:59:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0d1e718abf feat(ci): add 🧩 Subset label for explicit role selection on PRs
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).

- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 00:59:48 +02:00
Alejandro Roman d832e88cec Merge remote-tracking branch 'upstream/main' into feature/self-host-runners 2026-06-30 00:14:59 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0bcd63b658 fix(ci): make 🧩 Subset resolver comply with repo lint rules
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.

- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 00:08:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6ad17c07de feat(ci): add 🧩 Subset label for explicit role selection on PRs
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).

- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 23:10:50 +02:00
Alejandro Roman 1c80c2413d Merge remote-tracking branch 'upstream/main' into feature/self-host-runners 2026-06-29 11:36:40 +02:00
Alejandro Roman a6b9081dcb fix(ci): jinja-safe secrets, runner count/distro + clean state, update docs 2026-06-29 11:20:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c59cb3c959 docs(iteration): require 95% in-container confidence before redeploy
Add the pre-redeploy confidence gate to the compose and swarm loops and
allow bringing up the other deploy mode in parallel for comparison.
Sync the roundtrip cheatsheet prompt to the same rule.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 11:11:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 059232bd0d feat(web-app-checkmk): add Checkmk role with oauth2-proxy SSO + LDAP
Implements requirement 029: web-app-checkmk deploying checkmk/check-mk-raw
2.4.0p32 (Raw/Community) behind sys-svc-proxy. No native OIDC, so SSO uses the
platform oauth2-proxy gate + Checkmk header-auth (auth_by_http_header =
X-Remote-User, injected by the role proxy.conf.j2 from $authrequest_username =
Keycloak preferred_username), plus a native LDAP connection for user records.
No external DB. Agent receiver (8000) host-port mapped via a new
PORT_BANDS.local.agent band.

Auth/LDAP/proxy mechanisms source-verified against Checkmk v2.4.0p32
(cmk/gui/auth.py, userdb/ldap_connector.py, userdb/_connections.py) and the
infinito sso_proxy nginx chain: fixed the LDAP file path (user_connections.mk)
and directory_type ("openldap"). LDAP sync timing and classic-GUI Playwright
selectors remain live-deploy-validation items (env-blocked).

Validation: make autoformat clean; make test green (lint, test-lint, test-unit,
test-integration 190, test-external).

Refs: docs/requirements/029-web-app-checkmk.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 02:35:08 +02:00
kevinveenbirkenbachandClaude Opus 4.8 59f22e3bda feat(meta): static web-svc scrape/track opt-out
Static-file (autoindex) and 301-redirect/alias web-svc vhosts route no request through lua-resty-prometheus and carry no user-facing HTML, so they have no per-vhost app=<role> scrape target and no _paq tracker page. scrape:false (asset, cdn, css, html, file, collabora) drops them from PROMETHEUS_TARGET_ROLES_JSON; track:false (asset, cdn, file) drops them from MATOMO_TARGET_ROLES_JSON. suppression.md catalogs both markers. Real apps stay scrape/track targets so monitoring is default-on.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 02:16:49 +02:00
kevinveenbirkenbach 3a7f7e6f00 Merge branch 'main' into feature/swarm-nfs
# Conflicts:
#	.github/workflows/images-build-ci.yml
#	.github/workflows/release-version.yml
#	roles/web-app-matomo/meta/services.yml
#	roles/web-app-pihole/meta/server.yml
2026-06-28 22:53:15 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6ce6780575 refactor(meta): split server.yml into csp/domains/networks.yml
Extract the csp, domains, and networks blocks out of roles/*/meta/server.yml
into dedicated meta files, each mapping to its own top-level applications key
via the recursive meta-walk:

  csp:      -> meta/csp.yml      -> applications.<app>.csp
  domains:  -> meta/domains.yml  -> applications.<app>.domains
  networks: -> meta/networks.yml -> applications.<app>.networks

Residual nginx/proxy keys (status_codes, client_max_body_size, locations,
ip6_enabled, config_upstream_url) stay in server.yml; an emptied server.yml is
deleted. 98 server.yml -> 249 new files, 74 deleted.

- Consumers repointed applications.<app>.server.{csp,domains,networks} ->
  applications.<app>.{csp,domains,networks} across templates, vars, tasks,
  lookups and filters.
- Infra readers migrated: canonical_domains_map, application_domain_index,
  build_ldap_role_entries, service-registry overlay discovery,
  roles_with_service, and the file-based readers utils/domains/list.py +
  utils/meta/scan.py (read the now-deleted server.yml directly).
- Detection + mapping: _META_TOPICS gains csp/domains/networks;
  ROLE_FILE_META_{CSP,DOMAINS,NETWORKS} constants + ROLE_FILES entries.
- New self-expiring lint guard (test_server_meta_no_split_fields) fails if a
  server.yml still carries a split field; auto-fails from 2026-12-28 as a
  delete reminder.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:05 +02:00
kevinveenbirkenbachandClaude Opus 4.8 defa2d6db3 feat(web-app-semaphore): add Semaphore UI role with OIDC SSO + LDAP
Implements requirement 028: web-app-semaphore deploying semaphoreui/semaphore
v2.18.13 behind sys-svc-proxy with central PostgreSQL, native Keycloak OIDC
SSO, and LDAP login, plus guest/biber/administrator Playwright personas.

Auth/admin model verified against the upstream v2.18.13 source (config.go,
api/login.go, api/router.go, web/src/views/Auth.vue, config.schema.yaml,
cli/cmd/user_add.go): the OIDC callback matches by email and rejects local
users, the login form is LDAP-only when ldap_enable=true, and OIDC users are
never admin from a claim. The role therefore seeds a distinct break-glass
local admin and creates the Keycloak administrator as an external admin at
deploy time (semaphore user add --admin --external). Correct v2.x env names
(SEMAPHORE_LDAP_ENABLE/_SERVER/_BIND_DN/_BIND_PASSWORD/_SEARCH_DN, OIDC
provider JSON) confirmed against the source.

Validation: make autoformat clean; make test green (lint, test-lint,
test-unit, test-integration 190, test-external). Live full-matrix deploy +
Playwright still pending (host container runtime fault, environment-only).

Refs: docs/requirements/028-web-app-semaphore.md

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:40:26 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b9d86e176f test(lint): fail on invalid git branch names
Add tests/lint/repository/naming/test_branch_name.py validating the checked-out branch against the naming policy in branch.md. Rejects unknown prefixes (e.g. feat/ instead of feature/), non-kebab-case descriptions, and fix/ branches missing a ticket id; exempts main, master, and detached HEAD. Document the local enforcement in branch.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 21:13:46 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9539feb496 docs(requirements): add 028 web-app-semaphore role spec
Requirements for a web-app-semaphore role (Semaphore UI): native OIDC SSO via Keycloak, LDAP login, central PostgreSQL, single all-in-one container with embedded runner, always-on local-admin bootstrap plus OIDC claim->admin mapping, and Playwright persona coverage. Includes upstream research with cited sources and acceptance criteria.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 21:13:46 +02:00
kevinveenbirkenbachandClaude Opus 4.8 20ab8ee76b ci(fork): key trusted fork PRs off the '🛡️ Trusted' label
Switch the label that flips a fork PR to the trusted (PR-head, with
credentials) path from 'trusted-pr' to '🛡️ Trusted', in the
pull_request_target gating and the orchestrator text, plus the docs.
The trusted_pr input names are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:58:54 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6031633c43 fix(ci): unblock fork-PR image producer under pull_request_target
actions/checkout now refuses to fetch PR head/merge code under
pull_request_target without an explicit opt-in, which broke the fork
image build and the fork mirror job for every fork PR.

Set allow-unsafe-pr-checkout on the fork source checkouts (new
allow_unsafe_checkout input on images-build-ci.yml and
images-mirror-missing.yml, passed true by the fork producer jobs) so
the producer builds again — untrusted forks from the safe base merge
ref, trusted forks (trusted-pr label) from the PR head.

Harden the untrusted path: the fork build no longer inherits
organization secrets (it authenticates to GHCR with the per-job
github.token only, and the build needs no other secret), and Docker Hub
mirror credentials are passed only for trusted PRs (untrusted forks
mirror anonymously). Switch the GHCR logins to github.token so auth no
longer depends on secrets inheritance.

Docs updated in pipeline.md and authentication.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 19:58:53 +02:00
kevinveenbirkenbachandClaude Opus 4.8 238bc285c7 refactor(lookups): reach merged config via the lookup-loader SPOT
Consumer lookup plugins now reach the merged config through the Ansible
lookup loader instead of importing get_merged_applications /
get_merged_domains from utils.cache. The merge stays one cached
implementation surfaced via the applications/config/domains/users lookups;
consumers call lookup_loader.get(...).run() (the compose_networks idiom).

- get_merged_applications walks the whole meta/ tree recursively
  (meta/**/*.yml to applications.<app>.<path>, addons and any depth) and
  folds the raw meta/volumes.yml subtree onto applications.<app>.volumes
  after the templar render so the Jinja source: strings stay unrendered.
- 31 consumer lookups rerouted; providers (applications/config/
  applications_current_play/users/domains) and the raw-volume accessors
  (volume/nginx) keep the cache import behind # nocheck: lookup-cache-import.
- New lint test_no_cache_imports_in_lookups enforces the contract.
- nfs_flavor is now the flavor SPOT (svc-storage-nfs-server + nfs_client_src
  resolve via lookup('nfs_flavor')); removed the unused STORAGE_NFS_FLAVOR
  group_var. Extracted extract_default_credentials to
  utils/cache/schema_credentials.py to keep applications.py under the cap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 18:35:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cf3531b9e3 refactor(svc-storage-nfs-server): drive nfs flavor from meta/variants.yml
The inventory applications override in default.yml is dropped during inventory provisioning (CI ran kernel despite RUNTIME=github + a ganesha override), so select the flavor in meta/variants.yml where it reliably applies: ganesha for RUNTIME in [dev, act], kernel for github CI and production. meta/services.yml keeps kernel as the base default.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 07:41:12 +02:00
kevinveenbirkenbachandClaude Opus 4.8 539a078475 fix(svc-storage-nfs-server): select ganesha in DinD CI, kernel default
The github swarm legs picked the kernel NFS flavor inside the privileged DinD nfs-server, where kernel nfsd is not network-namespaced and wedges exportfs. Move flavor selection out of the role meta default and into the dev inventory: meta/services.yml now defaults to kernel (production); the development inventory overrides it to ganesha for the containerized runtimes (dev/act/github) and keeps kernel for real hosts (RUNTIME=host).

Add meta/variants.yml (kernel + ganesha) and a TODO to test both flavors via files/test.sh, plus server-side kernel-nfsd diagnostics in 12_collect_diagnostics.sh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 06:22:26 +02:00