Commit Graph
214 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.7 0b5672d7d5 ci(update): mint GitHub App token so update PRs trigger full CI
Workflow-driven update PRs opened with GITHUB_TOKEN do not start
further workflow runs, so the CI orchestrator never executed against
them. Each PR-opening job in update.yml now mints a short-lived
GitHub App installation token via actions/create-github-app-token@v2
and forwards APP_SLUG to open_update_pr.sh, which derives the
verified-commit identity from the App user.

The job-level permissions block drops to read-only since write access
is provided exclusively by the App token. Behaviour with the secrets
unset is fail-loud: the token step fails, the PR step is skipped,
and no PR with an unintended identity is ever created.

Requires the BOT_APP_ID and BOT_APP_PRIVATE_KEY repository secrets
documented in docs/contributing/tools/github/actions/secrets.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 00:50:25 +02:00
kevinveenbirkenbach a38ab671ec Merge branch 'main' into feature/web-app-kix 2026-05-19 17:00:02 +02:00
kevinveenbirkenbachandClaude Opus 4.7 dd1481a9b3 feat(bundles): add deploy-bundles / redeploy-bundles flow
- new utils.inventory.bundle_apps resolver aggregates and dedupes role
  groups from inventories/bundles/{servers,workstations}/<name>/inventory.yml
- new make targets deploy-bundles (fresh-purged) and redeploy-bundles
  (reuse-kept), driven by BUNDLES=... env, wrapped by
  scripts/tests/deploy/local/deploy/{bundles,redeploy-bundles}.sh
- unit tests for the resolver under tests/unit/utils/inventory/
- document the new targets in docs/contributing/tools/make.md,
  docs/contributing/actions/testing.md, inventories/bundles/README.md
- collapse multi-line Makefile comments to one-liners for consistency
- disable nextcloud bbb plugin auto-enable (commented out)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-19 16:55:00 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ab99f5dbc0 refactor(meta/services): rename ports.inter to ports.internal.<category>
The internal container port slot under <entity>.ports is now a
category-keyed map (matching the local/public shape) instead of a
single int. "internal" replaces "inter" because the latter is
ambiguous in English (could read as "between").

Affected:
- 8 role meta/services.yml (matrix, ollama, mig, shopware, oauth2-proxy,
  gitea, listmonk, sphinx) and every caller (vars, templates, group_vars).
- web-app-matrix: 8008 hardcoded across compose.yml.j2, homeserver.yaml.j2,
  tasks/01_docker.yml, and 7 mautrix bridges now reads
  MATRIX_SYNAPSE_CONTAINER_PORT from services.synapse.ports.internal.http.
- web-app-matrix: admin username/password lookups consolidated as
  MATRIX_ADMINISTRATOR_{USERNAME,PASSWORD} in vars/main.yml, sourced from
  lookup('users', 'administrator').
- Tooling: cli/meta/ports/suggest, utils/meta/scan, integration scan,
  migrate_meta_layout/role_builder updated to the new key name.
- Docs: design/role/services/layout.md, tools/ports-suggest.md,
  requirements 009/021, image/origin.md, PR template.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 11:35:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1fa2a50c1b docs(agents/cheatsheet): tighten Web Development Iteration prompt; drop trailing Use mode /caveman ultra from every prompt
Web Development Iteration prompt now spells out the in-container loop
explicitly: for every failing Playwright test, follow the inner-loop
procedure in role.md together with playwright.md; before every
redeploy run `make exec` and `make run` against the live stack and
fully fix and inspect every failure in-container; the next deploy
iteration only starts once every error is resolved and the fix is
empirically verified. Adds explicit prohibitions on commit/push and
on `ask`-mode approvals, and forbids premature termination — the
iteration is finished only when every role is green end-to-end.

Strips the `Use mode /caveman ultra` postscript from every other
prompt: it was an experimental mode marker that no longer applies
and only added noise. The prompts now stop at the standard
`act autonomously …` sentence.

Reformatted to comply with documentation.md: complete sentences
instead of semicolon chains, no em-dash thought breaks, RFC 2119
keywords used consistently.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 19:29:36 +02:00
kevinveenbirkenbachandClaude Opus 4.7 16bdbf03d7 docs: fix repo-wide drift against documentation.md style rules
Three classes of drift cleaned up across 85 .md files:

- Em-dashes: replaced ` — ` with the contextually correct separator
  (colon for `term: explanation`, period to split clauses across
  sentences/source lines, parentheses for asides, semicolon for
  tight-linked continuations). Two intentional exceptions remain
  in `docs/contributing/documentation.md` (the rule literal) and
  inside a fenced ```sh block in `docs/contributing/actions/debugging/ci.md`.
- SPOT self-labels: rewrote "This is the SPOT for X" to describe
  the subject directly ("This directory holds X" / "This file
  describes X"). Cross-references to other artefacts' canonical
  role remain unchanged.
- Heading-emoji position: moved leading-emoji headings (`# 🧩 Title`)
  to trailing position (`# Title 🧩`) per the headline rule that
  emojis MUST come after the title text.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:12:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 98457f98fd refactor(update): unify version bumpers under utils/update + cli/contributing/update + scripts/github/update
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.

utils/
  update/
    base.py        # semver primitives shared by both backends
    docker.py      # OCI registry lookups + version: line rewriter
    repository.py  # git ls-remote walker + ref: line rewriter

cli/contributing/update/
  docker/           # python -m cli.contributing.update.docker
  repository/       # python -m cli.contributing.update.repository

scripts/github/update/
  image_versions.sh
  repository_refs.sh

Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.

Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:12:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 14700e0658 feat(test-e2e-playwright): add PLAYWRIGHT_KEEP_ALL artefact-retention toggle
Forwards `PLAYWRIGHT_KEEP_ALL` from host env / GitHub Actions repo var
through `TEST_E2E_PLAYWRIGHT_KEEP_ALL` (inventory-overridable) into the
container and flips trace / screenshot / video to `on` for every test
when truthy.

SPOT for the propagation chain lives in
`docs/contributing/actions/testing/playwright.md#artefact-retention-`;
CI Configuration and the agent iteration loops link to it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:06:31 +02:00
kevinveenbirkenbachandClaude Opus 4.7 256e027525 docs: drop stale requirement refs + reflow soft-wraps to one sentence per line
Removes 67 live references to docs/requirements/<NNN>-*.md files across
43 contributor docs, role READMEs, code comments, task files, and tests.
Requirement docs are initial-creation-only artefacts; live references
freeze a snapshot that rots once the per-feature SPOT (a contributor
doc, a code module, a lint test) takes over the live behaviour.

In the same pass, fixes documentation.md violations on the touched
files: drops em-dashes used as thought breaks, removes "SPOT for ..."
self-labels, strips historical phrasing ("the legacy", "previously",
"used to", "is no longer used", "matching the historical"), and
reflows soft-wrapped sentences so each sentence stays on one logical
line.

The `test_inventory_vars_file_spot.py` whitelist for the literal
inventory path drops the docs/requirements/003 entry and instead
skips docs/requirements/ entirely; future requirement docs that
mention the literal path are then ignored automatically.

Verified by tests/lint/repository/test_no_stale_requirement_refs.py
(green tree-wide).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 05:00:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 951f6d36d2 chore(lint): unify autoformat/galaxy parallelism into one PARALLEL env var
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
  declare it at file start with default `true`; still env-overridable.
  Reword the worker comment so the literal `# shellcheck` token no
  longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
  GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
  is always fatal. NAMESPACE is an internal constant. Add the same
  PARALLEL env var (default `true`, env-overridable); falsy demotes the
  xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
  AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
  consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
  add the lint-galaxy parallelism note.

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:59:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 79409ebf10 test(tooling): integrate markdownlint-cli2, ansible-lint, galaxy-importer + fix the resulting baseline
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
  configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
  ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
  ensure_* functions (pip / npm with sudo / --user / --break-system-packages
  fallbacks). install_ansible_tools now also pulls ansible-lint and
  galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
  syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
  galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
  role, and treats `bare-name dependency` errors as success since our
  roles/ tree is not a Galaxy collection layout. All other Galaxy schema
  errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
  `make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
  now installs ansible-core + ansible-lint; ci-orchestrator wires both
  new lint workflows into lint-gate.

Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
  two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
  stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
  meta/main.yml plus _CANONICAL_COMPANY in
  tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.

Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
  across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
  TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
  list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
  renamed to "## User data resolution" (D3 self-label rule).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 23:46:37 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a26bfc4e94 refactor(cli): reorg admin/contributing tree + per-folder help system
Module moves (with imports, paths, argv literals, scripts, docs, and
parents[N] formulas updated in lockstep):

- cli/deploy             -> cli/administration/deploy
- cli/validate/inventory -> cli/administration/inventory/validate
- cli/vault              -> cli/administration/secrets/vault
- cli/administration/inventory/encrypt -> cli/administration/secrets/seal
- cli/mirror             -> cli/contributing/mirror

The new cli/administration/secrets cluster collocates the whole-file
ansible-vault wrapper next to the inventory-aware per-field sealer.
The user-facing CLI subcommand paths follow the package layout, e.g.
"infinito administration deploy dedicated <inv>" and
"infinito administration inventory validate <inv>".

Help system redesign in cli/core/{discovery,help,app}.py:

- Discovery now distinguishes runnable commands (directories with
  __main__.py) from category folders (directories that only group
  further commands). Empty support packages are filtered out.
- Each category folder MAY carry a README.md; its H1 heading is used
  as the title and its first paragraph as the one-line description in
  the help output.
- "infinito" / "infinito --help" prints only the top-level entries
  (one level deep). "infinito <category>" navigates into a folder and
  shows its sub-folders and commands one level deep, with the README
  title as the page heading and the invocation as a dim subtitle.
- New "--tree [N]" flag walks the tree under the (optional) path
  prefix recursively. N caps the depth (1-based); omit N for an
  unbounded walk. The output uses Unicode tree connectors, an emoji
  per entry (folder for categories, gear for commands), a globally
  aligned description column, and multi-line wrapped descriptions
  with tree-continuity bars on continuation lines.
- Category and command listings use the full invocation
  ("infinito meta callorder") rather than the bare leaf name, so the
  output is copy-pasteable.

Convention doc + 17 CLI READMEs:

- New docs/contributing/artefact/files/cli/readme_md.md describes the
  H1+paragraph layout the help system reads, and how it differs from
  role/docs READMEs.
- README.md added for cli/{administration, build, contributing, meta}
  and their immediate sub-folders, plus cli/meta/roles/applications
  and its resolution sub-folder. Each follows the "title with trailing
  emoji + single descriptive paragraph" convention.

Mirror resolver/sync description strings (the help system reads them
from argparse --help output) added so they no longer surface as
"options:" in the global index. cli/mirror/cleanup already had a
description and is unchanged.

Fixes uncovered by the move:

- import unittest.mock added to four dedicated-deploy tests so
  unittest.mock.patch resolves under stdlib unittest discovery
  (Python 3.13 no longer surfaces unittest.mock through "import
  unittest" alone).
- parents[N] index in the new
  cli/administration/inventory/validate/__init__.py corrected from
  3 to 4 to match the deeper layout.
- test_paths.INVENTORY_VALIDATOR_PATH assertion and test_runner
  substring assertions updated to match the new validator path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 20:17:09 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7d7301614d docs(changelog): per-release archive CLI + release docs + SPOT cleanup
Add cli/contributing/changelog/archive: a CLI that trims CHANGELOG.md
to the last N entries (default 7) and writes every older release to
its own file under docs/changelog/ named <padded-semver>-<date>.md
(7.0.0 → 007.000.000-2026-05-08.md). The padded form makes a directory
listing already chronological, and the index in CHANGELOG.md is
rebuilt from the archive directory listing on every run, so it never
drifts away from what is on disk. The tool is idempotent and never
overwrites an existing archive file.

Run the migration once to split the 44-entry CHANGELOG.md into 7
inline entries plus 37 archived per-release files, with the index at
the bottom of CHANGELOG.md linking each archive in descending order.

Add docs/contributing/actions/release.md as the contributor-facing
release procedure (versioning, active window vs. archive, cutting a
release end-to-end, CI hand-off on tag push) and add a docs README
under docs/changelog/ that explains the archive directory layout and
filename schema. Cross-link release.md from pipeline.md and add
"releases" to the actions/ scope description.

Documentation cleanup pass on every page touched in the previous
mapping refactor:

- Remove "is the SPOT for X" / "Single Point of Truth" self-labels
  across docs/contributing/, docs/agents/, and docs/changelog/. The
  rule in documentation.md forbids self-declared SPOT status; status
  is conferred by readers cross-linking, not by the page asserting
  it. Rephrase to descriptive phrasing ("This page covers X",
  "describes X", "lists X") throughout.
- Drop the orphaned docs/contributing/tools/actions.md duplicate
  (101 lines paralleling tools/github/actions/workflows.md, with no
  inbound links).
- Fix em-dashes used as thought breaks in dockerfile.md, origin.md,
  agents/style.css.md, and suppression.md per documentation.md.
- Unwrap soft-wrapped sentences in iam/common.md, iam/ldap.md,
  iam/oidc.md, image/origin.md, requirements.md, and dockerfile.md
  so each logical line carries one full sentence.

Add a unit test for the archive CLI under
tests/unit/cli/contributing/changelog/archive/test_main.py covering
_pad_version, _split_into_entries, idempotency, dry-run, no-overwrite
of existing archives, and the multi-run index rebuild.

Quote `# noqa:` in CHANGELOG.md prose with backticks so the
noqa-only-ruff-codes lint does not parse "for" as a non-ruff code.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 16:19:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 286408fa91 refactor(roles): role-mapping SPOT under utils/roles + cli inspector + generalist lint
Move utils/roles/applications/files.py to utils/roles/mapping.py and add a
data-driven role-type detector in utils/roles/type.py. Each entry in
ROLE_FILES carries per-role-type allowed/mandatory flags plus marker entries
that drive get_role_types(); a role MAY belong to several types (e.g. an
application that also ships a system service).

Add cli/meta/roles/mapping with files / types / markers subcommands so
contributors can query the schema in text or JSON. Move
cli/meta/applications/ to cli/meta/roles/applications/ and adjust PROJECT_ROOT
parents indices accordingly. Move docs/contributing/design/services/ to
docs/contributing/design/role/services/ and document the mapping in
docs/contributing/design/role/mapping.md per documentation.md.

Replace the narrow test_app_only_files with a generalist
tests/integration/roles/meta/test_mapping.py that walks every role and every
ROLE_FILES entry and reports forbidden files, missing mandatory files,
forbidden entries, and missing mandatory entries in one structured fail
report. meta/info.yml and meta/users.yml are now allowed across every role
type. Create the seven mandatory files surfaced by the generalist test
(README.md for desk-keepassxc, desk-obs, web-svc-cdn following readme_md.md,
plus tasks/main.yml stubs for svc-db-memcached, svc-db-redis, sys-svc-proxy,
web-app-socialhome).

Audit and rewrite touched markdown for documentation.md compliance: replace
em-dashes with fluent sentences, drop SPOT self-labels, fix all broken
relative links introduced by the services/ move.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 13:21:22 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9524c50928 feat(ci): diff-driven app whitelist for deploy tests
Narrow the test-deploy-{server,universal,workstation} matrix to roles
actually impacted by the branch's diff against origin/main, expanded
transitively over run_after + dependencies + shared services.

Why: full-matrix deploy CI on every PR is wasteful; explicit per-PR
whitelisting is error-prone. The resolver derives the affected set
automatically while remaining fail-safe.

Behaviour (in resolve_effective_whitelist.sh precedence order):
- whitelist input "__ALL__"  -> empty whitelist (force full deploy)
- whitelist input non-empty  -> verbatim
- whitelist input empty      -> diff vs origin/main
  - empty diff or any path outside roles/<role>/   -> __ALL__
  - all paths under roles/<role>/                  -> reverse closure

Fail-safe: any seed that is not modellable in CombinedResolver
(no application_id and not referenced by any role's run_after) and
any resolver runtime error MUST trigger a fall-back to __ALL__.
The closure never silently shrinks the deploy matrix.

test-deploy-local.yml is intentionally exempt (local/manual dispatch
keeps explicit whitelist semantics).

Documented in pipeline.md and workflows.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-05 22:49:41 +02:00
kevinveenbirkenbach 07247b3552 Merge branch 'main' into feature/role-meta-refactor
# Conflicts:
#	roles/web-app-bridgy-fed/config/main.yml
#	roles/web-app-decidim/config/main.yml
#	roles/web-app-joomla/config/main.yml
#	roles/web-app-magento/config/main.yml
#	roles/web-app-shopware/config/main.yml
#	roles/web-app-suitecrm/config/main.yml
2026-04-27 10:16:52 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ed04dad5d5 refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).

req-008 — role meta layout
  * Move every role's metadata under meta/<topic>.yml with file-root
    convention (no wrapping key matching the basename):
      - schema/main.yml + config/main.yml.credentials → meta/schema.yml
      - users/main.yml                                → meta/users.yml
      - config/main.yml.server                        → meta/server.yml
      - config/main.yml.rbac                          → meta/rbac.yml
      - config/main.yml.compose.services + every other top-level key
        of config/main.yml inlined under <entity>.<key>
                                                      → meta/services.yml
      - config/main.yml.compose.volumes               → meta/volumes.yml
  * Drop the legacy schema/, users/, config/ directories from every role.
  * meta/schema.yml: support nested credential keys, optional `default:`
    field (Jinja literal, not rendered/validated/regenerated), implicit
    `algorithm: plain`.
  * Materialised paths drop the `compose.` prefix:
      compose.services.<e>.<…> → services.<e>.<…>
      compose.volumes.<k>      → volumes.<k>
      <former-top-level>.<…>   → services.<entity_name>.<former-top-level>.<…>
  * Update every Python consumer (utils/cache, utils/manager, utils/applications,
    plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
    and task file to the new paths.
  * No legacy fallback. Repository-wide grep for `compose.services.` /
    `compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
    is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
  * Add tests/lint/repository/test_role_meta_layout.py guarding against the
    old paths returning, and an explicit lint guard that meta/variants.yml
    files do not carry a `compose:` wrapper.

req-009 — per-role networks/ports
  * Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
    dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
    dns_resolver}.
  * Delete group_vars/all/09_ports.yml entirely; ports now live on the
    service entity that exposes them at
    roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
  * Rename + int-cast every former top-level `services.<entity>.port`
    field to `services.<entity>.ports.inter` (single int).
  * Always-category-keyed maps for ports.local / ports.public; relay
    spans use {start, end} integers.
  * Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
    of truth for the per-<scope>.<category> band ranges, consumed by both
    the new CLI helpers and the lint test.
  * New CLI helpers (cli meta ports suggest, cli meta networks suggest)
    walk the live role tree gap-first then increment-fallback, with
    deterministic output and non-zero exit on capacity overflow.
  * cli create role consumes both helpers when scaffolding a new role.
  * Lint extensions: bare-int port shape rejection, host-bound port
    collision check (single ints + relay span integers, inter excluded),
    band-membership check with documented allow-list for the legacy
    web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.

req-010 — run_after / lifecycle migration
  * Move galaxy_info.run_after and galaxy_info.lifecycle out of
    meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
    lifecycle}; <primary_entity> is the value returned by
    get_entity_name(role_name).
  * For multi-entity roles whose primary entity has no compose service
    (web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
    gen roles), create a dedicated metadata-only top-level entry under
    <primary_entity>.
  * Introduce utils/roles/meta_lookup.py with get_role_run_after /
    get_role_lifecycle that resolve <primary_entity> once and read from
    meta/services.yml; degrade gracefully when meta/services.yml is
    absent OR the field is absent; raise MetaServicesShapeError when the
    file exists but is malformed.
  * Update every consumer (utils/service_registry, utils/invokable,
    cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
    lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
    cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
    init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
    use the helper.
  * Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
    values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
    run_after/lifecycle never on a non-primary entity; empty run_after
    lists rejected.

Also folded into this change set
  * services_disabler now writes `services.<svc>.{enabled,shared}` into
    host_vars instead of the old `compose.services.<svc>.{...}` path; the
    discover/find pair walks meta/services.yml and only treats entries
    carrying enabled/shared/provides as provider declarations (so
    metadata-only entries from req-010 do not create spurious provider
    rows).
  * service_registry skips `run_after` deps whose target was filtered
    out via SERVICES_DISABLED instead of failing the whole load.
  * cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
    consume the new applications.<role>.services.<svc>.image shape
    (mirror.md updated to match).
  * utils/domains/list.build_applications_from_roles now reads
    meta/server.yml directly (it had been silently returning {} since
    config/main.yml stopped existing).
  * utils/roles/resource_validation.filter_roles_by_min_storage reads
    meta/services.yml.<entity>.min_storage instead of the legacy
    compose.services path.
  * web-app-bluesky pre-clones the social-app build context BEFORE
    sys-svc-compose so the build context is on disk by the time
    sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
  * 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
    web-app-wordpress, web-app-keycloak) lose their `compose: services:`
    wrapper so deep-merge over meta/services.yml lands on the new
    file-root shape.

Validation
  * Unit tests:        1571 passed.
  * Lint tests:        50 passed.
  * Integration tests: 161 passed (1 skipped).
  * Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
    set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
    discourse,mattermost}.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 09:58:26 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3e30694ee3 ci(updates): gate auto-update PRs via repo var
Add CI_ENABLE_AUTO_UPDATES (default unset/false). Workflow update
jobs and Dependabot PRs are skipped/auto-closed unless the variable
is set to true, so update noise is opt-in.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 16:16:58 +02:00
kevinveenbirkenbach 8563df9400 chore: adopt git-maintainer-tools as dev dependency for git ops
Drop the project-local copy of scripts/git/sign-push.sh in favour of
the git-maintainer-tools CLIs (git-setup-remotes, git-sign-push),
which now ship as a standalone PyPI/GitHub package so the same helpers
can be reused across every maintainer repo without vendoring shell
scripts.

The package is declared in pyproject.toml's `dev` extras and lands on
$PATH via `make install-python-dev` (which already runs
`pip install -e .[dev]`). Consumers invoke the CLIs directly; there
are no make wrappers because doing so would just duplicate help text
the tool itself already prints.

Fallout:
  * scripts/git/sign-push.sh deleted.
  * Makefile: sign-push target removed (no replacement target).
  * .claude/settings.json: deny rule for `make sign-push*` replaced
    with direct `git-setup-remotes*` / `git-sign-push*` denies so the
    sandbox blocks the real binaries, not just the make alias.
  * CLAUDE.md: operator instruction points at the bare `git-sign-push`
    CLI + `make install-python-dev` for setup.
  * docs/contributing/artefact/git/remotes.md: new tool-focused page.
    Describes the layout and the two CLIs; does not duplicate raw git
    commands already covered by git-maintainer-tools.
  * docs/contributing/artefact/git/README.md: trimmed per readme_md.md
    (docs READMEs must not act as a file index).
  * docs/contributing/tools/make.md: Git section now documents the
    external CLIs and the install entry point, not a wrapper target.
2026-04-25 09:38:26 +02:00
kevinveenbirkenbach e87bd9d37b chore: adopt git-maintainer-tools as dev dependency for git ops
Drop the project-local copy of scripts/git/sign-push.sh in favour of
the git-maintainer-tools CLIs (git-setup-remotes, git-sign-push),
which now ship as a standalone PyPI/GitHub package so the same helpers
can be reused across every maintainer repo without vendoring shell
scripts.

The package is declared in pyproject.toml's `dev` extras and lands on
$PATH via `make install-python-dev` (which already runs
`pip install -e .[dev]`). Consumers invoke the CLIs directly; there
are no make wrappers because doing so would just duplicate help text
the tool itself already prints.

Fallout:
  * scripts/git/sign-push.sh deleted.
  * Makefile: sign-push target removed (no replacement target).
  * .claude/settings.json: deny rule for `make sign-push*` replaced
    with direct `git-setup-remotes*` / `git-sign-push*` denies so the
    sandbox blocks the real binaries, not just the make alias.
  * CLAUDE.md: operator instruction points at the bare `git-sign-push`
    CLI + `make install-python-dev` for setup.
  * docs/contributing/artefact/git/remotes.md: new tool-focused page.
    Describes the layout and the two CLIs; does not duplicate raw git
    commands already covered by git-maintainer-tools.
  * docs/contributing/artefact/git/README.md: trimmed per readme_md.md
    (docs READMEs must not act as a file index).
  * docs/contributing/tools/make.md: Git section now documents the
    external CLIs and the install entry point, not a wrapper target.
2026-04-24 21:05:40 +02:00
kevinveenbirkenbach b2914b7cd1 Merge branch 'main' into fix/users-applications.yml 2026-04-23 18:14:17 +02:00
Alejandro Roman 9cd39f0c39 feat(prometheus-comm): fix external tests for matrix HTTP 541 issue 2026-04-23 00:31:20 +04:00
kevinveenbirkenbachandClaude Opus 4.7 925dc830a1 chore(claude-settings): collapse Bash allow list to Bash(*) wildcard
Replaces 15 per-shape Bash entries that existed as sandbox auto-allow
workarounds with a single Bash(*) wildcard. Gating responsibility moves
entirely to ask (mutating gh/docker/git verbs) and deny (structurally
destructive shapes). Documentation updated across settings.md,
security.md, and sandbox.md to reflect the new model; em-dash clause
separators removed throughout per documentation.md style rules.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-22 00:30:35 +02:00
kevinveenbirkenbachandClaude Opus 4.7 162b152a0d chore(claude-settings): generalize python allowlist to multi-env, python/python3
Replaces narrow `Bash(PYTHONPATH=* python *)` with `Bash(*=* python* *)`
so test invocations carrying multiple env-var assignments
(e.g. `PYTHON=python3 PYTHONPATH=. python3 -m unittest ...`) and
`python3` binaries no longer trip the auto-allow heuristic. Sandbox
remains the primary containment layer, so widening the allow shape
adds no new blast radius.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 19:03:29 +02:00
kevinveenbirkenbachandClaude Opus 4.7 8ce9179706 fix(lookups,web-app-peertube): finish users/applications migration and harden OIDC plugin install
- Resolve merge conflict in web-app-peertube/vars/main.yml; migrate
  remaining `applications |` and `users.*` references to
  `lookup('applications')` / `lookup('users', ...)` in web-app-odoo and
  web-app-decidim (compose, env, playwright, setup tasks).
- web-app-peertube: split OIDC tasks into oidc/{enable,disable}.yml,
  add install_mem_limit filter + PEERTUBE_MEM_LIMIT /
  PEERTUBE_OIDC_PLUGIN_INSTALL_MEM_LIMIT vars to raise the cgroup ceiling
  during the one-off `npm run plugin:install` exec (prevents OOM-SIGKILL
  that was surfacing as "Container is restarting"); add
  upsert-oidc-plugin.sql.j2 and diagnostics.sh.
- CI: test-deploy-{local,server,universal,workstation}.yml tweaks plus
  scripts/github/{dump_runner_state,enlarge_swap}.sh for constrained
  runners.
- cli/meta/applications/ressources: new CLI entry with unit tests.
- Misc config/docs updates across dashboard, mailu, mattermost, asset,
  cdn, file roles and contributing docs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 10:14:21 +02:00
kevinveenbirkenbachandClaude Opus 4.7 97ee7e65ea Bump peertube cgroup mem_limit during plugin install
Plugin install via `container exec` shares the cgroup with the long-running
peertube node process. Combined RSS was tripping the 8g mem_limit and
SIGKILLing peertube, which surfaced as "Container is restarting" on
subsequent exec attempts.

- Temp raise mem_limit + unlimited swap for the install window, restore
  the compose-defined value on the happy path only
- Derive the bumped ceiling from PEERTUBE_MEM_LIMIT + install heap cap
  via a new role-local `install_mem_limit` filter (overhead=4 covers
  native RSS and spawned npm/node-gyp workers)
- Extract the SQL upsert into templates/upsert-oidc-plugin.sql.j2
- Move enable/disable OIDC tasks under tasks/oidc/ and rename the
  diagnostics script
- Document the extract-code convention in docs/contributing/tools/ansible.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-20 22:55:08 +02:00
kevinveenbirkenbach b5c4b8a3d9 Merge branch 'main' into fix/users-applications.yml 2026-04-19 22:21:16 +02:00
kevinveenbirkenbach da140094a1 docs(agents): split iteration loops and refactor Playwright SPOT
Split docs/agents/action/iteration.md into role.md, playwright.md,
workflow.md, and a scope-focused README. Move Certificate Authority
and Inspect rules into role.md. Rewrite playwright.md as a machine-
oriented state machine with explicit preconditions, procedure, exit
gates, and escape path.

Extract Playwright spec content MUSTs into the new contrib SPOT
docs/contributing/artefact/files/role/playwright.specs.js.md. Reduce
the agent-side spec and env.j2 pages to procedure only. Align the
framework SPOT and the test-e2e-playwright role README with the
central templates/package.json.j2 + files/playwright.config.js model
and images.playwright.version as single source of truth.

Sweep em dashes and sentence-case role headings across all touched
docs per documentation.md and readme_md.md rules.
2026-04-19 22:04:24 +02:00
kevinveenbirkenbach 2d91eb3368 Merge branch 'main' into fix/users-applications.yml 2026-04-19 15:37:37 +02:00
kevinveenbirkenbachandClaude Opus 4.7 85c008a90b chore(ci): add weekly CodeQL cron to backstop idle periods
Restores the weekly scan coverage previously provided by Default Setup,
so CodeQL analysis still runs when no CI triggers land on main.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 13:05:23 +02:00
kevinveenbirkenbachandClaude Opus 4.7 50bc41b449 chore(ci): route CodeQL through orchestrator only, dedupe CI docs
Remove the direct push-on-main trigger from security-codeql.yml so that
the concurrency group no longer cancels the orchestrator-invoked scan.
CodeQL now runs exactly once per CI run via ci-orchestrator.yml, and
its failure continues to block code-quality-gate and all downstream
deploy, install, and environment tests.

Split the CI documentation along SPOT lines: workflows.md owns the
per-workflow catalog (trigger, inputs, short description); pipeline.md
owns the flow (stages, gates, concurrency, fork handling). Drop the
duplicated entry-points table and per-stage workflow bullet lists from
pipeline.md in favour of cross-references. Fix the stale "push to
latest" wording that pre-dated the current branch prefix list.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 12:46:26 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5e75a4b0ad feat(git): add make sign-push and deny direct push in sandbox
Route pushes through scripts/git/sign-push.sh so the operator's GPG key
stays outside the sandbox. Script signs only @{u}..HEAD (or merge-base
with origin/main|master on first push), skips re-signing when all
commits already carry a good signature, and uses --force-with-lease on
rewritten history. settings.json denies git push*, git send-pack*, and
make sign-push* so the agent cannot trigger a push path directly.
CLAUDE.md instructs the agent to hand off to the operator instead.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 12:15:44 +02:00
kevinveenbirkenbach ccb3e8a96c Merge branch 'main' of github.com:infinito-nexus/core into fix/users-applications.yml 2026-04-19 03:29:31 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2abfc64636 chore(claude): broaden make allow shapes, per-domain WebFetch, deny git -C
Three related settings.json changes:

1. Replace narrow `Bash(*=* make *)` with three broader rules covering
   any prefix + make, plus composites with `> /tmp/*` redirect and
   `2>&1` stderr-merge. Needed so multi-env invocations with quoted
   multi-word values (e.g. `FOO=a BAR=b APPS="x y z" make target > /tmp/log 2>&1`)
   auto-allow without prompting — the prior single-env rule and the
   generic `* > /tmp/*` rules did not compose for this shape.

2. Replace inert `WebFetch(domain:*)` with explicit per-domain entries
   for `github.com` and `raw.githubusercontent.com`. Tested empirically:
   the wildcard form does not suppress the permission prompt, so a
   per-domain allowlist is the only mechanism that works.

3. Add `Bash(git -C *)` to deny. Blocks cross-repo escapes via the
   `-C <path>` flag, which otherwise lets the agent reach into other
   working trees under `additionalDirectories`. Operators needing a
   second repo must open a dedicated session there.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-19 03:27:04 +02:00
kevinveenbirkenbachandClaude Opus 4.7 433a4138dd feat(ci): recognize chore branch prefix
Treat chore/** branches like feature/fix/etc. for CI: trigger the
push pipeline on chore/** pushes and accept chore as a valid PR
branch prefix in full scope. This unblocks the chore/update-* PRs
that update.yml creates automatically, which previously failed
branch-prefix validation and never triggered the push pipeline.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 17:11:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 4f2dd95fe9 feat(ci): add CI_RUN_ON_MAIN gate, reorg actions docs
Add repository variable CI_RUN_ON_MAIN that gates whether pushes to
main trigger CI. When unset (default), pushes to main skip CI;
set to 'true' to enable. Pushes to other branches are unaffected.

Reorganize GitHub Actions docs into a dedicated tools/github/actions/
directory and consolidate every CI repository variable in a single
SPOT (configuration.md). Move actions.md to actions/workflows.md and
branch/configuration.md to actions/configuration.md, then update
every cross-reference. Trim README files to scope-only per
docs/contributing/artefact/files/docs/readme_md.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 16:31:52 +02:00
kevinveenbirkenbach c6b0c2609d Merge branch 'main' into fix/users-applications.yml 2026-04-18 13:44:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 1de1bf9b1d chore(claude): allow env-prefixed make, add nektosact domain
Add `Bash(*=* make *)` so `FOO=bar make <target>` invocations (notably
`make act-workflow` with ACT_WORKFLOW/ACT_MATRIX overrides) auto-allow
without prompting — mirrors the existing `PYTHONPATH=* python *` entry.
Add `*.nektosact.com` to sandbox network allowedDomains so `act` (the
local GitHub Actions runner invoked by `make act-workflow`) can reach
its update/catalog endpoint.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 12:00:12 +02:00
kevinveenbirkenbachandClaude Opus 4.7 980ab3de5e chore(claude): grant gh cache write, generic /tmp redirect allow
Add ~/.cache/gh to sandbox allowWrite so gh CLI no longer needs an
XDG_CACHE_HOME=/tmp/... prefix, and replace the five XDG-prefixed allow
entries with two generic `* > /tmp/*` redirect entries that cover any
command capturing output into /tmp. Sync sandbox.md / settings.md /
security.md and normalize the allowWrite path enumerations.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 11:25:16 +02:00
kevinveenbirkenbach 1e0cab4952 Merge branch 'main' into fix/users-applications.yml 2026-04-18 01:18:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9e0a7778e0 chore(claude): generalize gh run download, gate update-config via ask
Replace the one-off literal allow entry for a specific `gh run download`
invocation with the `XDG_CACHE_HOME=* gh run download *` wildcard that
mirrors the existing `view`/`list` env-prefix precedents, and move the
`Skill(update-config*)` wildcard from `allow` to `ask` so settings-file
mutations (permission rules, hooks, sandbox scope) always prompt — the
skill can otherwise self-escalate the agent's own permission surface.
Document both entries in settings.md with matching Env-Prefix and
Privilege-Escalation rationales.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 23:50:34 +02:00
kevinveenbirkenbach e32fb87f95 chore(claude): allow ~/.ssh read so git push works on ask
Removes ~/.ssh from sandbox.filesystem.denyRead so SSH auth (keys, known_hosts, config) succeeds inside the sandbox. Bash(git push*) stays in ask, which becomes the effective control point for pushes.

Documents the trade-off:
- security.md: updates Assumption 9 and adds Assumption 10 covering why ~/.ssh is readable, the cost, and the mitigations (passphrase-protected keys, key rotation on incident, settings.local.json opt-out, SSH-agent-socket alternative).
- sandbox.md: removes ~/.ssh from the Read Restrictions table and links back to Assumption 10.
2026-04-17 23:20:15 +02:00
kevinveenbirkenbach 6eeef0de30 docs(tools): move actions.md into tools/github/ and fix refs
Moves the GitHub Actions workflow catalog under tools/github/ alongside the other GitHub-scoped docs, and updates the internal workflow/artefact links plus the two external references in workflow.md and pipeline.md.
2026-04-17 23:00:25 +02:00
kevinveenbirkenbach 375119b414 docs(agents): clarify push/PR flow, target repo, run-id resolve
- pull-request.md: PRs must target canonical repo behind s.infinito.nexus/code; agent resolves redirect and compares against origin.
- push-trigger-pull.md: freeze <branch> once (propose current and wait if unnamed); forbid checkout/switch; handle non-fast-forward; fetch before diff; define distro-specific scope; capture <prev-run-id> to avoid stale-run watch; resolve <pr-number> via isDraft dispatch; null-guard.
- cheatsheet.md: push-trigger-pull prompt takes explicit <branch> placeholder.
2026-04-17 22:54:13 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2a1dfc04e3 chore(claude): generalize gh run/issue allow patterns
Collapse six literal XDG_CACHE_HOME=/tmp/gh-cache gh run/issue list
allow entries into XDG_CACHE_HOME=* gh run list *, XDG_CACHE_HOME=*
gh issue list *, gh run *, and gh issue *. Plain gh run * / gh issue *
are safe because every mutating verb is caught by gh * <verb>* ask or
gh issue transfer* deny. Env-prefix variants stay narrow (no broader
than list) because env prefix bypasses subcommand ask/deny patterns.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 21:27:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 297e8e0769 chore(claude): allow gh api *, ask mutation flags
Allow read-only gh api GETs (plus XDG_CACHE_HOME=* prefix variant) so
log-fetch invocations with env prefix + redirect stop prompting. Route
every implicit-POST shape through ask: -f/-F/--field/--raw-field/--input
body flags and graphql*. Path-based deny rules keep precedence.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 21:27:10 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a7cd24b9cb docs(actions): add SPOT catalog and push-trigger-pull procedure
Centralise the full GitHub Actions workflow catalog in
docs/contributing/tools/actions.md as the single source of truth, link
pipeline.md and workflow.md to it, and introduce
docs/agents/action/push-trigger-pull.md as the authoritative procedure
for the push to draft CI to ready-for-review loop. Retire the stale
flat docs/contributing/tools/agents/claude.md and drop docs README
page-inventories that violate the readme_md.md scope rule.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 17:40:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7218af12a8 docs(actions): add SPOT catalog and push-trigger-pull procedure
Centralise the full GitHub Actions workflow catalog in
docs/contributing/tools/actions.md as the single source of truth, link
pipeline.md and workflow.md to it, and introduce
docs/agents/action/push-trigger-pull.md as the authoritative procedure
for the push to draft CI to ready-for-review loop. Retire the stale
flat docs/contributing/tools/agents/claude.md and drop docs README
page-inventories that violate the readme_md.md scope rule.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 17:38:33 +02:00
kevinveenbirkenbachandClaude Opus 4.7 75d229a5f1 chore(claude): allow PYTHONPATH=* python *
Mirrors the XDG_CACHE_HOME=* gh run view * precedent: leading env-prefixes fall out of the auto-allow heuristic. Narrow namespace, sandbox boundaries still bound every write / read / network call.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 14:12:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 ed34db2e8c chore(claude): allow tail *
Parallel to head *: tail is the right-hand-side twin that lives after && / | and would otherwise decline auto-allow.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-17 14:12:23 +02:00