Commit Graph
12 Commits
Author SHA1 Message Date
kevinveenbirkenbach 9b2f870777 Merge branch 'main' into feature/swarm-nfs 2026-07-01 05:37:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c1c5230302 Merge branch 'main' into feature/swarm-nfs
Conflicts resolved:
- .claude/.gitignore: union of ignore patterns
- plugins/lookup/rbac_group_path.py: drop both unused conflicting imports
  (body fetches apps via lookup_loader)
- roles/sys-svc-compose/handlers/main.yml: keep HEAD's refactor (compose-up
  handlers live in compose.yml, included per DEPLOYMENT_MODE)
- utils/cache/base.py: keep HEAD's merge_with_defaults/FILE_TOKENS imports

Post-merge fixes for main's newly-enabled lints:
- pyproject.toml: grandfather 5 files under the BLE001 ratchet
- justify 6 pre-existing security # noqa markers (S108/S311/S701)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 22:41:39 +02:00
Alejandro Roman b85729297b Merge remote-tracking branch 'upstream/main' into feature/self-host-runners
# Conflicts:
#	cli/administration/deploy/development/down.py
2026-06-29 21:16:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cddb0dc59b feat(lint): require emoji-prefixed task/step/job names + convert existing
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.

- emit emoji names at the source: cli/build/include generator now writes
  "🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
  tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
  task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
  🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fc77d9d265 fix(security): resolve all open code-scanning alerts and guard against recurrence
Resolve the 80 open CodeQL/code-scanning alerts on this branch.

Security-sensitive fixes:
- cli/administration/deploy/development: break the unsafe import cycle by
  extracting the env/arg helpers into a leaf module `env.py`, giving the
  load-time DAG compose -> common -> deps -> env.
- web-app-bluesky login-broker server.js: prototype-pollution guard on
  cookie parsing, CR/LF + control-char log sanitisation, HTML-escaped
  exception text, and a ReDoS-bounded handle regex.
- web-app-baserow SSO: harden the open-redirect `next` handling.
- network diagnose probes: require TLS >= 1.2.
- actionlint installer: validate tar members against path traversal.
- decidim/apt-purge lint regexes: rewrite to avoid catastrophic backtracking.
- 6x Playwright URL regexes anchored; crs-k/stale-branches pinned to a commit SHA.

Mechanical cleanups: unused imports/globals/locals, empty-except comments,
import-and-import-from, implicit-string-concat, multiple-definition,
unreachable/useless-assignment.

False positives are intentionally left in place (Django settings patches,
Odoo __manifest__, side-effect imports, cross-module symbols).

Recurrence guards:
- import-linter contract (pyproject + tests/lint) locks the dev-deploy
  package acyclic; a new cycle now fails CI instead of only CodeQL.
- new lint test requires every bandit (S###) `noqa` to carry a justification.
- BLE001 (blind-except) added to ruff as a ratchet (tests/roles exempt,
  current production offenders grandfathered).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:42:03 +02:00
Alejandro Roman e6866faa08 refactor(svc-runner): modify test 2026-06-24 18:30:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3c07721303 feat(swarm-nfs): bootstrap Docker Swarm + NFS-backed shared volumes (#023)
Adds the svc-docker-swarm, svc-docker-swarm-manager, svc-storage-nfs-server
and svc-storage-nfs-client roles, the compose_volumes filter NFS support,
the test-deploy-swarm-nfs CI workflow with its swarm-nfs-pilot script set,
and the deploy_mode-aware compose/swarm wiring across mediawiki, mariadb
and sys-svc-* helpers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 18:57:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 fc2ac7597a refactor(lookup): rename status -> deployment, add runtime-aware deployed
`status` was too vague for what the lookup returns (apps in current
play). Rename to `deployment` and add two new fields:

* `runtime` — RUNTIME variable value
* `deployed` — runtime-aware authoritative set: equals `running`
  (whitelist or groups) for ephemeral CI runtimes (act, github),
  falls back to `groups` for persistent runtimes (host, dev). This
  matches reality: in CI the container is fresh so only whitelisted
  apps exist; in production a partial deploy still leaves prior
  apps live, so groups is the right set.

Switch the two webserver/csp healthchecks to `lookup('deployment').deployed`
so they only probe domains for apps that actually exist in the current
deploy. In production behaviour is unchanged.

Update remaining consumers (cli/build/include, web-app-keycloak,
test-e2e-playwright) for the rename. Existing `status` semantics
(`whitelist`, `running`, `groups`, `all`) preserved under the new
name.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:52:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7f221f91eb fix(cli/build/include): tolerate cross-UID overwrites of generated playbook entries
`cli build include` writes tasks/groups/*-roles.yml. Both
test-integration (in-container, nobody UID) and lint-ansible: setup
(host UID) run setup in parallel via scripts/make/parallel.sh and
race on the same file; `open(..., 'w')` then trips PermissionError
on the second writer. Unlink the file first when it exists but is
not writable — directory-level delete permission is host-owned in
our layout, so both UIDs can safely recreate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-23 18:52:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 944e782d84 refactor(applications): introduce status lookup + rbac_app_ids filter; rename allowed_applications → APPLICATIONS_WHITELIST
- plugins/lookup/status.py: new lookup returning {whitelist, running, groups, all}, cached per (whitelist, groups). Consumers replace direct group_vars references.
- web-app-keycloak/filter_plugins/rbac_per_app.py: add `rbac_app_ids` (intersect input list with invokable apps; sorted, deduped).
- web-app-keycloak/tasks/update/05c_per_app_mappers.yml: drop two set_fact tasks; consume KEYCLOAK_RBAC_{ALL,RUN}_APP_IDS as constants from vars/main.yml. Split orphan-purge scope (all host-deployed) from mapper-sync scope (current run).
- Rename Ansible-side `allowed_applications` → `APPLICATIONS_WHITELIST` (group_vars/all default, deploy CLI `-e` injection, generated include `when:` clause, debug task, test asserts). Python kwarg name unchanged (internal contract).
- test-e2e-playwright: `TEST_E2E_PLAYWRIGHT_ONLY_ROLES` consumes `lookup('status').running`; `TEST_E2E_PLAYWRIGHT_APPS` moved from set_fact to vars/main.yml constant.
- 01_constructor debug: collapse to single `status:` block over the lookup result.
- web-app-{opencloud,opentalk}/vars: drop dead `group_names | default([])` + fix latent `| lower`-on-list precedence bug.
- New lint `test_no_jinja_default_on_ansible_magic_var.py`: forbid `| default(...)` on Ansible magic vars (group_names, inventory_hostname, …); meta-scan + 7 fixture positive cases.
- Unit tests for `rbac_app_ids` (10) and `status` lookup (8).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-16 18:39:52 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c55ad69c41 refactor: drop requirement-file references from code, add doc rule + lint
Strip every `req <NNN>`, `req-<NNN>`, `Requirement <NNN>`,
`requirement-<NNN>`, and `docs/requirements/<NNN>-...md` mention from
non-Markdown source files. Comments anchored to requirement numbers
rot when requirements are renumbered or rewritten and conflate
present-state behaviour (which the comment should describe) with the
audit trail of how the code got there (which belongs in commit
messages, PR descriptions, and the requirement file itself).

* Document the rule in `docs/contributing/documentation.md` under
  "Comments". Code MUST describe the *why* in present-state language;
  contract enforcement lives in tests, not folklore.
* Clean 254 source files across `roles/`, `tests/`, `plugins/`,
  `cli/`, `scripts/`, `utils/`, `group_vars/`, and the repo-root
  `eslint.config.js`. Pure comment / docstring / log-message edits;
  no behavioural change.
* Add `tests/lint/repository/test_no_req_references_in_code.py` to
  enforce the rule going forward across `.js`, `.py`, `.yml`,
  `.yaml`, `.j2`, and `.sh` files. The legacy
  `test_no_stale_requirement_refs.py` is exempt as the only file
  whose purpose is to detect these very tokens.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:25:22 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3ffc7894bd refactor(cli): rename build/meta packages and move services resolver to utils
Package renames (dynamic discovery picks up new names automatically):
- cli/build/role_include/      -> cli/build/include/
- cli/build/roles_list/        -> cli/build/list/
- cli/meta/roles/lifecycle_filter/ -> cli/meta/roles/lifecycle/

Domain logic moved out of cli/ (no CLI surface, just a helper class):
- cli/meta/roles/applications/resolution/services/{resolver,errors}.py
  -> utils/roles/applications/services/{resolver,errors}.py
- drop dead services/yaml_utils.py + its only test

Bug fixes:
- cli/meta/{callorder -> roles/callorder} dropped its hard-coded
  parents[3] PROJECT_ROOT (broke after the directory move and resolved
  to <repo>/cli/); it now reuses utils.PROJECT_ROOT.

Help renderer (cli/core/help.py + colors.py):
- DIM color on the description column to separate it from the command name
- Blank line between the long command name and its wrapped description
- Italic invocation under the category heading + leading blank line
- New Style.ITALIC

Doc updates: path references in docs/requirements/00{8,10}*.md plus
soft-wrap fixes in the touched regions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 02:45:40 +02:00