Commit Graph
37 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.7 16bdbf03d7 docs: fix repo-wide drift against documentation.md style rules
Three classes of drift cleaned up across 85 .md files:

- Em-dashes: replaced ` — ` with the contextually correct separator
  (colon for `term: explanation`, period to split clauses across
  sentences/source lines, parentheses for asides, semicolon for
  tight-linked continuations). Two intentional exceptions remain
  in `docs/contributing/documentation.md` (the rule literal) and
  inside a fenced ```sh block in `docs/contributing/actions/debugging/ci.md`.
- SPOT self-labels: rewrote "This is the SPOT for X" to describe
  the subject directly ("This directory holds X" / "This file
  describes X"). Cross-references to other artefacts' canonical
  role remain unchanged.
- Heading-emoji position: moved leading-emoji headings (`# 🧩 Title`)
  to trailing position (`# Title 🧩`) per the headline rule that
  emojis MUST come after the title text.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 20:12:07 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0b5db3e421 fix(install/lint): auto-install npm so markdownlint/eslint installs work on virgin distros
`make install-lint` failed on every test-environment matrix runner
(manjarolinux/base, debian:bookworm, ubuntu:latest, fedora:latest,
quay.io/centos/centos:latest) with:

    npm not found; install Node.js/npm first to get markdownlint-cli2.
    make: *** [Makefile:227: install-lint] Error 1

The two npm-driven installers (`install_markdownlint_with_npm` and
`ensure_eslint`) bailed out as soon as `npm` was missing instead of
following the same auto-install pattern used elsewhere in the
script (e.g. `ensure_command shellcheck` / `shfmt` / `ruff`).

* Add an `npm:<manager>` mapping to `install_command`. pacman /
  apt-get / dnf / yum get the candidate list `npm nodejs`; brew
  gets `node`. `install_package_candidates` returns on the first
  candidate that installs, which covers every matrix distro
  (Arch's `npm` pulls `nodejs` as a hard dep; Debian/Ubuntu's `npm`
  Depends: nodejs; Fedora/EL9+ ship `npm` as a direct package; brew's
  `node` bundles npm).
* `install_markdownlint_with_npm` and `ensure_eslint` now call
  `ensure_command npm` when `npm` is missing, before failing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 16:25:23 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0e877caafe build(lint/javascript): introduce ESLint + eslint-plugin-playwright
Wires JavaScript linting into the project for the first time, scoped to
the Playwright specs and persona helpers under `roles/**/files/**/*.js`.

Setup
-----
- `package.json` + `package-lock.json` pin eslint v9, eslint-plugin-
  playwright v2, @eslint/js, globals.
- `eslint.config.js` (flat config) loads `eslint:recommended` and
  `playwright/flat/recommended`. Browser globals are injected so
  `page.evaluate(() => window.foo)` callbacks lint cleanly. Project-
  idiomatic patterns are explicitly turned off with rationale: the
  service-gating contract (req 006/019) uses test.skip on disabled
  services; persona helpers branch on isVisible/isServiceEnabled before
  expect; some specs deliberately use waitForTimeout / networkidle for
  OIDC settle. `no-empty` allows the empty-catch idiom; the autofixer
  for `prefer-web-first-assertions` and `no-wait-for-navigation` is
  unsafe in this codebase (broken diff in dashboard.js), so both are
  off pending a hand-migration pass.
- `scripts/lint/javascript.sh` mirrors the markdown.sh shape (skip on
  missing, exit 127 with "make install-lint" hint).
- `scripts/lint/autoformat.sh` gets a `run_eslint` worker (treats rc≤1
  as OK, like markdownlint).
- `scripts/install/lint.sh` gets `ensure_eslint` (npm-local install in
  the repo root, matching the markdownlint pattern), a `javascript`
  group, and `install_javascript_tools` is wired into the `all` group.
- `Makefile` exposes `lint-javascript` and folds it into the aggregate
  `lint` target.
- `.github/workflows/lint-javascript.yml` follows the lint-makefile.yml
  shape (Node 20, npm cache, `npm ci`, `make lint-javascript`); the
  ci-orchestrator wires it into the lint pipeline + lint-gate `needs:`.
- `.gitignore` adds `node_modules/`.

Real lint findings fixed in this commit
---------------------------------------
- `roles/web-app-decidim/files/playwright.spec.js` was using
  `runBiberFlow` / `runAdminFlow` without ever requiring `./personas`
  (would have failed at runtime). Adds the require.
- Same file: two nutzlose `\\'csrf-token\\'` escapes in a double-quoted
  query selector — eslint flagged, fixed.
- `roles/web-app-moodle/files/playwright.spec.js`:
  `expect(await locator.count()).toBe(0)` →
  `await expect(locator).toHaveCount(0)` (Playwright auto-retrying
  assertion; safe `prefer-to-have-count` autofix).

Lint baseline: 0 errors, 285 warnings (mostly `no-unused-vars` and
`playwright/expect-expect` — addressable as follow-up cleanup, not
blocking).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 14:55:04 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c52e4269bf chore(tooling): integrate mbake (Makefile linter/formatter) + working-tree drift sync
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
  ensure_mbake (mbake is pip-installable, parses our Makefile correctly
  whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
  reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
  (mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
  `.PHONY:` declarations so mbake's parser can validate it; add the
  previously-missing `install-python-dev` and `build-dependency` PHONY
  targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
  `^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
  + make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
  lint-gate.needs.

Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
  galaxy-importer when it runs in roles/).

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:32:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 79409ebf10 test(tooling): integrate markdownlint-cli2, ansible-lint, galaxy-importer + fix the resulting baseline
Tooling integration
* add .markdownlint-cli2.jsonc, .ansible-lint, .ansible-lint-ignore
  configs (permissive day-1: cosmetic rules disabled, semantic ones on)
* extend scripts/install/lint.sh with generic install_pip_pkg helper +
  ensure_ansible_lint / ensure_galaxy_importer / ensure_markdownlint_cli2
  ensure_* functions (pip / npm with sudo / --user / --break-system-packages
  fallbacks). install_ansible_tools now also pulls ansible-lint and
  galaxy-importer; new `markdown` install group.
* extend scripts/lint/ansible.sh to run ansible-lint after the existing
  syntax-check.
* add scripts/lint/markdown.sh and scripts/lint/galaxy.sh runners. The
  galaxy runner parallelizes via xargs (-P=$(nproc)), prints OK/FAIL per
  role, and treats `bare-name dependency` errors as success since our
  roles/ tree is not a Galaxy collection layout. All other Galaxy schema
  errors still fail.
* Makefile: new lint-markdown and lint-galaxy targets, both wired into
  `make lint`.
* CI: new workflows lint-markdown.yml and lint-galaxy.yml; lint-ansible.yml
  now installs ansible-core + ansible-lint; ci-orchestrator wires both
  new lint workflows into lint-gate.

Galaxy schema fixes
* shorten the canonical `galaxy_info.company` block from three lines to
  two ("Kevin Veen-Birkenbach\nhttps://www.veen.world", 45 chars) so it
  stays under Galaxy's hard 50-char limit. Updated everywhere: 229 role
  meta/main.yml plus _CANONICAL_COMPANY in
  tests/lint/repository/test_role_meta_main_galaxy_schema.py.
* shorten 19 over-long role descriptions to <=255 chars.
* add stub READMEs for sys-svc-compose-ca and sys-utils-service-loader.

Markdown baseline fixes
* markdownlint-cli2 --fix run repo-wide (formatting normalization
  across CHANGELOG, role READMEs, docs).
* manual MD041 fixes: prepend H1 in 15 files (CHANGELOG, LICENSE,
  TODO.md helpers, role README templates, etc.).
* MD025 / MD003 / MD001 / MD032 manual fixes for heading-style and
  list-spacing edge cases.
* role README content polish: "## Single Point of Truth (SPOT)" headings
  renamed to "## User data resolution" (D3 self-label rule).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 23:46:37 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9ffd118264 fix(skills): mirror .agents/skills to .claude/skills instead of symlink
The previous symlink approach left .claude/skills as a real empty dir in
some setups, which collides with the agent sandbox (denyWithinAllow on
.claude/skills expects a file path, not a directory) and breaks every
Bash invocation. Copy the installed skills tree into .claude/skills so
both locations are real, independent copies — no symlink involved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 18:32:20 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6364b8949f fix(install/arch): init pacman keyring before pacman -Syu
Move `pacman-key --init` + `--populate` ahead of the first `pacman -Syu`
call in `build_and_install_arch`. The previous guard
`pacman-key --list-sigs` always returned 0 on a fresh Manjaro/Arch image,
so the keyring init never ran and the upgrade of `archlinux-keyring`
failed with `==> ERROR: There is no secret key available to sign with.`,
breaking the manjarolinux/base test-environment matrix entry.

Refs: https://github.com/infinito-nexus/core/issues/167

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 12:28:35 +02:00
Environment TestandClaude Opus 4.6 e2dbbceb6a chore(sandbox): add cross-distro installer for bwrap/socat via make agent-install
Adds scripts/install/sandbox.sh, a distro-aware installer for the OS-level
dependencies the Claude Code sandbox backend needs (bubblewrap, socat),
and exposes it through a new `make agent-install` target. Documents the
target in docs/contributing/tools/make.md, scripts/install/README.md, and
the Claude sandbox guide so contributors on a fresh host can bring up a
working sandbox without hand-installing distro packages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 18:06:57 +02:00
kevinveenbirkenbachandClaude Opus 4.6 f04bae680e chore(skills): symlink .claude/skills -> .agents/skills
Claude Code discovers project skills under .claude/skills/, but
`skills experimental_install` writes to .agents/skills/. Add a
relative symlink so installed skills (caveman, compress, ...) are
picked up without extra copy step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-16 11:30:41 +02:00
Environment TestandClaude Sonnet 4.6 6c8547c6a4 feat(skills): add agent skills install/update pipeline and caveman docs
- Add scripts/install/skills/install.sh to restore skills from
  skills-lock.json via npx skills experimental_install
- Add scripts/install/skills/update.sh to update all skills by
  re-running skills add for each source in skills-lock.json
- Add make install-skills and make update-skills targets
- Add .github/workflows/update-skills.yml for daily automated updates
  matching Dependabot schedule (00:00 UTC), opens PR on lock changes
- Add docs/agents/meta/caveman.md as SPOT for caveman mode, startup
  notification, and token-budget tier progression
- Update AGENTS.md with RFC 2119-compliant skills installation check
- Update docs/agents/action/commit.md: replace ALWAYS with MUST
- Update pyproject.toml: respect-gitignore = true, exclude .agents
- Track skills-lock.json and add .agents to .gitignore

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 12:54:43 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 e84bb3fe06 fix(env): unset cached env vars before reload and guard commit test against dirty worktree
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 02:08:39 +02:00
kevinveenbirkenbach c97f1d84e0 fix(install): ensure venv python is used for pip and pre-commit
Reload env layer in install scripts after venv creation so `PYTHON`
points to the virtualenv instead of the system interpreter.

* Reload env in `python.sh` before installing dependencies
* Reload env in `pre-commit.sh` before installing hooks
* Reset env cache flags to force re-evaluation
* Keep centralized environment resolution (no hardcoding)

Fixes PEP 668 `externally-managed-environment` errors and missing
`pre_commit` module during hook installation.
2026-04-08 21:21:37 +02:00
kevinveenbirkenbach b0b8874bad feat(python): reload env after venv creation to ensure pip uses virtualenv
Fixes an issue where `PYTHON` could still point to the system interpreter
when `scripts/install/python.sh` runs after `install-venv`.

* Reload env layer if venv exists and PYTHON is not yet pointing to it
* Reset env caching flags to force re-evaluation
* Preserve central environment resolution logic (no hardcoding)

This resolves Debian/PEP 668 `externally-managed-environment` errors during pip install.
2026-04-08 20:30:35 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 821c3c53b6 feat(pre-commit): introduce pre-commit hook enforcement and modular environment test suite
- Add .pre-commit-config.yaml running make test on non-doc changes
- Add pre-commit to pyproject.toml dev dependencies
- Add scripts/install/pre-commit.sh for hook registration
- Add install-python-dev Makefile target; wire into environment-bootstrap
- Rename setup-development → mark-development (env.ci override marker)
- Refactor scripts/tests/environment.sh into modular 00–10 scripts with orchestrator
- Add 09_commit.sh validating pre-commit enforcement and --no-verify bypass
- Update docs/agents/action/commit.md and testing.md for new hook behavior
- Update makefile.md, setup.md, installation.md accordingly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 16:58:24 +02:00
kevinveenbirkenbach 8a75959c11 Merge branch 'main' into feature/makefile-wsl2
# Conflicts:
#	.github/workflows/test-development.yml
#	Makefile
#	roles/svc-db-postgres/files/set_postgres_superuser_password.sh
#	scripts/administration/apparmor/restore.sh
#	scripts/administration/apparmor/teardown.sh
#	scripts/dns/setup.sh
#	scripts/install/package.sh
#	scripts/install/venv.sh
#	scripts/tests/development.sh
2026-03-24 20:20:27 +01:00
Alejandro Roman 06d3dd8fe9 fix(makefile-wsl): add envsubst and docker compose plugin to dev smoke test containers 2026-03-22 00:19:03 +01:00
Alejandro Roman 82744f40df fix(makefile-wsl): fix bootstrap failures in distro containers without systemd 2026-03-21 23:07:30 +01:00
Alejandro Roman 9b4315d8f2 feat(makefile-wsl): add package installer to bootstrap development environment 2026-03-21 22:27:32 +01:00
Alejandro Roman ce91178db1 fix(makefile-wsl): fix issues on tests related to wsl implementation with no guards 2026-03-21 16:08:28 +01:00
Alejandro Roman da6e87191b fix(makefile-wsl): Fix the tesl app error and the ca trust 2026-03-21 16:08:28 +01:00
Alejandro Roman 9d1fae39f9 fix(makefile-wsl): root permisions to allow creation of files 2026-03-21 16:08:28 +01:00
kevinveenbirkenbach dcfd7016ae refactor workflow apt installs into script 2026-03-21 02:22:38 +01:00
kevinveenbirkenbach 44ea2df452 Fix dev workflow image build and docker repo bootstrap 2026-03-19 19:11:51 +01:00
kevinveenbirkenbach 3b11d7f469 feat(packaging): add distro packages and install via package metadata 2026-03-19 02:53:15 +01:00
kevinveenbirkenbach 38b1d08ef7 Improve distro compatibility and dev environment testing 2026-03-18 20:00:33 +01:00
kevinveenbirkenbach aed48d3231 Fix actionlint installer 2026-03-18 14:47:39 +01:00
kevinveenbirkenbach a715970506 Refactor lint tooling and shell scripts 2026-03-18 12:37:31 +01:00
kevinveenbirkenbach 575474f5bb Refactor Python bootstrap to single installer SPOT 2026-02-22 01:20:23 +01:00
kevinveenbirkenbach 98e1f39291 fix(install): add retry logic for pip installs in python setup script
Add retry wrapper (7 attempts, 20s delay) around pip install
commands to mitigate transient network errors (e.g. GitHub 403,
rate limits, temporary connectivity issues) during CI builds.

https://chatgpt.com/share/e/69933fb2-9c94-8013-815d-4f9509b0456f
2026-02-16 17:03:18 +01:00
kevinveenbirkenbach 49da7376a9 ci(ansible): add galaxy-to-git fallback for collection installation
Split collection requirements into Galaxy and Git sources and
introduce retry logic with Git fallback to harden CI against
transient Ansible Galaxy API failures.
2026-01-25 11:03:19 +01:00
kevinveenbirkenbach ea39b5b215 ci(ansible): add retry logic for galaxy collection install
Retry ansible-galaxy collection installation up to 10 times with
randomized backoff to mitigate transient Galaxy API outages.

https://chatgpt.com/share/6975e68c-1ae4-800f-810c-91521230f55b
2026-01-25 10:46:45 +01:00
kevinveenbirkenbach a89f74fac0 Included extra modul parameter 2026-01-21 16:02:18 +01:00
kevinveenbirkenbach 370938c71b fix(venv): use bootstrap python to create virtualenv
Avoid using the venv interpreter before it exists.
Create the venv with system python3 and only rely on venv/bin/python afterwards.

https://chatgpt.com/share/69702711-790c-800f-bbe0-812afdf9d975
2026-01-21 02:08:23 +01:00
kevinveenbirkenbach 18acafa9dc ci(images): build & push full+slim CI images
- Build matrix now includes variants: full + slim
- Tag slim images with -slim suffix and use Dockerfile target selection
- Use dedicated gha cache scopes per distro+variant
- Update release checks/retag/promote scripts to handle slim images too

https://chatgpt.com/share/69701a4e-b000-800f-be7e-162dcb93b1d2
2026-01-21 01:46:52 +01:00
kevinveenbirkenbach 0928022731 More python calls from correct context 2026-01-21 01:09:01 +01:00
kevinveenbirkenbach 6c84807778 feat(test-local): add all-app inventory init and run targets
Add two local test helpers to run full-stack deployments for all discovered apps:

- make test-local-inventory-init-all: bring up dev stack and create inventory
  (/etc/inventories/local-full-<type>/servers.yml) including all apps
- make test-local-run-all: run deploy using the pre-initialized inventory

Also mark various scripts as executable (chmod +x) to ensure consistent
behavior across environments.

https://chatgpt.com/share/6970018e-1244-800f-9345-ca1596061729
2026-01-20 23:43:10 +01:00
kevinveenbirkenbach 2344a04958 refactor(make): move install and setup logic into scripts
- extract Ansible collection install into scripts/install/ansible.sh
- extract venv bootstrap into scripts/install/venv.sh
- extract python deps install into scripts/install/python.sh
- extract project setup logic into scripts/setup.sh
- simplify Makefile targets to delegate to scripts
- add bootstrap target for full environment setup

https://chatgpt.com/share/696ff8cd-32d4-800f-bdd5-a34159aac1c0
2026-01-20 22:50:02 +01:00