Commit Graph
14682 Commits
Author SHA1 Message Date
Environment TestandClaude Sonnet 4.6 7cae9b15fb fix(dependabot): use ** glob for recursive auto-discovery of dependency files
All ecosystems now scan all subdirectories automatically instead of
hardcoded paths, so new dependency files are picked up without
manual configuration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 01:07:27 +02:00
Environment TestandClaude Sonnet 4.6 3ca2e2f62f ci(orchestrator): add images-mirror-missing as dependency of test-development
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 00:27:37 +02:00
Environment TestandClaude Sonnet 4.6 1530788dbe feat(mirror): add repository segment to GHCR image path
Image names now follow {owner}/{repository}/mirror/{registry}/{name}
instead of {owner}/mirror/{registry}/{name}.

- GHCRProvider: add `repository` param + `add_args()`/`from_args()` classmethods
  as SPOT for CLI argument registration and provider construction
- sync, resolver: use GHCRProvider.add_args() / from_args() (no duplication)
- Workflows: resolve repository name via scripts/meta/resolve/repository/name.sh
  and pass --ghcr-repository to sync and publish
- mirrors.sh: require and forward GHCR_REPOSITORY env var
- deploy/development/mirrors.py: derive repository from GITHUB_REPOSITORY
- Tests: update expected paths, argv and env fixtures

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 00:20:41 +02:00
Environment TestandClaude Sonnet 4.6 317287983a fix(mirror): resolve merge conflicts with fix/playwright-pull
Keep new multi-registry logic (feature/pre-commit):
- Merge is_mirrorable_image(), _detect_registry(), image_source() and
  canonical_image_name() into utils/docker/image_discovery.py
- iter_role_images() now scans both config/main.yml and vars/main.yml
  and uses image_source() for full registry-prefixed URLs
- Delete cli/mirror/util.py (superseded by utils/docker/image_discovery)
- model.py re-exports ImageRef from utils.docker.image_discovery
- sync/__main__.py: revert fix/playwright-pull's if/else src URL to
  simple `src = f"docker://{img.source}"` (source already contains
  the full registry prefix)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 23:44:58 +02:00
Environment TestandClaude Sonnet 4.6 e8a5896881 feat(mirror): include registry domain in package name and fix multi-registry source URLs
- Add `registry` field to `ImageRef` (default `"docker.io"`)
- Populate `registry` from `_detect_registry()` in `iter_role_images()`
- `GHCRProvider.image_base()` and `GiteaProvider.image_base()` now produce
  `{prefix}/{registry}/{name}` paths (e.g. `mirror/docker.io/nextcloud`,
  `mirror/quay.io/keycloak/keycloak`) instead of flattening slashes to dashes
- Fix `mirror()` in both providers: use `docker://{image.source}` directly
  instead of always prepending `docker://docker.io/`, which broke quay.io/ghcr.io images
- Update tests to reflect new naming convention and add quay.io test case
- Add missing __init__.py files in test subdirectories

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 22:57:33 +02:00
Environment TestandClaude Sonnet 4.6 3e0d9662a6 merge(mirror): resolve conflicts between feature/pre-commit and fix/playwright-pull
- Keep HEAD's registry-aware src construction in sync/__main__.py
  (docker://docker.io/{source} for Docker Hub, docker://{source} otherwise)
  to stay compatible with image_discovery.py's ImageRef.registry field
- Accept deletion of cli/mirror/util.py (content moved to utils/docker/image_discovery.py)
- Revert test_sync.py source format to library/nextcloud:31.0.0 (HEAD format)
- Retain publish/__main__.py improvements from feature/pre-commit:
  graceful 404 handling, no blind retries on client errors, pkg_url usage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 22:33:07 +02:00
Environment TestandClaude Sonnet 4.6 3f19ec1df4 fix(mirror): extend registry support and harden publish visibility handling
Extend image mirroring to cover quay.io and ghcr.io in addition to Docker Hub,
and make the publish step non-fatal when the token lacks write:packages scope.

cli/mirror/util.py:
- Add _detect_registry(), _strip_registry_prefix(), is_mirrorable_image()
  to support Docker Hub, quay.io, ghcr.io, and mcr.microsoft.com
- Update canonical_image_name() to strip any known registry prefix so that
  quay.io/keycloak/keycloak -> keycloak/keycloak (mirror/keycloak-keycloak)
- Add image_source() returning full registry-prefixed pull reference
  (docker.io/library/postgres:16, quay.io/keycloak/keycloak:latest, …)
- iter_role_images() now uses is_mirrorable_image + image_source, picking up
  the 17 previously skipped quay.io/ghcr.io images

cli/mirror/sync/__main__.py:
- Use docker://{img.source} directly (no hardcoded docker.io/ prefix)

cli/mirror/publish/__main__.py:
- Detect HTTP 404 on PATCH as _InsufficientTokenError (no retry, graceful
  GHA warning + exit 0) instead of retrying 5x and failing the workflow
- Use pkg.get("url") from listing response as canonical PATCH target URL
- Use safe="/" in urllib.parse.quote so mirror/name stays as literal path
- Do not retry on non-transient 4xx errors (400/401/403)

tests/unit/cli/mirror/test_sync.py:
- Update ImageRef.source to new docker.io/library/… format

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 22:27:11 +02:00
Environment TestandClaude Sonnet 4.6 1db58af7fd fix(docker): replace ENV-based ENTRYPOINT with RUN symlink in espocrm and suitecrm
The ENV approach failed at runtime because the ca-trust-wrapper does not
propagate container ENV variables when executing the wrapped ENTRYPOINT,
causing $ESPOCRM_ENTRYPOINT_SCRIPT_DOCKER / $SUITECRM_ENTRYPOINT_SCRIPT_DOCKER
to remain unexpanded ("not found" error in CI).

Replace with a RUN-time symlink (ln -sf) so the dynamic ARG path is baked
into the image filesystem layer. ENTRYPOINT now uses the fixed /entrypoint.sh
path, which is independent of ENV propagation and cache state.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 21:48:18 +02:00
Environment TestandClaude Sonnet 4.6 7303fc892c docs(agents): add SERVICES_DISABLED proposal to Role Loop in iteration guide
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 20:53:00 +02:00
Environment Test 5bc6ce74b5 debug: add disk diagnostics before playwright 2026-04-09 20:39:24 +02:00
Environment Test 1c96eb7ee9 refactor: extract compose docker prune script 2026-04-09 20:13:43 +02:00
Environment TestandClaude Sonnet 4.6 9ea2ad6ff1 refactor(docker): introduce utils/docker/image_discovery as single SPOT for image enumeration
- Move iter_role_images + all helpers + ImageRef from cli/mirror/util.py and
  cli/mirror/model.py into utils/docker/image_discovery.py; add source_file field
  to ImageRef to distinguish config/main.yml vs vars/main.yml entries
- Delete cli/mirror/util.py; update all callers to import from utils.docker.image_discovery
- Move utils into subpackages: utils/docker/ (image_ref, paths_utils, hadolint) and
  utils/roles/ (dependency_resolver, resource_validation); update all imports
- Reorganise tests into subpackages: tests/lint/docker/dockerfile/, tests/integration/docker/,
  tests/unit/utils/docker/, tests/unit/utils/roles/; fix parents[N] offsets after relocation
- tests/lint/docker/test_image_versions.py now uses iter_role_images (same SPOT as mirror)
- Add vars/main.yml images.<name>.{image,version} support for non-application roles
  (test-e2e-playwright, sys-ctl-hlth-csp); eliminate hardcoded duplicates in defaults/tasks
- Add web-app-matrix matrix-chatgpt-bot and web-app-mini-qr mini-qr entries in config/main.yml;
  replace hardcoded image strings in vars/templates with lookup('config', ...) references
- Fix sync/__main__.py log line to use correct registry prefix for non-Docker-Hub images
- Update docs/contributing/code/role/dockerfile.md with image declaration rules and SPOT reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 19:32:31 +02:00
Prageeth Panicker 1055b41d1a fix(web-app-decidim): fix deployment issues and verify local deployment
- Use ghcr.io/decidim/decidim image (pre-generated app)
- Fix container_port to 3000 (Puma internal port)
- Fix REDIS_URL to use redis service name
- Fix OIDC.CLIENT.ISSUER_URL (was OIDC.CLIENT.ISSUER)
- URL-encode database password in DATABASE_URL
- Add RAILS_HOSTS for host authorization
- Remove /etc/timezone volume mounts (not a directory)
- Render compose.yml and env explicitly in tasks/main.yml
- Fix changed_when in 01_setup.yml to use stdout_lines
- Use defaults_service_provider.platform.titel for org name
- Add biber vars to playwright.env.j2
- Expand playwright tests with biber scenarios
- Verified: Decidim accessible at decidim.infinito.example
2026-04-09 14:01:12 +00:00
Environment TestandClaude Sonnet 4.6 4ab67a1b8a fix(ci): increase test-environment timeout from 180 to 270 minutes
The full test suite (two complete runs of 01–10_teardown) consistently
takes ~212 minutes; the previous 180-minute limit caused premature
cancellation at 08_deploy_performance.sh in the second run.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 15:21:30 +02:00
Environment TestandClaude Sonnet 4.6 0d8c6b29a7 test(lint): add test for ARG declarations used in multi-stage FROM instructions
Detects the bug where an ARG referenced in a FROM instruction is declared
inside a build stage instead of the global scope (before the first FROM),
which causes Docker to treat the variable as empty and fail with
"UndefinedArgInFrom" / "invalid reference format".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 13:12:46 +02:00
Environment TestandClaude Sonnet 4.6 16fbd3aa9a refactor(bookwyrm): move hardcoded python:3.11-bookworm to config/vars as ARG
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 13:06:12 +02:00
Environment TestandClaude Sonnet 4.6 92793aafba fix(docker): move SHOPWARE_PHP_IMAGE/VERSION ARGs to global scope for multi-stage FROM
ARG declarations used in FROM instructions must be in the global scope (before
the first FROM). Declaring them inside a build stage makes them invisible to
subsequent FROM instructions, causing "UndefinedArgInFrom" and an invalid
reference format error at build time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 13:01:02 +02:00
Environment TestandClaude Sonnet 4.6 fa43addbb5 docs(role): rename readme.md to readme_base.md and update link
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 12:49:31 +02:00
Environment TestandClaude Sonnet 4.6 3e22dc2844 refactor(docker): migrate 3 remaining Dockerfile.j2 without control-flow logic
- svc-db-mariadb: replace Dockerfile.j2 with files/Dockerfile using ARG declarations;
  add MARIADB_IMAGE/MARIADB_VERSION to compose args block
- web-svc-libretranslate, web-app-prometheus: delete empty Dockerfile.j2 files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 12:43:19 +02:00
Environment TestandClaude Sonnet 4.6 f7eb675432 feat(lint): fail on Dockerfile.j2 without Jinja2 control-flow logic; add dockerfile.md
- test_dockerfile_templates.py: emit warning for all Dockerfile.j2 files;
  fail for files without {% %} control-flow logic (must migrate to files/Dockerfile)
- Add docs/contributing/code/role/dockerfile.md as SPOT for Dockerfile placement,
  ARG variable handling, and lint behaviour

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 12:41:41 +02:00
Environment TestandClaude Sonnet 4.6 1c026ce6ef fix(docker): remove quoted FROM and fix ENTRYPOINT ARG expansion in 4 roles
- Remove quotes from FROM instruction in confluence, jira, suitecrm, espocrm Dockerfiles
  (Docker does not strip quotes — quoted image references are invalid)
- Promote ENTRYPOINT_SCRIPT_DOCKER ARG to ENV in espocrm and suitecrm so the
  runtime shell can resolve the path (ARG is not expanded in ENTRYPOINT exec form)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 12:40:28 +02:00
Environment TestandClaude Sonnet 4.6 f1b7825455 docs(roles): add role README standard and refactor affected READMEs
Add docs/contributing/code/role/readme.md defining the required section
order, formatting rules, and the explicit prohibition of emojis in role
README headings. Reference the new guide from documentation.md.

Apply the standard to the four roles touched in the previous commit:
remove emojis from all headings, fix duplicate paragraph in WordPress,
correct typo in OpenProject compose template (crom/cental → from/central),
and align Credits sections to the canonical format.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 12:13:36 +02:00
Environment TestandClaude Sonnet 4.6 30f3b88f38 refactor(docker): replace Dockerfile.j2 with plain files/Dockerfile for 4 roles
Replace templated Dockerfiles with plain Dockerfiles for web-app-littlejs,
web-app-openproject, web-app-roulette-wheel, and web-app-wordpress.
Variables are passed as Docker ARG without defaults, keeping config/main.yml
and vars/main.yml as the single source of truth. Build args are declared
in compose.yml.j2 directly after the build.yml.j2 lookup. Image and version
for roulette-wheel and littlejs are added to config/main.yml and exposed
via vars/main.yml.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 11:56:15 +02:00
Environment TestandClaude Sonnet 4.6 bfc22d3ce1 docs(workflow): rewrite as prose, apply RFC 2119, add repository configuration section
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 11:33:05 +02:00
Environment TestandClaude Sonnet 4.6 b1f2ccd471 fix(mirror): include link to ghcr.md in GHCR_PAT warning annotation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 11:27:18 +02:00
Environment TestandClaude Sonnet 4.6 eb1b20be30 fix(mirror): use GHA warning annotation and document GHCR_PAT requirement for personal accounts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 11:25:37 +02:00
Environment Test 4de93a285c lint 2026-04-09 10:57:01 +02:00
Environment TestandClaude Sonnet 4.6 ff277c88d9 fix(mirror): emit clear warning when GITHUB_TOKEN cannot list user packages
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 10:50:09 +02:00
Environment TestandClaude Sonnet 4.6 b99e77beeb fix(mirror): pass explicit visibility param for user/packages (required by GITHUB_TOKEN)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 10:43:35 +02:00
Environment TestandClaude Sonnet 4.6 8ffe57da4f fix(mirror): use /user/packages endpoint for personal accounts (GITHUB_TOKEN incompatible with /users/{name}/packages)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 10:38:07 +02:00
Environment TestandClaude Sonnet 4.6 3ea40bbfc5 fix(mirror): fall back to user endpoint when org endpoint returns 404
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 10:25:58 +02:00
Environment TestandClaude Sonnet 4.6 a721e87212 fix(tests): replace lint-based pre-commit test with a proper failing unit test
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 02:33:22 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 e84bb3fe06 fix(env): unset cached env vars before reload and guard commit test against dirty worktree
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 02:08:39 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 434aa81ee2 fix(tests): ensure git safe.directory is configured before each environment step
Add ensure_git_safe_directory helper to lib.sh that detects dubious
ownership warnings in mounted containers and adds safe.directory to
the global git config. Call it on lib load and before each step in
the orchestrator.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 01:05:11 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 eee3a4487c fix(tests): update shopware integration test for files/Dockerfile migration
The test was still pointing to templates/Dockerfile.j2 and checking for
Jinja2 syntax. Update path to files/Dockerfile and assertion to ARG/Docker
variable syntax after the Dockerfile.j2 → files/Dockerfile refactor.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 00:52:39 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 3f9e3b0179 docs(agents): clarify --no-verify scope and apply RFC 2119 keywords
Permission to use --no-verify is single-use per commit. Generic commit
requests MUST NOT qualify. Applied MUST/MUST NOT conventions consistently.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 00:36:14 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 f5a81736cf refactor(roles): convert Dockerfile.j2 templates to plain files/Dockerfile
Replace Jinja2 Dockerfile templates with static Dockerfiles using Docker
ARG statements. Variables are now passed as build args in compose.yml.j2,
following the existing pattern from web-app-bridgy-fed. Affects 12 roles:

- svc-db-openldap
- web-app-bookwyrm
- web-app-confluence
- web-app-espocrm
- web-app-jira
- web-app-joomla
- web-app-mediawiki
- web-app-pretix
- web-app-shopware
- web-app-suitecrm
- web-app-xwiki
- web-svc-simpleicons

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 00:28:37 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 9eef2a8ce4 chore(deps): update Docker image versions
- baserow: 2.1.6 → 2.2.0
- funkwhale/front + api: 2.0.0 → 2.0.1
- mattermost: 11.5.1 → 11.6.0
- prometheus: v3.2.1 → v3.11.1
- shopware/opensearch: 3.5.0 → 3.6.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 00:15:56 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 f4b116224e fix(ci): enable CodeQL file coverage on pull requests
Opts out of the April 2026 default change that skips file coverage
computation on PRs by setting CODEQL_ACTION_FILE_COVERAGE_ON_PRS=true.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 00:11:11 +02:00
kevinveenbirkenbach bbaf119640 split git bootstrap into dedicated environment step 2026-04-08 23:42:09 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 70055f7861 refactor(mirror): extract visibility logic into standalone publish module
Move _set_public/ensure_public out of GHCRProvider into a new
cli.mirror.publish module that lists all non-public mirror packages
via the GitHub API and sets them public in bulk. The workflow runs
this as a separate step after mirroring. Mirror sync itself no longer
touches visibility.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 23:19:15 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 813725b412 fix(ci): add push trigger to CodeQL workflow for Security tab visibility
CodeQL results only appear on the Security tab when the workflow is
triggered by a push event directly. Adds push trigger on main alongside
the existing workflow_call (double run on main push is harmless).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 22:57:26 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 c48ee9ddbf docs(security): add GHCR authentication setup guide and cross-links
Document PAT creation, secret configuration, fork PR restrictions,
and troubleshooting. Link from ci.md and repository.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 22:39:23 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 9588f4baf6 test(ci): add debug auth mode to pull_request_target job
Add debug step to prepare-fork-merge-ref job which runs exclusively
via pull_request_target to verify secrets are passed in that context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 22:25:36 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 9a2878e109 refactor(ci): extract debug auth mode step into separate script
Move inline shell from entry-pull-request-change.yml into
scripts/meta/debug/auth_mode.sh and pass secrets via env vars.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 22:08:34 +02:00
kevinveenbirkenbach c97f1d84e0 fix(install): ensure venv python is used for pip and pre-commit
Reload env layer in install scripts after venv creation so `PYTHON`
points to the virtualenv instead of the system interpreter.

* Reload env in `python.sh` before installing dependencies
* Reload env in `pre-commit.sh` before installing hooks
* Reset env cache flags to force re-evaluation
* Keep centralized environment resolution (no hardcoding)

Fixes PEP 668 `externally-managed-environment` errors and missing
`pre_commit` module during hook installation.
2026-04-08 21:21:37 +02:00
kevinveenbirkenbach b0b8874bad feat(python): reload env after venv creation to ensure pip uses virtualenv
Fixes an issue where `PYTHON` could still point to the system interpreter
when `scripts/install/python.sh` runs after `install-venv`.

* Reload env layer if venv exists and PYTHON is not yet pointing to it
* Reset env caching flags to force re-evaluation
* Preserve central environment resolution logic (no hardcoding)

This resolves Debian/PEP 668 `externally-managed-environment` errors during pip install.
2026-04-08 20:30:35 +02:00
kevinveenbirkenbach db44c998c2 fix codeql workflow coverage 2026-04-08 18:14:10 +02:00
kevinveenbirkenbach 3dc243fb0d fix dependabot coverage and environment test scripts 2026-04-08 18:01:42 +02:00
Prageeth Panicker 73e74d0caf feat(web-app-decidim): add Decidim participatory democracy platform
- Add new Ansible role web-app-decidim
- Register port 8067 in group_vars/all/11_ports.yml
- Supports PostgreSQL, Redis, Sidekiq background workers
- OIDC SSO via Keycloak integration
- NGINX reverse proxy via sys-stk-full
- Health checks and resource limits configured
- E2E Playwright tests included
- Lifecycle: beta
2026-04-08 15:57:15 +00:00