Commit Graph
829 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbach 3f626a71c1 Merge branch 'main' into feature/service-refactoring 2026-04-14 17:05:51 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 1bcb7cfbb0 Merge branch 'main' into feature/service-refactoring
Resolve conflicts:
- docs/agents/action/iteration.md: keep both new bullets (SERVICES_DISABLED
  proposal + make test requirement)
- docs/contributing/design/services.md: deleted (renamed to
  docs/contributing/code/services.md in this branch)
- group_vars/all/20_services.yml: keep deletion from this branch (req-001)
- tests/lint/ansible/roles/test_service_core_first_task_run_once.py:
  use build_service_registry_from_roles_dir (req-001) with repo_root()
  from main for robust path resolution

Fix broken relative links caused by rename and deletion of files:
- docs/contributing/design/architecture.md: services.md → ../code/services.md
- docs/contributing/design/exceptions.md: load_frontend_service.yml →
  load_service.yml
- docs/requirements/001-service-registry-refactoring.md: fix paths for
  unit.md, integration.md, principles.md, documentation.md, AGENTS.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:58:51 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 be6a310d59 refactor(services): implement role-local service registry (req-001)
- Delete group_vars/all/20_services.yml; all service metadata now lives
  in each role's config/main.yml under compose.services.<name>
- Rename generic 'database' service key to explicit 'mariadb'/'postgres'
  across all app roles; remove legacy type: field
- Add utils/service_registry.py and plugins/lookup/service_registry.py
  as the single source of truth for service discovery
- Add utils/database_service.py to centralize DB service resolution
- Delete service_load_enabled filter and service_should_load lookup;
  replaced by service lookup plugin
- Update sys-utils-service-loader to run in constructor stage and use
  generic load_service.yml; remove load_frontend_service.yml
- Add _service_loader_preloading guard to sys-svc-compose,
  sys-svc-webserver-core and sys-ctl-alm-compose to prevent
  health-checks/alarms from firing during preload phase
- Replace manual load_app.yml calls for shared DB/Mailu with assert-
  based preload validation in sys-svc-rdbms and sys-svc-mail
- Update networks.yml.j2 templates to resolve LDAP/AI networks via
  service_registry instead of hardcoded role names
- Migrate all tests to new service discovery API; delete obsolete tests
- Update docs/contributing/code/services.md to reflect role-local model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:33:20 +02:00
Environment TestandClaude Sonnet 4.6 053a13d631 fix(utils): remove unused utils/gha compatibility shim
No code references utils.gha anywhere. Removing resolves CodeQL
py/unused-import alert #262.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 11:55:21 +02:00
Environment TestandClaude Sonnet 4.6 eebb3b9788 refactor(utils): move utils/docker/image_*.py into utils/docker/image/ subpackage
- Move image_discovery.py → utils/docker/image/discovery.py
- Move image_ref.py → utils/docker/image/ref.py
- Add utils/docker/image/__init__.py
- Update all imports and references across cli/, tests/, utils/, and docs/

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 19:00:02 +02:00
Environment TestandClaude Sonnet 4.6 bf8c535988 feat(update): add docker image version updater CLI, workflow, and lint tests
- Add utils/docker/version_updater.py with CLI for updating Docker image versions
- Add cli/fix/docker_image_versions module wrapping the updater
- Add .github/workflows/update.yml replacing update-skills.yml
- Add scripts/github/ helpers: check_git_changes, install_update_python, open_update_pr, update_docker_image_versions
- Add tests/lint/filesystem/ for scripts/ and utils/ folder file-type enforcement
- Add tests/lint/ansible/ for legacy module usage and vars usage checks
- Reorganize tests/lint/repository/ and move test_vars_usage_in_yaml.py to ansible/
- Add artefact docs: files/github/workflow.md, scripst.md, utils.md
- Apply documentation.md standards to all modified markdown files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 18:38:39 +02:00
Environment TestandClaude Sonnet 4.6 aa33256cde feat(mirror): split resolver output, move images to defaults, add lookup plugin
- resolver: output both applications: (config/main.yml compose images) and
  images: (defaults/main.yml images) keyed by source_file
- image_discovery: scan defaults/main.yml instead of vars/main.yml for images:
- mirror_overrides: write mirror URLs to applications.*.compose.services.*
  (mirror_policy-aware) and images_overrides.*.* (always merged, if_missing)
- roles/sys-ctl-hlth-csp, roles/test-e2e-playwright: move images: from
  vars/main.yml to defaults/main.yml for correct Ansible variable precedence
- plugins/lookup/image.py: new Ansible lookup plugin that prefers
  images_overrides over role-local images defaults
- docs: add artefact/image.md and artefact/mirror.md as SPOTs; add
  code/role/image.md contributor guide; link from ci.md
- tests: update test_util, test_mirror_overrides; add test_resolver,
  test_publish (404/401 coverage), test_image (lookup plugin)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 11:17:42 +02:00
Environment TestandClaude Sonnet 4.6 317287983a fix(mirror): resolve merge conflicts with fix/playwright-pull
Keep new multi-registry logic (feature/pre-commit):
- Merge is_mirrorable_image(), _detect_registry(), image_source() and
  canonical_image_name() into utils/docker/image_discovery.py
- iter_role_images() now scans both config/main.yml and vars/main.yml
  and uses image_source() for full registry-prefixed URLs
- Delete cli/mirror/util.py (superseded by utils/docker/image_discovery)
- model.py re-exports ImageRef from utils.docker.image_discovery
- sync/__main__.py: revert fix/playwright-pull's if/else src URL to
  simple `src = f"docker://{img.source}"` (source already contains
  the full registry prefix)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 23:44:58 +02:00
Environment TestandClaude Sonnet 4.6 9ea2ad6ff1 refactor(docker): introduce utils/docker/image_discovery as single SPOT for image enumeration
- Move iter_role_images + all helpers + ImageRef from cli/mirror/util.py and
  cli/mirror/model.py into utils/docker/image_discovery.py; add source_file field
  to ImageRef to distinguish config/main.yml vs vars/main.yml entries
- Delete cli/mirror/util.py; update all callers to import from utils.docker.image_discovery
- Move utils into subpackages: utils/docker/ (image_ref, paths_utils, hadolint) and
  utils/roles/ (dependency_resolver, resource_validation); update all imports
- Reorganise tests into subpackages: tests/lint/docker/dockerfile/, tests/integration/docker/,
  tests/unit/utils/docker/, tests/unit/utils/roles/; fix parents[N] offsets after relocation
- tests/lint/docker/test_image_versions.py now uses iter_role_images (same SPOT as mirror)
- Add vars/main.yml images.<name>.{image,version} support for non-application roles
  (test-e2e-playwright, sys-ctl-hlth-csp); eliminate hardcoded duplicates in defaults/tasks
- Add web-app-matrix matrix-chatgpt-bot and web-app-mini-qr mini-qr entries in config/main.yml;
  replace hardcoded image strings in vars/templates with lookup('config', ...) references
- Fix sync/__main__.py log line to use correct registry prefix for non-Docker-Hub images
- Update docs/contributing/code/role/dockerfile.md with image declaration rules and SPOT reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 19:32:31 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 0ce6b1347c refactor(annotations): move to utils/annotations, add step summary support
- Move utils/gha/annotations.py → utils/annotations/message.py
- Move scripts/gha/summarize_annotations.py → utils/annotations/summarize.py
- Keep utils/gha/annotations.py as compatibility shim
- Update all imports across codebase to utils.annotations.message
- Wire tee + summary step into test-code-lint, unit, and integration workflows
- Add unit tests for utils/annotations/summarize (20 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 14:05:24 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 e1966c16da refactor(gha): centralize annotation helpers in utils/gha/annotations.py
Replace copy-pasted ::warning prints and _gha_escape helpers across lint
tests and utils with a shared utility that emits GHA workflow commands in
CI and plain [WARNING]/[ERROR]/[NOTICE] lines locally. Update
test_role_resource_validation to capture stdout instead of stderr.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 12:32:06 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 5e970ff5bd feat(gha): introduce utils/gha/annotations for GitHub Actions workflow annotations
- Add utils/gha/annotations.py with warning/error/notice helpers and *_each
  variants that emit one annotation per list item
- Use warning() in GHCRProvider._set_public for missing GHCR_PAT case
- Expose GHCR_PAT_SET env var in mirror step to distinguish missing-PAT from
  real API failures
- Add unit tests for all annotation functions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 10:59:46 +02:00
kevinveenbirkenbach c3fb04e91a refactor: share application dependency resolution 2026-04-01 15:13:04 +02:00
kevinveenbirkenbach 2b0ed2eacf Merge branch 'feature/pixelfed-playwright' into feature/taiga-ldap 2026-03-28 09:22:30 +01:00
kevinveenbirkenbach 827c12c903 refactor(ci): extract hadolint SARIF check into utils 2026-03-26 16:21:55 +01:00
kevinveenbirkenbach 2f0edbb2fc Fix oauth2 proxy cookie secret generation 2026-03-26 01:15:08 +01:00
kevinveenbirkenbach 66bbffe963 Fix docker image reference scan hints 2026-03-25 19:24:53 +01:00
kevinveenbirkenbach 289a05ac13 Merge branch 'main' into feature/lint 2026-03-23 10:23:08 +01:00
kevinveenbirkenbach 2e37a3689f Merge branch 'main' into feature/lint
# Conflicts:
#	Dockerfile
#	scripts/administration/ghcr/cleanup_ci_images.sh
#	scripts/meta/wait/image.sh
2026-03-22 21:51:23 +01:00
kevinveenbirkenbach d8218babfc Fix DNS hosts fallback and domains CLI 2026-03-21 09:12:24 +01:00
kevinveenbirkenbach 189267496d Rename module_utils package to utils 2026-03-21 02:05:39 +01:00
kevinveenbirkenbach 562603a8cd Restructured libraries 2025-07-17 16:38:20 +02:00
kevinveenbirkenbach 2d276cfa5e Optimized Dockerfile and github workflow 2025-07-16 09:38:19 +02:00
kevinveenbirkenbach 241c5c6da8 Optimized test_main.py 2025-07-16 08:25:16 +02:00
kevinveenbirkenbach ac72544b72 Added validation for deploy application ids 2025-07-14 09:52:53 +02:00
kevinveenbirkenbach f07557c322 Implemented dict renderer to resolve assets 2025-07-12 23:25:31 +02:00
kevinveenbirkenbach d43fdc63ea Optimized inventory validator for wrong groups 2025-07-11 01:34:44 +02:00
kevinveenbirkenbach c160c58a5c Restructured CLI logic 2025-07-10 21:26:44 +02:00