Commit Graph
367 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbach e9f95318e7 Merge branch 'infinito-nexus:main' into fix/nextcloud-talk 2026-04-15 01:46:38 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 f1ea036bd2 fix(ci): resolve test-development, nextcloud, and container CA failures
- plugins/lookup/config.py: honor default by setting skip_missing_app=not strict
  so lookup('config', app, path, default) returns the default when the app is
  missing from the applications dict, instead of raising during early constructor
  phases. Fixes 5 test-development jobs.

- roles/web-app-nextcloud/files/playwright.spec.js: replace unsupported
  .toSatisfy() with a boolean check inside expect.poll() + .toBe(true).
  Fixes Deploy web-app-nextcloud.

- roles/sys-svc-container/files/container.py: when the target image has no
  ENTRYPOINT, inject the internal CA as a volume plus NODE_EXTRA_CA_CERTS,
  SSL_CERT_FILE, REQUESTS_CA_BUNDLE and CURL_CA_BUNDLE instead of falling
  back to a plain 'docker run' without CA trust. Fixes npm
  UNABLE_TO_VERIFY_LEAF_SIGNATURE inside Microsoft Playwright images for
  moodle, prometheus and onlyoffice deploys.

- tests/unit/roles/sys-svc-container/files/test_container.py: cover the new
  no-ENTRYPOINT CA-injection path.

Note: web-app-mini-qr 'lookup ghcr.io: no such host' is not addressed here
and likely needs a separate investigation of CoreDNS forwarding on the CI
runner.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 23:07:23 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 1bcb7cfbb0 Merge branch 'main' into feature/service-refactoring
Resolve conflicts:
- docs/agents/action/iteration.md: keep both new bullets (SERVICES_DISABLED
  proposal + make test requirement)
- docs/contributing/design/services.md: deleted (renamed to
  docs/contributing/code/services.md in this branch)
- group_vars/all/20_services.yml: keep deletion from this branch (req-001)
- tests/lint/ansible/roles/test_service_core_first_task_run_once.py:
  use build_service_registry_from_roles_dir (req-001) with repo_root()
  from main for robust path resolution

Fix broken relative links caused by rename and deletion of files:
- docs/contributing/design/architecture.md: services.md → ../code/services.md
- docs/contributing/design/exceptions.md: load_frontend_service.yml →
  load_service.yml
- docs/requirements/001-service-registry-refactoring.md: fix paths for
  unit.md, integration.md, principles.md, documentation.md, AGENTS.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:58:51 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 be6a310d59 refactor(services): implement role-local service registry (req-001)
- Delete group_vars/all/20_services.yml; all service metadata now lives
  in each role's config/main.yml under compose.services.<name>
- Rename generic 'database' service key to explicit 'mariadb'/'postgres'
  across all app roles; remove legacy type: field
- Add utils/service_registry.py and plugins/lookup/service_registry.py
  as the single source of truth for service discovery
- Add utils/database_service.py to centralize DB service resolution
- Delete service_load_enabled filter and service_should_load lookup;
  replaced by service lookup plugin
- Update sys-utils-service-loader to run in constructor stage and use
  generic load_service.yml; remove load_frontend_service.yml
- Add _service_loader_preloading guard to sys-svc-compose,
  sys-svc-webserver-core and sys-ctl-alm-compose to prevent
  health-checks/alarms from firing during preload phase
- Replace manual load_app.yml calls for shared DB/Mailu with assert-
  based preload validation in sys-svc-rdbms and sys-svc-mail
- Update networks.yml.j2 templates to resolve LDAP/AI networks via
  service_registry instead of hardcoded role names
- Migrate all tests to new service discovery API; delete obsolete tests
- Update docs/contributing/code/services.md to reflect role-local model

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 19:33:20 +02:00
Environment TestandClaude Sonnet 4.6 aa33256cde feat(mirror): split resolver output, move images to defaults, add lookup plugin
- resolver: output both applications: (config/main.yml compose images) and
  images: (defaults/main.yml images) keyed by source_file
- image_discovery: scan defaults/main.yml instead of vars/main.yml for images:
- mirror_overrides: write mirror URLs to applications.*.compose.services.*
  (mirror_policy-aware) and images_overrides.*.* (always merged, if_missing)
- roles/sys-ctl-hlth-csp, roles/test-e2e-playwright: move images: from
  vars/main.yml to defaults/main.yml for correct Ansible variable precedence
- plugins/lookup/image.py: new Ansible lookup plugin that prefers
  images_overrides over role-local images defaults
- docs: add artefact/image.md and artefact/mirror.md as SPOTs; add
  code/role/image.md contributor guide; link from ci.md
- tests: update test_util, test_mirror_overrides; add test_resolver,
  test_publish (404/401 coverage), test_image (lookup plugin)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-10 11:17:42 +02:00
Environment TestandClaude Sonnet 4.6 9ea2ad6ff1 refactor(docker): introduce utils/docker/image_discovery as single SPOT for image enumeration
- Move iter_role_images + all helpers + ImageRef from cli/mirror/util.py and
  cli/mirror/model.py into utils/docker/image_discovery.py; add source_file field
  to ImageRef to distinguish config/main.yml vs vars/main.yml entries
- Delete cli/mirror/util.py; update all callers to import from utils.docker.image_discovery
- Move utils into subpackages: utils/docker/ (image_ref, paths_utils, hadolint) and
  utils/roles/ (dependency_resolver, resource_validation); update all imports
- Reorganise tests into subpackages: tests/lint/docker/dockerfile/, tests/integration/docker/,
  tests/unit/utils/docker/, tests/unit/utils/roles/; fix parents[N] offsets after relocation
- tests/lint/docker/test_image_versions.py now uses iter_role_images (same SPOT as mirror)
- Add vars/main.yml images.<name>.{image,version} support for non-application roles
  (test-e2e-playwright, sys-ctl-hlth-csp); eliminate hardcoded duplicates in defaults/tasks
- Add web-app-matrix matrix-chatgpt-bot and web-app-mini-qr mini-qr entries in config/main.yml;
  replace hardcoded image strings in vars/templates with lookup('config', ...) references
- Fix sync/__main__.py log line to use correct registry prefix for non-Docker-Hub images
- Update docs/contributing/code/role/dockerfile.md with image declaration rules and SPOT reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 19:32:31 +02:00
kevinveenbirkenbach 5db1ff8e7f fix(services): resolve transitive shared asset dependencies 2026-04-01 23:01:09 +02:00
kevinveenbirkenbach c3fb04e91a refactor: share application dependency resolution 2026-04-01 15:13:04 +02:00
kevinveenbirkenbach 00193b4a23 Rename the canonical services map to SERVICE_REGISTRY 2026-03-31 21:43:29 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 c74bd98969 feat(services): introduce canonical field for shared-role service entries
Adds canonical: cdn to css/javascript entries in 20_services.yml so that
role-based reverse lookup (web-svc-cdn → cdn) is deterministic. Updates
_build_role_to_key to use the canonical field. Adds an integration test
verifying bidirectional resolution and a lint test enforcing canonical
consistency rules (exactly one primary, no chaining, matching roles).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 15:58:54 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 84323aa471 refactor(cdn): replace cdn_paths/cdn_urls filter plugins with cdn lookup plugin
Eliminates scope-dependent CDN variable bugs by introducing a `cdn` lookup
plugin that resolves paths and URLs at call-time using an explicit application_id
argument. Deletes the old cdn_paths.py and cdn_urls.py filter plugins and their
tests; adds a new unit test for the lookup plugin's _cdn_paths and _to_url_tree
helpers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 15:10:59 +02:00
kevinveenbirkenbachandClaude Sonnet 4.6 96649d8c9b refactor(services): centralize frontend service loading with generic lookup plugin
Introduces a SPOT for all frontend service loading in sys-front-inj-all/tasks/01_services.yml,
driven by a new generic `service` lookup plugin that resolves services by key or role name
and returns {id, role, enabled, shared, needed}. Needed = enabled AND shared (direct or
transitive). The canonical key→role mapping moves to group_vars/all/20_services.yml
(previously scattered hardcoded rules). Removes per-role service loads from bigbluebutton,
nextcloud, and dashboard. Renames 01_dependencies.yml→01_services.yml and
08_services.yml→08_daemons.yml. Adds flush_handlers after each service load.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 08:27:21 +02:00
kevinveenbirkenbach 189267496d Rename module_utils package to utils 2026-03-21 02:05:39 +01:00
kevinveenbirkenbach 003a3a0b84 Merge branch 'main' into feature/lint
Resolve merge conflicts, keep the cleanup script move to scripts/image, and fix the resulting lint regressions.
2026-03-21 01:36:58 +01:00
kevinveenbirkenbach de555f6bc3 add get_url_retry plugin and migrate get_url usages 2026-03-19 01:58:11 +01:00
kevinveenbirkenbach c6f5fdea15 refactor plugins layout and migrate uri calls to uri_retry 2026-03-19 01:41:07 +01:00