Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- plugins/lookup/config.py: honor default by setting skip_missing_app=not strict
so lookup('config', app, path, default) returns the default when the app is
missing from the applications dict, instead of raising during early constructor
phases. Fixes 5 test-development jobs.
- roles/web-app-nextcloud/files/playwright.spec.js: replace unsupported
.toSatisfy() with a boolean check inside expect.poll() + .toBe(true).
Fixes Deploy web-app-nextcloud.
- roles/sys-svc-container/files/container.py: when the target image has no
ENTRYPOINT, inject the internal CA as a volume plus NODE_EXTRA_CA_CERTS,
SSL_CERT_FILE, REQUESTS_CA_BUNDLE and CURL_CA_BUNDLE instead of falling
back to a plain 'docker run' without CA trust. Fixes npm
UNABLE_TO_VERIFY_LEAF_SIGNATURE inside Microsoft Playwright images for
moodle, prometheus and onlyoffice deploys.
- tests/unit/roles/sys-svc-container/files/test_container.py: cover the new
no-ENTRYPOINT CA-injection path.
Note: web-app-mini-qr 'lookup ghcr.io: no such host' is not addressed here
and likely needs a separate investigation of CoreDNS forwarding on the CI
runner.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds inventory consistency guard that fails fast when SERVICES_DISABLED
conflicts with an already-deployed inventory, and ensures the env var is
forwarded through the dedicated runner, development exec command, and
local test scripts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add 07_admin_identities.yml: resolves LDAP-backed admin identities
for OCC calls that require the actual NC username, not the LDAP DN
- vars: add NEXTCLOUD_HPB_TURN/STUN_ACTIVE_CONFIG selecting standalone
or onboard TURN based on enabled flags; add TALK_ADMIN_URL
- config: talk container gated on spreed plugin flag
- env.j2 + playwright.env.j2: expose active TURN/STUN config and
Talk admin URL for container env and E2E tests
- playwright.spec.js: extend with Talk TURN/STUN E2E assertions
- schema: add spreed.enabled validation
- coturn compose: move relay port range before TLS block so ports are
always set regardless of TLS mode
- docs/requirements: document TURN relay requirement
- test_resolver: cover new resolution paths
Resolve conflicts:
- docs/agents/action/iteration.md: keep both new bullets (SERVICES_DISABLED
proposal + make test requirement)
- docs/contributing/design/services.md: deleted (renamed to
docs/contributing/code/services.md in this branch)
- group_vars/all/20_services.yml: keep deletion from this branch (req-001)
- tests/lint/ansible/roles/test_service_core_first_task_run_once.py:
use build_service_registry_from_roles_dir (req-001) with repo_root()
from main for robust path resolution
Fix broken relative links caused by rename and deletion of files:
- docs/contributing/design/architecture.md: services.md → ../code/services.md
- docs/contributing/design/exceptions.md: load_frontend_service.yml →
load_service.yml
- docs/requirements/001-service-registry-refactoring.md: fix paths for
unit.md, integration.md, principles.md, documentation.md, AGENTS.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Delete group_vars/all/20_services.yml; all service metadata now lives
in each role's config/main.yml under compose.services.<name>
- Rename generic 'database' service key to explicit 'mariadb'/'postgres'
across all app roles; remove legacy type: field
- Add utils/service_registry.py and plugins/lookup/service_registry.py
as the single source of truth for service discovery
- Add utils/database_service.py to centralize DB service resolution
- Delete service_load_enabled filter and service_should_load lookup;
replaced by service lookup plugin
- Update sys-utils-service-loader to run in constructor stage and use
generic load_service.yml; remove load_frontend_service.yml
- Add _service_loader_preloading guard to sys-svc-compose,
sys-svc-webserver-core and sys-ctl-alm-compose to prevent
health-checks/alarms from firing during preload phase
- Replace manual load_app.yml calls for shared DB/Mailu with assert-
based preload validation in sys-svc-rdbms and sys-svc-mail
- Update networks.yml.j2 templates to resolve LDAP/AI networks via
service_registry instead of hardcoded role names
- Migrate all tests to new service discovery API; delete obsolete tests
- Update docs/contributing/code/services.md to reflect role-local model
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add image_version filter plugin to extract bare semver from image tag
- Derive TEST_E2E_PLAYWRIGHT_VERSION via image_version filter
- Convert files/package.json to templates/package.json.j2 using the
derived version so image tag and devDependency stay in sync
- Use lookup('image', 'playwright', 'ref') in run_one.yml instead of
building the image ref manually from package.json
- Add unit tests for image_version filter
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move sys, unittest.mock and main to top-level imports. Replace inline
module imports and mod.main() calls with the already-imported main().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Apply documentation.md rules across staged files (emojis after headlines, RFC 2119 keywords, correct link text, Credits format)
- Remove expired ChatGPT share URLs from 39+ role READMEs and templates
- Remove orphaned text after link deletion: bare # @see, {# @see #}, Discussion: #}, empty Chat Conversation sections, dangling "Conversation is here available.", "Refer to the conversation with ChatGPT ()", "See ChatGPT Conversation", "ChatGPT Session Transcript" references
- Remove empty ## Further Resources and ## Credits sections where no links remain
- Fix URL reachability test: treat 401/403/405 as ok, 429/500/503 as warn, Timeout as warn; add # nocheck: url suppression (inline and line-above)
- Add lint test for broken relative and repo-root-absolute markdown links
- Add # nocheck: url to AUR git clone URLs, CSP allowlist entries, and other non-browser URLs
- Restore docs.infinito.nexus/setup in documentation.md MUST NOT example
- Fix Mobilizon geospatial endpoints (Addok, Photon) reverted to original values with # nocheck: url
- Add integration test rule: MUST NOT write unit tests for integration test logic
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Change the URL reachability test to fail on 4xx, non-500 5xx, and
request errors (DNS, connection). Only HTTP 500 now emits a warning
annotation, distinguishing temporary upstream breakage from dead links.
Add a unit test for the probe classification logic.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move emojis after headline text (CONTRIBUTING.md, actions/, design/, tools/, tests/)
- Replace em dashes with colons, semicolons, or complete sentences throughout
- Add missing emojis to all headlines in exceptions.md, principles.md, services.md, shell.md, setup.md, pull-request.md, commit.md, act.md, docker.md, pr/README.md
- Update H1 titles to reflect renamed directories (actions/, design/)
- Fix broken relative links in workflow.md after directory moves
- Fix full-path link texts in services.md (style.css.md, javascript.js.md)
- Fix broken principles.md link to lint.md after testing/ → actions/testing/ move
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds cli.mirror.cleanup for deleting GHCR container packages by prefix
and visibility via the GitHub Packages REST API. Supports dry-run mode.
Requires a classic PAT with delete:packages scope (GITHUB_TOKEN from
gh auth or GitHub Actions does not include this scope).
Also adds unit tests and a usage guide at
docs/contributing/flow/security/mirror-cleanup.md.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces GHCR_PAT-based authentication with GITHUB_TOKEN across all
mirror and build workflows. Packages pushed from a public repository
with GITHUB_TOKEN are automatically linked to the repository and
inherit its public visibility — no publish step required.
Removes cli.mirror.publish (used a non-existent GitHub REST API
endpoint for changing package visibility), the associated unit tests,
and the GHCR_PAT debug helper. Cleans up all GHCR_PAT/GHCR_USERNAME
references and rewrites the GHCR authentication documentation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Image names now follow {owner}/{repository}/mirror/{registry}/{name}
instead of {owner}/mirror/{registry}/{name}.
- GHCRProvider: add `repository` param + `add_args()`/`from_args()` classmethods
as SPOT for CLI argument registration and provider construction
- sync, resolver: use GHCRProvider.add_args() / from_args() (no duplication)
- Workflows: resolve repository name via scripts/meta/resolve/repository/name.sh
and pass --ghcr-repository to sync and publish
- mirrors.sh: require and forward GHCR_REPOSITORY env var
- deploy/development/mirrors.py: derive repository from GITHUB_REPOSITORY
- Tests: update expected paths, argv and env fixtures
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep new multi-registry logic (feature/pre-commit):
- Merge is_mirrorable_image(), _detect_registry(), image_source() and
canonical_image_name() into utils/docker/image_discovery.py
- iter_role_images() now scans both config/main.yml and vars/main.yml
and uses image_source() for full registry-prefixed URLs
- Delete cli/mirror/util.py (superseded by utils/docker/image_discovery)
- model.py re-exports ImageRef from utils.docker.image_discovery
- sync/__main__.py: revert fix/playwright-pull's if/else src URL to
simple `src = f"docker://{img.source}"` (source already contains
the full registry prefix)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add `registry` field to `ImageRef` (default `"docker.io"`)
- Populate `registry` from `_detect_registry()` in `iter_role_images()`
- `GHCRProvider.image_base()` and `GiteaProvider.image_base()` now produce
`{prefix}/{registry}/{name}` paths (e.g. `mirror/docker.io/nextcloud`,
`mirror/quay.io/keycloak/keycloak`) instead of flattening slashes to dashes
- Fix `mirror()` in both providers: use `docker://{image.source}` directly
instead of always prepending `docker://docker.io/`, which broke quay.io/ghcr.io images
- Update tests to reflect new naming convention and add quay.io test case
- Add missing __init__.py files in test subdirectories
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extend image mirroring to cover quay.io and ghcr.io in addition to Docker Hub,
and make the publish step non-fatal when the token lacks write:packages scope.
cli/mirror/util.py:
- Add _detect_registry(), _strip_registry_prefix(), is_mirrorable_image()
to support Docker Hub, quay.io, ghcr.io, and mcr.microsoft.com
- Update canonical_image_name() to strip any known registry prefix so that
quay.io/keycloak/keycloak -> keycloak/keycloak (mirror/keycloak-keycloak)
- Add image_source() returning full registry-prefixed pull reference
(docker.io/library/postgres:16, quay.io/keycloak/keycloak:latest, …)
- iter_role_images() now uses is_mirrorable_image + image_source, picking up
the 17 previously skipped quay.io/ghcr.io images
cli/mirror/sync/__main__.py:
- Use docker://{img.source} directly (no hardcoded docker.io/ prefix)
cli/mirror/publish/__main__.py:
- Detect HTTP 404 on PATCH as _InsufficientTokenError (no retry, graceful
GHA warning + exit 0) instead of retrying 5x and failing the workflow
- Use pkg.get("url") from listing response as canonical PATCH target URL
- Use safe="/" in urllib.parse.quote so mirror/name stays as literal path
- Do not retry on non-transient 4xx errors (400/401/403)
tests/unit/cli/mirror/test_sync.py:
- Update ImageRef.source to new docker.io/library/… format
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move iter_role_images + all helpers + ImageRef from cli/mirror/util.py and
cli/mirror/model.py into utils/docker/image_discovery.py; add source_file field
to ImageRef to distinguish config/main.yml vs vars/main.yml entries
- Delete cli/mirror/util.py; update all callers to import from utils.docker.image_discovery
- Move utils into subpackages: utils/docker/ (image_ref, paths_utils, hadolint) and
utils/roles/ (dependency_resolver, resource_validation); update all imports
- Reorganise tests into subpackages: tests/lint/docker/dockerfile/, tests/integration/docker/,
tests/unit/utils/docker/, tests/unit/utils/roles/; fix parents[N] offsets after relocation
- tests/lint/docker/test_image_versions.py now uses iter_role_images (same SPOT as mirror)
- Add vars/main.yml images.<name>.{image,version} support for non-application roles
(test-e2e-playwright, sys-ctl-hlth-csp); eliminate hardcoded duplicates in defaults/tasks
- Add web-app-matrix matrix-chatgpt-bot and web-app-mini-qr mini-qr entries in config/main.yml;
replace hardcoded image strings in vars/templates with lookup('config', ...) references
- Fix sync/__main__.py log line to use correct registry prefix for non-Docker-Hub images
- Update docs/contributing/code/role/dockerfile.md with image declaration rules and SPOT reference
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move _set_public/ensure_public out of GHCRProvider into a new
cli.mirror.publish module that lists all non-public mirror packages
via the GitHub API and sets them public in bulk. The workflow runs
this as a separate step after mirroring. Mirror sync itself no longer
touches visibility.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move utils/gha/annotations.py → utils/annotations/message.py
- Move scripts/gha/summarize_annotations.py → utils/annotations/summarize.py
- Keep utils/gha/annotations.py as compatibility shim
- Update all imports across codebase to utils.annotations.message
- Wire tee + summary step into test-code-lint, unit, and integration workflows
- Add unit tests for utils/annotations/summarize (20 tests)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace copy-pasted ::warning prints and _gha_escape helpers across lint
tests and utils with a shared utility that emits GHA workflow commands in
CI and plain [WARNING]/[ERROR]/[NOTICE] lines locally. Update
test_role_resource_validation to capture stdout instead of stderr.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add utils/gha/annotations.py with warning/error/notice helpers and *_each
variants that emit one annotation per list item
- Use warning() in GHCRProvider._set_public for missing GHCR_PAT case
- Expose GHCR_PAT_SET env var in mirror step to distinguish missing-PAT from
real API failures
- Add unit tests for all annotation functions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
/users/{username}/packages does not support PATCH — it is a read-only endpoint
for public data. The correct endpoint for updating visibility of the authenticated
user's packages is /user/packages (without username).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Root cause: GITHUB_TOKEN in Actions is repo-scoped and cannot change package
visibility in user/org namespaces — all 58 mirror operations failed with 404.
- Use GHCR_PAT (which has write:packages scope) as GITHUB_TOKEN in the mirror
step, falling back to GITHUB_TOKEN if GHCR_PAT is not set
- Remove ensure_public() call from the already-exists skip path (wrong place:
existing images were already made public when first mirrored)
- Update test to assert ensure_public is not called for skipped images
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove redundant `images-mirror-missing` job from entry-pull-request-change.yml
(mirroring already runs via ci-orchestrator for same-repo PRs)
- Set GHCR package visibility to public after each successful mirror via GitHub API
- Add `ensure_public` hook to base class (no-op for other providers)
- Also call `ensure_public` for already-existing images to fix any that are still private
- Pass GITHUB_TOKEN explicitly to the mirror step
- Add unit tests for _set_public (user/org endpoint fallback) and sync skip-path
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- switch icon.source from remote URLs to local static paths
- add local Simple Icons directory creation on host
- mount local static directory into dashboard container
- introduce DASHBOARD_SIMPLEICONS_* variables for paths and sync base URL
- download required SVGs into local static directory during refresh
- update Playwright test to validate cached image paths and dimensions
- enhance add_simpleicon_source filter:
- return local static SVG paths instead of remote URLs
- validate non-empty local_static_dir
- update unit tests for new behavior and error handling
- remove dependency on TLS lookup for Simple Icons base URL
Replace stale unit test README with a pointer to the canonical unit.md guide.
Add index README to tests/ and per-directory READMEs for e2e, integration,
lint, and regression — each linking to the matching docs/contributing page.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eliminates scope-dependent CDN variable bugs by introducing a `cdn` lookup
plugin that resolves paths and URLs at call-time using an explicit application_id
argument. Deletes the old cdn_paths.py and cdn_urls.py filter plugins and their
tests; adds a new unit test for the lookup plugin's _cdn_paths and _to_url_tree
helpers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces a SPOT for all frontend service loading in sys-front-inj-all/tasks/01_services.yml,
driven by a new generic `service` lookup plugin that resolves services by key or role name
and returns {id, role, enabled, shared, needed}. Needed = enabled AND shared (direct or
transitive). The canonical key→role mapping moves to group_vars/all/20_services.yml
(previously scattered hardcoded rules). Removes per-role service loads from bigbluebutton,
nextcloud, and dashboard. Renames 01_dependencies.yml→01_services.yml and
08_services.yml→08_daemons.yml. Adds flush_handlers after each service load.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
