Switch the label that flips a fork PR to the trusted (PR-head, with
credentials) path from 'trusted-pr' to '🛡️ Trusted', in the
pull_request_target gating and the orchestrator text, plus the docs.
The trusted_pr input names are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
actions/checkout now refuses to fetch PR head/merge code under
pull_request_target without an explicit opt-in, which broke the fork
image build and the fork mirror job for every fork PR.
Set allow-unsafe-pr-checkout on the fork source checkouts (new
allow_unsafe_checkout input on images-build-ci.yml and
images-mirror-missing.yml, passed true by the fork producer jobs) so
the producer builds again — untrusted forks from the safe base merge
ref, trusted forks (trusted-pr label) from the PR head.
Harden the untrusted path: the fork build no longer inherits
organization secrets (it authenticates to GHCR with the per-job
github.token only, and the build needs no other secret), and Docker Hub
mirror credentials are passed only for trusted PRs (untrusted forks
mirror anonymously). Switch the GHCR logins to github.token so auth no
longer depends on secrets inheritance.
Docs updated in pipeline.md and authentication.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 'Set up Docker Buildx' step boots BuildKit, which pulls
moby/buildkit from registry-1.docker.io. A transient Docker Hub
timeout there had no retry (only the build step did), so the whole
build job failed at setup.
Pre-pull the BuildKit image with a 7x/20s retry before the buildx
setup, and pin the action to that exact tag via driver-opts. Once the
image is local, buildx falls back to it if the bootstrap pull fails
('pulling failed, using local image'), so a flaky Docker Hub no longer
breaks setup. Applied to both images-build-ci.yml and
release-version.yml.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Consumer lookup plugins now reach the merged config through the Ansible
lookup loader instead of importing get_merged_applications /
get_merged_domains from utils.cache. The merge stays one cached
implementation surfaced via the applications/config/domains/users lookups;
consumers call lookup_loader.get(...).run() (the compose_networks idiom).
- get_merged_applications walks the whole meta/ tree recursively
(meta/**/*.yml to applications.<app>.<path>, addons and any depth) and
folds the raw meta/volumes.yml subtree onto applications.<app>.volumes
after the templar render so the Jinja source: strings stay unrendered.
- 31 consumer lookups rerouted; providers (applications/config/
applications_current_play/users/domains) and the raw-volume accessors
(volume/nginx) keep the cache import behind # nocheck: lookup-cache-import.
- New lint test_no_cache_imports_in_lookups enforces the contract.
- nfs_flavor is now the flavor SPOT (svc-storage-nfs-server + nfs_client_src
resolve via lookup('nfs_flavor')); removed the unused STORAGE_NFS_FLAVOR
group_var. Extracted extract_default_credentials to
utils/cache/schema_credentials.py to keep applications.py under the cap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
self._templar.template('{{ var }}') no-ops on untrusted strings in ansible 2.19+, so the nfs_* lookups emitted literal Jinja into /etc/exports and exportfs failed (the fsid=0 export never came up). The base lookups now read resolved inputs from 'variables'. nfs_flavor/nfs_client_src read the flavor from the applications variable (the variant-baked config provisioning bakes under applications:), not a re-merge of the role files via get_merged_applications. Adds a real-Templar regression test that fails on the old code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Matomo 5.x rejects the SitesManager.getAllSites API call with 401 when only the browser session cookie is sent; it requires token_auth. Surface the admin API token into the Playwright env (MATOMO_API_TOKEN) and pass it in the request POST body so it stays out of the URL/logs. Mode-agnostic test/env change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
enable_framed_pages=1 disables Matomo's JS frame-buster so the dashboard can iframe-embed it (CSP frame-ancestors still gates who may frame). Shared web-svc consumers (logout, cdn, file) render body_filter.lua in the earlier web-svc service-loader bucket, before web-app-matomo writes its API token, so the tracker is gated off and never re-rendered; after the token is persisted, re-run sys-front-inj-all for each web-svc matomo consumer so the tracker lands. The loop is restricted to web-svc-* because re-running it for a web-app consumer outside its own play would dereference unloaded role vars and crash; web-svc roles declare no javascript injection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Simpleicons: the reachability probe ran inside the controller-side Jinja filter, which has no route to the node's published port in swarm, so every card silently fell back to Font Awesome. Move the probe to a STACK_HOST-delegated uri_retry step and make the filter pure assembly. Compose output is byte-identical.
Tile reachability: add a per-card services.dashboard.iframe capability (default = enabled) so apps whose SPA top-redirects out of an embed (Keycloak admin console, iframe:false) open in a new tab; roles_with_service emits the flag into the target manifest and the spec branches on it. For iframe:false targets the spec also asserts the HTTP layer permits framing (no blocking X-Frame-Options, CSP frame-ancestors allows the dashboard) so the flag can never mask a real header block. Fix the Tab-button locator and reveal it from fullscreen before asserting.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve the 80 open CodeQL/code-scanning alerts on this branch.
Security-sensitive fixes:
- cli/administration/deploy/development: break the unsafe import cycle by
extracting the env/arg helpers into a leaf module `env.py`, giving the
load-time DAG compose -> common -> deps -> env.
- web-app-bluesky login-broker server.js: prototype-pollution guard on
cookie parsing, CR/LF + control-char log sanitisation, HTML-escaped
exception text, and a ReDoS-bounded handle regex.
- web-app-baserow SSO: harden the open-redirect `next` handling.
- network diagnose probes: require TLS >= 1.2.
- actionlint installer: validate tar members against path traversal.
- decidim/apt-purge lint regexes: rewrite to avoid catastrophic backtracking.
- 6x Playwright URL regexes anchored; crs-k/stale-branches pinned to a commit SHA.
Mechanical cleanups: unused imports/globals/locals, empty-except comments,
import-and-import-from, implicit-string-concat, multiple-definition,
unreachable/useless-assignment.
False positives are intentionally left in place (Django settings patches,
Odoo __manifest__, side-effect imports, cross-module symbols).
Recurrence guards:
- import-linter contract (pyproject + tests/lint) locks the dev-deploy
package acyclic; a new cycle now fails CI instead of only CodeQL.
- new lint test requires every bandit (S###) `noqa` to carry a justification.
- BLE001 (blind-except) added to ruff as a ratchet (tests/roles exempt,
current production offenders grandfathered).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the kernel-flavor swarm ENOENT: no export carried fsid=0, so the NFSv4 mount of the nested /srv/nfs/infinito-state sub-export had no pseudo-root anchor and the server returned NFS4ERR_NOENT on the PUTROOTFH+LOOKUP walk. crossmnt could not fix it (it only crosses a boundary mid-walk, never sets the starting filehandle). Kernel now exports the state dir itself at fsid=0 and clients mount the bare v4 root ':/' (zero LOOKUPs). ganesha keeps Pseudo=Path and v3 mounts the full path. The self-bind is kept so fsid=0 rides a real mount boundary (stable st_dev) under overlayfs DinD.
The NFS derivations move to per-value lookup plugins (nfs_flavor, nfs_client_src, nfs_fstype, nfs_mount_opts, nfs_state_path) backed by pure helpers in utils/storage/nfs.py with unit tests; group_vars STORAGE_NFS_* become thin lookup wrappers. The dev/act->ganesha flavor rule stays the single SPOT in the role config; nfs_flavor reads the merged value so client and server agree.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In swarm the sibling exporters (alertmanager, cadvisor, node-exporter, blackbox) and the healthz backend are unreachable on the compose container_name / host.docker.internal / 127.0.0.1 addresses, so prometheus marked every target down and /healthz/ready returned 502. Add a scrape_target lookup emitting <host>:<port> per mode (swarm tasks.<entity>_<service>:<internal_port>, compose container_name/compose_host), reading the internal port from services.<svc>.ports.internal; declare those ports on the four exporters; route both healthz probes through the proxy_pass lookup via one set variable. Compose output stays byte-identical.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Kernel nfsd only crosses the self-bind mountpoint into the distinct-fsid state export when the parent export carries NFSEXP_CROSSMOUNT (nfsd_mountpoint()). Without crossmnt the v4 controller/node mount of the state path returns ENOENT even though the export is present. Add crossmnt to the node and controller-traversal parent exports.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the inline "nfs4 if version>=4 else nfs" fstype expression (repeated across svc-storage-nfs-client and svc-storage-nfs-server) with a single STORAGE_NFS_FSTYPE in group_vars/all/15_storage.yml, next to the other STORAGE_NFS_* values it derives from.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The inventory applications override in default.yml is dropped during inventory provisioning (CI ran kernel despite RUNTIME=github + a ganesha override), so select the flavor in meta/variants.yml where it reliably applies: ganesha for RUNTIME in [dev, act], kernel for github CI and production. meta/services.yml keeps kernel as the base default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A launcher-managed app (discourse) renders an empty services map, so 'docker compose up' fails with 'no service selected'. The wrapper now returns no command for an 'up' against a zero-service project (main() exits 0); a compose that fails to parse is not treated as serviceless, so real errors still surface.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The github swarm legs picked the kernel NFS flavor inside the privileged DinD nfs-server, where kernel nfsd is not network-namespaced and wedges exportfs. Move flavor selection out of the role meta default and into the dev inventory: meta/services.yml now defaults to kernel (production); the development inventory overrides it to ganesha for the containerized runtimes (dev/act/github) and keeps kernel for real hosts (RUNTIME=host).
Add meta/variants.yml (kernel + ganesha) and a TODO to test both flavors via files/test.sh, plus server-side kernel-nfsd diagnostics in 12_collect_diagnostics.sh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The host-side rm in 01_start_nfs.sh ran in the (often non-root) act runner and could not delete the prior run's root-owned NFS writes, so stale state survived every redeploy. A stale root-owned matomo config.ini.php that the uid-33 web user can no longer rewrite kept matomo unhealthy, so the role's post-health chown never ran and the swarm service never converged. Clear the export contents from inside the privileged container, the only context that can delete root-owned files, so every cluster start begins from an empty export.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NFSv4 has no MOUNT protocol; the kernel nfsd only crosses from the base export into the nested distinct-fsid state export at a REAL mount boundary (nfsd_lookup_dentry -> nfsd_cross_mnt, gated on nfsd_mountpoint()). /srv/nfs/infinito-state is a plain same-filesystem subdirectory, not a mountpoint, so the cross is never attempted, the fsid=2 filehandle is never built, and the controller's direct mount returns NFS4ERR_NOENT. crossmnt/nohide cannot help (only consulted inside the cross, never reached); exportfs/showmount only show the export table, not the v4 pseudo-walk, which is why the child looked exported yet was unreachable. Bind-mount the state subdir onto itself (kernel flavor, before exportfs) so it becomes a real boundary; the existing fsid=2/no_root_squash child export then resolves. Ganesha (userspace, per-Export handle) was never affected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cleanup step ran silently, so a no-op run was indistinguishable
from a broken one. Add a config banner, per-deletion lines, and a
per-package plus grand-total summary with a kept-reason breakdown, and
an explicit message when nothing matches. Add DRY_RUN support.
Deletion rule: delete a version only when it carries a ci-* tag, has no
semver (v*) or latest tag, and is older than MAX_AGE_DAYS. Released
images and untagged multi-arch child layers are never touched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
crs-k/stale-branches publishes no moving v8 major tag, so @v8 was
unresolvable and the 'Delete inactive branches' job failed at setup.
Pin to v9.0.1.
The step also passed a 'protected-branches' list, which is not a valid
input for this action (it relies on GitHub branch-protection/rulesets),
so it was silently ignored. Replace it with the action's real
include-filter 'branches-filter-regex' so main/master/develop/dev and
release/*, hotfix/* branches are excluded from deletion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The controller mount of the nested state export failed with server 'No such file or directory': over NFSv4 the controller must LOOKUP-traverse the base /srv/nfs export (real export, fsid=1) to reach the nested /srv/nfs/infinito-state (fsid=2), but a controller outside svc-swarm-node is absent from the base client list, so traversal is denied. Re-export the base read-only to SVC_STORAGE_NFS_SERVER_CONTROLLER_CIDRS (empty by default -> no extra exposure; in-node controllers are already listed; the swarm test sets the lab subnet). Worker nodes keep base rw via their explicit-IP lines (exports(5) host-specificity wins).
Add nfs_export_all_rw seboolean for the kernel flavor so RedHat SELinux-enforcing hosts (fedora/centos) serve the export; install python3-libselinux and re-gather the selinux fact first so the boolean is not silently skipped when the binding is absent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Real GitHub Actions and production use the kernel NFS flavor (RUNTIME not in dev/act); only local act uses ganesha. The kernel exports.j2 set no fsid, so the export base and its nested state subdir (one device, a bind-mount under test) get colliding auto-derived NFSv4 fsids; the controller's mount of /srv/nfs/infinito-state then fails with the server returning 'No such file or directory'. Add fsid=1 (base) and fsid=2 (state subdir), mirroring the ganesha flavor's Export_Id 1/2.
Extend the swarm test 12_collect_diagnostics.sh to dump the nfs-server /etc/exports, exportfs -v, ganesha.conf, export tree, nfsd status, and the controller's NFS reachability, so an export failure is diagnosable from the job log instead of only the client-side ENOENT.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The merged main renamed the inventory variant filter (filter_plan_to_variant -> filter_plan_to_variants) and added the shared variant_select selector. Adapt cli.administration.deploy.swarm.matrix to consume the variant CSV bundle via variant_select (add_variant_args + apply_variant_filter) instead of its bespoke single-int parser, so it iterates exactly the runner's bundle of rounds and the stale import no longer breaks.
Mirror test-deploy-compose.yml in the swarm workflow: pass the variant bundle to the orchestrator (re-passed through sudo so env_reset can't drop it), show the variant in the job name, and suffix the playwright artifact with variant_slug. Also singularise the discover job names in both workflows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
After the main merge the discover job emits objects ([{apps,variant,variant_slug}]) for the variant matrix, but the swarm workflow still consumed them via the string-form 'matrix: apps:', so matrix.apps became an object, act produced no legs, the deploy job never ran, and act exited 0: every local swarm roundtrip false-passed.
Switch the swarm matrix to 'include:' (matching the compose workflow) so matrix.apps is the app id again, and guard roundtrip.sh to require the 'Matrix-deploy <app> across variant rounds' marker in the log or exit 1, so a skipped deploy can never pass as green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The dev inventory resolves DIR_VAR_LIB via lookup('env','INFINITO_DIR_VAR_LIB'), but compose.yml never forwarded the var into the infinito container, so it resolved empty inside the DiD: relative FILE_TOKENS made the mailu no-reply token write and read at diverging paths and the sys-svc-mail assert failed.
Forward the key in the infinito service environment block, and add a lint test that fails when any lookup('env','INFINITO_*') in inventories/development/*.yml is not forwarded in compose.yml, so this class of bug cannot recur.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A maintainer-applied trusted-pr label opts a fork PR out of the cautious fork-prereq dance: the orchestrator skips the wait for privileged fork images, and the pull_request_target build switches its source from the base merge ref to the raw PR head (head.repo.full_name@head.sha). The image keeps the merge-SHA tag so the pull_request orchestrator and its downstream consumers find it unchanged.
Security gates: only the label flips the build source (label-application is collaborator-only on GitHub, so fork authors cannot grant it); build-orchestration scripts and the helper checkout always come from the base repo at github.sha, only the Docker build context is the fork head; persist-credentials stays false and USE_NIX_TOKEN stays disabled for fork-sourced builds; fork pull_request runs keep their GitHub-enforced read-only token. Untrusted fork PRs and same-repo PRs are unchanged.
Wiring: ci-orchestrator gains a trusted_pr boolean input resolved once into a WAIT_REQUIRED job env; images-build-ci gains an optional source_repository input; both pull_request and pull_request_target carry a labeled trigger so applying the label starts fresh runs that observe it. Documented in pipeline.md (Trusted fork PRs) and workflows.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a lint-packages target/workflow that parses the generated packaging metadata with each ecosystem's native parser (dpkg-parsechangelog, rpmspec, bash -n for the PKGBUILD), failing on any parser stderr so warnings cannot slip through. Wired into the make lint fan-out and ci-orchestrator's lint gate; an install group provisions the parsers (best-effort, cross-distro).
Hardens package_mirror so the generated changelogs parse without warnings: every RPM body line gets a '- ' continuation prefix (a column-0 '*' was read as a new %changelog entry -> 'bad date'), and the archived-releases notice is folded into the oldest kept entry instead of trailing after the final entry (free text there made dpkg-parsechangelog warn). Regenerates packaging/debian/changelog and the Fedora spec accordingly.
Adds a test-lint check (test_package_changelog_completeness) asserting both package changelogs list every CHANGELOG.md + docs/changelog release and that their inline window matches CHANGELOG.md, plus unit regressions for the RPM continuation prefix and the Debian footer fold.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
format_debian_entry indented only the first body line per entry, leaving the rest at column 0. dpkg's changelog parser treats un-indented change lines as badly formatted, loses the entry boundary and reports 'cannot parse maintainer email address ""', failing the .deb build in build-ci-images. Every non-empty body line is now indented by two spaces.
trim_and_archive additionally dropped the '# Changelog' H1 preamble on every rewrite (it rebuilt content from the version entries only), which would break the MD041 markdown lint on the next trim. The preamble before the first version header is now preserved.
Regenerated packaging/debian/changelog and packaging/fedora/infinito-nexus.spec from CHANGELOG.md (untrimmed). Also adds the missing YAML document-start to web-app-pihole/tasks/main.yml.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
entry-push-latest.yml gains a sync-main job that force-pushes the current repository's main from a source repo (default infinito-nexus/core) before CI policy and deploy discovery derive scope from origin/main. The source is configurable and can be disabled via CI_SYNC_MAIN_SOURCE_REPOSITORY; the same-repository case and disabled values skip the sync.
The sync-main checkout uses fetch-depth: 0 and the push is --force to avoid shallow-update and non-fast-forward rejections. Docs warn that main is force-overwritten and must not be used as a working branch on synced repositories.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A role must not delete its own data, and the per-volume wipe added inside sys-svc-compose's nfs_prep additionally no-op'd because a selectattr filter emptied its loop. Revert nfs_prep and main.yml to the pre-create-only original, and move the clean-state guarantee into 01_start_nfs.sh: it wipes the NFS export dir before the server starts, so every swarm test run begins from empty state and stale per-app volume data cannot bleed across runs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
