Resolve the 80 open CodeQL/code-scanning alerts on this branch.
Security-sensitive fixes:
- cli/administration/deploy/development: break the unsafe import cycle by
extracting the env/arg helpers into a leaf module `env.py`, giving the
load-time DAG compose -> common -> deps -> env.
- web-app-bluesky login-broker server.js: prototype-pollution guard on
cookie parsing, CR/LF + control-char log sanitisation, HTML-escaped
exception text, and a ReDoS-bounded handle regex.
- web-app-baserow SSO: harden the open-redirect `next` handling.
- network diagnose probes: require TLS >= 1.2.
- actionlint installer: validate tar members against path traversal.
- decidim/apt-purge lint regexes: rewrite to avoid catastrophic backtracking.
- 6x Playwright URL regexes anchored; crs-k/stale-branches pinned to a commit SHA.
Mechanical cleanups: unused imports/globals/locals, empty-except comments,
import-and-import-from, implicit-string-concat, multiple-definition,
unreachable/useless-assignment.
False positives are intentionally left in place (Django settings patches,
Odoo __manifest__, side-effect imports, cross-module symbols).
Recurrence guards:
- import-linter contract (pyproject + tests/lint) locks the dev-deploy
package acyclic; a new cycle now fails CI instead of only CodeQL.
- new lint test requires every bandit (S###) `noqa` to carry a justification.
- BLE001 (blind-except) added to ruff as a ratchet (tests/roles exempt,
current production offenders grandfathered).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
crs-k/stale-branches@v9.0.1 rejects days-before-stale >= days-before-delete (the v8 'delete immediately' behaviour with days-before-delete: 0 is gone), so the daily cleanup failed with 'Invalid inputs'. Set days-before-stale: 350 and days-before-delete: 360 - deletion still triggers at 360 days of inactivity, with a 10-day stale warning before it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The compose and swarm inventory collectors hard-failed at the 'Zip generated
inventory' step because they looked in the wrong place. Inventory is written on
the runner host at $INFINITO_INVENTORY_DIR (resolve.sh yields
$HOME/inventories/github in CI), but:
- zip_compose_for_upload.sh ran 'docker cp' from infinito_nexus_* containers,
which dedicated.sh's cleanup trap removes before the always() step runs, so it
never found a container.
- zip_swarm_for_upload.sh globbed /tmp/inv*, which the matrix orchestrator never
populates; it defaults its base to $INFINITO_INVENTORY_DIR.
Both now source scripts/meta/env/load.sh to resolve INFINITO_INVENTORY_DIR and
zip that dir plus its per-round '-<n>' variants.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.
- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.
- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).
- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The merged subset feature failed test-lint: a .py under scripts/, a local parents[N] PROJECT_ROOT, a direct yaml.safe_load, an uncached read, and narration comments.
- move scripts/meta/resolve/pr/subset_roles.py -> cli.meta.ci.subset_roles (scripts/ is .sh-only); dispatcher invokes it via python -m
- parse via utils.cache.yaml.load_yaml_str; import utils.PROJECT_ROOT
- test invokes the module via -m against real roles; nocheck the tempfile read
- strip narration comments; docs document HOW not WHY
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The maintainer-only manual CI dispatch (workflow_dispatch) cannot be triggered by fork contributors. The 🧩 Subset label closes that gap: when present on a PR, the detect-affected-roles job resolves the deploy whitelist from an explicit roles: YAML block in the PR body instead of the diff, giving fork PRs the same narrowed matrix a maintainer would dispatch manually. Without the label, behavior is unchanged (the dispatcher falls through to the diff resolver, output byte-identical).
- subset_roles.py: parse and strictly validate the PR-body roles block (invalid YAML / empty / unknown id / path-traversal all hard-fail)
- pr_affected_roles.sh: single dispatcher selecting subset vs diff resolver
- entry-pull-request-change.yml: wire the label and PR body into the job
- PR templates, pipeline/workflows docs and READMEs, integration tests
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conflicts resolved:
- .claude/.gitignore: union of ignore patterns
- plugins/lookup/rbac_group_path.py: drop both unused conflicting imports
(body fetches apps via lookup_loader)
- roles/sys-svc-compose/handlers/main.yml: keep HEAD's refactor (compose-up
handlers live in compose.yml, included per DEPLOYMENT_MODE)
- utils/cache/base.py: keep HEAD's merge_with_defaults/FILE_TOKENS imports
Post-merge fixes for main's newly-enabled lints:
- pyproject.toml: grandfather 5 files under the BLE001 ratchet
- justify 6 pre-existing security # noqa markers (S108/S311/S701)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the now-disallowed `# Workaround:` comment above the Docker apt-repo task
(caveat/workaround were dropped from the comment-lint markers).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The redis/pg `--network` flags were gated behind `not SWARM_CLUSTER_AVAILABLE`,
so in swarm the standalone launcher container was not on the redis overlay when
`rake db:migrate` runs during `./launcher rebuild` -> "Connection refused" to
redis-central. The post-rebuild network-connect handler runs too late. Emit the
flags in both modes (overlays are attachable); compose output is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The plugin copy used the bare `joomla` container name, which does not exist on
swarm (services are <stack>_<svc>.<slot>.<id>). Use JOOMLA_CONTAINER_ADDRESS via
shell so the swarm resolve-container-id subshell expands, matching the role's
other container steps.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On swarm the bare compose container names (erpnext-backend, ...) do not exist,
so `container restart` failed with "No such container". Split the restart into a
compose path (container restart) and a swarm path (service update --force
<entity>_<service>), extracted into 02b_restart_frappe.yml to keep the bootstrap
block within the size limit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shared curl healthcheck has no start_period, so swarm probes Jenkins
immediately; Jenkins LTS boots slowly (plugins + JCasC), the first probes fail,
the task is marked unhealthy and e2e gets a 502. Add start_period: 10m.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Under swarm the ollama task can be rescheduled, so the container id resolved for
`ollama pull` was stale by exec time. Wait for the service to converge first
(swarm-only; a no-op in compose), mirroring web-svc-collabora.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The getfider/fider image is debian-slim with no wget, so the previous
`wget -qO- /_health` healthcheck never passed. Compose tolerates unhealthy
containers so it went unnoticed; swarm reaps the unhealthy task (exit 137) and
the service never converges. Switch to `./fider ping`, the image's own
HEALTHCHECK, which works in both modes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
setfacl returns "Operation not supported" on NFS exports, hard-failing the
backup user setup in the swarm-nfs topology. Extract the ACL tasks into
03b_acl.yml and include them only when storage.backend \!= 'nfs'; chown
backup:backup + 0700 already enforce the same posture on NFS.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`docker stack deploy` rejects `cgroup: host` ("Additional property cgroup is not
allowed"), aborting the swarm deploy. Emit it via the compose_only lookup so the
key is dropped in swarm and kept in compose.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove caveat and workaround from the comment-lint marker vocabulary and delete
the comments that used them across the tree. Move test_comments_valid's own
git-safe.directory note from a marked comment into the function docstring.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Docker env-files take everything after '=' literally, so cosmetic column
alignment (KEY= value) leaks the spaces into the value (the cause of the
MariaDB "Access denied for user ' friendica'" failure). Add a lint scanning
every *env.j2 for whitespace before/after '=' or trailing whitespace, and strip
the existing offenders in espocrm, friendica, nextcloud and phpldapadmin.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add skip: [compose, swarm] to its meta/services.yml so the deploy discovery
drops it from both test matrices; the role stays deployable outside CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ruff format line-wrapping only (the files were committed under --no-verify, so
the format hook had not run). No behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zip_swarm_for_upload.sh only checked the base /tmp/inv, but the matrix
orchestrator writes per-round dirs /tmp/inv-<n> (and runs as root), so the
artifact was always empty. Collect every /tmp/inv* dir with sudo and de-root
the staged copy before zipping.
Both the swarm and compose collectors now hard-fail (exit 1) when they find no
inventory to copy, instead of echoing and exiting 0, so a silently-missing
artifact surfaces a real provisioning/path bug.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the [DEPRECATION WARNING] entries the compose/swarm deploys emitted:
- apt_repository (removed in ansible-core 2.25): write docker.list via copy +
an explicit apt update. deb822_repository is unusable here because
apt_repo.sh's sanitize_sources deletes .sources every run.
- INJECT_FACTS_AS_VARS (default deprecated in 2.24): use ansible_facts['<name>']
instead of the auto-injected top-level ansible_os_family / virtualization_* /
default_ipv4 vars.
- community.mysql.* (removed in community.mysql 6.0.0): switch to ansible.mysql
and pin the collection in requirements.galaxy.yml.
Add a compose deploy gate (assert_no_deprecation_warnings.sh + workflow step +
test) that fails on [DEPRECATION WARNING] in the deploy log, and a lint test
(test_no_deprecated_ansible_patterns) that forbids the three patterns above so
they cannot return.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Driver roles configure the host (kernel modules, lid switch, vendor drivers)
and are meaningless as swarm services. Add `skip: [swarm]` to each drv-* role's
meta/services.yml so the swarm discovery (--skip-mode swarm) drops them; compose
deploys still run them.
Add tests/lint/ansible/roles/test_drv_roles_skip_swarm.py to enforce the policy
for the whole class: every drv-* role must list swarm in its skip list, so
future driver roles cannot silently enter the swarm matrix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert the two resolve_host_cid.yml includes in 02b_updates.yml from the
[playbook_dir, ...] | path_join form to lookup('path_absolute', ...), matching
the repo-wide sweep.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Autoformat (mbake/yamllint) re-indented sequence items under their mapping key
across the regenerated meta files and wrapped one long URL scalar in the matomo
inject task. Whitespace/formatting only, no semantic change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:
[playbook_dir, '<rel>'] | path_join -> lookup('path_absolute', '<rel>')
playbook_dir ~ '/<rel>' -> lookup('path_absolute', '<rel>')
{{ playbook_dir }}/<rel> -> lookup('path_absolute', '<rel>')
Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.
Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.
Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In swarm the object updates run delegated to KEYCLOAK_HOST_NODE, but that fact
was resolved early in the login step. Over a long deploy swarm can reschedule
keycloak to another node, leaving the update delegated to a node that no longer
runs keycloak, so kcadm fails with "Failed to create Keycloak object". The
cross-node rescue recovered, but with any_errors_fatal the rescued failure still
aborted the play (exit 2) after a slow recovery loop.
Re-resolve the live hosting node at the start of the update block and run the
update routines with a fresh delegate_to binding, so the first attempt targets
the node currently running keycloak. The rescue stays as a fallback for a
reschedule during the update itself.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Autoformat (mbake/yamllint) re-indented sequence items under their mapping key
in three meta files. Whitespace only, no semantic change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The --failed swarm/compose/total filters keyed the deploy-job regex on the
literal words "Compose"/"Swarm", which no real CI job carries. Job names are
"<glyph> <app> <variant>" (e.g. "... / test-deploy-swarm / 🐝 web-app-x 0,1"),
so the regex matched nothing and the CLI reported "Nothing failed" for runs
full of failures. app_for_job in the role-runtime path had the identical bug
(returned the shard token instead of the app id).
Match on the 🐳/🐝 glyph instead, tolerate the orchestrator caller prefix and
the " <variant>" shard suffix, and route both consumers through one parser
(runs.app_of_job). Centralize the naming as SPOTs: MODE_GLYPHS in runs.py and
ORCHESTRATOR_PREFIX plus a workflow-sourced deploy_job_name() helper in tests,
so fixtures render real names from the workflow YAML and cannot drift back to
job names GitHub never emits.
Add an integration test that renders names from test-deploy-{compose,swarm}.yml
and asserts the parser extracts app+mode for both modes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
