Commit Graph
14682 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.7 9beee8dc03 refactor(web-app-keycloak): SPOT per-app mapper naming via filter plugin
Add `roles/web-app-keycloak/filter_plugins/rbac_per_app.py` with two
filters that centralise the strings that vary per application:

  kc_per_app_mapper_name(app_id) -> "ldap-roles-<app_id>"
  kc_per_app_ldap_filter(app_id) -> "(&(objectClass=groupOfNames)(cn=<app_id>-*))"

Consume them from:

* `vars/main.yml::KEYCLOAK_PER_APP_LDAP_ROLES_MAPPER` for the mapper
  name and the LDAP filter, instead of inlining the string templates.
* `tasks/update/ldap/per_app_mapper_one.yml` as the `MAPPER_NAME` env
  variable handed to the upsert script.
* `files/scripts/per_app_mapper/upsert_app_mapper.sh`, which now reads
  `MAPPER_NAME` from the environment instead of building it from
  `APP_ID`.

A future rename of the per-app mapper convention is now a one-line
change in the filter plugin and stays consistent across vars, tasks,
and shell scripts. Unit tests cover both filters and the
`FilterModule` registration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 09:25:13 +02:00
kevinveenbirkenbachandClaude Opus 4.7 927469c51b refactor(web-app-keycloak): extract per-app mapper shell logic into .sh files
Move the inline `ansible.builtin.shell:` blocks from
update/05c_per_app_mappers.yml and update/ldap/per_app_mapper_one.yml
into dedicated scripts under
`files/scripts/per_app_mapper/`. Each task is now a thin
`ansible.builtin.script:` invocation that passes its parameters via
`environment:`, so the shell logic stays linter-friendly, easier to
re-run by hand, and decoupled from Ansible's YAML quoting.

No behaviour change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 09:04:19 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d009a35116 feat(rbac): hierarchical /roles/<app>/<role> + service-gated Playwright + Discourse round-trip
Implements requirements 005, 006, and 007.

005: per-application Keycloak `group-ldap-mapper`s anchor each app's
role groups under `/roles/<application_id>/<role_name>` (and, for
tenant-aware apps, `/roles/<application_id>/<tenant>/<role_name>`).
The flat-CN LDAP layout keeps each CN globally unique so Keycloak's
group-tree resolver never collides; per-app mappers use the LDAP
`description` as the visible Keycloak group name to keep the path
free of redundant prefixes. Migration purges legacy pre-005 entries
and forces a Keycloak `logout-all` so stale tokens drop pre-005
group paths.

006: per-service `<SERVICE>_SERVICE_ENABLED` env flags and the
`service-gating.js` helper let role specs `skipUnlessServiceEnabled`
cleanly when a shared service is missing or `SERVICES_DISABLED` is
set; `MODE_CI` markers (`INFINITO_MAKE_DEPLOY`, `GITHUB_ACTIONS`,
`ACT`) propagate through the deploy CLI into the test stage.

007: WordPress->Discourse round-trip Playwright scenario gated on the
`web-app-discourse` deployment.

WordPress Playwright runs 5/5 green on the single-site flow; the
multisite + Discourse scenarios are correctly skipped when their
service prerequisites are absent. The `rbac_group_path` lookup is the
single sanctioned producer for OIDC group paths and is wired into
every consumer (yourls, lam, minio, openwebui, pgadmin, phpldapadmin,
phpmyadmin, prometheus, xwiki).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-25 08:51:01 +02:00
kevinveenbirkenbachandCopilot Autofix powered by AI 329edad548 Potential fix for code scanning alert no. 274: Empty except (#199)
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-04-25 08:24:52 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d91ae53d43 docs(git/remotes): align with git-maintainer-tools v1.1.0 push routing
The tool now pins branch.main.pushRemote to origin, so main pushes go
upstream while other branches keep landing on the personal fork. The
layout table and the new Push Routing section reflect this and list
the three git-config keys the tool writes. Also tighten the sandbox
rule to MUST per the docs RFC-2119 style.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 00:08:53 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3f0b5481cc fix(ci): use github.event.ref in delete-branch concurrency group
On `delete` events the workflow runs from the default-branch context, so
`github.ref_name` evaluated to "main" and the concurrency group collided
with entry-push-latest.yml's main group — cancelling the merge-commit
CI: Push run on main. `github.event.ref` carries the actually deleted
branch name, so the group matches the intended push-CI group.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 23:18:21 +02:00
kevinveenbirkenbach fab099c1c1 feat(version-updater): support flavored semver tags and centralise logic
Docker Official Image tags of the form `<semver>-<flavor>` (e.g. the
joomla:5.4.5-php8.3-apache pin) were previously skipped by the
`web-app` image version checker because its strict regex only matched
pure `x.y.z` shapes. As a result the auto-update workflow and the
external warning test never proposed a newer 5.4.x-php8.3-apache tag,
and flavored pins silently drifted out of date.

Widen the regex in `utils/docker/version_updater.py` to accept the
`<semver>-<flavor>` shape, with the flavor suffix treated as an opaque
discriminator: a candidate tag is only considered an upgrade when it
shares the *same* flavor. That guarantees a 5.4.5-php8.3-apache pin
never gets silently switched to 5.4.6-php8.4-apache or 5.4.6-fpm.

Refactor `tests/external/docker/test_image_versions.py` to import
directly from the util: ~130 lines of duplicated semver / Docker Hub /
GHCR helper code deleted, and the file now has exactly one code path
to keep in sync with the one used by the auto-update CLI
(`cli/fix/docker_image_versions`).

Chain impact:
  * `.github/workflows/update.yml` -> `scripts/github/update_docker_image_versions.sh`
    -> `python3 -m cli.fix.docker_image_versions` -> same util. No
    workflow change needed; the next scheduled update run will pick
    up flavored upgrades automatically.
  * Live external run now reports 5 previously-silent upgrades:
    web-app-joomla/joomla, web-app-magento/php, web-app-bridgy-fed,
    web-app-suitecrm/php, web-app-shopware/redis.

Tests: existing `test_latest_semver_respects_depth` still covers the
pure-semver case; three new unit tests pin the flavor behaviour
(`is_semver` accepts flavored tags, `version_key/depth/flavor` strip
the flavor, `latest_semver` refuses cross-flavor upgrades).

Also drop an unused `autocreate_users: true` marker at the top of
`roles/web-app-joomla/config/main.yml` that was left over from an
earlier LDAP-autocreate experiment.
2026-04-24 23:04:47 +02:00
kevinveenbirkenbach 8babec8340 Merge pull request #197 from infinito-nexus/alert-autofix-272
Potential fix for code scanning alert no. 272: Unused variable, import, function or class
2026-04-24 22:51:22 +02:00
kevinveenbirkenbach 8a3d236389 Merge pull request #198 from infinito-nexus/alert-autofix-277
Potential fix for code scanning alert no. 277: Explicit returns mixed with implicit (fall through) returns
2026-04-24 22:42:57 +02:00
kevinveenbirkenbach b0c1335f58 fix(web-app-joomla): tolerate Joomla 5.x cleanup-phase crashes
The Joomla 5.x CLI installer's post-install cleanup step can crash
with rc=1 while the core install has already written configuration.php
successfully. Observed on Debian (run 24897749979) as:

    In InstallCommand.php line 177:
      Call to a member function deleteInstallationFolder() on null

Our existing retry loop treated rc=1 as a failure and kicked off a
destructive retry: the next attempt `rm -f`s the freshly written
configuration.php and then fails because the installer self-deleted
its own entry file (`installation/joomla.php`) during the crashed
cleanup.

Extend the tolerance to also match the new Joomla 5.x null-ref
symptom in stderr. The task-local `vars:` block is now the SPOT for
the outcome classification: `_joomla_install_soft_success` and
`_joomla_install_git_tree_refusal` encode the full soft-success /
hard-failure policy, and the three filter clauses (`until`,
`changed_when`, `failed_when`) reference those booleans instead of
duplicating `is search(...)` expressions.

Distinct from the earlier infinito-nexus/core#194 pipefail regression
(already fixed in 3fdbe8d9e) and from the "Unable to delete
installation folder" case already handled. Both pre-existing guards
stay in place.
2026-04-24 21:52:13 +02:00
kevinveenbirkenbach d0244e1601 Merge pull request #196 from kevinveenbirkenbach/chore/use-git-maintainer-tools
chore: adopt git-maintainer-tools as dev dependency for git ops
2026-04-24 21:30:09 +02:00
kevinveenbirkenbachandCopilot Autofix powered by AI aefe7dfde3 Potential fix for code scanning alert no. 277: Explicit returns mixed with implicit (fall through) returns
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-04-24 21:23:25 +02:00
kevinveenbirkenbachandCopilot Autofix powered by AI 07816ecbb7 Potential fix for code scanning alert no. 272: Unused variable, import, function or class
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-04-24 21:20:50 +02:00
kevinveenbirkenbach 0236e82f4e Merge pull request #195 from infinito-nexus/alert-autofix-278
Potential fix for code scanning alert no. 278: Empty except
2026-04-24 21:16:45 +02:00
kevinveenbirkenbach e87bd9d37b chore: adopt git-maintainer-tools as dev dependency for git ops
Drop the project-local copy of scripts/git/sign-push.sh in favour of
the git-maintainer-tools CLIs (git-setup-remotes, git-sign-push),
which now ship as a standalone PyPI/GitHub package so the same helpers
can be reused across every maintainer repo without vendoring shell
scripts.

The package is declared in pyproject.toml's `dev` extras and lands on
$PATH via `make install-python-dev` (which already runs
`pip install -e .[dev]`). Consumers invoke the CLIs directly; there
are no make wrappers because doing so would just duplicate help text
the tool itself already prints.

Fallout:
  * scripts/git/sign-push.sh deleted.
  * Makefile: sign-push target removed (no replacement target).
  * .claude/settings.json: deny rule for `make sign-push*` replaced
    with direct `git-setup-remotes*` / `git-sign-push*` denies so the
    sandbox blocks the real binaries, not just the make alias.
  * CLAUDE.md: operator instruction points at the bare `git-sign-push`
    CLI + `make install-python-dev` for setup.
  * docs/contributing/artefact/git/remotes.md: new tool-focused page.
    Describes the layout and the two CLIs; does not duplicate raw git
    commands already covered by git-maintainer-tools.
  * docs/contributing/artefact/git/README.md: trimmed per readme_md.md
    (docs READMEs must not act as a file index).
  * docs/contributing/tools/make.md: Git section now documents the
    external CLIs and the install entry point, not a wrapper target.
2026-04-24 21:05:40 +02:00
kevinveenbirkenbachandCopilot Autofix powered by AI baddc1fec8 Potential fix for code scanning alert no. 278: Empty except
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-04-24 19:26:04 +02:00
kevinveenbirkenbach 3fdbe8d9e2 fix(web-app-joomla): use bash for CLI install argv (dash lacks pipefail)
The retry-loop install task introduced in f7d32109e used
`argv: [sh, -lc, …]` with `set -o pipefail` in the body. On Debian/Ubuntu
CI runners `/bin/sh` is dash, which aborts with
`sh: 1: set: Illegal option -o pipefail`, so all 5 retries die on the
very first line and the Joomla deploy fails.

Switch argv[0] to `bash` so the task works on every distro and document
the constraint inline. Regression is tracked in infinito-nexus/core#194;
the retry-loop itself was the fix for #186.

Add tests/lint/ansible/test_no_sh_pipefail_in_ansible_tasks.py guarding
the three ways this class of bug can come back:

  * `command` with `argv: […, sh, -lc, body_with_pipefail]`
  * `args.executable` set to any non-bash shell
  * removal of the global `ansible_shell_executable: /bin/bash` default
    from group_vars/all/00_general.yml
2026-04-24 16:32:27 +02:00
kevinveenbirkenbach a7cce51af7 Merge pull request #193 from infinito-nexus/update/docker-image-versions-20260424
update(images): update Docker image versions
2026-04-24 15:39:22 +02:00
github-actions[bot] 45675fde83 update(images): update Docker image versions 2026-04-24 13:27:50 +00:00
kevinveenbirkenbach 38ae9b86dc Merge pull request #159 from kevinveenbirkenbach/fix/users-applications.yml
refactor(lookups): reduce applications/users to lookup-only access
2026-04-24 13:40:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f0f802faf0 docs(requirements): add 005/006/007 and tighten Playwright spec guide
Add three follow-up requirements that extend the work landed in 004:

- 005 WordPress Multisite support + LDAP per-application OU refactor:
  introduces opt-in rbac.tenancy schema, migrates every rbac-carrying
  role onto the ou=<application_id>,ou=roles,... hierarchy, adds a
  central rbac_group_path lookup plugin as the only sanctioned way to
  compute OIDC group paths, mandates one-time LDAP rewrite plus a
  Keycloak logout-all to flush stale tokens, and specifies the
  WordPress mu-plugin contract for per-site roles and grant_super_admin
  semantics.

- 006 Service-gated Playwright tests: per-service <NAME>_SERVICE_ENABLED
  env flags, shared helper at roles/test-e2e-playwright/files/
  service-gating.js with hard-fail on unknown services, test.skip() with
  flag-named reason, explicit per-role refactor checkboxes for every
  existing spec, and a new MODE_CI variable (GitHub Actions + nektos/act
  + make-driven deploy) replacing the accidental DOCKER_IN_CONTAINER
  proxy at the test-e2e-playwright include site.

- 007 WordPress -> Discourse post round-trip: Playwright scenario that
  publishes a WP post and asserts the corresponding Discourse topic
  exists, gated on DISCOURSE_SERVICE_ENABLED, with teardown that covers
  draft and partial states and an API key scoped for topic delete.

Also update the role Playwright spec guide to carry the Option B
service-gating rules (per-service boolean flags, helper-only reads,
hard-fail on unknown services) so contributor and requirement SPOTs
stay in sync.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 13:25:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 d710b19692 docs(web-app-wordpress): add README for mu-plugins directory
Document the WordPress must-use plugins convention, the criteria for
adding code here (security/integration contract, must-always-load), and
the deploy path via 05_mu_plugins.yml, so future contributors understand
why this directory exists and what belongs in it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 07:35:23 +02:00
kevinveenbirkenbachandClaude Opus 4.7 bebae545eb feat(rbac): generic OIDC group RBAC for WordPress (requirement 004)
Auto-provision one OpenLDAP group per role declared under each application's
`rbac.roles` in `config/main.yml`, gated on `group_names` so only apps
deployed on the host contribute. Keycloak syncs the groups from LDAP, emits
the `groups` claim via the shared RBAC client scope, and WordPress maps that
claim to a local role in a mu-plugin (auto-loaded, cannot be deactivated
from the admin UI). No direct LDAP bind from WordPress.

Non-obvious design choice: the `groups` client scope MUST be a *default*
client scope, not optional. When optional + `include.in.token.scope=false`,
Keycloak 26 drops the group-membership mapper from userinfo even when the
scope is explicitly requested, so the mu-plugin receives no `groups` and
falls back to `subscriber` for every user. ID/access tokens still carry
the claim, but WP consumes userinfo.

Playwright verifies the three-role cycle (subscriber/editor/administrator)
serially and idempotently per requirement 004: the add step runs via the
Keycloak admin UI (detects "already a member" via a disabled picker
checkbox and skips); teardown runs via the Keycloak Admin REST API. Only
memberships the test itself added are removed, so any pre-existing
membership is preserved. Each hop (super-admin, biber, WP-admin) runs in
its own browser context so WP session cookies and Keycloak SSO cookies
cannot leak.

Resolves: docs/requirements/004-generic-rbac-ldap-auto-provisioning.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 07:26:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 cc0a4301d9 docs(requirements): add 004 — generic RBAC role auto-provisioning via OpenLDAP
Defines the end-to-end contract for auto-provisioning LDAP groups for
every role that declares an rbac: block, gated on group_names, plus
the OIDC flow from Keycloak into WordPress and the Playwright
verification that exercises the full pipeline for three exemplary
WordPress roles (subscriber, editor, administrator).

Key decisions captured in the criteria:

- Scope covers every role under roles/ with an rbac.roles block, not
  just web-app-* — svc-*, web-svc-* etc. included when they declare
  rbac.
- Gating: a role's groups are provisioned only when application_id is
  in group_names on the current host.
- Naming is the flat pattern <application_id>-<role_name>; full DN
  stays under the existing ou=roles container (no new OU introduced),
  which preserves every existing <application_id>-administrator DN
  exactly so current bind/search filters keep working.
- Administrator is still implicitly added even when not listed in
  rbac.roles, matching today's build_ldap_role_entries behavior.
- Keycloak sync + OIDC groups claim must deliver WordPress role
  mapping without any alternative path (no direct LDAP bind from WP);
  if the current OIDC plugin config doesn't deliver that, it is fixed
  in the same work item.
- Idempotency is required at two levels: re-running the provisioning
  is a no-op, and the Playwright test preserves biber's starting
  group membership across a full run.
- The "serial" constraint applies only to the test: the three role
  sequences (subscriber → editor → administrator) run one after
  another so a regression in one mapping does not mask another.
- Iteration follows the Role Loop + Playwright Spec Loop with
  SERVICES_DISABLED="matomo,email".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 00:27:22 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f722e676ca refactor(service-lookup): rename 'needed' flag to 'required'
Rename the third flag returned by lookup('service', ...) from 'needed'
to 'required' across code, tests and design docs. Same semantics: True
when some deployed app has the service with both enabled: true AND
shared: true, directly or transitively through its own enabled service
dependencies.

The new name reads as a contract ("this service is required by a real
consumer") rather than a suggestion ("something needs it"), which
matches the stricter-than-enabled semantic the flag already carries.

Touchpoints:
- plugins/lookup/service.py: _is_service_needed → _is_service_required,
  any_needed var, dict key in _compute_flags, docstring rationale.
- roles/sys-utils-service-loader/tasks/main.yml: caller switched to
  .required.
- roles/web-app-wordpress/config/main.yml + tasks/plugins/wp-discourse.yml:
  wp-discourse plugin gate uses lookup('service', 'discourse').required;
  compose.services.discourse declared as shared-service dependency so
  the service loader auto-loads web-app-discourse when inventory sets
  enabled: true. The legacy include_tasks: utils/load_app.yml in
  plugins/wp-discourse.yml is removed — the service loader now handles
  it.
- tests/unit/plugins/lookup/test_service.py: 25 assertions and 6 test
  method names renamed (test_needed_* → test_required_*,
  test_transitive_needed_resolved,
  test_direct_provider_is_not_needed_without_shared_flag).
- tests/integration/services/test_transitive_dependencies.py: one
  assertion renamed.
- docs/contributing/design/services/base.md: Lookup-Plugins section
  replaces the 'needed' bullet with an expanded 'required' entry that
  documents when consumers should gate on it and why plain enabled is
  not enough.

Also moves wordpress-app-specific keys (title, max_upload_size,
plugins) under compose.services.wordpress to match the convention
the other role keys already follow, and updates vars/main.yml lookup
paths accordingly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-24 00:06:47 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6977eb3f96 refactor(web-app-wordpress): nest app-specific keys under compose.services.wordpress
Move title, max_upload_size and the plugins block from the top of
config/main.yml into compose.services.wordpress, matching the convention
the other service-specific keys (version, image, name, cpus, …) already
follow for this role.

- title value now references service_provider.platform.titel (the
  scalar) instead of the platform dict, so the rendered Blog title
  reads e.g. "Blog by Infinito.Nexus" rather than a dict repr.
- vars/main.yml lookups updated to the new paths using the existing
  'compose.services.' ~ entity_name ~ '.X' pattern that the rest of
  this role already uses.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 23:45:45 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b56554a6f3 test(lookups): assert every defined lookup plugin is actually used
Symmetric to tests/integration/filters/test_usage.py: enumerates every
plugins/lookup/<name>.py that declares a LookupModule class and fails
if any of them is never referenced in production code. Three invocation
styles count as usage:

- lookup('<name>', ...) — canonical Jinja/YAML form.
- query('<name>', ...) — list-returning variant.
- from plugins.lookup.<name> import ... — direct Python import, used
  by utils/runtime_data.py which embeds several LookupModule classes
  as regular helpers instead of dispatching through Ansible.

Matches under tests/ alone count as "only used in tests" and still fail
the build. An unused lookup is a maintenance hazard: it stays in the
search path, rots silently, and confuses plugin browsing.

Uses the inverted-scan approach from the filter-usage test (one pass
over all matching project files, a single master alternation regex
covering every defined lookup name), so cost scales with M_files not
N_lookups × M_files. Walk and file contents are served from
tests.utils.fs process-level caches per the testing.md rule.

All 27 currently-defined lookups pass — no migration backlog.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 23:37:28 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3fefaf8cf4 fix(web-app-prometheus): move compose_volumes() to after all services
compose_volumes() emits a top-level `volumes:` key. The template placed
the call in the middle of the services: section (between `application:`
and `blackbox-exporter:`), which broke the services nesting — subsequent
service definitions (blackbox-exporter, alertmanager, cadvisor,
node-exporter) ended up as top-level keys instead of services-nested
ones. `compose config --quiet` rejected this:

  volumes.alertmanager additional properties 'logging', 'mem_limit',
  'mem_reservation', 'pids_limit', 'image', 'restart', 'volumes',
  'command', 'container_name', 'cpus', 'env_file' not allowed

Fix: move the compose_volumes() call to after all service definitions,
matching the convention already used by web-app-gitea, web-app-mastodon,
svc-db-openldap and the other roles that use this filter.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 23:27:47 +02:00
kevinveenbirkenbachandClaude Opus 4.7 e6eb5115bf test(utils/fs): extend cache coverage + invert filters/test_usage scan
Second-wave rollout of tests/utils/fs:

- 6 tests switched to the shared content cache via read_text() (replacing
  raw open()/Path.read_text()): roles/includes/test_existence,
  roles/dependencies/test_unnecessary, vars/test_facts_not_overridden,
  services/test_sys_requires_id, lint/ansible/roles/
  test_service_core_first_task_run_once, lint/ansible/roles/
  test_compose_resource_limits.

- 4 unit-lookup tests moved their fixture root from <repo>/.tmp_test_* to
  tempfile.TemporaryDirectory() so transient files land in /tmp (matching
  the AGENTS.md rule). Manual recursive-delete teardowns removed —
  TemporaryDirectory.cleanup() covers that path safely.

- filters/test_usage.py: algorithmic inversion. Old loop was
  O(N_filters × M_files × 4_regex); new scan walks files once, applies two
  master-alternation regexes (bare `| name` and `{% filter name %}`) plus
  one combined regex over all Python callables, and records hits per filter
  via the capture group. The two dropped Jinja patterns were strict
  subsets of the bare pattern, so the hit set is identical — only the
  complexity changes (3.6s → 0.2s, ~18x).

- docs/agents/action/testing.md: new "Filesystem Scans — Use the Shared
  Cache" section codifies the rule so future agent-written tests route
  through tests.utils.fs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 22:19:06 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9c8cc1ddcb test(utils/fs): cache project file walks + contents across test run
Adds tests/utils/fs.py with process-level @lru_cache for both the
project-wide os.walk and per-file read_text(), then migrates 26 tests
(24 modified + 2 new) that previously re-walked/re-read the tree
independently to share one cached pass.

Headline win: tests/integration/jinja/test_variable_usage.py drops from
~26s to ~1.3s (≈19x), jinja cluster as a whole ~14x. Other migrated
tests benefit from the shared content cache when the same file is read
by multiple scans in one pytest run.

Non-caching logic changes (kept intentionally):
- test_variable_usage.py: added `if varname not in content: continue`
  fast-path before the line loop (pure short-circuit, identical result).
- test_test_file_naming.py: exempt tests/utils/ from the test_*.py
  prefix rule so the new shared helpers don't trip the lint.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 20:29:54 +02:00
kevinveenbirkenbach b2914b7cd1 Merge branch 'main' into fix/users-applications.yml 2026-04-23 18:14:17 +02:00
kevinveenbirkenbach 69f0ce81bd Merge pull request #153 from AlejandroRomanIbanez/feature/communication-prometheus
feat(prometheus): integrate Prometheus monitoring and alerting across all apps
2026-04-23 16:27:58 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7709ad2151 fix(networks): sort web-app-decidim into correct 104.x subnet block
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 13:20:57 +02:00
kevinveenbirkenbach f7d32109e6 fix(web-app-joomla): retry stub-rm + install as one step to absorb image-init race
The joomla docker image populates /var/www/html asynchronously on first
container start: index.php appears early, but a stub configuration.php
can land a short time later. Between the install-check and the install
itself, the stub has been observed to appear in CI (~0.7s gap), which
then makes the CLI installer refuse with

  [ERROR] configuration.php already present\! Nothing to install, exiting.

An earlier fix added a single "rm -f configuration.php" task between
the check and the install, but CI still caught the stub re-appearing
between rm and install when image init was slow enough. Bundle the
stub removal and the CLI install into a single shell step so every
retry re-runs BOTH: the install can lose the race at most for its own
attempt; the 5x15s retry window covers the image-init settling period.

Also rewrite the step to use the `container` / `compose` wrappers
instead of raw `docker exec`, matching the repo-wide convention
enforced by tests/integration/compose/test_no_raw_docker.py.

Validated via make deploy-fresh-purged-apps APPS="web-app-joomla
web-app-peertube" SERVICES_DISABLED="matomo,email" FULL_CYCLE=true:
Pass 1 ok=1576 failed=0, Pass 2 ok=1444 failed=0, joomla 1/1
Playwright scenario green in both passes.
2026-04-23 12:50:08 +02:00
kevinveenbirkenbach e40ff954f0 fix(web-app-peertube): back-fill application-table NOT NULL defaults
Peertube v8.x (and intermittently under the mutable `production-bookworm`
tag) ships a migration that declares `application.nodeVersion` and
`application.nodeABIVersion` as NOT NULL without column defaults, while
the bootstrap `Application.loadApplication()` only INSERTs
`migrationVersion` into the single-row `application` table. On a fresh
database that INSERT fails with

  null value in column "nodeVersion" of relation "application"
  violates not-null constraint

and peertube crash-loops before it can add the OIDC plugin or serve
traffic. The failure blocks both the local fresh-purged deploy and the
corresponding CI job.

Patch it at the database level once peertube has created the table on
its first crashing startup: wait for `public.application` to exist, run
an ALTER TABLE that sets defaults on the two offending columns
(`nodeVersion='', nodeABIVersion=0`), and wait for the container to
come back healthy. Peertube itself overwrites the values with real
node/ABI version strings on every subsequent boot, so the defaults only
cover the bootstrap INSERT.

Validated via make deploy-fresh-purged-apps APPS="web-app-joomla
web-app-peertube" SERVICES_DISABLED="matomo,email" FULL_CYCLE=true:
Pass 1 ok=1576 failed=0, Pass 2 ok=1444 failed=0, peertube 3/3
Playwright scenarios green in both passes.
2026-04-23 12:45:37 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f0475e393f test(lint): forbid literal no_log: true in Ansible YAML
Adds a lint test that scans roles/, tasks/, playbooks/ for
`no_log: true` (and the YAML-truthy variants `yes`/`on`, all cases).
Hard-coded `no_log: true` blocks operators from surfacing masked
values when diagnosing a failing task — the fix is to route every
no_log through `MASK_CREDENTIALS_IN_LOGS`, which defaults to true
but can be flipped for a single play with
`-e MASK_CREDENTIALS_IN_LOGS=false`.

Failure message points at the exact file:line and shows the
required replacement form:

    no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 10:46:24 +02:00
kevinveenbirkenbachandClaude Opus 4.7 e26ce8a630 test(email-lookup): isolate token store from container filesystem
setUp/tearDown patches `runtime_data._load_store_users` to return `{}`
so `_hydrate_users_tokens` can't pull a real `no-reply` token out of
`/var/lib/infinito/secrets/tokens.yml` (or any DIR_SECRETS / DIR_VAR_LIB
candidate that happens to exist in the test container). Without this,
`test_returns_plugin_defaults_when_no_vars` failed inside the CI
container with `password='395d6871414ffcbfbc3253db32328001'` because
`email._compute('password')` reads `users['no-reply'].tokens['web-app-mailu']`
and a prior deploy had persisted that token to disk. Patches the loader
rather than env vars because `_resolve_tokens_file` iterates four
candidate sources and any one of them existing reintroduces the leak.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 10:17:18 +02:00
Alejandro Roman 5d63017271 fix(Prometheus): add new apps and fix a dns conflicts 2026-04-23 12:03:16 +04:00
kevinveenbirkenbach 6f2265b7a5 fix(web-app-peertube): pin image to explicit release tag
Pin the compose image to an explicit release tag instead of the mutable
`production-bookworm` one. CI picked up a rolling-tag build that
crash-loops peertube on first startup (`null value in column
"nodeVersion" of relation "application" violates not-null constraint`
from the initial `INSERT INTO "application" (migrationVersion) VALUES
(0)`), which then blocks the OIDC plugin install with "Container is
restarting".
2026-04-23 09:40:50 +02:00
kevinveenbirkenbachandClaude Opus 4.7 f85a9e5f7e docs(agents/iteration): clarify SERVICES_DISABLED syntax + APPS rule
Spell out the verbatim env-var form (`SERVICES_DISABLED="matomo,email"`)
and call out that the value is a comma-separated provider-key list — not
a glob or a `web-app-*.compose.services.*` path. Tighten the side-effect
note so the auto-removed provider roles are explicitly excluded from
`APPS`, not just from "the deploy".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 08:39:51 +02:00
kevinveenbirkenbachandClaude Opus 4.7 14cdbcd8d3 docs(agents/iteration): machine-readable matomo+email toggle rule
Restructure the matomo/email provider disable rule into a labeled
key/value block (WHEN/ACTION/DEFAULT/ON YES/ON NO/SIDE EFFECT/PERSIST)
so agents can parse it deterministically. Switch the disable mechanism
from per-app `compose.services.*` flags to appending `matomo,email` to
`SERVICES_DISABLED` on the deploy command.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 08:12:24 +02:00
kevinveenbirkenbach 1f27f4a7cd Merge pull request #192 from infinito-nexus/dependabot/docker_compose/coredns/coredns-1.14.3
build(deps): bump coredns/coredns from 1.14.2 to 1.14.3
2026-04-23 07:48:44 +02:00
dependabot[bot] 267ee758b7 build(deps): bump coredns/coredns from 1.14.2 to 1.14.3
Bumps coredns/coredns from 1.14.2 to 1.14.3.

---
updated-dependencies:
- dependency-name: coredns/coredns
  dependency-version: 1.14.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-04-23 04:54:33 +00:00
kevinveenbirkenbach 2e280a0434 test(web-app-peertube): add Playwright spec + env for landing and OIDC flows
- playwright.spec.js: cover landing domain from applications lookup,
  administrator OIDC login+logout via dashboard, and biber OIDC
  login+logout via dashboard (biber persona required by the MUSTs).
- playwright.env.j2: expose APP_BASE_URL, OIDC_ISSUER_URL,
  PEERTUBE_BASE_URL, and administrator/biber credentials rendered
  from the users lookup.

Validated in the Pass 1 fresh-purged deploy: peertube 3/3 passed
(landing 1.0s, administrator OIDC 2.5s, biber OIDC 2.4s).
2026-04-23 00:41:27 +02:00
kevinveenbirkenbach 2bea93edaa test(web-app-nextcloud): add biber OIDC login scenario + expose biber creds
- playwright.spec.js: add "biber logs into nextcloud via OIDC and logs
  out" scenario covering the non-administrator persona required by the
  MUSTs; parameterize loginToStandaloneNextcloud on username/password,
  add a logoutStandaloneNextcloud helper, and wrap biber's first login
  in a retry so the LDAP first-login-provisioning delay does not flake.
- playwright.env.j2: expose BIBER_USERNAME and BIBER_PASSWORD so the
  spec can read biber credentials from the rendered env.
- docs/contributing/.../playwright.specs.js.md: codify the MUST that
  existing scenarios may not be deleted while optimizing a spec.
- roles/web-app-nextcloud/docs/LDAP.md: document the biber first-login
  caveat against fresh Nextcloud + LDAP deployments.

Validated via scripts/tests/e2e/rerun-spec.sh web-app-nextcloud:
3 passed (talk admin settings, dashboard to nextcloud login, biber).
2026-04-23 00:39:23 +02:00
kevinveenbirkenbach ba5e8297d3 Merge pull request #190 from infinito-nexus/update/docker-image-versions-20260422
update(images): update Docker image versions
2026-04-23 00:11:10 +02:00
github-actions[bot] 0721d5f765 update(images): update Docker image versions 2026-04-22 20:37:46 +00:00
Alejandro Roman d2d394c30c feat(prometheus-comm): fix unit test CI 2026-04-23 00:31:25 +04:00
Alejandro Roman 58aa904129 feat(prometheus-comm): fix last comments and fix new issue in matrix 2026-04-23 00:31:24 +04:00
Alejandro Roman 9869d4416a feat(prometheus-comm): fix mattermost and yourls playwright 2026-04-23 00:31:24 +04:00