Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.
Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a database-style object-store abstraction: the objstore lookup resolves the active engine (seaweedfs|minio) per consumer and yields a uniform S3 payload, objstore_secret_key is auto-generated like database_password, sys-svc-objstore delegates per-consumer provisioning to the resolved provider's objstore_init entrypoint (lazy-loading the provider when it was not preloaded), and compose/network/volume plumbing supports both a shared central instance and an embedded storage-only instance per consumer.
Add the web-app-seaweedfs provider role: Filer and Master UIs behind the admin-only oauth2-proxy via a role-local nginx and proxy.conf.j2 (S3 API domain stays ungated, admin UIs deny with 403 when SSO is disabled), per-consumer S3 identities from s3.json, idempotent bucket provisioning via weed shell, a /24 on the central-provider subnet range, Playwright coverage for admin/biber/guest plus consumer bucket existence, and unit tests for the objstore lookup and the per-engine compose volumes.
Mark web-app-minio lifecycle: unsupported, move its mc client image into a dedicated services.client entry, align it with the objstore credential contract, and make its consumer provisioning idempotent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.
Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).
Breaking change for any external override that still sets the old
name — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure 1:1 rename (60 insertions / 60 deletions across 30 files) to
align the variable names with the role that owns them and to drop the
misleading ``docker_git_`` prefix — these knobs configure the
compose-stack's git clone step, not the docker CLI:
- docker_git_repository_address → compose_repository_url
- docker_git_repository_version → compose_repository_ref
- docker_git_repository_pull → compose_pull_repository
- docker_repository_path → compose_repository_path
``ref`` better captures git refs (branches/tags/SHAs are not
"versions"); ``url`` is the idiomatic name for the clone source.
Touches every consumer (web-app-akaunting/bigbluebutton/dashboard/
fediwall/hugo/littlejs/mini-qr/minio/openproject/openwebui/
roulette-wheel/taiga + svc-db-openldap/svc-prx-openresty/sys-svc-rdbms
+ web-svc-collabora/coturn/logout/onlyoffice + sys-front-inj-all), the
role's own defaults/tasks, the bigbluebutton compose_mods filter and
its unit test, the declared-in-services lint, and the hugo
requirements doc.
Breaking change for any external override that still sets the old
names — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests/lint/ansible/test_no_redundant_default_on_module_io.py which
flags `<reg>.{stdout,stderr,rc} | default(...)` inside `changed_when:`,
`failed_when:`, or `until:` lines. Those conditionals only fire after
the module has actually executed, so the fallback can never apply --
keeping it just hides bugs and noise.
Cleans up the 21 existing offenders across 14 role files. The taiga
upsert-admin task additionally gains a `not ASYNC_ENABLED and ...`
short-circuit so its async-poll-0 path no longer leans on a silent
empty fallback for stdout/stderr; with the short-circuit, the
`default('')` is genuinely dead code.
Cross-task references (a later task reading a registered result of an
earlier task that may have been skipped via `when:`) stay out of scope
and remain a legitimate use of `| default(...)`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
ensure_mbake (mbake is pip-installable, parses our Makefile correctly
whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
(mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
`.PHONY:` declarations so mbake's parser can validate it; add the
previously-missing `install-python-dev` and `build-dependency` PHONY
targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
`^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
+ make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
lint-gate.needs.
Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
galaxy-importer when it runs in roles/).
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Eliminates the last 102 violations of test_project_root_import by
making PROJECT_ROOT in each package's __init__.py the single source
and inlining every wrapper helper.
Migrations
==========
- ~70 files: ``Path(__file__).resolve().parents[N]`` → ``from <pkg>
import PROJECT_ROOT`` (single-expression shape, scripted).
- ~12 files: two-statement helpers (``T = Path(__file__).resolve();
R = T.parents[N]``) and multi-line ``os.path.abspath(os.path.join(...
os.pardir...))`` chains rewritten by a follow-up scripted pass.
- ~20 files migrated by hand for shapes the scripts couldn't safely
rewrite (helper functions like ``_repo_root_from_test_file``,
embedded ``os.pardir`` chains, etc.).
Helpers removed (inlined to PROJECT_ROOT)
=========================================
- All ``def repo_root_from_here() / _project_root_from_here()`` and
same-shape variants in cli/ (``cli/deploy/development/{down,common,
build,deploy}.py``, ``cli/meta/categories/invokable/command.py``,
``cli/meta/applications/resolution/{combined/repo_paths,
dependencies/__main__,run_after/__main__}.py``).
- All zero-arg ``def <name>() -> Path: return PROJECT_ROOT`` helpers
(utils/invokable.py, utils/roles/resource_validation.py, cli/meta/
runtime/__init__.py, cli/meta/callorder/__main__.py, plus a dozen
test files).
- 4 ``PROJECT_ROOT = PROJECT_ROOT`` self-assignments removed from
test files where the class attribute became dead after migration.
Test-mock updates
=================
``patch.object(repo_paths, 'repo_root_from_here', return_value=ROOT)``
→ ``patch.object(repo_paths, 'PROJECT_ROOT', ROOT)`` (27 patches across
8 unit-test files); same shape for ``patch('utils.invokable._repo_root',
return_value=…)``.
Suppressions documented
=======================
5 files marked ``# nocheck: project-root-import`` for the catalog's
documented exemptions:
- 2 lints walking from a discovered glob match (<role>/templates/...)
up to its role dir, NOT the repo root.
- ``cli/validate/inventory/__main__.py`` invoked as standalone
subprocess (relative imports unavailable).
- ``plugins/filter/value_generator.py`` loaded by Ansible outside the
project package (relative imports broken).
- ``roles/test-e2e-playwright/filter_plugins/discover_playwright_roles.py``.
Bugs surfaced and fixed in the same commit
==========================================
The migration uncovered two latent role-content violations that were
silently masked by buggy ``_repo_root()`` helpers in two integration
tests (the helpers walked upward looking for marker dirs and returned
``tests/integration/`` instead of the repo root, so the tests scanned
nothing under ``roles/`` for years):
- roles/web-app-minio: extracted 6-task OIDC block into 02_oidc.yml +
``include_tasks`` (the documented "Good" pattern from
test_block_size; parsing cost is paid only when OIDC is enabled).
- roles/web-app-bluesky/tasks/01_core.yml: collapsed 3 sequential
``when: BLUESKY_WEB_ENABLED | bool`` tasks into one ``block:`` with
the guard at block level (4 → 2 occurrences, under threshold).
Two semantic-bug fixes carried alongside
========================================
- cli/core/app.py: ``cli_dir`` was previously ``parents[1]`` (= cli/),
not the repo root; rewritten to ``PROJECT_ROOT / "cli"`` to match
the variable's intent.
- cli/make/__main__.py: pre-existing bug — ``repo_root`` was actually
the ``cli/`` directory (so ``cli/Makefile`` was the search target,
which never existed). Now correctly uses PROJECT_ROOT.
- scripts/lint/sort_claude_settings.sh: invokes utils/claude/
settings_sort.py as a module (``python -m``) so its
``from . import PROJECT_ROOT`` resolves.
Verification
============
- ``ruff check .`` → clean
- ``tests.lint.repository.test_project_root_import`` → OK (102 → 0)
- 1621 unit tests → all pass
- Integration suite: 44 failures (baseline, pre-existing, none from
this commit), 1 skipped
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements requirement docs/requirements/013-alpha-to-beta-promotion.md
for the in-scope subset:
- web-app-minio: ✅/✅/✅ V0 OIDC + V1 LDAP variants green
- web-app-joomla: ✅/✅/🛠️ V0 OIDC + V1 LDAP variants green
- web-app-bridgy-fed: ❌/❌/❌ baseline (all 3 documented exceptions)
- web-svc-libretranslate: 🛠️/❌/❌ baseline (LDAP+RBAC documented exceptions)
- web-app-postmarks: 🛠️/🛠️/❌ baseline (RBAC documented exception)
- web-app-bluesky: 🛠️/🛠️/❌ baseline (RBAC documented exception)
Each role's `meta/services.yml.<entity>.lifecycle` is now `beta`. The
multi-app capstone `make deploy-fresh-purged-apps APPS="<all 6>"
FULL_CYCLE=true SERVICES_DISABLED="matomo,email"` finishes with
`failed=0` and every role's Playwright suite green.
Per-role highlights:
* MinIO. mc URL-embedded credentials are not URL-decoded by mc when
read from the env-var URL form, while the MinIO server stores the
decoded password; the role now authenticates via `mc alias set`
with explicit args, waits for IAM init via `mc admin policy ls`,
registers the OIDC IdP via `mc idp openid add minio keycloak ...`
(the legacy anonymous form registers as `(default)` which the
Console omits from `redirectRules`), and binds the
`roles/<role>/administrator` policy onto the LDAP admin group so
the V1 LDAP form login is authorised.
* Bridgy-fed. Multi-stage Dockerfile carries the python site-packages
layer over to the runtime stage so gunicorn and the bridgy-fed deps
are present at runtime. Removes the colon from
`meta/main.yml.galaxy_info.description` so the dashboard config
rendering does not produce invalid YAML.
* Joomla. Adds `oidc:` enable + `ldap:` block in meta/services.yml,
matrix-deploy variants for the two auth flows, and an extended
Playwright env exposing OIDC issuer, biber persona, canonical
domain, and the `<service>_SERVICE_ENABLED` gate flags.
* LibreTranslate. Adds `container_port=5000`, drops the
`dmbs_excl.yml.j2` depends_on partial that produced an empty
`'': condition: service_healthy` map, and pins
`LT_LOAD_ONLY=en,de` so the deploy gate's readiness probe does not
race the upstream argos-translate model mirror.
* Postmarks. Built from scratch as a full role (was a stub): tasks,
Node-based Dockerfile that clones ckolderup/postmarks, compose
template, env with SESSION_SECRET / ADMIN_KEY pulled from
meta/schema.yml credentials, server.yml with subnet, Playwright
env + spec.
* Bluesky. Adds `files/playwright.spec.js` and
`templates/playwright.env.j2` for the deploy gate. The Keycloak
event-listener bridge to PDS `com.atproto.server.createAccount`
remains to be built in follow-up work; until then the spec
exercises baseline reachability and the OIDC/LDAP/RBAC gaps stay
documented in `roles/web-app-bluesky/README.md` per
lifecycle.md.
Shared infrastructure:
* `roles/test-e2e-playwright/tasks/run_one.yml`. The wait-for-app
probe now accepts 4xx as proof of life; the prior `<400` boundary
rejected legitimate 404 landing surfaces (bridgy-fed serves
federation endpoints under specific paths, not `/`). 5xx remains
the only signal of an unhealthy app.
README.md SSO/RBAC exception sections per
lifecycle.md (`docs/contributing/design/services/lifecycle.md`):
* web-app-bluesky — RBAC exception
* web-app-bridgy-fed — full SSO/RBAC exception
* web-app-postmarks — RBAC exception
* web-svc-libretranslate — LDAP + RBAC exception
Out of scope for this commit (deferred to follow-up requirements):
* Bluesky Keycloak event-listener SPI for PDS account auto-
provisioning (needs Java SPI build pipeline + JAR mount + realm
wiring).
* LibreTranslate sidecar oauth2-proxy on the UI subpath only (must
not gate the API endpoints).
* Postmarks sidecar oauth2-proxy in front of the web UI.
* Joomla community OIDC plugin install + group-mapping config.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop deprecated docker_compose_file_creation_enabled flag (defaults + all consumers)
- Always run docker-compose file routines and bring stack up (caller can disable auto-flush via docker_compose_flush_handlers)
- Treat docker-compose.yml.j2 as optional via first_found (skip if missing)
- Clean up role template vars (remove removed flag, fix alignment)
- Record new E2E CI failures for Akaunting (502) and Baserow (timeout)
Fixes BigBlueButton CI failure caused by missing .env in repository:
Previously file routines could be skipped, so /opt/docker/bigbluebutton/services/repository/.env
was never created/linked and scripts/generate-compose failed with:
"cat: .env: No such file or directory" and "EXTERNAL_IPv4 is not set in .env".
With file management always enabled, the env file is copied/linked before compose generation.
https://chatgpt.com/share/69681be9-f944-800f-9b67-8afdc3a256fc
This commit updates multiple roles to ensure compatibility with Ansible 2.20.
Several include paths and task-loading mechanisms required adjustments,
as Ansible 2.20 applies stricter evaluation rules for complex Jinja expressions
and no longer resolves certain relative include paths the way Ansible 2.18 did.
Key changes:
- Replaced legacy once_finalize.yml and once_flag.yml with the new structure
under tasks/utils/once/finalize.yml and tasks/utils/once/flag.yml.
- Updated all include_tasks statements to use 'path_join' with playbook_dir,
ensuring deterministic and absolute file resolution across roles.
- Fixed all network helper includes by converting direct relative paths such as
'roles/docker-compose/tasks/utils/network.yml' to proper Jinja-evaluated paths.
- Normalized MATOMO_* variable names for consistency with the updated variable
scope behavior in Ansible 2.20.
- Removed deprecated patterns that were implicitly supported in Ansible 2.18
but break under the more strict variable and path resolution model in 2.20.
These changes are part of the full migration step required to ensure the
infinito-nexus roles remain stable, deterministic, and forward-compatible with
Ansible 2.20.
Details of the discussion and reasoning can be found in this conversation:
https://chatgpt.com/share/69300a8d-24d4-800f-bec0-e895a695618a
Replaced default 'item' with explicit loop variable 'minio_proxy' to prevent
conflicts with inner roles (e.g., sys-stk-front-proxy using with_first_found).
This resolves the undefined variable error and removes the Ansible warning
about the loop variable already being in use.
Reference: https://chatgpt.com/share/692c5778-2b24-800f-b203-5b6b2c9b1d4b
- Use dockerized mc with MC_HOST_minio (stateless), no temp files/dirs
- Create only RAW policy name with slash to match Keycloak claim
- Split policy: s3:* on S3 ARNs; admin:* on Resource "*"
- Add mc vars (image, MC_HOST components) to vars/main.yml
- Remove unused Ollama dependency block from tasks
Refs: ChatGPT conversation → https://chatgpt.com/share/68d1eab9-a35c-800f-aa81-76fb2101bd93
- Added OIDC and LDAP feature flags in config
- Introduced API/Console URL vars for proxy alignment
- Implemented automatic MinIO policy creation for OIDC admin group
- Replaced static env.J2 with dynamic env.j2 (OIDC-aware)
- Added policy.json.j2 template with full admin rights
- Cleaned up tasks to use stdin instead of file for mc policy apply
Ref: https://chatgpt.com/share/68d1d3ef-ca84-800f-abe2-11ab70e20c4e
• Add svc-ai category under roles and load it in constructor stage
• Create new 'svc-ai-ollama' role (vars, tasks, compose, meta, README) and dedicated network
• Refactor former AI stack into separate app roles: web-app-flowise and web-app-openwebui
• Add web-app-minio role; adjust config (no central DB), meta (fa-database, run_after), compose networks include, volume key
• Provide user-focused READMEs for Flowise, OpenWebUI, MinIO, Ollama
• Networks: add subnets for web-app-openwebui, web-app-flowise, web-app-minio; rename web-app-ai → svc-ai-ollama
• Ports: rename ai_* keys to web-app-openwebui / web-app-flowise; keep minio_api/minio_console
• Add group_vars/all/17_ai.yml (OLLAMA_BASE_LOCAL_URL, OLLAMA_LOCAL_ENABLED)
• Replace hardcoded include paths with path_join in multiple roles (svc-db-postgres, sys-service, sys-stk-front-proxy, sys-stk-full-stateful, sys-svc-webserver, web-svc-cdn, web-app-keycloak)
• Remove obsolete web-app-ai templates/vars/env; split Flowise into its own role
• Minor config cleanups (CSP flags to {}, central_database=false)
https://chatgpt.com/share/68d15cb8-cf18-800f-b853-78962f751f81
