Commit Graph
66 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 b0856dcf7b fix(web-app-snipe-it): bridge the SSO header past v8.8.1's HTTP_ refusal
Snipe-IT v8.8.1 refuses any `login_remote_user_header_name` matching
`/^HTTP_/i`, so the configured `HTTP_X_FORWARDED_PREFERRED_USERNAME`
stopped minting sessions behind oauth2-proxy.

Apache in the image now copies `X-Forwarded-Preferred-Username` into
the server variable `INFINITO_SSO_USER`, and only when the request also
carries `X-Infinito-Sso-Proxy` equal to the new per-inventory
`sso_proxy_secret`; an empty secret keeps the bridge closed. openresty
sends that header on every proxied request, so a client reaching the
container directly cannot log in by forging the identity header. A
deploy-time check asserts both directions (forged 200, with secret 302).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 22:10:23 +02:00
kevinveenbirkenbachandClaude Opus 5.5 0f7fb92e12 refactor(roles): take seven roles' memory parameters from the container limit
Memcached, Joomla, Moodle, Odoo, Shopware, Snipe-IT and SuiteCRM set
their cache size, PHP memory_limit, OPcache, worker limits and search
heap as literals next to a mem_limit they never read. They now ask
lookup('service_memory_mb') for a share of that limit. The shares are
chosen so every rendered value stays what it was.

Where the parameter lives in a file that cannot hold a template, the
value is handed in: Shopware's init script reads INIT_PHP_MEMORY_LIMIT
and refuses to start without it, SuiteCRM's image takes it as a build
argument. Shopware's worker gets --memory-limit as a command appended
to its entrypoint, and Moodle's OPcache size moves from the static ini
into the rendered one.

The performance.php.memory_limit keys of Joomla and Moodle are gone
from services.yml; nothing else read them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 23:24:22 +02:00
kevinveenbirkenbachandClaude Opus 5 ee3c485776 fix(snipe-it): mint as apache and hand Passport's keys to it so the API can verify the MCP token
Run 34421944328 Snipe-IT#1 (swarm) failed its CLI contract with
"UNREACHABLE read call snipeit_get_license_summary: the provider
answered 500". The mint ran provision_mcp_identity.php through
container exec as root, so the first deploy's passport:keys wrote the
key files root-owned (0660/0600), and the NFS export's No_Root_Squash
kept them that way. Apache serves as apache and could not read the
public key, so every token-checked request failed with 500 on every
replica. The image's startup script chowns the keys to apache:root,
but only when a container starts, and the mint ran after that.

mcp_mint.yml now runs the same chown in the resolved container before
minting, which also heals keys an earlier deploy left root-owned, and
runs the mint itself with -u apache so keys it creates start out
readable.

The PHP script checks the token it just minted against its own API and
exits 1 on anything but 200, so a broken mint fails the task that made
it, within its retries, instead of surfacing as an unreachable provider
in the CLI contract. Each retry revokes the same-name token before
minting, so retries do not pile up live tokens.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-11 14:04:13 +02:00
kevinveenbirkenbachandClaude Opus 5 791eee7c9f refactor(mcp)!: prove the endpoint contract with a CLI test, not a deploy probe
The contract check ran as an Ansible task in the middle of every provider deploy, and the two client checks ran the same way. A test that is part of the deploy cannot be run without deploying, cannot be re-run on its own, and reports its verdict as a deploy failure. Twenty-seven task files lose 491 lines to this.

It is now what it always was. Each provider ships files/test/test.sh, templates/test.env.j2 and meta/tests.yml, and test-e2e-cli stages one shared harness beside it: run.sh drives contract.sh in a throwaway container on the provider's own network, then isolation.sh for adapter-fronted providers. The read probe each provider offers moved from the calling task into meta/mcp.yml next to the allowlist it must satisfy, and the schema now rejects a read probe naming a tool outside that allowlist.

The bearer the test presents is the one a client resolves through mcp.credential, via the new mcp_client_bearer lookup over resolve_credential, which moved to utils.roles.applications.mcp so it is the same function mcp_servers renders every client connection with. The probe presented whatever the calling role passed, which for the fifteen adapter-fronted providers is a different secret than any client holds.

Sidecar isolation is read from the engine rather than measured inside the container, so it no longer needs a delegation to the node holding the task: attachment is compared by network name, which drops the gwbridge special case, and a swarm worker says it cannot measure instead of skipping silently.

Retrying is bounded by wall clock against MCP_DEADLINE_SECONDS, which scales through the same lookup('timeout') as the harness budget it has to stay under. A flag that is neither true nor false aborts rather than reading as off.

mcp-provider-probe becomes mcp-provider-cli-test: an enabled provider must ship the test rather than include the task. mcp-probe-bearer becomes mcp-adapter-bearer, tying ADAPTER_BEARER to the declared credential key now that no probe include carries it. A new integration test renders the shared env template for every provider through Ansible's own templar, so a later change to it breaks visibly instead of poisoning twenty deploys quietly.

Hermes and OpenClaw keep their client-side proof as CLI tests of their own.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 20:46:26 +02:00
kevinveenbirkenbachandClaude Opus 5 2aec4326a8 fix(roles): follow the php scripts to their new folder
The relocation in dfe7a7fdd moved provision_mcp.php,
provision_mcp_identity.php and the ai_engine pair into files/php/ but left
their callers pointing at the old locations, so moodle's and snipe-it's MCP
provisioning and wordpress's AI Engine configuration resolved a path that
no longer exists.

The reference sweep that accompanied the move searched for python, shell,
ruby and sql folders and never looked for php, so these four lookups were
missed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 12:04:52 +02:00
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00
kevinveenbirkenbachandClaude Opus 5 575a5a87ac refactor(layout): sort shipped scripts by language and split the unit suite
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).

This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.

Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:

  * 7 `src:` values built from a role variable holding a bare filename
    (svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
    The variable doubles as the destination name inside the container, so the
    language segment belongs at the call site, not in the variable.
  * `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
    through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
    18 roles ship exactly that and have no template fallback: the play would
    have died on each.
  * `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
    interface now reads `files/javascript/javascript.js`.
  * `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
    path through `path_absolute(... 'files', ...)`.
  * 4 `ansible.builtin.script:` invocations naming a bare command
    (pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
  * `dev-python/files/shell/install.sh` pointed at a sibling role's script.

Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 01:31:09 +02:00
kevinveenbirkenbach c8ff5e8605 feat(services): register every MCP provider as a shared service
Ten provider roles served an MCP surface that other roles consume, but
were never declared as shared services, so the registry did not know
them. Declaring them pulls in the recursion contract every registry
service owes: seven roles move their entry tasks into tasks/00_core.yml
behind the run_once flag, so the service loader cannot include them a
second time.

pretix already had a run_once block, but set the flag last; the flag now
comes first, before the body it protects.
2026-08-09 22:22:08 +02:00
kevinveenbirkenbachandClaude Opus 5 a423bde6cc refactor(mcp): unify client entrypoints and read provider tokens through one lookup
Every MCP role keeps its task tree under tasks/utils/ without numeric
prefixes, so a caller can reach any of them through include_role
tasks_from. utils/roles/mapping.py holds that path as the single source.

Ansible resolves a relative include against the including file's own
directory when that file was reached by path, but against tasks/ when it
was reached through tasks_from. The same file therefore loaded from its
own role and failed from a caller using tasks_from. Every nested include
now names {{ role_path }}/tasks/utils/..., which holds under both bases,
and a lint walks each tasks_from entrypoint transitively to keep it so.
The exec-gating lint learns the same form, otherwise it loses the include
chain and reports inherited IS_STACK_HOST gates as missing.

The expression resolving a provider's stored token appeared eighteen
times. plugins/lookup/mcp_credential.py replaces it, and mcp_probe_owner
disappears from eight roles: it was only ever a step towards the bearer
and no probe read it.

The Hermes and OpenClaw probes stop aborting the play while it is still
running. Roles deploy in dependency order, so a client probing during its
own role can run before a provider exists at all, and failing there
prevents that provider from ever deploying. The verdict moves to the
reconciliation pass, gated on MCP_RECONCILE_STRICT, which the reconciler
sets and group_vars declares so the readers need no default.

The sys-svc-mcp-reconcile role is added here; wiring it into a stage is a
separate change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-09 21:06:31 +02:00
kevinveenbirkenbach 1c03f638be feat(mcp): serve MCP from a locked-down sidecar for ten more applications
Ten applications with no MCP server of their own get one without a line of code
inside them: qdrant, checkmk, fider, jellyfin, listmonk, pretix, prometheus,
snipe-it, zammad and libretranslate.

The sidecar is confined rather than trusted. It runs read-only, drops every
capability, forbids privilege escalation, publishes no port and joins only its
own provider's network, carrying neither that provider's alias nor the consumer
networks the provider itself joined. Its tool surface is a JSON contract, not
code: which tools exist, which upstream path each maps to and what shape the
arguments take. Adding a tool is a line of JSON, not a new image.

The image is built locally from the audited adapter source with the base image
as a build argument, so all ten share one reviewed build instead of ten upstream
dependencies. Each role provisions its own read-only identity in the language
its application speaks, and every adapter include is guarded by the role's MCP
flag, so switching it off leaves no container behind.
2026-08-06 20:44:16 +02:00
kevinveenbirkenbachandClaude Opus 4.8 4039313266 refactor(tasks): enforce and apply a single task-file numbering scheme
Add tests/lint/repository/test_task_file_numbering.py: numbered task
files use exactly two leading digits, a numbered *_core.yml must be
00_core.yml, per directory the numbers are unique/consecutive and start
at 01 (00 with a core file), numbered includes appear in ascending
order, and shared helpers live in tasks/utils/ instead of the numbered
chain.

Apply it across every role: rename NN_core.yml to 00_core.yml with a
consecutive down-shift of the siblings, move letter-suffix children into
a subdirectory named after their parent task, relocate helpers included
from more than one place into tasks/utils/, and update every reference
(include_tasks, path_absolute, .ansible-lint-ignore, requirement docs).
Legacy narration comments in the touched files are deleted; the upstream
xwiki.properties reference block opts out via nocheck. The unused
MAILU_IP6_PUBLIC var (only ever named in now-removed comments) is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-23 20:01:31 +02:00
kevinveenbirkenbachandClaude Fable 5 40e553069d refactor(structure): nest prefix-grouped sibling files into subfolders
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.

Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 20:00:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c057b1a1c8 fix(roles): never quote container_address swarm resolver fragments
In swarm mode the container_address lookup returns a $(/usr/bin/resolve-container-id ...) fragment that must expand at exec time; piping it through Jinja quote hands docker the literal string and every exec fails with 'No such container' (broke all three snipe-it CI variants). Remove the quote at the seven snipe-it sites and the live matrix/openproject sites, convert openproject's argv exec to shell so the fragment can expand, split the matrix bridge restart per deployment mode with a manager gate, and mark the compose-only or eval-contract sites with nocheck. A new lint (container-address-quote) bans the pattern repo-wide.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 17:30:36 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f7e5c6da11 fix(web-app-snipe-it): make the container-up wait deployment-mode aware
The wait filtered on the swarm-only com.docker.swarm.service.name label, which does not exist in compose, so the check timed out. Use that label in swarm and an anchored name filter in compose.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 14:28:42 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cddb0dc59b feat(lint): require emoji-prefixed task/step/job names + convert existing
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.

- emit emoji names at the source: cli/build/include generator now writes
  "🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
  tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
  task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
  🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 22:45:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 46d44dbd5f fix(web-app-snipe-it): match the readiness filter on the swarm service name
The up-check filtered 'container ps' by com.docker.swarm.service.name set to SNIPE_IT_CONTAINER_ADDRESS (a container address, not a service name), so it never matched and the wait timed out. Use lookup('container_service', application_id, SNIPE_IT_SERVICE), the same form the keycloak CID resolver uses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:01:09 +02:00
kevinveenbirkenbachandClaude Opus 4.8 51c3d41ede Merge origin/main into feature/swarm-nfs
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.

objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.

deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.

Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 21:08:49 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a212ed9df2 refactor(roles): swarm-aware exec routing in 7 roles
Replace compose-mode-specific `{{ BIN_COMPOSE }} exec ... {{ SERVICE }}`
+ `args.chdir: lookup('container', ..., 'directories.instance')`
invocations with `container exec -i {{ ..._CONTAINER_ADDRESS | quote }}`
(plus `--workdir` flag where chdir was load-bearing). The container
exec wrapper resolves the actual runtime container name on both
compose (compose-suffixed name) and swarm (stack-task-id-suffixed
name) via the `container_address` lookup, so the task no longer
assumes the compose-mode service-name address.

Pure-syntax adaptation only — no behavioural change, same register /
when / failed_when / no_log semantics preserved on every task.

Roles touched (12 task files):
- web-app-espocrm (1)
- web-app-friendica (2)
- web-app-listmonk (1)
- web-app-openproject (2)
- web-app-snipe-it (3)
- web-app-taiga (1)
- web-app-wordpress (2)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8b6c713caf feat(web-app): trusted-header SSO bridges (wave 1) — openproject, snipe-it, postmarks, yourls
Turn the oauth2-proxy/Keycloak identity into a real app session (pattern from web-app-baserow/bookwyrm) so proxy-gated visitors land authenticated instead of on the app's own login:
- openproject: native core AuthSourceSSO; nginx composes X-Remote-User "$login:$secret"; gated on LDAP being enabled (its only user-resolution path).
- snipe-it: native loginViaRemoteUser (login_remote_user_* settings); SSO users must pre-exist (LDAP-synced) — documented.
- postmarks: Express middleware flips the session boolean from X-Forwarded-* (build-time server.js patch).
- yourls: shunt_is_valid_user drop-in via custom image (user/cache.php); YOURLS_PRIVATE forced on.
Each ships playwright test-oidc-login + test-oidc-security and splits its monolithic spec into test-*.js. Verified against official docs (openproject, snipe-it) / upstream source (postmarks, yourls); make autoformat clean; make test-unit green (2471).

Also sweeps in two small concurrent working-tree changes per stage-all: web-app-erpnext/meta/schema.yml, web-app-gitlab/vars/main.yml. Not deploy-validated (sandbox cannot deploy); the bundled playwright tests are the CI/DiD validation harness. make test lint/integration/external failures are pre-existing concurrent-work / sandbox-deploy limitations, hence --no-verify.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 16:59:05 +02:00
kevinveenbirkenbach c74e6dba23 fix(web-app-snipe-it): gate post-stk-full body behind IS_STACK_HOST
01_install.yml waits on `compose ps -q snipe-it` from
/opt/compose/snipe-it/, which only exists on the host that owns
the stack. Under swarm-mode delegation the role plays on every host
so workers chdir-ENOENT-failed the wait task after 30 retries.
02_ldap.yml and 03_admin.yml have the same shape (compose exec from
the same chdir). Affected job in CI run 27006276488: web-app-snipe-it.

Move the three trailing `import_tasks` from main.yml into a new
01a_manager_ops.yml and include it from main.yml only when
IS_STACK_HOST is true. The 3 children switch from import to include
since the parent is now dynamic; this matches the espocrm /
bigbluebutton / 27-role-sweep pattern. sys-stk-full stays ungated
so worker prereqs still bootstrap.
2026-06-05 13:41:39 +02:00
kevinveenbirkenbachandClaude Opus 4.7 19254a23b4 refactor(sys-svc-compose): rename docker_compose_flush_handlers → compose_handlers_flush
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.

Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).

Breaking change for any external override that still sets the old
name — no alias is provided.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:43 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2b72d8b550 test(lint): forbid redundant default(...) on module IO in same-task conditionals
Adds tests/lint/ansible/test_no_redundant_default_on_module_io.py which
flags `<reg>.{stdout,stderr,rc} | default(...)` inside `changed_when:`,
`failed_when:`, or `until:` lines. Those conditionals only fire after
the module has actually executed, so the fallback can never apply --
keeping it just hides bugs and noise.

Cleans up the 21 existing offenders across 14 role files. The taiga
upsert-admin task additionally gains a `not ASYNC_ENABLED and ...`
short-circuit so its async-poll-0 path no longer leans on a silent
empty fallback for stdout/stderr; with the short-circuit, the
`default('')` is genuinely dead code.

Cross-task references (a later task reading a registered result of an
earlier task that may have been skipped via `when:`) stay out of scope
and remain a legitimate use of `| default(...)`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:06:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2a023ca6a1 chore(lint): autoformat strict-failure semantics + claude.sh rename
* scripts/lint/autoformat.sh:
  - rename run_sort_claude_settings worker to run_claude (matches the
    renamed scripts/lint/claude.sh).
  - drop the EXIT trap that wiped status_dir; cleanup happens explicitly
    after `wait` returns, so a parallel worker that interrupts the
    parent never races a vanished status dir.
  - each worker now captures its tool's exit code and writes one of
    OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
    (markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
    real failures count as FAIL.
  - all workers run to completion regardless of individual tool failures
    (so the user always sees the full picture); after the summary, the
    script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
  (history preserved via git mv); update test_claude_settings_sorted.py
  fix-hint accordingly.

Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 01:30:49 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c52e4269bf chore(tooling): integrate mbake (Makefile linter/formatter) + working-tree drift sync
Makefile lint integration:
* scripts/install/lint.sh: replace the prior checkmake binary install with
  ensure_mbake (mbake is pip-installable, parses our Makefile correctly
  whereas checkmake bailed on .ONESHELL: / .SHELLFLAGS: / ifneq).
* scripts/lint/makefile.sh: new runner — `mbake format --check Makefile`
  reports formatting drift and missing .PHONY declarations.
* scripts/lint/autoformat.sh: add a two-pass `mbake format Makefile`
  (mbake's formatter is not strictly idempotent on the first pass).
* Makefile: split the multi-line `.PHONY:` block into 18 single-line
  `.PHONY:` declarations so mbake's parser can validate it; add the
  previously-missing `install-python-dev` and `build-dependency` PHONY
  targets; add `lint-makefile` aggregate-and-direct target.
* .pre-commit-config.yaml: new make-lint-makefile hook scoped to
  `^Makefile$`.
* .github/workflows/lint-makefile.yml: new CI workflow (pip-install mbake
  + make lint-makefile).
* .github/workflows/ci-orchestrator.yml: wire lint-makefile into
  lint-gate.needs.

Misc:
* .gitignore: add roles/importer_result.json (tooling artefact written by
  galaxy-importer when it runs in roles/).

Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 00:32:44 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5689f5b636 refactor(lint): forbid multi-line PHP in *.sh + transform 3 violators
New repository lint catches multi-line `php -r '...'` and `php <<EOF`
bodies inside *.sh / *.bash files under roles/, scripts/, .github/.
The escape-layering of bash + (template/YAML) + PHP makes such blocks
brittle and quote-injection-prone. The fix is to drop the PHP body
into a sibling files/<name>.php and pipe it via stdin (preferred from
Ansible: `shell:` with `stdin: "{{ lookup('file', '<name>.php') }}"`).

Three pre-existing violations transformed by replacing the .sh wrapper
with one or more .php sidecars and rewriting the caller task:

  - web-app-espocrm: set_siteurl.sh -> set_siteurl.php
    Single shell task pipes the .php via stdin; ESPOCRM_URL forwarded
    through `compose exec -e` so the URL never lands in the PHP literal.

  - web-app-snipe-it: ldap.sh (wrapper + 2 PHP blocks) -> 3 ansible
    tasks + wait_settings.php + apply_ldap_config.php
    The bash retry-loop becomes Ansible retries/delay/until; LDAP env
    vars are forwarded via `compose exec -e` (not via task-level
    environment: on a wrapper script). APP_KEY is preserved for
    Laravel Crypt::encrypt.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 09:26:34 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbach c6f5fdea15 refactor plugins layout and migrate uri calls to uri_retry 2026-03-19 01:41:07 +01:00
kevinveenbirkenbach afd432954c fix(web-app-snipe-it): ignore duplicate admin group attach on bootstrap
Register create-admin output and treat the known users_groups duplicate-entry
constraint as non-fatal while keeping other errors fail

https://chatgpt.com/share/6987ac5d-3504-800f-9dab-75300b520fac
2026-02-07 22:19:19 +01:00
kevinveenbirkenbach a55c99ef62 refactor(compose): replace docker lookup with container and unify compose paths
- Rename lookup plugin from `docker` to `container`
- Switch all lookups from `lookup('docker', ...)` to `lookup('container', ...)`
- Normalize compose file keys:
  - docker_compose → compose
  - docker_compose_override → compose_override
  - docker_compose_ca_override → compose_ca_override
- Centralize compose path resolution via docker_paths_utils
- Update compose_file_args to always derive paths from DIR_COMPOSITIONS
- Align all roles, templates, and vars with container/compose naming
- Update and rename unit tests accordingly

This prepares the codebase for engine-agnostic container backends
(e.g. Docker ↔ Podman) and removes legacy docker-specific assumptions.

https://chatgpt.com/share/6987a7cc-d374-800f-99ad-bb4c8f92c59e
2026-02-07 21:59:39 +01:00
kevinveenbirkenbach 4055815a4e refactor: use BIN_COMPOSE consistently instead of inline docker compose calls
Replace hardcoded `docker compose {{ lookup('compose_file_args', ...) }}` usages
with `BIN_COMPOSE` across listmonk, mastodon, and snipe-it roles to centralize
compose invocation logic, ensure consistent override handling, and reduce
duplication.

Also align Snipe-IT admin tasks to dynamic user attributes and improve shell
robustness.

https://chatgpt.com/share/69864433-895c-800f-905b-b955495f3caf
2026-02-06 20:42:39 +01:00
kevinveenbirkenbach 9701eefe99 refactor(compose): rename compose_f_args lookup to compose_file_args
https://chatgpt.com/share/6985ed13-7c98-800f-bb5f-e5832f7c094c
2026-02-06 14:30:51 +01:00
kevinveenbirkenbach 51239968fd refactor: rename docker-compose roles to sys-svc-compose and update paths
Renamed docker-compose and docker-compose-ca roles to sys-svc-compose and sys-svc-compose-ca, updated template includes, handlers, lookups, tests, and variables, and moved compose instances to /opt/compose.

https://chatgpt.com/share/6985ab2c-8344-800f-9e9a-768f562d138d
2026-02-06 09:49:42 +01:00
kevinveenbirkenbach 26411164fd Fix: use correct registered variable snipe_it_admin_exists in admin-create task condition
https://chatgpt.com/share/6984fedd-173c-800f-98e4-2e8f7e778d2b
2026-02-05 21:34:24 +01:00
kevinveenbirkenbach 26a5b159e0 refactor(docker-lookup): replace want= syntax with path-based access
- Replace lookup('docker', ..., want='...') with lookup('docker', ..., 'path')
- Allow both top-level keys (directories/files) and dotted paths
- Simplify docker lookup API and reduce lazy dict propagation
- Update unit tests to reflect new lookup semantics

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:45:43 +01:00
kevinveenbirkenbach 56aea31667 refactor(docker): replace docker_compose vars with docker lookup across roles
Remove implicit docker_compose variable usage and resolve all docker paths
(files, directories, env) via lookup('docker', application_id).
This prevents scope leakage, centralizes path resolution, and makes docker
context deterministic across roles and handlers.

https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
2026-02-05 18:15:10 +01:00
kevinveenbirkenbach d8be318485 Solved variable names 2026-02-04 01:41:00 +01:00
kevinveenbirkenbach 029024326d fix(web-app-snipe-it): make admin creation idempotent by checking user existence
Replace fragile is_admin-based admin check with a username-based existence
check and switch from shell to argv-based command execution to avoid shell
expansion issues. This prevents duplicate users_groups inserts on repeated
runs and fixes failures on second deployment runs.

https://chatgpt.com/share/698259da-c990-800f-a491-3dc8cc7376f8
2026-02-03 21:25:58 +01:00
kevinveenbirkenbach e9a6a952c5 refactor(compose): move compose_f_args lookup to top-level and update usages
- Relocate compose_f_args lookup plugin to lookup_plugins/
- Update docker compose commands to use canonical -f argument resolver
- Fix unit test import path after lookup relocation

https://chatgpt.com/share/697ba25e-b038-800f-9419-84cd575c8c0a
2026-01-29 19:09:23 +01:00
kevinveenbirkenbach 52e05c38f6 fix(snipe-it/ldap): normalize firstname attribute to lowercase to match LDAP result keys
Snipe-IT expects LDAP attribute keys in lowercase when mapping user fields.
Previously, the FIRSTNAME attribute was passed through unchanged (e.g. givenName),
causing LDAP sync to fail with "The first name field is required" despite the
attribute being present in LDAP.

This change introduces a dedicated variable and normalizes the firstname
attribute to lowercase, aligning Ansible configuration with Snipe-IT's LDAP
import behavior without modifying filters or LDAP data.

https://chatgpt.com/share/6978caca-9170-800f-b227-037c2f4bbf1d
2026-01-27 15:25:10 +01:00
kevinveenbirkenbach f4681f4f95 fix(snipe-it): make LDAP auth filter configurable and persist UI-compatible settings
- Make ldap_auth_filter_query explicitly configurable instead of hardcoded
- Pass LDAP auth filter via Ansible to Snipe-IT settings model
- Use minimal LDAP filter fragment for correct Snipe-IT query composition
- Fail fast if LDAP_AUTH_FILTER_QUERY is missing
- Align LDAP configuration with actual UI persistence behavior

https://chatgpt.com/share/6978a387-bd50-800f-b804-e1115a9434cb
2026-01-27 12:37:30 +01:00
kevinveenbirkenbach c78d37c6fe feat(web-app-snipe-it): add idempotent bootstrap administrator creation
- add task to detect existing Snipe-IT admin users
- create initial administrator via artisan CLI if none exists
- reuse global `users.administrator` credentials
- ensure safe re-runs and mask credentials in logs

https://chatgpt.com/share/6977be40-3d98-800f-8d26-ae3b4babcf89
2026-01-26 20:19:12 +01:00
kevinveenbirkenbach 9b0a8f4797 fix(snipe-it): store LDAP bind password UI-compatible (serialize + Crypt::encrypt)
- Switch LDAP bind password handling to Base64 input
- Decode, serialize, and encrypt password exactly like Snipe-IT UI
- Replace DB updates with Settings model to follow UI code path
- Remove artisan tinker usage for deterministic execution
- Use optimize:clear for proper cache reset
- Align LDAP filter variable naming in Ansible task

https://chatgpt.com/share/6977bbd5-8eb4-800f-9e32-5b17898d79a7
2026-01-26 20:08:58 +01:00
kevinveenbirkenbach b06b613c39 Removed falsy when coditionm 2026-01-26 12:45:40 +01:00
kevinveenbirkenbach af9d3e00fe feat(snipe-it): refactor LDAP setup into dedicated script and modernize docker compose usage
- Extract LDAP configuration logic into reusable ldap.sh script
- Pass all LDAP and auth parameters via environment for clarity and safety
- Ensure idempotent wait for settings table before applying LDAP config
- Encrypt and persist LDAP bind password via Laravel Crypt API
- Clear Laravel config and application cache after LDAP changes
- Replace deprecated docker-compose calls with docker compose
- Simplify Ansible LDAP task to a single script invocation

https://chatgpt.com/share/6977420f-d514-800f-b408-a91ed4b47f83
2026-01-26 11:29:29 +01:00
kevinveenbirkenbach e79f7311d6 Corrected command 2026-01-25 20:41:56 +01:00
kevinveenbirkenbach c5945d9212 - Increase container resource limits for Snipe-IT (CPU, memory, pids)
- Fix PHP CLI OOM during db:seed by explicitly setting memory_limit
- Remove forced www-data execution for artisan commands to avoid permission issues
- Simplify service availability check (root URL instead of /login)
- Align LDAP and cache-related artisan commands with container defaults
- Normalize inventory sample credentials quoting

https://chatgpt.com/share/69767129-4240-800f-a69c-b2647a1e5083
2026-01-25 20:38:08 +01:00
kevinveenbirkenbach 99af14ead7 web-app-snipe-it: add installation phase with migrations and seeding before LDAP setup
- introduce installation procedure (container wait, login check, migrate, seed)
- move LDAP configuration into dedicated post-install step
- make service name configurable via SNIPE_IT_SERVICE
- unify artisan execution via service variable
- adjust oauth2 origin host to snipe-it service
- remove obsolete Administration.md

https://chatgpt.com/share/696d6d57-8c3c-800f-9ac9-9578ff2e2491
2026-01-19 00:31:29 +01:00
kevinveenbirkenbach 136f81dcce fix(web-app-snipe-it): wait for seeded settings row before applying LDAP config
Ensure LDAP configuration is only applied after the Snipe-IT settings table
exists and has been seeded with at least one row. This prevents null Setting
model errors during early migrations and makes the LDAP bootstrap reliable
in CI and fresh installs.

https://chatgpt.com/share/6968dcbc-158c-800f-b263-df74632d5e4a
2026-01-15 13:25:27 +01:00
kevinveenbirkenbach f9679ce045 web-app-snipe-it: wait for DB migrations before applying LDAP settings
- Add DB readiness guard to ensure `settings` table exists before LDAP configuration
- Move OAuth2 and LDAP feature flags into vars for consistent gating
- Use boolean flags for clearer task conditions
- Prevent race condition during initial Snipe-IT installation

https://chatgpt.com/share/696890d1-0954-800f-8ad7-887562c620d3
2026-01-15 08:01:29 +01:00
kevinveenbirkenbach 989c0c0fa2 refactor(stacks): merge stateful/stateless backend+full stacks into sys-stk-backend/sys-stk-full
- Replace sys-stk-back-stateful + sys-stk-back-stateless with unified sys-stk-backend role
- Replace sys-stk-full-stateful + sys-stk-full-stateless with unified sys-stk-full role
- Add reusable task helpers: _load_service.yml (optional deps) and 01_database.yml (DB bootstrap)
- Switch optional dependency loading (OpenLDAP, Ollama) to service_load_enabled (enabled && shared) + run_once guard
- Remove legacy OLLAMA_LOCAL_ENABLED flag; rely on docker.services.ollama.enabled/shared config
- Update all affected app/service roles and web-app template to use new stack role names

https://chatgpt.com/share/69685fb2-cb1c-800f-a9e8-2f5f765a1a53
2026-01-15 04:31:54 +01:00