svc-db-postgres 02_init.yml asserts application_id == 'svc-db-postgres' AND database_consumer_id \!= 'svc-db-postgres'. When 01_core is skipped (run-once), the include_role application_id var does not reliably reach 02_init through a consumer's nested looped include (web-app-matrix's per-bridge DB seeding), so the assert fails. Pin application_id on the 02_init include so it holds for any caller. On the matrix side, freeze the consumer id eagerly before the provider include so database_consumer_id stays the matrix consumer instead of lazily resolving to svc-db-postgres once application_id is pinned.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The integration_zammad Connect flow reached the partner /oauth/authorize with a malformed client_id, so Zammad's Doorkeeper rejected the unknown client UID and 302-redirected to its homepage (the spec then captured the partner root, not the authorize page). The encrypted-appconfig helper stored the credentials via setValueString(..., lazy: true) WITHOUT sensitive: true, which mangled the client_id on read. Store it sensitive so the provisioned client_id round-trips intact and the OAuth authorize carries the correct UID.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Joomla OIDC plugin's server-side discovery/token cURL to Keycloak hardcoded the CA path to /etc/ssl/certs/ca-certificates.crt, which only carries the Infinito root CA once the CA-trust wrapper has run update-ca-certificates in the container. The wrapper is skipped on Joomla because the locally-built joomla_custom image does not exist when the CA-trust override is generated, so PHP cURL never trusts auth.infinito.example and OIDC fails with cURL error 60. Resolve the CA bundle from the CURL_CA_BUNDLE/CA_TRUST_CERT/SSL_CERT_FILE env vars (which compose-ca sets and bind-mounts reliably regardless of the wrapper), falling back to the system bundle. setVerifyHost/setVerifyPeer stay true — TLS verification is not relaxed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pixelfed's Laravel OIDC token exchange to Keycloak uses php-curl, which verifies against libcurl's OS bundle (/etc/ssl/certs/ca-certificates.crt), not the CURL_CA_BUNDLE/SSL_CERT_FILE env vars the compose_ca wrapper exports. The custom docker-entrypoint now installs the injected internal CA via update-ca-certificates (installing ca-certificates first if absent) so the OIDC callback's server-side TLS succeeds instead of failing with cURL error 60.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The wordpress (php-apache/Debian) image lacks ca-certificates, so the sys-ca-selfsigned with-ca-trust.sh wrapper's update-ca-certificates path is skipped and the internal self-signed CA never lands in /etc/ssl/certs/ca-certificates.crt — the bundle PHP libcurl verifies against. The daggerhart OIDC plugin then fails server-side TLS with 'cURL error 60: unable to get local issuer certificate' (confirmed in the Playwright artifact). Add ca-certificates so update-ca-certificates runs and the CA is trusted. compose_ca.py alone is insufficient because PHP libcurl uses the OS bundle, not the CURL_CA_BUNDLE/SSL_CERT_FILE env vars.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The account addon spec opened /odoo/accounting (the journal dashboard, which in Odoo 19 has no .o_account_dashboard and not the asserted invoice vocabulary) and timed out even though the account module is installed. Navigate to /odoo/invoicing — the canonical account.move customer-invoices action — and assert the invoices list/kanban plus account-specific vocabulary (customer invoice / register payment / invoice date / tax excluded). Verified green live against Odoo 19 via 'make compose-playwright role=web-app-odoo'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The lazy object-store provider include cleared compose_repository_url with an inline literal "" to stop the consumer's repo coordinates leaking into the provider, but the compose-repository lint requires the URL to be a Jinja expression paired with compose_repository_ref. Express the empty clearing as "{{ '' }}" and pair it with an empty compose_repository_ref — same runtime effect (the object-store provider is image-based, no git clone), now lint-clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Upstream requirements.txt pins brevity at git+...@new-tlds while granary requires brevity at the same git URL without a ref; pip treats the two direct URLs as conflicting and aborts the build. Strip the @new-tlds ref so both requirers agree (brevity master is new-tlds minus one trivial additive commit).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The account addon spec asserted bare .o_control_panel/.o_breadcrumb, which render on every Odoo page, so it passed green even when the account module was enabled-but-not-installed. Require Accounting domain vocabulary (Customer Invoices / Vendor Bills / Chart of Accounts / Journal Entries) and text-gate the generic containers, mirroring the crm/stock specs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The admin gunicorn at localhost:8080/api/v1 may not be ready when the token tasks run; add until/retries(30)/delay(2). Create uses curl -sS -f so the until requires an HTTP 2xx, and failed_when still fails the play loudly after retries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mastodon emits inline initial-state/bootstrap <script> blocks server-side; the reverse proxy cannot nonce/hash upstream HTML, so the CSP healthcheck flagged a refused-to-execute violation. Set script-src-elem.unsafe-inline (script-src-attr stays hardened), matching every fediverse SPA sibling (pixelfed/bookwyrm/friendica/mobilizon).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When no XWiki addons are enabled (XWIKI_PLUGINS empty), the probe-results set_fact loop runs zero times and _xwiki_probe_results stays undefined, so the selectattr in the 'all requested extensions installed' assert raised AnsibleUndefinedVariable. Default it to [] so 'nothing requested' means 'nothing missing'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Nextcloud runs pretty-URLs (front_controller_active), so each integration app emits its oauth-redirect WITHOUT /index.php at runtime. The partner OAuth apps were registered with only the /index.php variant, so exact-match validation rejected the real callback and the per-user connect silently failed while the deploy and Playwright specs went green. Register both variants (the no-/index.php one first). Build the list on the consumer side (Doorkeeper newline-join for gitlab, native JSON callback_urls array for mattermost, Doorkeeper for openproject) to avoid newline-in-argv.
SuiteCRM is unaffected (OAuth2 password grant, no redirect_uri); the Nextcloud->OpenProject client callback is an OpenProject URL and is left untouched. Also drop the Doorkeeper>=5 hashed-secret fallback in the gitlab provisioner so a missing plaintext secret fails loudly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Provision a Doorkeeper OAuth application on the partner Zammad and write its client_id/client_secret into integration_zammad (Nextcloud-encrypted), so the per-user OAuth connect flow has the credentials the generic SPOT cannot supply. Register both the pretty-URL and /index.php oauth-redirect variants, and drop the hashed-secret fallback so a missing plaintext secret fails loudly instead of persisting an unusable hash.
Strengthen the integration_peertube spec: pin the admin 'instances' value to the exact deployed partner host (new PEERTUBE_BASE_URL) instead of merely 'not Nextcloud', and add a functional OCS search-providers check that fails when the PeerTube provider is not registered. PeerTube has no per-user OAuth (verified upstream: no client_id/oauth_instance_url config, no /oauth/authorize route), so the removed Connect->/oauth/authorize assertion tested a non-existent flow.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The image probe (docker inspect + 'docker run --entrypoint /bin/sh') conflated a timeout/kill under load with 'image has no /bin/sh', skipping the CA-trust wrapper entrypoint and leaving only env vars (CURL_CA_BUNDLE/SSL_CERT_FILE) that PHP libcurl ignores -> cURL 60 on server-side OIDC. has_sh is now tri-state: a timeout (rc==124) maps to None ('assume shell present', apply the wrapper); only a clean non-zero rc disables it. Hang protection (timeout+killpg) unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A consumer (e.g. web-app-openproject) sets compose_repository_url + pull=true
and database_consumer_id; sys-svc-objstore then lazily loads the shared
object-store provider (web-app-seaweedfs), which inherited those and tried to
clone the consumer's repo / target the consumer's DB under its own
application_id, failing the play. Pin the provider's own context (empty repo
url, no pull, provider's own database_consumer_id) on the lazy provider
include so it runs self-contained; the consumer's own clone is unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Drop the RuntimeBudget guard and its INFINITO_MAX_RUNTIME /
INFINITO_VARIANT_TIME_OVERFLOW env handlers: the deploy round loop now runs
every variant unconditionally instead of skipping (cut) or aborting (fail)
when the projected runtime would exceed the budget. Removes runtime_budget.py,
both env handlers + their registration, the budget calls in deploy.py, the
obsolete unit tests + the TestHandlerBudget class, and the docs reference.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The deploy-matrix discovery ran `python3 -m utils.github.variant_bundles`
with the runner system python, which lacks PyYAML, so importing get_variants
failed (ModuleNotFoundError: No module named yaml). Source
scripts/meta/env/python.sh to use the venv `make install` populates (pip
install -e . puts PyYAML there), matching how the rest of the toolchain
resolves PYTHON.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The personal-security enable control for twofactor_nextcloud_notification
only renders once the provider is activatable, which needs a SECOND active
session to receive the approval notification — impossible in a single-session
E2E (the provider state is correctly false there). The addon hook already
proves registration via occ twofactorauth:state; the browser-observable
equivalent is that the provider state element is surfaced, so assert
toBeAttached, not toBeVisible. Verified in-container: nextcloud v1 suite
31 passed, 0 failed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The handler COMMENT rendered a 188-char line in .env; the env-file-schema
lint caps non-nocheck comment lines at 140. Trim it. Also collapses two
ruff-format lines in the variant-split unit tests. Follow-up to the
runner-split variant feature; no behaviour change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
add_header without `always` only emits on 2xx/3xx, so error/startup
pages (e.g. an app's "initializing" 5xx) lacked Strict-Transport-Security
— which failed the xwiki HSTS E2E even though the settled site was fine.
`always` emits the header on every response; it only broadens emission
(never removes it), so it is strictly safe across all HTTPS vhosts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
compose_ca.py ran ~3 docker calls per service sequentially; on a DiD
daemon saturated by a many-service app's compose up (e.g. BigBlueButton,
16 services) the inject blew the CA-trust handler's timeout and failed
the whole deploy.
- run(): Popen + kill the whole process group on timeout so a wedged
docker/buildx grandchild cannot hold the captured pipe open and defeat
the timeout (subprocess.run's own timeout was inert in that case).
- /bin/sh probe: DEVNULL + short bound, so a wedged probe has no pipe to
block on.
- gather_image_meta(): read per-image (inspect + probe) CONCURRENTLY
(8 workers) up front; the render loop then only consults the cache.
- unreadable image (missing, or inspect stalled under load) -> env-only
injection (mount CA + set SSL_*/CURL_CA_BUNDLE env, skip the wrapper);
never build or hard-fail, so the inject always completes.
- handlers/main.yml: timeout backstop on the "Generate CA trust override"
handler.
- test_compose_ca.py: updated to the new run() signature + gather_image_meta.
Proven on the heaviest round: nextcloud variant-0 full_cycle went green
(failed=0, sync + async) with BigBlueButton + xwiki + all bonded partners.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Thread a single `variant` selector (CSV) end-to-end so CI can split a
role's matrix deploy across runners.
- variant_select.py (SPOT, shared by init + deploy): one --variant taking
a CSV (2 pins one round, 0,1,2 is a runner bundle, empty = full matrix);
replaces the old separate --variant/--variants.
- planner.filter_plan_to_variants filters the plan to a round subset.
- CI discovery (utils/github/variant_bundles.py) splits roles with more
than INFINITO_VARIANT_BUNDLE_SIZE (default 3) variants into bundles of
consecutive indices, one matrix runner each (5 -> 0,1,2 + 3,4), threaded
via the variant env. test-deploy-{local,universal,workstation,server}.yml
switch to matrix.include with a comma-free variant_slug for artifact/job
names (GitHub expressions have no replace()).
- runtime_budget honours INFINITO_VARIANT_TIME_OVERFLOW (cut|fail) when a
runner cannot finish its slice within INFINITO_MAX_RUNTIME.
Unit tests + docs (variants.md, deploy.md) included.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
OnlyOffice declares fileslibreofficeedit (and richdocuments) incompatible and
disables them on install, so the LibreOffice editor's E2E spec failed wherever
OnlyOffice was co-deployed. Enable fileslibreofficeedit only when the OnlyOffice
addon is off (addons.onlyoffice.enabled), and add the required service-coupled
addon integration hook.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
mariadb.enabled is a static literal in meta/services.yml, so variants must not
pin it (test_variants_no_static_override). Regenerate variants.yml overriding
only the dynamic flag per service (mariadb keeps shared-only). With mariadb
correctly always-on and flowise+openwebui sharing ollama, the bond<1 partners
repack into 4 bins -> 5 variants (v0 coupled baseline + 4 partner bins), each
under the 73GB/37GB host budget.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_file_spot flags hard-coded role-file paths; import the SPOT constant from
utils.roles.mapping and build the path from it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The empty-string fallback in os.environ.get(KEY, '') is forbidden for INFINITO_*
vars (test_no_infinito_get_defaults) — defaults live in default.env/handler. Read
it with no default and let _parse_duration_seconds accept None.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bond<1 partner bins (and the coupled v0 baseline) sit just under these
limits; the previous 32/72 cap was the single-all-on-host figure that the
6-variant distribution replaces. Honest accounting (every enabled service still
counts) — no bond-zeroing or per-variant exemption.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_order now only pulls bond>=1 (tightly-coupled, same-host) services into a
role's closure; bond<1 partners deploy on their own host and are excluded, so a
later variant may enable them without variant 0 having to list them.
New test_bond_baseline enforces the placement: a bond>=1 dynamic flag must be
pinned true in variant 0 (the host baseline); a bond<1 flag must be pinned true
in at least one variant. Escape: '# nocheck: bond-baseline'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An addon's enabled flag must defer to the owning role's service flag
(lookup config services.<x>.enabled), a literal true/false, or api lookups —
never a raw '<role>' in group_names test. group_names belongs in
meta/services.yml (the SPOT); addons read the resolved flag so one source
decides partner presence. New lint scans roles/*/meta/addons/*.yml (87 files)
with a '# nocheck: addon-enabled-group-names' escape.
deck.yml is the one legitimate exception (anti-dependency on web-app-taiga,
which has no nextcloud service flag) and is nochecked accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
integration_moodle has no Nextcloud-33-compatible release on the app store
(absent for nc 30/32/33) and integration_suitecrm requires nc <= 24, so both
bridges cannot work on the deployed Nextcloud 33. Set enabled/shared to false
(nocheck: dynamic-flag, like minio) so neither partner is pulled or bridged,
with a TODO to re-enable once a compatible plugin ships. Verified against the
Nextcloud app store nc-33 compatibility list; all other bridges (discourse,
matrix, peertube, mattermost, mastodon, zammad, openai, openproject, gitlab,
github, google, jira, bbb, xwiki) have real nc-33 releases and stay enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add INFINITO_MAX_RUNTIME (env handler: 6h on real GitHub Actions, 48h otherwise)
and a RuntimeBudget guard in the dev-deploy variant loop. Before each next
variant round the guard projects elapsed + longest round so far + 30m buffer;
if it would exceed the budget it stops and emits a CI warning annotation
(never fails — the rounds that ran stay deployed).
- utils/env/handlers/infinito_max_runtime.py: new handler (canonical SPOT for
the var; uses ctx.on_gha, materialised into .env by the generator).
- cli/administration/deploy/development/runtime_budget.py: RuntimeBudget +
duration parser (6h/48h/90m/3600/3600s), extracted to keep deploy.py lean.
- deploy.py: loop uses budget.exhausted()/start_round()/end_round().
- tests for both the handler and RuntimeBudget.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_limit.py hard-capped meta/variants.yml at MAX_VARIANTS (3) with no opt-out.
Allow a role whose extra matrix rounds are justified (e.g. distributing many
bridged partners so each variant stays within the memory budget) to opt out via
a '# nocheck: variant-limit' comment in its meta/variants.yml.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Making the partner roles shared providers (4243d76c1) added enabled: to their
primary entity, which trips the playwright-service-flag rule (an entity with
enabled: needs a <NAME>_SERVICE_ENABLED flag in templates/playwright.env.j2).
The primary entity is the role's own app, covered by the role's own Playwright
spec, so add '# nocheck: playwright-service-flag' on each entity:
bigbluebutton, flowise, matrix, mattermost, moodle, openproject, openwebui,
peertube, xwiki, zammad.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A role's vars/main.yml is auto-loaded into play scope and persists, so proxy
override vars (webserver_*) leaked into the central front-proxy render of OTHER
roles (re-evaluated against the wrong application_id) — silently applying one
role's value elsewhere, or hard-failing when the lazy lookup hit a role lacking
the key (the seaweedfs<->matrix client_max_body_size crash).
- Templates (sys-svc-proxy html/upload/ws_generic, matrix synapse homeserver)
resolve client_max_body_size from the role's own server config
(lookup('config', application_id, 'server.client_max_body_size', '100m')).
- Move every role-level webserver_* out of vars/main.yml into task-scoped
include params (9 roles); they apply only to that role's proxy and can't leak.
- sys-stk-front-proxy resets webserver_* to safe defaults via task-scoped vars
on the render tasks, so a leaked value from any other source is neutralized;
a role's own include params (#20) still override the reset (precedence-tested,
direct + nested).
- Add test_no_leaky_proxy_role_vars lint forbidding webserver_* in vars/main.yml
(suppressable via '# nocheck: leaky-proxy-var').
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
web-app-seaweedfs overrides meta/server.yml without a client_max_body_size key.
Consumers that resolve webserver_client_max_body_size via
lookup('config', application_id, 'server.client_max_body_size') (e.g.
web-app-matrix, whose lazy role var is evaluated while seaweedfs' own proxy is
rendered with application_id=web-app-seaweedfs) then hit a hard lookup failure.
Declare the field on seaweedfs itself (0 = unbounded, matching its S3 object
store nature and the proxy's api branch) instead of papering over it with a
lookup default, so every consumer resolves seaweedfs' own value.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
test_service_core_first_task_run_once requires tasks/01_core.yml to set the
run_once flag (include_tasks: utils/once/flag.yml) as its very first task so the
guard is in place before the role body runs and the variant-aware loader cannot
recurse. Move the flag include from the end to the top in the shared-provider
roles that had it last (bigbluebutton, flowise, gitlab, matrix, mattermost,
moodle, openproject, openwebui, peertube, suitecrm, xwiki).
Also give the template-only providers svc-db-redis and svc-db-memcached the
run-once guard (main.yml includes 01_core.yml when run_once_<role> is undefined;
01_core.yml sets the flag) so they satisfy the shared-provider run_once guard.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an integration guard that, for every role and every meta/variants.yml
entry, sums the deduplicated resource footprint (own containers plus shared
dependencies, each service counted once) via the shared
utils.roles.applications.services.resources implementation and fails when a
variant exceeds mem_reservation 32 GB or mem_limit 48 GB, to avoid an OOM kill
at deploy time. The failure message recommends disabling services in the
offending variant or moving them to other variants.
Currently surfaces 6 over-budget variants (nextcloud, shopware, openproject,
flowise, gitlab, openwebui v0) that still need their footprint trimmed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the parse/collect/aggregate logic out of the ressources CLI into
utils/roles/applications/services/resources.py so the CLI and the upcoming
variant resource-budget guard share one implementation instead of duplicating
it. The CLI's query/__main__ and the unit tests now import from utils; the
per-module CLI files (parse.py, collect.py, aggregate.py) are removed.
No behavior change: the 43 ressources unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Declare the 14 services that consuming roles reference but no role provided
(bigbluebutton, xwiki, openproject, gitlab, mattermost, matrix, zammad,
openwebui, flowise, peertube, moodle, suitecrm, redis, memcached) as shared
providers: set enabled/shared/bond on each entity so the service registry
registers them and the dangling references resolve.
Add test_shared_service_provider.py (lint): fail when a referenced shared
service has no provider and its name matches a role's entity/provider that does
not register it (suppressable via '# nocheck: shared-provider').
Guard the 9 web-app shared services with the run-once 01_core.yml pattern
(main.yml includes 01_core.yml only when run_once_<role> is undefined; the core
flags itself via utils/once/flag.yml) so they deploy once when pulled in by
multiple consumers. svc-db-redis/memcached are template-only (no runtime tasks).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rework cli/meta/roles/applications/ressources into single-responsibility
modules (parse, collect, aggregate, query, render) behind a thin __main__,
with mirrored unit tests, and extend it:
- resolve template-gated shared services to the provider role's entity via the
service registry, so a role's full dependency footprint is listed
- bond + depth columns; entries without bond default to bond 1
- load each service once by default; --unshared lists every occurrence
- --order [asc|desc] FIELD, --filter 'bond<=0.5 & cpus>=1 & mem_limit>=512m'
- --depth N (0 = unlimited recursion over shared deps)
- --variant N applies the same meta/variants.yml overlay as inventory creation,
reusing utils.cache.applications.get_variants (single source, no duplication)
- --sum [FIELD...] shows a SUM row over all or selected fields
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
