Every task after the seed step reads `Decidim::Organization.first`
(ensure/default_pages.rb, set_organization_host.rb, ensure/admin_user.rb,
configure_oidc.rb), and only `rails db:seed` ever creates one. Decidim guards
its core seeds with `if !Rails.env.production? || ENV.fetch("SEED", nil)`
(decidim-core db/seeds.rb, v0.31.6) and this role sets no SEED, so in
production the seed step creates nothing, and `failed_when: false` hides it.
The next task then calls `CreateDefaultPages.call(nil)`, whose
`StaticPage.find_or_create_by!(organization:, slug:)` raises, and
set_organization_host.rb would follow with `nil.host =`.
The regular matrix deploys RAILS_ENV=development, where the seeds run, so
only the README replay step, which provisions a production inventory, gets
here. It has not got this far yet: the healthcheck stopped it first
(87ce146d8).
ensure/organization.rb creates the organization when there is none, with the
attributes Decidim's two creation paths agree on (the core seeds and
Decidim::System::CreateOrganization), plus the help pages and content blocks
CreateOrganization adds. It leaves out the admin invitation, because
ensure/admin_user.rb creates the administrator itself, and tos_version,
which the terms-of-service page sets when the next task creates it. Name and
reference prefix come from DECIDIM_ORGANIZATION_NAME, the host from the same
domain the other tasks use.
The task runs only where the seeds skip, with ENVIRONMENT == 'production'. In
development nothing changes, and a seed that fails there still surfaces at the
next task instead of being covered by a bare organization.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Decidim#1 of run 33923190220 fails on "Wait for web container to be healthy"
after twenty attempts with stdout: unhealthy. The same job passes that wait
twice before it: the container is healthy after roughly thirty seconds on the
first pass and after about three and a half minutes on the second, and on the
third it never gets there inside the five minutes the task allows. At 19:01:41
every decidim container is created fresh, so each pass migrates against a
database the previous pass left behind.
1518512fcf set start_period to 180s from a measurement on an empty database,
where migrate took ~37s and replicas reached healthy at 133s and 123s. That
number was right for what it measured. A third redeploy is a different
scenario, and pass 2 already needed ~210s, past the window.
start_period moves to 10m, joining the six roles that already use it, so swarm
stops reaping a task that is still migrating. The converge wait moves from
twenty to sixty retries at fifteen seconds, giving 900s over a 600s window; it
only costs that time when the container genuinely never comes up.
The healthcheck itself is not at fault. The rescue diagnostics captured
FailingStreak 0 with exit code 0 on curl -f http://127.0.0.1:3000/up, so the
probe works once the app serves. An earlier attempt in this series excluded
/up from HostAuthorization and force_ssl on the theory that the probe was
being refused; passes 1 and 2 pass with that initializer in place, which
refutes it, and that change is dropped.
Every script a role ships now lives under `files/<language>/`, and the unit
suite is split the same way: `tests/unit/python/` beside new sibling trees for
the other languages. `tests/lint/filesystem/test_role_files_layout.py` keeps it
that way; directories whose shape is dictated from outside carry a `.nocheck`
naming the constraint (Docker build contexts, WordPress mu-plugins, the Joomla
extension, Odoo addons, Playwright suites, the `files/test/test.sh` harness
that `test-e2e-cli` resolves generically).
This cannot be applied in halves. Moving a file and repointing what names it
are the same change: 14 references resolve only at deploy time, so the tree is
green either before or after, never between.
Those 14 were found by static resolution over every reference class, not by the
test suite -- a full `make quality-high` was green while all of them were
broken:
* 7 `src:` values built from a role variable holding a bare filename
(svc-bkp-remote-2-local, decidim, espocrm, friendica, joomla, pixelfed x2).
The variable doubles as the destination name inside the container, so the
language segment belongs at the call site, not in the variable.
* `sys-service/tasks/04_base.yml` resolved every consumer's systemd script
through `first_found(..., errors='strict')` over `files/script.{sh,py}`.
18 roles ship exactly that and have no template fallback: the play would
have died on each.
* `sys-front-inj-javascript` fell back to `<role>/files/javascript.js`; the
interface now reads `files/javascript/javascript.js`.
* `sys-front-inj-logout/templates/head_sub.j2` built the cache-busting mtime
path through `path_absolute(... 'files', ...)`.
* 4 `ansible.builtin.script:` invocations naming a bare command
(pull.py, prune.sh, set_postgres_superuser_password.sh, ganesha-serve.sh).
* `dev-python/files/shell/install.sh` pointed at a sibling role's script.
Depth-dependent paths inside the moved trees were bumped accordingly: 59
`parents[N]` computations, 12 relative climbs, and the per-role recover CLI,
whose path `cli/administration/recover/recoverers.py` still assembled from the
pre-move location.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A handler name was doing two jobs at once: the line an operator reads in the
play output, and the identifier a `notify:` binds to. The two pull in opposite
directions -- the log wants "restart the SSH service, with diagnostics", the
binding wants a token that survives quoting, YAML flow style and a shell
round-trip -- and a name that changes for readability silently changes which
handler runs.
Every handler that a notify reached by name now carries a `listen:` with that
exact wording, so the binding is explicit before anything is renamed. The
topics themselves become identifiers: `compose up` reads `compose-up`,
`Restart NFS ganesha` reads `restart-nfs-ganesha`, 22 of them across both
sides of every notify. Where the topic is assembled at runtime, both branches
moved together: the conditional in sys-service carries two literals, and the
LDAP renderer interpolates `import-{{ folder }}-ldif-files`, which is what
makes those handlers reachable at all.
Freed from the binding, the names take the emoji prefix every other task line
already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, each resolved by combining both sides rather than picking one.
compose.yml passes both env vars through to the container: swarm-nfs adds
INFINITO_SWARM_MGR_IP, this branch adds INFINITO_TOR_SOCKS_PORT, and neither
substitutes for the other.
daemon.json.j2 takes the data-root block from swarm-nfs and keeps HEAD's dns
key. The incoming side writes networks.internet.dns directly; lookup(
'container_dns') already returns that value as its clearnet entry and prepends
the docker0 bridge on a Tor node in compose mode, so the incoming form is a
strict subset and dropping it loses nothing.
The WordPress CA-trust addon spec gains skipUnlessServiceEnabled("sso") from
swarm-nfs alongside HEAD's gotoOnion and resolveTimeout, and asserts on the
Plugin Name the mu-plugin header declares - the mustuse screen renders that,
not the filename - with resolveTimeout applied to the literal the incoming
side left bare.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An untyped inspect searches every docker object kind, and a role's overlay network carries the same entity name as its main container, so the call can resolve the network and report a missing State key instead of naming the container. That is how the openresty assert died on every swarm job of run 30709070050.
Extend container-exec-resolver to cover inspect, stripping raw markers and quoted spans first so the flag walker survives Go template format strings. Add container-inspect-type, scoped to roles/ because the collision follows the entity naming convention and the outer act namespace carries no role networks. Sweep 40 call sites; suppress the delegate_to-resolved CIDs and the compose co-location probes with a stated reason.
Move the five container rules into tests/lint/ansible/shell/container/ and drop the container_ prefix from their filenames. Register both markers in the suppression docs, where the resolver marker was missing entirely.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
decidim 0.31.6 dropped the /health_check route (rails routes now serves
the Rails 7.1 /up endpoint), so the container healthcheck 404-curled
forever and the pre-migration healthy-gate deadlocked against the
pending-migration 500. Point the healthcheck at /up and reorder setup
to running -> migrate -> healthy.
OIDC discovery failed on onion nodes because the swd gem forces
URI::HTTPS regardless of the issuer scheme, dialing the plaintext-only
onion issuer on :443 (TLS alert 112). Pin SWD/WebFinger url_builder to
URI::HTTP when the issuer is http; https issuers are untouched.
First-time OIDC registration 500ed on needs_tos_accepted because a
fresh organization has no terms-of-service static page; ensure the
default pages idempotently before setting the organization host.
The spec now logs in through the shared performKeycloakLoginForm
helper and derives the Keycloak host from OIDC_ISSUER_URL instead of a
hardcoded clearnet domain.
Verified end to end: full onion closure deploy failed=0, decidim suite
6 passed / 3 gated skips.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce plugins/lookup/path_absolute.py as the single source for the
repo-root anchor: lookup('path_absolute', 'roles/...') returns an absolute
path under playbook_dir, joining every term. Replace all hand-built
playbook_dir path constructions across .yml/.yaml and .j2 templates with it:
[playbook_dir, '<rel>'] | path_join -> lookup('path_absolute', '<rel>')
playbook_dir ~ '/<rel>' -> lookup('path_absolute', '<rel>')
{{ playbook_dir }}/<rel> -> lookup('path_absolute', '<rel>')
Unrelated repo_path symbols in cli/meta/roles/applications/resolution and the
single bare discover_playwright_roles(playbook_dir) call are left untouched.
Add tests/lint/ansible/test_playbook_dir_via_lookup.py: fails the build if any
of the three construction forms reappears in .yml/.yaml/.j2, pointing callers
at the lookup. Unit test covers the plugin.
Touching these files pulled their pre-existing narration comments into
test_comments_valid scope; clean them (delete restatement/banners, keep real
trip-wires as Workaround/Upstream/Invariant markers). Drop caution, important
and pitfall from the comment-lint marker vocabulary and remove the comments
that used them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/lint/repository/test_name_emoji_and_length.py: every Ansible task
name and GitHub Actions workflow/job/step name must start with an emoji (first
char not 0-9/A-Za-z) and be <= 80 chars.
- emit emoji names at the source: cli/build/include generator now writes
"🔧 setup <app>" / "🚿 flush handlers after <app>" so generated
tasks/groups/*-roles.yml satisfy the convention too
- convert existing task/step/job/workflow names to emoji-prefixed, choosing a
task-fitting icon per action (📦 install, ✅ ensure, 🔍 probe, 🥾 bootstrap,
🚀 deploy, 🧩 service-module load, ⬇️ checkout, 📂 include-routines, ...)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the per-role swarm host-node/container-id resolve duplicated across fider, matomo, decidim, keycloak (login + realm-import), collabora and listmonk with two shared task utils under roles/sys-svc-compose/tasks/utils/swarm/. resolve_host_cid sets <prefix>_HOST_NODE / <prefix>_LOCAL_CID (swarm: converge + node + cid delegated to the node; compose: inventory host + given cid); wait_converged is the bare convergence gate.
Teach test_variable_definitions to read templated set_fact keys like {{ prefix }}_HOST_NODE as a *_HOST_NODE name pattern (guarded so a pure {{ x }} key cannot become a catch-all), and add the playbook_dir-relative include load-failure exemptions to .ansible-lint-ignore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The setup execs ran 'container exec {{ DECIDIM_CONTAINER }}' (a compose-only
container name) and probed health via community.docker.docker_container_info,
which returns undefined in swarm and broke the until-expr with "object of type
has no len()". Resolve the hosting node + the local container ID per mode
(01_setup/{swarm,compose}.yml, selected by a DEPLOYMENT_MODE-driven include) and
run every exec/inspect delegated to that node against the resolved CID, mirroring
the web-app-matomo resolve pattern. The six rails steps
(migrate/seed/org-host/admin-user/oidc) are logically unchanged; only the
container targeting moved.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Across roles/web-app-*, the typical body shape is:
- include_role: sys-stk-full (or sys-svc-compose) cluster prereqs
- <host-fs writes, container exec, compose exec> stack-host only
Under swarm-mode delegation the second group only makes sense on the
host that holds the compose stack (manager), but the role plays on
every host in groups[<role>] so workers abort on missing paths.
Two workflow sweeps (gate-manager-ops + gate-manager-ops-01core, 58
agents total, discover/plan/adversarial-verify per role) plus three
manual fixes surfaced 27 roles to refactor. Each role keeps
sys-stk-full ungated so worker-side prereqs (sys-svc-container, NFS
client) still bootstrap; the trailing manager-only block either moves
into a new tasks/02_manager_ops.yml included with
when: IS_STACK_HOST | bool (SUB_FILE pattern) or gets the gate
inlined on the one or two trailing tasks (INLINE pattern). In
compose mode IS_STACK_HOST stays true so single-host deploys are
unchanged.
Manual catches the workflow agents missed:
- web-app-fediwall: utils/once/flag.yml lives AT THE END of 01_core,
not at the start; preserved verbatim outside the gate so workers
still flip the run_once_<role> fact.
- web-app-xwiki: a pre-existing block(...) wraps sys-stk-full plus
_flush_config.yml; gated only the _flush_config.yml include so
sys-stk-full keeps running on workers.
- web-app-dashboard: 02_javascript.yml (creates /opt/compose/dashboard/
JS dirs) ran BEFORE sys-stk-full and was ungated; added the gate.
Roles touched (INLINE): akaunting, jenkins, magento, mobilizon
Roles touched (SUB_FILE): baserow, dashboard, decidim, erpnext,
fediwall, friendica, gitea, jitsi, keycloak, listmonk, littlejs,
mailu, matomo, moodle, odoo, openproject, opentalk, pixelfed,
shopware, suitecrm, taiga, xwiki, zammad
Lint scanner now flags `community.docker.docker_container_exec` — it has
a 1:1 `compose exec`/`container exec` replacement and pinning to Docker
defeats engine-swappability. Other `community.docker.*` modules (network
create, host_info, container start with image+args) have no clean wrapper
equivalent and stay legal.
Migrations:
- svc-db-mariadb (01_core, 02_init, 03_reset_root_password): replace
module calls with `container exec -i` via `ansible.builtin.command`,
pipe SQL via stdin. Health probes share a single SELECT 1 file.
- web-app-decidim (01_setup): same migration, preserving env vars as
`-e KEY=VAL` argv elements and stdin for rails-runner snippets.
Extract all SQL to `roles/svc-db-mariadb/files/sql/*.sql` as plain SQL
(no Jinja). Values flow as MariaDB user variables (`@db_name`, `@db_pwd`,
`@root_pwd`, …) via a `SET` prefix in the task stdin. Identifier
substitution (db/user/encoding names) uses `PREPARE` + `CONCAT(@var, …)`
since `@vars` cannot bind identifiers directly. Password values use
`QUOTE(@var)` inside SQL and Jinja `replace("'", "''")` on the task side.
Pin MariaDB to "11.8" (current LTS major). Required for `CREATE DATABASE`
in prepared statements, supported since 10.5.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pure 1:1 rename (69 insertions / 69 deletions across 66 files) to
finish the ``docker_*`` → ``compose_*`` alignment started in
8373468e9. The flag configures whether the compose include flushes
handlers right after run; the ``docker_`` prefix was misleading
(handlers are Ansible-side, not docker-side) and the variable is part
of the same ``sys-svc-compose`` SPOT cluster as ``compose_repository_*``.
Touches the role default, the implementation
(``sys-svc-compose/tasks/utils/up.yml``), all 60+ consumer roles,
the network-utility wrapper + its README, and the
``test_network_create_via_util.py`` lint (both the docstring guidance
and the asserted diff).
Breaking change for any external override that still sets the old
name — no alias is provided.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picked up by the make-test setup phase regenerating generated artefacts
during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Playwright drives Decidim's first-time OIDC registration (TOS + newsletter
modal) so the biber user is created on first login and reused idempotently
on subsequent runs. Ruby snippets previously inlined in tasks/01_setup.yml
and patch.py are now separate .rb files under files/ruby/, copied into the
image alongside patch.py. patch.py now also fixes Decidim 0.31.3's broken
omniauth_registrations_path view helper (both new.html.erb and
new_tos_fields.html.erb).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Resolve merge conflict in web-app-peertube/vars/main.yml; migrate
remaining `applications |` and `users.*` references to
`lookup('applications')` / `lookup('users', ...)` in web-app-odoo and
web-app-decidim (compose, env, playwright, setup tasks).
- web-app-peertube: split OIDC tasks into oidc/{enable,disable}.yml,
add install_mem_limit filter + PEERTUBE_MEM_LIMIT /
PEERTUBE_OIDC_PLUGIN_INSTALL_MEM_LIMIT vars to raise the cgroup ceiling
during the one-off `npm run plugin:install` exec (prevents OOM-SIGKILL
that was surfacing as "Container is restarting"); add
upsert-oidc-plugin.sql.j2 and diagnostics.sh.
- CI: test-deploy-{local,server,universal,workstation}.yml tweaks plus
scripts/github/{dump_runner_state,enlarge_swap}.sh for constrained
runners.
- cli/meta/applications/ressources: new CLI entry with unit tests.
- Misc config/docs updates across dashboard, mailu, mattermost, asset,
cdn, file roles and contributing docs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Add copy task in tasks/main.yml to stage patch_decidim.py into build context
- Pass HOST_REL and DOCKER path as build ARGs via compose.yml.j2
- Use COPY + RUN in Dockerfile with no inline code
- patch_decidim.py remains single source of truth for unit tests
- Add custom Dockerfile to install omniauth_openid_connect gem and patch
Decidim's OmniAuth initializer for OpenID Connect provider registration
- Patch secrets.yml to add openid_connect section to all environments
- Patch omniauth_helper.rb to return correct icon for openid_connect
- Fix redirect_uri to use APPLICATION_HOST env var for correct callback URL
- Fix SSL cert trust by relying on compose.ca.override.yml at runtime
- Add OIDC omniauth settings to organization (enabled + icon only, no
encrypted credentials - credentials read from env vars at runtime)
- Add biber OIDC login/logout Playwright test and profile access test
- Add SSO button visibility test when OIDC is enabled
- Fix SSO button selector to use href-based locator
- Use form.evaluate() POST with CSRF token for data-method=post links
- Add OIDC_ENABLED to playwright.env.j2
- Add DECIDIM_CUSTOM_IMAGE var for custom built image
- Update compose.yml.j2 to build custom image with OIDC patches
- Remove sidekiq from config (disabled service)
- Remove biber user creation - users access via OIDC
- Refactor tasks/main.yml to use sys-stk-full with flush_handlers pattern
- Fix database service key: 'database' -> 'postgres'
- Restore dmbs_excl.yml.j2 include in compose template
- Add subnet to 10_networks.yml to fix bare default network key
- Remove SSL cert env vars (handled by NGINX proxy)
- Keep DECIDIM_FORCE_SSL for correct URL generation
- Use robust selectors in Playwright (getByLabel + input[type=password])
- Remove biber from Playwright tests and env template
- Set organization host to match DECIDIM_BASE_URL domain on every deploy
- Use state: attached for session_user_email waitFor
- Root cause: Decidim redirected to /system/admins/sign_in when org host
did not match the access domain, causing #session_user_email not to exist
- Add decodeDotenvQuotedValue helper to playwright.spec.js
- Add dotenv_quote to all values in playwright.env.j2
- Use robust selectors (getByLabel + input[type=password])
- Handle cookie consent banner before login
- Fix user.email typo (was use.email) in setup tasks
- Only update password when user is new or password changed
- Use save! to fail loudly on invalid config
- Fix login form submission using form.evaluate() to bypass hidden button
- Add administrator and biber user creation with accepted_tos_version
- Use double quotes for passwords to handle special characters
- Only set password on new users to avoid reuse validation errors
- Simplify conversation tests to verify UI accessibility
- Disable sidekiq service (not needed for basic deployment)
- Add allowed_hosts.rb.j2 initializer to fix Rails host authorization
- Mount allowed_hosts initializer into decidim container
- Fix DATABASE_URL password encoding for special characters
- Strip trailing slash from RAILS_HOSTS
- Remove Dockerfile.j2 (using prebuilt image)
- Use ENVIRONMENT variable instead of hardcoded production
- Use ghcr.io/decidim/decidim image (pre-generated app)
- Fix container_port to 3000 (Puma internal port)
- Fix REDIS_URL to use redis service name
- Fix OIDC.CLIENT.ISSUER_URL (was OIDC.CLIENT.ISSUER)
- URL-encode database password in DATABASE_URL
- Add RAILS_HOSTS for host authorization
- Remove /etc/timezone volume mounts (not a directory)
- Render compose.yml and env explicitly in tasks/main.yml
- Fix changed_when in 01_setup.yml to use stdout_lines
- Use defaults_service_provider.platform.titel for org name
- Add biber vars to playwright.env.j2
- Expand playwright tests with biber scenarios
- Verified: Decidim accessible at decidim.infinito.example
