DISCOURSE_LDAP_AUTH_PLUGIN_REPOSITORY and DISCOURSE_LDAP_AUTH_PLUGIN_REF date
from fd22d34559, when every role mirrored its git URL into a per-role
REPOSITORY/REF pair. 84a4b04f0 pointed the clone loop in config.yml.j2 at
plugin_config.config.repository and plugin_config.config.ref instead, reading
the addon declaration directly, and left the mirror without a reader.
They are not merely noise. Both are unconditional top-level vars whose
lookup('config', ...) carries no fourth argument and is therefore strict, and
both address a path that exists only while this one addon keeps this one id.
Renaming or removing discourse-ldap-auth would have failed every Discourse task
at resolution time over a value no caller wanted.
The surviving LDAP path is untouched: DISCOURSE_LDAP_ENABLED reads
services.ldap.enabled, and stack_host/02_credentials.yml takes the site
settings through DISCOURSE_PLUGINS['discourse-ldap-auth'].config.site_settings.
A tree-wide grep for DISCOURSE_LDAP_AUTH_PLUGIN now returns nothing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
params.version sat commented out in app.yml, so Discourse silently inherited
tests-passed and every bootstrap could pull a different application revision
with nothing recording which.
Declaring it as services.discourse.version would have been invisible to both
updaters: utils/update/docker.py reaches version: only through
iter_role_images, which skips any service without an image: (discovery.py,
"if not image or not version: continue"), and Discourse is built by the
launcher and has none. A nested application entry carrying repository: and
ref: lands in utils/update/repository.py instead, whose walk_repo_ref_pairs
descends the whole tree; the collector now reports
discourse/application | discourse.git | v2026.8.0
so git ls-remote --tags keeps the pin current rather than freezing it, and the
external URL test sees the repository through iter_non_ignored_files. This is
the shape 7b15c61dfc established when it migrated version/branch keys to ref.
v2026.8.0 is the newest stable tag: v2026.9.0 answers 404, only
v2026.9.0-latest exists. The auto-bump will not drift onto those pointers,
version_flavor("v2026.8.0") is empty and latest_semver with that flavor picks
the plain tag back.
Pinning the core while the plugins clone from their default branch is safe
because upstream solves it: docker_manager, discourse-akismet and
discourse-activity-pub each ship a .discourse-compatibility map, and the
bootstrap checks out the plugin commit matching an older core. The newest
threshold in those maps is 2026.4.0-latest, below this pin, so the default
branch is the intended pairing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three notifiers pointed at one handler, and that handler always ran
./launcher rebuild. A rotated database password therefore cost a full image
build with bundle install and asset precompile: 961.82s in pass 1 and 13m16s
in pass 2 of job 103878400427, on a run that then missed its wall-clock
budget by 201s.
The launcher draws the line itself. set_template_info reads app.yml on every
call and turns env: into -e flags that run_start hands to docker run, while
templates:, params: and hooks: are consumed during bootstrap only. An env:
change needs a new container, not a new image. run_start takes the docker run
branch only when no container exists, which is why the start path still stops
and removes it first.
The digest task is the discriminator: it records a sha256 over every top-level
key except env:, so a credential-only change leaves it untouched and the
bootstrap handler is never notified. A hash rather than the content because
that section carries the OIDC client secret and the administrator password.
The guard against running both paths lives in a fact cleared per role run. A
register from inside the included file would have outlived the play and, in a
later one, silently skipped the recreate while the config change never reached
the container.
from_yaml adds no failure mode: the launcher parses the same file with
YAML.load, so anything it rejects was already a broken deployment. start
without an image is structurally impossible, since a running container
implies its image and a missing one notifies the bootstrap.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.
The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.
Resolutions that were not unions:
Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.
plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.
compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.
tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.
Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.
01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.
Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.
roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.
config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.
READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.
Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Discourse AI is configured through an LlmModel row rather than settings, so the
addon upserts one via rails: provider open_ai, the gateway's chat-completions
URL, the model the gateway serves, and this role's virtual key. It then binds
that row to ai_default_llm_model and to the bot, because a model that exists
but is bound to nothing serves no surface.
llm_models.url is POSTed verbatim, so DISCOURSE_LITELLM_CHAT_ENDPOINT carries
the full /v1/chat/completions path rather than a base URL. Every other consumer
in this requirement takes a base and appends; getting this one wrong produces a
404 at prompt time, long after a green deploy.
Three things had to be cleared before any of that could work.
Discourse refuses to call a private address unless the host is in
allowed_internal_hosts, and it fails as an opaque provider error rather than as
a blocked request. The script appends the gateway host to that list and leaves
existing entries alone, which matters because the OIDC bootstrap in
config.yml.j2 appends the discovery host to the same pipe-separated setting.
Both sides check membership before adding, so they compose instead of
clobbering each other.
The launcher-managed container sits outside the compose graph and does not join
the gateway network on its own. handlers/stack_host.yml adds
DISCOURSE_LITELLM_NETWORK to its network list and config.yml.j2 passes the
matching --network flag, both only while litellm is enabled.
The bootstrap aborted before the plugin could install at all. after_code cloned
every enabled plugin unconditionally, but a bundled plugin already sits in
$home/plugins, git exits non-zero on the clone, and the whole bootstrap stops.
discourse-ai is bundled in current images, so it now skips plugins whose source
is bundled.
mem_limit goes from 2g to 4g. `rails r` boots a second Rails inside the
container already running unicorn and sidekiq, and at 2g the kernel kills it
mid-run: measured in deploy-036-11 and again in deploy-ai-v2-7, where the task
died with "Killed" and the deploy ended at exit 137. Upstream documents 4 GB as
Discourse's own minimum, so this corrects an under-provisioned limit rather than
working around the task. The variant resource budget still passes.
Verified: the OOM with the exact task and exit code across two independent runs,
the bundled-plugin abort, and that the budget test holds at 4g.
Not verified: the registration has never completed, because every attempt so far
died at the OOM first. The eight-fact read-back assertion and
addons/discourse-ai.spec.js are both entirely unexercised, the spec because the
addon bridge gate had it skipping.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
meta/schema.yml became meta/secrets.yml, but its content still landed at
applications.<app>.credentials — the one meta file whose payload did not sit
under a root named after the file. Every other topic obeys that rule:
meta/volumes.yml goes to volumes, meta/rbac.yml to rbac. Credentials now go to
secrets.credentials.
The path exists exactly once, in utils/manager/credential_key.py, and
OVERRIDE_SECTION is derived from its two segments rather than spelled beside
them, so the dotted form used by --set and the nested form used by the
inventory cannot drift. Six consumers read it: the loader, the generator, the
CLI override parser, the two emit helpers, and the four lookup validators.
--set now accepts exactly one key shape,
applications.<app>.secrets.credentials.<key>. The three short forms are
rejected instead of guessed at: the same short key names different credentials
on different apps. That also retired primary_app_id, which only existed to
disambiguate them.
Three defects surfaced while wiring this up. _apply_one_role_schema still wrote
to the old root, so an app's own credentials would have moved while its
providers' stayed behind. web-app-pgadmin reads through
lookup('applications', ...).credentials.<key>, an access path no regex over
lookup('config', ...) could see, and would have deployed with an empty postgres
password. And _META_NON_GENERIC_TOPICS still listed 'schema' — a bare topic
name, invisible to any path search — so the generic file-root loop assigned the
raw schema under secrets and the follow-up write landed inside the cached
document, truncating keycloak's four credentials to one for every later
reader.
tests/integration/ansible/lookups/config/test_credentials_roundtrip.py pins both
directions of the seam and the cache invariant that last defect broke.
Existing inventories still carry the pre-rename root; migrating them is
deliberately out of scope here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Container-name vars derived their value inconsistently (hard-coded literals, entity_name composites) instead of the role-owned SPOT. Add a lint requiring every *_CONTAINER var to resolve through lookup('config', app, 'services.<x>.name'), the bare entity_name form, or the engine lookup; give each multi-container worker a name: in meta/services.yml (with realistic resource limits) and switch its var to the lookup. Extend the engine lookup with a shared-aware 'container' field (central provider container when shared, embedded <entity>-<engine> otherwise) and gate zammad's embedded memcached sidecar on not-shared so its container_name never collides with the central memcached.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The five docker network connect/disconnect tolerances were inline stderr substring chains, inconsistent between sites (the handler variants matched case-sensitively without a default). The docker_network_noop filter is the single marker source per operation; disconnect additionally tolerates absent networks/containers, which is the proven swarm failure ('No such container: postgres' when the central DB is a swarm service). Drop the now-orphaned DISCOURSE_ERROR_NOT_CONNECTED.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
network connect failed_when only matched one idempotent stderr phrasing; also accept 'already attached to network'. Fix the platform-logo truthiness gate (a URL string is never a bool) and drop a now-unused error var.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The discourse launcher is a standalone container (docker run via ./launcher), not a swarm service, so it cannot attach to swarm-scoped overlay networks. In swarm it was left without a reachable network for its shared pg/redis/openresty peers.
The shared network util now creates a node-local bridge instead of an overlay when the role declares networks.local.force_bridge in meta/server.yml, read via the config lookup with a false default so every other caller stays byte-identical. Discourse sets force_bridge: true and connects the rebuilt launcher to the shared overlays (web/pg/redis) so swarm services stay reachable; the launcher config drops the explicit --network flags in swarm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
In swarm the standalone ./launcher container was created with only --network=DISCOURSE_NETWORK. With a shared redis/db the engine resolves DISCOURSE_REDIS_HOST/DISCOURSE_DB_HOST to the central aliases (redis-central, the postgres name) which live on the svc-db-redis / svc-db-postgres overlays, not the discourse network. So 'rake db:migrate' during './launcher rebuild' aborted with Redis::CannotConnectError 'redis://redis-central:6379' (resolving to 127.0.0.1:6379, refused). The compose-only db-bridge handler is skipped in swarm and 05_network's connect runs after the rebuild, so neither can help a migrate that runs during the rebuild. Attach the launcher to DISCOURSE_REDIS_NETWORK (when redis is shared) and DISCOURSE_PG_NETWORK (when the db is shared) at creation time, gated on the engine local lookup so embedded backends keep their sidecar on the discourse network. Multiple --network at docker run is honored on the targeted Docker 28/29.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
About 38 web-app/web-svc roles now consume the central svc-db-* and svc-dns engines via lookup('engine', ...) instead of embedded sidecars: meta/services flags, variant matrices, and env/conf hosts point at the shared services. Lets these apps run unpinned and NFS-shareable in swarm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
docker_manager, discourse-activity-pub, discourse-akismet, discourse-ldap-auth (bridges ldap) declared as mechanism: plugin; config.yml.j2 + vars/main.yml read the new addons path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migrate every role's meta/volumes.yml from ad-hoc dict-of-named-volumes
to a canonical dict-of-dicts schema where the YAML key is the semantic
short name and the entry body carries type / name / source / mounts /
nfs / mode / read_only. The optional 'name:' field is the docker volume
name (defaults to the key); the previous 'docker_name:' field is gone.
What's in this commit (210 paths):
Infrastructure (23)
- New filter plugins/filter/{compose,container}_volumes.py — render the
top-level volumes:/configs:/secrets: section and per-service mount
blocks from the canonical meta. container_volumes uses the SEMANTIC
name as per-service mount source so it resolves against compose_volumes'
top-level dict key (docker_name as source produced 'undefined volume'
on swarm).
- New lookup plugins/lookup/{compose,container,volume}_volumes.py.
volume(application_id, name) returns a normalised single entry;
volume(application_id) returns the whole canonical dict.
- utils/cache/applications.py keeps a sibling registry
_CANONICAL_VOLUMES_BY_ROLE outside the merged applications payload so
raw Jinja source: strings never enter the templar walk (root-cause of
the 200x cache-miss + recursive get_merged_applications storm).
- utils/roles/applications/mounts.py — schema validator + normaliser;
list-shape is rejected.
- plugins/lookup/nginx.py: _volume_path_by_name reads canonical directly.
- scripts/tests/deploy/swarm/_context.sh: NFS_VOLUMES extraction handles
both legacy dict and canonical dict shapes.
- roles/svc-prx-openresty/handlers/restart_and_validate.yml: gate
compose-only validate+reload behind a single when-block; swarm uses
service-update instead (bind-mounts of a FILE pin source inode at
task-create time, so atomic write-tmp+rename leaves the running task
on a deleted inode).
- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch to
lookup('volume', application_id) | dict2items so NFS subdir
pre-creation works under the new cache split.
Tests (12)
- New strict lints flagging inline mount sections in compose templates,
single-file bind mounts, and legacy lookup('config', ..., 'volumes.X.Y')
call sites — all volume-mount surfaces MUST flow through container_volumes
/ compose_volumes / volume lookups.
- Integration: test_volumes_compose_chain + test_volume_constants_are_lookups
accept both the legacy and the new lookup('volume', app, 'X').name forms
during the migration window.
- Unit tests for the new lookup, filter, mounts helper, plus updated
nginx lookup test.
- Prometheus role-wiring test reads canonical meta for the
alert_rules.yml bind mount.
Role migrations (175)
- 62 meta/volumes.yml converted to dict-of-dicts canonical shape
- 63 templates/compose*.yml.j2 replace inline volumes:/configs:/secrets:
with lookup('container_volumes', application_id, '<service>') |
indent(N) per service + a single trailing lookup('compose_volumes',
application_id) for the top-level block
- 50 vars/main.yml drop dead volume constants and refactor 23 remaining
legacy lookup('config', ..., 'volumes.X.Y') to lookup('volume',
application_id, 'X').Y across 9 roles
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-locates every git-clone URL with its owning entity in the role's
`meta/services.yml` so the unified `image`/`version` ↔ `repository`/`ref`
naming convention covers secondary repos (plugins, build helpers,
bootstrap sources) just as cleanly as the primary entity source.
Migrations:
- sys-aur, pkgmgr (non-invokable bootstrap roles): `meta/services.yml`
gains `repository:`/`ref:` on the role's own entity; `vars/main.yml`
renders them via `lookup('config', role_name, …)` because these
roles never set `application_id`.
- web-app-{akaunting,bigbluebutton,bluesky,discourse,roulette-wheel,
taiga}: declare the secondary git URL on the most fitting entity
(`akaunting` itself for the docker-helper repo; `view` for bluesky's
social-app web client; `plugins.discourse-ldap-auth` for that
Discourse plugin; `oidc` for taiga's OIDC plugin). Each role gains
named `<ROLE>_REPOSITORY` / `<ROLE>_REF` constants in `vars/main.yml`
so tasks and templates stay readable instead of inlining a
`lookup('config', application_id, …)` string at every call site.
- svc-db-postgres `templates/Dockerfile.j2`: keep the `pgvector`
extension clone but mark the line with `{# nocheck: repository-declared #}`
since pgvector is an extension source, not the role's primary repo.
New integration test
`tests/integration/roles/meta/services/test_repository_declared_in_services.py`
walks every role's text files and fails if a literal `https://…/<…>.git`
URL is not declared as a `repository:` value somewhere in that role's
`meta/services.yml` (recursive — top-level entity, sub-entity, plugin
map). Legitimate exceptions opt out with `# nocheck: repository-declared`
on the same line or the preceding line; Jinja templates may use the
`{# … #}` form so a `\`-continued Dockerfile shell command stays valid.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Unify the source-repo half of the entity naming convention introduced
by `0cc37d0fd`. Every entity that declares `repository:` MUST now
also declare `ref:` (git ref — tag, branch, or commit SHA). The
overloaded `version:` (when used as git ref) and the alternative
`branch:` key are gone.
Roles migrated:
* web-app-bigbluebutton: `version: main` → `ref: main`
(consumed in tasks/01_core.yml as `docker_git_repository_version`).
* web-app-discourse: `version: main` → `ref: main`;
`DISCOURSE_VERSION` renamed to `DISCOURSE_REF`.
* web-app-bridgy-fed: `branch: main` → `ref: main`;
`BRIDGY_REPO_BRANCH` renamed to `BRIDGY_REF` across vars, the
Dockerfile ARG, and the compose build-args.
* web-app-postmarks: identical migration to bridgy-fed.
New guard `tests/integration/roles/meta/services/test_repository_requires_ref.py`
walks every `meta/services.yml` and fails when an entity declares
`repository` without `ref` (or vice versa), so the next role that
forks the source-build pattern cannot regress to either half-
declaration without `make test` catching it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two bugs in one config option:
1. `SiteSetting.enable_local_logins = {{ ... | bool | string }}` emitted
Python's capitalized `True`/`False`, which Ruby parses as undefined
constants → `uninitialized constant True` aborts `./launcher rebuild`
(CI run 25634428082 deploy job for web-app-discourse).
2. `enable_local_logins` was set to the *raw* `disable_local_logins`
value, so when the config truly meant "disable", the live setting
was stuck on "enable", and vice versa.
Renames the field across `meta/services.yml`, `vars/main.yml`, and
`templates/config.yml.j2` to its positive form and applies De Morgan to
keep the original semantics:
disable_local_logins = oidc.enabled AND NOT ldap.enabled
local_login_enabled = NOT oidc.enabled OR ldap.enabled
Truth table is preserved across all four (oidc, ldap) combinations; the
LDAP-needs-the-local-mask path stays open.
The template now uses `| string | lower` so the rendered Ruby literal
is `true`/`false`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL,
declare it at file start with default `true`; still env-overridable.
Reword the worker comment so the literal `# shellcheck` token no
longer appears at start-of-line and trips SC1072/SC1073.
* scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE /
GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure
is always fatal. NAMESPACE is an internal constant. Add the same
PARALLEL env var (default `true`, env-overridable); falsy demotes the
xargs fan-out to one worker.
* docs/contributing/environment/variables.md: collapse the four
AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both
consumers.
* docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere;
add the lint-galaxy parallelism note.
Plus working-tree drift across roles/, group_vars/, inventories/, and
tasks/ that accumulated from the make-test setup phase regenerating
generated artefacts during recent test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the enable_local_logins=false setting out of the OIDC-only
conditional and gate it on a new services.discourse.disable_local_logins
flag instead. The flag evaluates to true when OIDC is the sole auth
path (Keycloak in group_names, OpenLDAP not in group_names); with
LDAP running in parallel the local-login form stays active so the
LDAP plugin's username/password hook keeps working.
- services.yml: services.discourse.disable_local_logins jinja predicate
- vars/main.yml: DISCOURSE_DISABLE_LOCAL_LOGINS via lookup('config', ...)
- config.yml.j2: standalone {% if DISCOURSE_DISABLE_LOCAL_LOGINS %} block;
the OIDC-only block keeps enable_passkeys=false / username_change_period
/ openid_connect_* unchanged
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Five web-app roles read their own `credentials.database_password` via
`lookup('config', application_id, …)` from `vars/main.yml`. The
credential is wired in by the inventory's apply_schema step rather
than declared in each role's `meta/schema.yml`, so the static
lookup-config check cannot see it. Add the `lookup-config-path`
marker to each call so the role-local pass stays clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously the OIDC RP-initiated logout redirect landed back on Discourse
itself (application_id), which only ends the local session and leaves
other SSO apps signed in. Route it through the central
`web-svc-logout` service instead, mirroring how Keycloak already wires
its FRONTCHANNEL_LOGOUT_URL, so OIDC logout terminates all cross-app
sessions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Rename lookup plugin from `docker` to `container`
- Switch all lookups from `lookup('docker', ...)` to `lookup('container', ...)`
- Normalize compose file keys:
- docker_compose → compose
- docker_compose_override → compose_override
- docker_compose_ca_override → compose_ca_override
- Centralize compose path resolution via docker_paths_utils
- Update compose_file_args to always derive paths from DIR_COMPOSITIONS
- Align all roles, templates, and vars with container/compose naming
- Update and rename unit tests accordingly
This prepares the codebase for engine-agnostic container backends
(e.g. Docker ↔ Podman) and removes legacy docker-specific assumptions.
https://chatgpt.com/share/6987a7cc-d374-800f-99ad-bb4c8f92c59e
Replace usages of the get_app_conf filter with the config lookup plugin
across roles and templates. Adjust quoting where required to keep YAML
and Jinja syntax valid. No functional changes intended.
https://chatgpt.com/share/69860678-cef8-800f-8402-7fe6231fdc2d
The TLS lookup has been renamed from `tls_resolve` to `tls` to better
reflect its responsibility as a strict TLS state resolver.
This refactor updates all usages across the codebase, including:
- lookup plugins and internal helpers
- roles, vars, tasks, and templates
- tests, stubs, and error messages
- documentation and changelog references
Behavior, lookup API, and semantics remain unchanged.
https://chatgpt.com/share/6985e635-5a94-800f-b794-937e566ee308
- Replace lookup('docker', ..., want='...') with lookup('docker', ..., 'path')
- Allow both top-level keys (directories/files) and dotted paths
- Simplify docker lookup API and reduce lazy dict propagation
- Update unit tests to reflect new lookup semantics
https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
Remove implicit docker_compose variable usage and resolve all docker paths
(files, directories, env) via lookup('docker', application_id).
This prevents scope leakage, centralizes path resolution, and makes docker
context deterministic across roles and handlers.
https://chatgpt.com/share/6984ccea-8290-800f-a6d7-ec0e94df95a0
- Pin Discourse repository to configurable version instead of implicit HEAD
- Add TLS/self-signed CA env + volume injection to container config
- Make force_https depend on resolved TLS state
- Replace fragile printf-based admin creation with safe heredoc
- Ensure admin user creation works with arbitrary password characters
- Expose DISCOURSE_VERSION via role vars for reproducible deploys
https://chatgpt.com/share/6980a49e-3000-800f-b6a4-cb6a3e15a3d8
- Discourse:
- Allow explicit http font-src to diagnose mixed-content font loading
- Fix OIDC RP-initiated logout redirect to use resolved base URL
- WordPress:
- Make core install idempotent by seeding wp core if volume is empty
- Refactor install task to use explicit role variables
- Normalize docker service variable paths
- pgAdmin:
- Remove implicit defaults in get_app_conf lookups for clarity
- General:
- Align variable naming and formatting across roles
https://chatgpt.com/share/6980820e-6670-800f-9812-9b87debf1dee
- Replace database_application_id with database_consumer_id
- Use explicit consumer context for database resolution
- Fix recursive Jinja variable loop in sys-svc-rdbms
- Decouple discourse database network from global database_type
https://chatgpt.com/share/696bd40d-ebfc-800f-9ba8-1e6be919a629
