The web frontend, the controller and the recorder were pinned to v2.9.0,
v0.34.1 and v0.16.2, a mix of the 26.2 and the 26.1 line, and excluded
from the image updater with nocheck: docker-version. Nothing moved them.
They now sit on the set OpenTalk publishes as product version 26.2.1 in
ot-setup (v2.9.2, v0.34.5, v0.17.5), and each pin names that bundle as its
version source, so the three move together with the next product version.
The markers are gone.
Recorder v0.17 no longer reads recording jobs from RabbitMQ or logs in
with an OIDC service account. The controller starts recordings on the
recorder's REST API and both sides authenticate with API keys:
recorder.toml.j2 names the controller and the recorder's own key,
controller.toml.j2 enables roomserver.modules.recording, the secret
recorder_api_secret replaces recorder_oidc_client_secret and the Keycloak
client opentalk-recorder is no longer created.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The lookup writes overrides.get(key, effective_default), so a service that
repeats what it would have inherited changes nothing and only hides the day the
default moves. Forty seven such lines stood in the tree.
Almost all of them came from the migration that moved the literal blocks into
meta/services.yml. The values were carried over verbatim so the move could be
proven lossless, which was right for the proof and left the no-ops behind.
The defaults they are measured against are the effective ones, not the base
class. Curl probes every minute, Connect grants a fifteen minute start window,
and a prefix raises timeout, retries and start_period further. Comparing against
Probe alone would have called out every service that simply agrees with its
flavor.
Eight services lose interval: 10s as well and fall back to their flavor's
cadence, thirty seconds, or a minute for decidim's curl. Since start_interval
exists, interval no longer touches convergence at all: it is steady state load
for the life of the deployment. That takes the five databases, matomo, snipe-it
and decidim from 360 probes an hour each to 120. It buys that with slower
failure detection, 180s to 540s for the databases, which is acceptable because
an unhealthy container fails the deploy whichever number stands there.
Twelve deviations that do cost something now carry their reason. Where the
mechanism is known it is named: ejabberdctl and rabbitmq-diagnostics start an
Erlang CLI node, mmctl spawns a binary and talks to the local socket, php artisan
boots Laravel on every probe, keycloak's management endpoint answers from the
JVM.
Two say instead that the value was carried over and never measured. './fider
ping' and pgrep both fit the five second default and no reason for ten is
visible, so the comment records that there is no evidence rather than inventing
one. They mark where a measurement belongs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
42ebd9ea71 gave the healthcheck SPOT start_interval, the cadence docker uses
while start_period is still running. A block a template writes out itself
bypasses the SPOT and never gained it. Measured across the repository: 46 such
blocks, 40 of them already carrying start_period, not one carrying
start_interval.
Without it a probe fires only every interval, so a service ready after two
seconds is reported healthy at the next tick and a stack converges no faster
than its coarsest quantisation. That lands in sys-svc-compose's swarm converge
wait, 1551.7s per CI job over 38 stack deploys.
Twenty five of the blocks move into meta/services.yml and render through
container_healthcheck. Each keeps an explicit test rather than picking a flavor:
a flavor would replace the command instead of carrying it across.
The move is lossless by construction. probes.py block() writes
overrides.get(key, default) for interval, timeout, retries, start_period and
start_interval, so a declaration repeating the literal values renders the same
block plus the missing cadence. Checked per block against the old values: the
only difference is start_interval 5s, nothing changed, nothing lost.
Twenty one stay literal. Seventeen interpolate a template variable, whose value
resolves in a scope meta/services.yml does not share, so moving them would be a
rewrite rather than a transcription. Four interpolate nothing and still cannot
move: svc-db-memcached and svc-db-redis probe every second in their swarm
service partial, where a 5s start cadence would report healthy later than the
block already does, and the sys-svc-rdbms and keycloak sso_proxy partials render
for the consuming role's application_id, so their declaration has no single
meta/services.yml to live in. Each carries a nocheck naming its reason.
All twenty one gain start_interval. Three also gain start_period, without which
the key does nothing at all, since it only applies while the start period runs.
That widens their failure tolerance rather than narrowing it: bluesky from 90 to
120 seconds, magento from 50 to 80, the mariadb partial from 180 to 210.
pixelfed spelled its probe as a bare string and now spells it CMD-SHELL. Docker
reads both as a shell command; only the YAML shape changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
83c1c729de gave every provider a group_names-gated services entry per MCP
client, including the clients it refuses with mcp_consumer: false. The gate
is what makes the inventory resolver plan provider and client into one round,
so 19 of the repo's 82 co-deploy pairs started a client for a pairing the
provider had already declined. flowise is a live MCP client of qdrant,
homeassistant and prometheus; the other 17 providers were bringing up its
node stack for nothing.
The entries now read enabled: false, shared: false, mcp_consumer: false, and
the variants that pinned them to true are gone. Setting those variants to
false instead is not an option: test_variants_no_static_override forbids any
variant override of a literal base value in either direction.
bond leaves with them, and a new lint holds that line: collect_role_resources
skips a literally disabled entry before it reads any resource key, so a bond
there describes a cost nobody pays. Thirteen entries unrelated to MCP carried
one and are cleared too.
test_mcp_consumer_services_entry demanded the group_names shape for every
client entry, refused ones included, and now demands the opposite there.
dynamic-flag keeps its rule and takes its own documented per-key exemption.
make docs redraws twenty graphs: the edges go, the nodes stay as
'openwebui ❌' and 'flowise ❌', so the refusal is visible rather than read as
an integration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every variants.yml header states the rule the bond feeds: variant 0 carries each
tightly-coupled (bond>=1, same-host) service, while loosely-coupled (bond<1,
separate-host) partners are excluded from the closure and distributed across
later variants to keep every host under budget. tor was declared bond: 1 in all
101 consumers, so it had to sit in every variant-0 closure.
It does not belong there. The onion is a separate-host provider reached over the
node's hidden service, the same shape as the bridged partners that already carry
0.5, and f1dff50fb established both the reasoning and the precedent when it
dropped xwiki and bigbluebutton from 1 to 0.5: bridged partners with their own
role and their own spec, whose bond declaration was the outlier rather than
their placement.
This also unblocks the variant-level tor contract. Requiring a non-baseline
variant to pin the onion on while tor still counted as tightly coupled would
have pulled variant 0's closure requirements along with it.
Verified: tests/lint 336 passed / 307 subtests and tests/integration/roles 107
passed / 610 subtests, including test_resource_budget over every variant's
deduplicated footprint and the bond-aware test_order closure.
Not measured, and the honest cost: f1dff50fb's own caveat applies unchanged
here. Whether redistributing a service out of the baseline closure buys back
wall-clock is not established; role-runtimes has the per-role seconds and no run
has been made with this shape.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
75 commits: the socialhome role reimplemented against the project it actually
deploys, native-login specs for funkwhale and shopware, a window.open IdP
handoff and a settings-link fallback in the shared persona helpers, a swarm
converge budget that gives up on a terminal task, and a long tail of role
fixes.
Eleven conflicts. The shared persona helpers (keycloak.js, logout.js) take the
incoming structure — they gained real behaviour this branch never had — with
this branch's resolveTimeout convention re-applied over it and
confirmKeycloakLogoutIfPrompted re-attached at every return point; without it
the Keycloak SSO session outlives the logout. playwright.config.js keeps this
branch's version: both sides set actionTimeout, but ours routes through an env
var and 15s is too short behind an onion. socialhome's spec takes the incoming
content — 2dbd6eb10 found the role described the wrong upstream project
entirely — and keeps gotoOnion and expectHstsWhenTls, since demanding HSTS over
a plaintext onion is a false failure. Both READMEs were regenerated rather than
merged by hand.
Three collisions arrived with no conflict marker at all, because each side
changed a different place in a different file:
- group_vars/all/18_swarm.yml declared swarm_stack_converge_retries twice.
Same value, so nothing breaks today; editing the documented one would have
silently changed nothing.
- svc-net-tor and web-app-socialhome both claimed 192.168.74.0/24. Socialhome,
the newer role, moves to 192.168.75.0/24.
- web-app-socialhome/meta/schema.yml arrived as a new file, so ef5ea7cb5's
rename never saw it. Its credential would not have resolved and the role
would have deployed with an unusable administrator account.
The incoming specs also needed this branch's conventions: raw timeouts wrapped,
four page.goto calls routed through gotoOnion, a request.get given an
onion-aware timeout, and 21 roles marked playwright-service-flag over the tor
block this branch added to them.
make quality green on the merge result.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3f51d4ed9 bumped the recorder from v0.16.2 to v0.17.1. Upstream 0.17 drops
the rabbitmq transport for REST and reshapes [controller]: MR !746, listed
under the 0.17.0 changelog, replaces domain and insecure with url and a
mandatory api_key. recorder.toml.j2 still writes the old shape, and 0.17
also started warning about unknown configuration fields. Back to v0.16.2
until the template is migrated.
The frontend, the controller and the recorder are released as one
coordinated set, so all three carry a nocheck marker now. Bumping them
individually is what produced this. LiveKit and RabbitMQ keep updating:
they version independently of OpenTalk.
The marker sits after the value; utils/update/docker.py resolves it with
the default same-or-above placement, which the suppression catalog lists
for docker-version.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three conflicts, none resolved by picking a side wholesale:
- Makefile: took the incoming ACT_INPUTS, which routes through SWARM_DISTROS
instead of inlining the same expression a second time.
- test-e2e-playwright/02_run_one.yml: took the incoming rc handling. An absent
rc now counts as failure; the previous default(0) let an aborted Playwright
run pass silently, contradicting the debug message right above it.
- web-app-peertube spec: combined both. Kept gotoOnion and resolveTimeout,
which this branch's lint enforces, and adopted the incoming retry-inside-poll
for the OIDC button plus its networkidle wait.
The declared values were guesses that ran up to 60x the size of the image they belong to: seaweedfs asked for 30GB against a 104MB image, nextcloud for 20GB against 353MB. Because min_storage feeds the storage guard, the runner packer and the sufficient-storage filter, those numbers made hosts look too small and pruned caches that deploys did not need to lose.
Every service that declares min_storage together with image and version now carries the measured compressed transfer size of that image plus 200MB of headroom. Sizes were read from the registries through utils.docker.registry.manifest_transfer_size, the same path the headroom test uses, so the declarations and the gate agree by construction. Values are written in MB throughout, matching the existing 500MB / 100MB / 50MB style.
119 declarations across 89 roles. Range: 206MB (memcached) to 3473MB (ollama); four exceed 1GB. The 84 min_storage entries that name no image are untouched, since there is nothing to measure them against.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Per-role deploy-mode opt-out moves from meta/tests.yml 'skip:' to meta/services.yml <entity>.modes.{compose,swarm}.enabled on the primary entity. get_role_skip now derives from it; get_role_mode_enabled added. A new lint requires every role shipping templates/*compose*.yml.j2 to declare both mode flags. Migrates 107 services.yml and drops 11 skip-only tests.yml. Also removes the variant_status deployability gate so swarm runs one job per variant unconditionally (variant_status.py and deployable_variant_indices deleted, complexity model simplified). Adds the i8matix alias.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
coturn, bigbluebutton, jitsi and opentalk move WebRTC media over UDP (STUN/TURN, SRTP, JVB, LiveKit SFU), which Tor cannot carry. Pin their tor service block to enabled/shared false in meta/services.yml and in every variant, so no deploy scope or variant ever injects onion domains for them and every consumer lookup renders their clearnet URLs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Controller v0.34 restructured its configuration: the top-level livekit,
turn, stun and log keys are gone, livekit now lives under the mandatory
roomserver section (run internally via kind=internal) and
http.service_api_keys became required for the roomserver integration
(new service_api_secret credential). TURN moves to the livekit embedded
server on the stun_turn band port 3479 since the controller no longer
brokers coturn to clients. Verified by booting controller v0.34.1 with
the rendered template (settings load, startup reaches the DB connect)
and livekit v1.13.3 with the TURN block (TURN server starts). LiveKit
and controller ports resolve from the services.yml ports SPOT instead
of hardcoded values.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bump every outdated image pin and repository ref to latest except
web-app-nextcloud (kept on 33-fpm-alpine). Covers keycloak, gitlab (6
services), matrix, opentalk, jira, confluence, espocrm, magento
opensearch, elasticsearch, rabbitmq, qdrant, ollama, seaweedfs,
semaphore, wordpress, taiga, bluesky (image + social-app ref), bookwyrm
ref, roulette-wheel, simpleicons, registry cache/docker, and the
svc-runner base image. Includes major jumps (elasticsearch 9, rabbitmq
4, opensearch 3, espocrm 10, registry 3, node 26, ubuntu 26.10) that
are pin-only and still need a deploy validation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename the role, its test tree, the primary entity key
(volume-2-local), the derived vars (BKP_VOLUME_2_LOCAL_*,
SYS_SERVICE_BACKUP_VOLUME_2_LOCAL), the run-once fact, and every
group_names consumer flag, include path, doc, and lint reference. The
container_backup service key and the on-disk backup-docker-to-local
tree keep their names: both are contracts with existing consumer
entries, snapshot trees, and the baudolo pull tooling.
Operators must rename the svc-bkp-container-2-local inventory group and
disable the previous versioned units on already-provisioned hosts; the
pre-deploy snapshot glob derives its base from the new role name and
will not find them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every invokable stack role that declares named volumes in meta/volumes.yml now carries the standard container_backup consumer entry: enabled resolves via 'svc-bkp-container-2-local' in group_names, shared stays literally true since there is no per-consumer backup instance. Playwright/dynamic-flag/bond-baseline suppressions ride on the block per the established idioms; stale explanatory comments in the touched files are dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the meta/services.yml key across all roles, the roles_with_default_placement lookup (now roles_with_placement), the get_role_default_placement / iter_roles_with_default_placement helpers, and every consumer (swarm closure, compose_volumes filter, nfs_prep, container deploy template, constructor stage, extend_inventory) plus their tests and docs.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Onion routing is transport-layer with no persona/UI surface, so tor cannot be gated from the per-persona Playwright specs. Mark the tor block with '# nocheck: playwright-service-flag' in the 60 domain roles that ship a playwright.env.j2, per test_env_services_match's documented exemption for services that legitimately cannot be Playwright-gated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Domain-serving roles reference the svc-net-tor provider via a tor: block. Consumers declare enabled (dynamic group_names gate) and shared: true (static — onion is always the shared node model, inline nocheck: dynamic-flag). The provider svc-net-tor additionally declares exclusive/primary defaults that consumers inherit via resolve_service_config unless overridden.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The controller refuses to start without an [oidc] (or legacy
[keycloak]) section (upstream settings validation:
OidcConfigurationMissing), so an sso-disabled OpenTalk cannot exist.
The all-off variant only ever passed because the old include resolver
ignored variant pins and pulled keycloak in via run_after; once
2f9a54cea enforced the pins, variant 1 deployed without keycloak and
the unguarded kcadm provisioning crashed in compose while controller
and recorder crashlooped on the unreachable issuer in swarm.
Pin services.sso to enabled/shared true (mirroring the hard ldap:false
next to it), drop the sso pins from both variants, and move the
keycloak client provisioning into 03_keycloak_client.yml behind an
OPENTALK_OIDC_ENABLED include gate as a defensive skip.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The embedded minio/seaweedfs object store emitted only compose-native resource
keys (cpus/mem_limit) and no replica pin, so under 'docker stack deploy' it got
the default replica count and raced its own state across nodes. Gate the
compose-native resource block to compose and add a swarm deploy: block pinned to
replicas=1 (deploy.yml.j2 self-gates to swarm, so compose emits nothing new).
Give the embedded seaweedfs an explicit name to avoid colliding with a
standalone web-app-seaweedfs sharing the cluster.
The kcadm provisioning tasks ran 'container exec' against a local container id
that only exists on the node hosting Keycloak; in swarm that node is not
necessarily the one running the play. Resolve the Keycloak host node and its
local container id, delegate_to that node, and split the controller restart into
'container restart' (compose) vs 'container service update --force' (swarm).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Take the host IP as an optional third element of each term
([service, protocol, host_ip]) instead of an ip= kwarg, and build the whole
ports: block from declared ports.local/public (host) + ports.internal
(container). Convert every compose template across 84 roles, folding the
remaining raw var/numeric/specific-ip mappings (including mailu's per-interface
mail ports bound to MAILU_IP4_PUBLIC). Multi-term calls render one term per line.
Add the compose-ports-must-use-container-ports lint: it forbids any raw
"<host>:<published>:<container>" publish item under a ports: block and requires
every term to declare ports.internal. Remove
tests/lint/roles/test_swarm_internal_ports.py, now subsumed.
Fix a container_depends_on rendering regression. The lookup self-indented
(default indent=4), so the added 4-space placement + | indent(4) triple-indented
the block. Flip the lookup default to column 0 (consistent with
container_networks), make every call use the 4-space + | indent(4) form, and add
a lint enforcing it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
About 38 web-app/web-svc roles now consume the central svc-db-* and svc-dns engines via lookup('engine', ...) instead of embedded sidecars: meta/services flags, variant matrices, and env/conf hosts point at the shared services. Lets these apps run unpinned and NFS-shareable in swarm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`bond` is a 0..1 factor for how strongly a service is tied to the overall
integration: bond: 1 = always required, bond: 0 = omittable. Groundwork for
tooling that later scores service importance and decides what can be dropped.
Named `bond` (not `bound`) to avoid clashing with the resource bounds/limits.
Add test_service_bond.py (lint): every enabled + shared service must declare a
valid `bond` (number in 0..1). Seed `bond: 1` as a placeholder on all 638
enabled+shared service entries across 88 roles, to be refined later.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.
objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.
deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.
Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
