fix(compose): bootstrap before notify, render last

Two branch-only commits each armed a defect in the compose-up handler
topic, and both failed in run 33785198875.

63f7f962bf registered the topic owner at the top of sys-svc-rdbms'
dedicated.yml so the notify two tasks below stopped aborting with
ERROR_ON_MISSING_HANDLER. The notification then survived to the
unconditional meta: flush_handlers at sys-svc-container/tasks/00_core.yml:53,
reached from sys-svc-compose's own 00_core before that role installs
BIN_COMPOSE, and long before 02_stack_host_prep installs BIN_CA_INJECT.
Matrix#1 died there on ca-inject with rc 2 after seven retries. That play
carries exactly one compose notification (job 100756596907, log line
11046), so closing this window is sufficient, not merely necessary.
dedicated.yml now enters 00_core.yml under the role's run-once flag
before it notifies, so the flush drains an empty queue and the flag then
makes main.yml skip the body on the real entry. The registration itself
stays ungated: gating it would skip the owner from the second app on.

95186163aa hoisted the sys-svc-compose include to sys-stk-backend
position 1. The consumer therefore rendered and notified compose-up
before sys-svc-objstore lazily loaded the provider app role, whose own
unconditional flush consumed the notification and ran the single
registered handler in the provider's chdir. mastodon and peertube never
got a stack, in compose and in swarm alike; the victim's own guard task
appears zero times in all four logs. Restoring the include below rdbms,
objstore and engine puts the notify after every foreign crossing. Two
pre-hoist runs in exactly that order are green on record.

The test keeps its cases and gains one. It selects the render include by
the absence of tasks_from instead of by first occurrence, so an early
registration-only include stays legal while a hoisted render still fails.
The new case pins the bootstrap: entry point, position before the first
notify, and the run-once guard whose removal would let 01_reset.yml tear
down DIR_COMPOSITIONS once per dedicated-DB app.

Same class, pre-existing, not fixed here: svc-db-qdrant notifies and
flushes three tasks before its first compose entry; web-app-magento pairs
a dedicated database with a shared object store; web-app-seaweedfs still
flushes a topic it does not own; svc-runner enters 00_core.yml unguarded.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kevinveenbirkenbachandClaude Opus 5 committed 2026-09-04 01:16:38 +02:00
1 parent 70feb319be
commit 252b3e58b2
3 files changed
+41 -14

No files matched your search

+7 -7
View File
@@ -18,13 +18,6 @@
- "sso_proxy_enabled={{ lookup('sso', application_id, 'is_proxy_gated') }}"
when: MODE_DEBUG | bool
- name: "🧩 For '{{ application_id }}': Load sys-svc-compose"
include_role:
name: sys-svc-compose
vars:
compose_handlers_flush: false
database_consumer_id: "{{ _sys_stk_backend_database_consumer_id }}"
- name: "🧩 For '{{ application_id }}': Load central RDBMS"
include_role:
name: sys-svc-rdbms
@@ -51,6 +44,13 @@
vars:
engine_consumer_id: "{{ application_id }}"
- name: "🧩 For '{{ application_id }}': Load sys-svc-compose"
include_role:
name: sys-svc-compose
vars:
compose_handlers_flush: false
database_consumer_id: "{{ _sys_stk_backend_database_consumer_id }}"
- block:
- name: "🔧 set sso_proxy_application_id (needed due to lazy-loading issue)"
set_fact:
+6
View File
@@ -5,6 +5,12 @@
tasks_from: none.yml
handlers_from: main
- name: "🥾 For '{{ application_id }}': bootstrap the compose host once"
ansible.builtin.include_role:
name: sys-svc-compose
tasks_from: 00_core.yml
when: run_once_sys_svc_compose is not defined
- name: "✅ Ensure env dir exists: {{ _container_env_dir }}"
ansible.builtin.file:
path: "{{ _container_env_dir }}"
@@ -23,20 +23,28 @@ def _role_include(task: dict) -> dict:
return {}
def _included_roles(tasks: list) -> list[str]:
names = []
def _role_includes(tasks: list) -> list[dict]:
includes = []
for task in tasks:
include = _role_include(task)
if include.get("name"):
names.append(include["name"])
return names
includes.append(include)
return includes
class TestHandlerRegistrationOrder(unittest.TestCase):
def test_the_backend_loads_the_handler_owner_before_any_other_role(self) -> None:
order = _included_roles(load_yaml_str(read_text(str(BACKEND))))
def test_the_backend_renders_the_stack_after_every_foreign_provider(self) -> None:
includes = _role_includes(load_yaml_str(read_text(str(BACKEND))))
order = [include["name"] for include in includes]
self.assertIn(COMPOSE_OWNER, order)
self.assertEqual(order[0], COMPOSE_OWNER)
render = next(
index
for index, include in enumerate(includes)
if include["name"] == COMPOSE_OWNER and not include.get("tasks_from")
)
for provider in ("sys-svc-rdbms", "sys-svc-objstore", "sys-svc-engine"):
with self.subTest(provider=provider):
self.assertLess(order.index(provider), render)
def test_the_owner_is_reloaded_directly_before_the_flush(self) -> None:
tasks = load_yaml_str(read_text(str(BACKEND)))
@@ -56,6 +64,19 @@ class TestHandlerRegistrationOrder(unittest.TestCase):
self.assertEqual(loader.get("name"), COMPOSE_OWNER)
self.assertEqual(loader.get("handlers_from"), "main")
def test_the_notifier_bootstraps_the_compose_host_before_its_first_notify(self) -> None:
tasks = load_yaml_str(read_text(str(RDBMS_DEDICATED)))
bootstrap = next(
index
for index, task in enumerate(tasks)
if _role_include(task).get("tasks_from") == "00_core.yml"
)
notify = next(index for index, task in enumerate(tasks) if task.get("notify"))
self.assertLess(bootstrap, notify)
self.assertIn(
"run_once_sys_svc_compose is not defined", str(tasks[bootstrap].get("when"))
)
def test_the_owner_still_holds_every_compose_topic_that_is_notified(self) -> None:
owner = ""
for path, content in iter_project_files_with_content(