fix(test-e2e-cli): exec on the node that holds the container

hermes_hermes.1 ran on a worker while run_one.yml ran its script on the loop
host. resolve_container_id.sh finds nothing node-locally on the manager, falls
through to the cluster-wide branch and returns the task's ContainerID, which
only the owning node's daemon knows, so `container exec` answered
`No such container: 65801ca259a1` twenty times over 116 seconds. Run
33785198875, job 100756596935.

Resolve the owning node and run there. meta/tests.yml gains
container_service_key for the two roles that exec into their own container; a
role without the key keeps the loop host, and role_tests returns the default
for a missing path, so absence is a polarity rather than an error.

The reset at run_one.yml:27-30 is unconditional and precedes every
delegate_to. Without it the first loop iteration dies with
'CLI_HOST_NODE' is undefined, because _cli_apps starts with a role that has no
key; the same task closes the fact leak that include_tasks leaves open between
iterations. The resolve include is gated on an explicit length check rather
than truthiness, or a keyless role would hang in resolve_host_cid's converge
wait, and it caps that wait at 30 x 2s instead of inheriting 600 x 2s: the
deploy is long finished by the time the CLI stage runs, so failing fast beats
waiting twenty minutes.

Staging, render and run all delegate. Delegating only the run would turn
`No such container` into `No such file or directory`, since the assets would
stay on the loop host. Both templates now carry CLI_LOCAL_CID: with the exec
on the owning node the id is node-local, so container_address's deferred
`$( ... )` is not needed, and it never survived `set -a && . test.env` anyway,
which evaluates it once at source time.

The stage directory is removed on the delegate too. tasks/main.yml clears it
without delegation, i.e. only on the loop host, and the task's own name calls
its contents decodable DB secrets.

Measured locally rather than argued: self-delegation leaves file, copy with
content, copy with src and shell byte-identical, including mode; the loop
reset gives keyless roles an empty container and keeps CLI_HOST_NODE on the
loop host across five iterations; and container_service_key resolves to
hermes, openclaw and empty strings through the same walk cli.timeout already
uses in production.

Not fixed, and named so it is not forgotten: web-app-joomla, web-app-yourls
and web-app-penpot exec from an IS_STACK_HOST gate without resolve_host_cid,
so a task of theirs landing on a worker hits the same error from the same
source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

roles/test-e2e-cli/tasks/run_one.yml

@@ -4,6 +4,7 @@
     _cli_env_template: "{{ lookup('path_absolute', 'roles', application_id, 'templates/test.env.j2') }}"
     _cli_stage_dir: "{{ TEST_E2E_CLI_STAGE_BASE_DIR }}/{{ application_id }}"
     _cli_test_script: "{{ TEST_E2E_CLI_STAGE_BASE_DIR }}/{{ application_id }}/test.sh"
+    _cli_service_key: "{{ lookup('role_tests', application_id, 'cli.container_service_key', default='') | trim }}"
 
 - name: "✅ Assert CLI test assets exist for {{ application_id }}"
   ansible.builtin.stat:
@@ -23,23 +24,41 @@
   loop_control:
     label: "{{ item.item }}"
 
+- name: 🧭 Default the CLI host to the loop host for {{ application_id }}
+  ansible.builtin.set_fact:
+    CLI_HOST_NODE: "{{ inventory_hostname }}"
+    CLI_LOCAL_CID: ""
+
+- name: "🔍 Resolve the node that holds {{ application_id }}'s test container"
+  ansible.builtin.include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/swarm/resolve_host_cid.yml') }}"
+  vars:
+    resolve_service_key: "{{ _cli_service_key }}"
+    resolve_output_prefix: CLI
+    resolve_compose_cid: "{{ lookup('container_address', application_id, _cli_service_key) }}"
+    resolve_converge_retries: 30
+    resolve_converge_delay: 2
+  when: _cli_service_key | length > 0
+
 - name: 📁 Create stage directory for {{ application_id }}
   ansible.builtin.file:
     path: "{{ _cli_stage_dir }}"
     state: directory
     mode: "0700"
+  delegate_to: "{{ CLI_HOST_NODE }}"
 
 - name: 📦 Stage CLI test assets for {{ application_id }}
   ansible.builtin.copy:
     src: "{{ lookup('path_absolute', 'roles', application_id, 'files/test') }}/"
     dest: "{{ _cli_stage_dir }}/"
     mode: "0755"
+  delegate_to: "{{ CLI_HOST_NODE }}"
 
 - name: 📦 Stage the shared CLI harness for {{ application_id }}
   ansible.builtin.copy:
     src: "{{ lookup('path_absolute', 'roles/test-e2e-cli/files/shared') }}/"
     dest: "{{ _cli_stage_dir }}/shared/"
     mode: "0755"
+  delegate_to: "{{ CLI_HOST_NODE }}"
 
 - name: 🧭 Reset the per-role deployment-mode override
   ansible.builtin.include_vars:
@@ -74,6 +93,7 @@
     dest: "{{ _cli_stage_dir }}/test.env"
     mode: "0600"
     content: "{{ lookup('template', _cli_env_template) }}"
+  delegate_to: "{{ CLI_HOST_NODE }}"
 
 - name: 🚀 Run CLI test script for {{ application_id }}
   ansible.builtin.shell:
@@ -88,6 +108,7 @@
   register: _cli_run
   changed_when: false
   failed_when: false
+  delegate_to: "{{ CLI_HOST_NODE }}"
 
 - name: 📄 Print CLI test output on failure for {{ application_id }}
   ansible.builtin.debug:
@@ -103,3 +124,10 @@
   ansible.builtin.set_fact:
     _cli_failures: "{{ (_cli_failures | default([])) + [application_id] }}"
   when: _cli_run.rc != 0
+
+- name: 🧹 Remove the delegated stage dir of {{ application_id }}
+  ansible.builtin.file:
+    path: "{{ _cli_stage_dir }}"
+    state: absent
+  delegate_to: "{{ CLI_HOST_NODE }}"
+  when: CLI_HOST_NODE != inventory_hostname

roles/web-app-hermes/meta/tests.yml

@@ -1,3 +1,4 @@
 ---
 cli:
   timeout: 600
+  container_service_key: hermes

roles/web-app-hermes/templates/test.env.j2

@@ -1,3 +1,3 @@
-MCP_CLIENT_CONTAINER={{ lookup('container_address', application_id, 'hermes') }}
+MCP_CLIENT_CONTAINER={{ CLI_LOCAL_CID }}
 MCP_SERVER_IDS='{{ HERMES_MCP_SERVERS | map(attribute='id') | list | to_json }}'
 MCP_TEST_ENABLED={{ lookup('config', application_id, 'mcp.enabled') | string | lower }}

roles/web-app-openclaw/meta/tests.yml

@@ -1,3 +1,4 @@
 ---
 cli:
   timeout: 600
+  container_service_key: openclaw

roles/web-app-openclaw/templates/test.env.j2

@@ -1,2 +1,2 @@
-MCP_CLIENT_CONTAINER={{ lookup('container_address', application_id, 'openclaw') }}
+MCP_CLIENT_CONTAINER={{ CLI_LOCAL_CID }}
 MCP_TEST_ENABLED={{ lookup('config', application_id, 'mcp.enabled') | string | lower }}

tests/integration/roles/test-e2e-cli/test_mcp_env_render.py

@@ -140,6 +140,8 @@ def _render(
         "applications": get_application_defaults(roles_dir=PROJECT_ROOT / "roles"),
         "application_id": application_id,
         "group_names": group_names,
+        "CLI_HOST_NODE": "stack-host",
+        "CLI_LOCAL_CID": f"{application_id}-container",
         "playbook_dir": str(PROJECT_ROOT),
         "compose_mode": mode,
         "DEPLOYMENT_MODE": mode,