perf(proxy): reload OpenResty for a vhost instead of redeploying the task

Adding a domain redeployed the whole OpenResty task. Measured in the green onion job odoo 0: 26 redeploys at 41 to 43 seconds each, 18.1 minutes of a 4h44m job. A reload costs one to two seconds.

It is safe for a vhost because of where the file lands. nginx.py builds files.domain as servers_dir + protocol + domain.conf, and servers_dir is /etc/nginx/conf.d/servers/ - inside the conf.d DIRECTORY bind. A directory keeps its inode across the atomic write-and-rename that stack_host_template performs, so the running master already sees the new file. Only nginx.conf is bound as a FILE, whose inode the task pins at create time; that path keeps the redeploy, as do the default server config and the site-is-down case.

History matters here: a9ca2ae68 once made the handler reload for everything, including nginx.conf, and f9dea42c6 replaced it with the force-recreate. That replacement rode along in a commit about apt mirrors and explains itself nowhere, so the only recorded reason is the file header - the file bind and its pinned inode. That reason does not apply to a vhost in the directory bind, which is why this reload is confined to that case.

The reload cannot prove it took, so it does not have to. After the flush the domain is probed once, and anything other than 200, 301 or 302 notifies the full redeploy. The worst case is therefore one HTTP probe on top of the behaviour this replaces, never a domain left unserved behind a green deploy.

OpenResty runs replicated 1/1 (services.txt of a real deployment), so the exec reaches the only instance and no replica can keep stale config.

Not verified: the reload itself was never executed against a running stack.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

roles/svc-prx-openresty/handlers/main.yml

@@ -1,4 +1,8 @@
 ---
 - name: Restart and validate OpenResty
   listen: restart openresty
-  include_tasks: restart_and_validate.yml
+  include_tasks: restart.yml
+
+- name: Reload OpenResty
+  listen: reload openresty
+  include_tasks: reload.yml

roles/svc-prx-openresty/handlers/reload.yml

@@ -0,0 +1,31 @@
+---
+# A vhost lands in directories.configuration.servers, which sits inside the
+# conf.d DIRECTORY bind. A directory keeps its inode across an atomic file
+# write, so the running container already sees the new file and SIGHUP is
+# enough -- unlike nginx.conf, which is bound as a FILE and whose inode the
+# task pins at create time (see restart.yml).
+- name: Validate OpenResty configuration before reloading
+  ansible.builtin.shell: >
+    container exec {{ lookup('container_address', 'svc-prx-openresty', 'openresty') }} openresty -t -q
+  register: openresty_test_reload
+  changed_when: false
+  retries: 10
+  delay: 3
+  until: openresty_test_reload.rc == 0
+  failed_when: >
+    openresty_test_reload.rc != 0 and
+    ('is not running' not in ((openresty_test_reload.stderr | default('')) | lower)) and
+    ('no such container' not in ((openresty_test_reload.stderr | default('')) | lower)) and
+    ('is restarting' not in ((openresty_test_reload.stderr | default('')) | lower))
+
+- name: Reload OpenResty
+  ansible.builtin.shell: >
+    container exec {{ lookup('container_address', 'svc-prx-openresty', 'openresty') }} openresty -s reload
+  register: openresty_reload
+  changed_when: openresty_reload.rc == 0
+  failed_when: >
+    openresty_reload.rc != 0 and
+    ('is not running' not in ((openresty_reload.stderr | default('')) | lower)) and
+    ('no such container' not in ((openresty_reload.stderr | default('')) | lower)) and
+    ('is restarting' not in ((openresty_reload.stderr | default('')) | lower))
+  when: openresty_test_reload.rc == 0

roles/sys-stk-front-proxy/tasks/main.yml

@@ -95,7 +95,7 @@
     webserver_websocket_location: ""
     webserver_client_max_body_size: "{{ lookup('config', front_proxy_application_id, 'server.client_max_body_size', '100m') }}"
   register: nginx_conf
-  notify: restart openresty
+  notify: reload openresty
 
 - block:
     - name: "🔍 Check if {{ lookup('domain', front_proxy_application_id) }} is reachable"
@@ -120,3 +120,28 @@
 
 - name: "🔄 Restart Webserver for '{{ front_proxy_domain_conf_dst_resolved }}'"
   ansible.builtin.meta: flush_handlers
+
+- name: "🔍 Verify '{{ front_proxy_domain }}' is served after the reload"
+  uri_probe:
+    url: "{{ lookup('tls', front_proxy_application_id, 'url.base') }}"
+    status_code:
+      - 200
+      - 301
+      - 302
+  register: front_proxy_reload_check
+  failed_when: false
+  changed_when: false
+  when: nginx_conf.changed
+
+- name: 🔄 Fall back to a full redeploy when the reload did not take
+  ansible.builtin.command:
+    cmd: "true"
+  notify: restart openresty
+  when:
+    - nginx_conf.changed
+    - front_proxy_reload_check.status is defined
+    - front_proxy_reload_check.status not in [200, 301, 302]
+
+- name: "🔄 Flush the fallback redeploy for '{{ front_proxy_domain }}'"
+  ansible.builtin.meta: flush_handlers
+  when: nginx_conf.changed