mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 10:59:59 +00:00
kevinveenbirkenbachandClaude Opus 5
a8cea953f8
The compose flush recreates a dedicated postgres moments before this probe
runs, and its healthcheck sits on a 10s interval, so health=healthy cannot have
reported yet. The probe returned empty, the realign task below is gated on that
output being non-empty, and it skipped silently. The rotated POSTGRES_PASSWORD
therefore never reached the surviving data volume, and ./launcher rebuild
discourse died minutes later:
PG::ConnectionBad: connection to server at "192.168.9.2", port 5432 failed:
FATAL: password authentication failed for user "discourse"
Measured in the failing job: the engine started 01:29:44.757, postgres logged
"database system is ready to accept connections" at 01:29:46.737, and the probe
ran at 01:29:48.419 - 1.68s after the socket was already live, and 3.66s after
a start whose first healthcheck could not fire before ~10s.
Running is the predicate the operation needs, not a loosening.
set_postgres_superuser_password.sh reaches the cluster over the container local
socket by peer auth, which is exactly what lets a rotated credential be written
into an engine nobody can log into any more, and it carries its own readiness
loops: 30 x 30 x 2s around SHOW hba_file and 15 x 2s of TCP auth retry. Against
that 60s budget the observed start-to-ready times were 1.98s, 4.05s and 5.51s
across three engines, an 11x margin.
The other call site, sys-svc-compose post_deploy, already waits for healthy
before reaching this file, so running matches there too and its behaviour is
unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
37 lines
1.9 KiB
YAML
37 lines
1.9 KiB
YAML
---
|
|
- name: "🔍 Find the compose file rendering '{{ application_id }}'s own engine"
|
|
ansible.builtin.stat:
|
|
path: "{{ lookup('path_absolute', 'roles', application_id, 'templates/compose.yml.j2') }}"
|
|
register: _app_db_compose_template
|
|
delegate_to: localhost
|
|
run_once: true # nocheck: run-once the path is host-independent, one stat answers for the play
|
|
changed_when: false
|
|
when: (compose_mode_force | default('') or DEPLOYMENT_MODE) == 'compose'
|
|
|
|
- name: "🔑 Realign a dedicated postgres for '{{ application_id }}'"
|
|
when:
|
|
- (compose_mode_force | default('') or DEPLOYMENT_MODE) == 'compose'
|
|
- lookup('database', application_id, 'local') | bool
|
|
- lookup('database', application_id, 'type') == 'postgres'
|
|
- _app_db_compose_template.stat.exists | default(false)
|
|
block:
|
|
- name: "🔍 Find the running engine of '{{ application_id }}'"
|
|
ansible.builtin.command: >-
|
|
container ps -q
|
|
--filter name=^{{ lookup('database', application_id, 'container') }}$
|
|
--filter status=running
|
|
register: _app_db_engine
|
|
changed_when: false
|
|
|
|
- name: "🔑 Realign the password a dedicated postgres kept from its initdb"
|
|
ansible.builtin.script: > # nocheck: container-address-quote script evals the fragment itself
|
|
{{ lookup('path_absolute', 'roles/svc-db-postgres/files/shell/set_postgres_superuser_password.sh') }}
|
|
{{ lookup('database', application_id, 'container') | quote }}
|
|
{{ lookup('database', application_id, 'password') | quote }}
|
|
{{ lookup('database', application_id, 'username') | quote }}
|
|
register: _app_db_realign
|
|
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
|
|
changed_when: _app_db_realign.stdout is search('\\[CHANGED\\]')
|
|
failed_when: _app_db_realign.rc != 0 or _app_db_realign.stdout is search('\\[FAIL\\]')
|
|
when: (_app_db_engine.stdout | trim) != ''
|