mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 19:01:54 +00:00
Prageeth Panicker
dc3d30acaa
Problem: Every compose service logs to journald through sys-svc-container/templates/base.yml.j2. The daily journal health check grepped the whole journal for "error", so routine application errors from containers triggered the host alert (email/Telegram) and buried real host problems. Change: - Read the last day of the journal as JSON (MESSAGE, CONTAINER_ID, SYSLOG_IDENTIFIER, _SYSTEMD_UNIT) and skip every entry that carries a CONTAINER_ID field. Docker daemon and system errors are still reported. - Report each match as one readable "source: message" line instead of raw journal output. - Stream the journalctl output line by line instead of buffering a whole day of logs in memory. - Exit 2 with journalctl's stderr if journalctl itself fails, instead of silently reporting "no errors". - Rewrite the check in Python (files/python/script.py, deployed by sys-service like the other sys-ctl-hlth roles) and decode non-UTF-8 messages. - Fix the success message, which claimed "All docker containers are healthy". Impact: Container log errors no longer raise this host alert on any installation. Container health stays covered by sys-ctl-hlth-container; application log analysis belongs in a log platform such as Wazuh. system_service_suppress_flush is unchanged until a deploy confirms that no host errors remain. Tests: Unit tests run the script against a fake journalctl (8 cases): container error ignored, daemon error reported, identifier fallback, non-UTF-8 message, journalctl failure, many container lines streamed past, clean journal passes. The core cases fail against the previous script.