Problem:
Every compose service logs to journald through
sys-svc-container/templates/base.yml.j2. The daily journal health
check grepped the whole journal for "error", so routine application
errors from containers triggered the host alert (email/Telegram) and
buried real host problems.
Change:
- Read the last day of the journal as JSON (MESSAGE, CONTAINER_ID,
SYSLOG_IDENTIFIER, _SYSTEMD_UNIT) and skip every entry that carries
a CONTAINER_ID field. Docker daemon and system errors are still
reported.
- Report each match as one readable "source: message" line instead
of raw journal output.
- Stream the journalctl output line by line instead of buffering a
whole day of logs in memory.
- Exit 2 with journalctl's stderr if journalctl itself fails, instead
of silently reporting "no errors".
- Rewrite the check in Python (files/python/script.py, deployed by
sys-service like the other sys-ctl-hlth roles) and decode non-UTF-8
messages.
- Fix the success message, which claimed "All docker containers are
healthy".
Impact:
Container log errors no longer raise this host alert on any
installation. Container health stays covered by sys-ctl-hlth-container;
application log analysis belongs in a log platform such as Wazuh.
system_service_suppress_flush is unchanged until a deploy confirms
that no host errors remain.
Tests:
Unit tests run the script against a fake journalctl (8 cases):
container error ignored, daemon error reported, identifier fallback,
non-UTF-8 message, journalctl failure, many container lines streamed
past, clean journal passes. The core cases fail against the previous
script.
A compose row has two behaviours, with or without the mesh, but its title
showed three: no glyph on a role's larger variants, 🛣️ or 🚇 on its smallest.
No glyph and 🛣️ deploy the same thing, so every compose title now reads 🛣️
or 🚇 like a swarm title, and host titles stay without one. Only the smallest
variant still rotates onto the mesh.
A &vpn pin on a row no mode of which can take the mesh now aborts the run
instead of quietly deploying 🛣️, and a regular row pinned to the mesh only
rotates onto a mode that can carry it.
495d23dbde raised an admin context with a fresh in-memory bean, which fixed
hasSaveAccess() and moved the failure one guard further. User::verify_data()
compares $this->id == $current_user->id, and between two unsaved beans that is
'' == '', so a deployment with a single administrator takes the last-admin
branch for the very account CreateUser is in the middle of creating. The branch
reads the legacy $mod_strings, which this request never loads.
That is why the two SuiteCRM jobs of CI run 38818071194 looked like two
failures: the LDAP variant ended in ErrorException "Trying to access array
offset on value of type null" at modules/Users/User.php:1678 and answered
HTTP 500 to POST /login, while the SAML variant took the same verify_data()
failure, avoided the 500 because $mod_strings happened to be loaded, redirected
to index.php?action=Error&module=Users and left no session, so its spec timed
out waiting for the SPA instead. Both are the first login of a persona that has
no local row yet; administrator passes in both because it already has one.
Use the stored system user, which has a real id, so hasSaveAccess() passes,
restrictAdminOnlyFields() early-returns, setIsAdmin()'s reloaded bean resolves
and '' == '1' skips the last-admin branch.
Assign $current_user before calling getSystemUser(), not from its return value.
User::retrieve() runs loadPreferences() after filling the bean, and both
UserPreference::loadPreferences() and reloadPreferences() dereference
$current_user->id unguarded, so retrieving while the caller's anonymous
$current_user is still in place raises "Attempt to read property id on null"
and answers HTTP 500 one line further on. Assigning first also means the only
session preferences key written is the system user's own, never the key of the
account signing in.
The patch header records both failure modes, because the patch outlives this
commit and is applied at image build time with -F0, so whoever reads it next
when a hunk stops matching needs to know what it was for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 207cdc70ff)
The 🐳 Workspace (ubuntu:latest) job on main (run 38879556215, 1b36ef6120)
failed test-unit-python in the hook's second make test:
AssertionError: Regex didn't match: '^ localhost=ok/\d+ms$'
not found in ' localhost=ok/-1612931482679767779ms'
ubuntu:latest is Ubuntu 26.04, whose date is uutils coreutils. uutils
0.7.0 ignores the field width in %3N and prints all nine nanosecond
digits, so `date +%s%3N` yields a 19-digit value instead of epoch
milliseconds. Against main's sampler, uutils 0.7.0 date reports
localhost=ok/7081859ms, and an unpadded nanosecond value turns the
difference negative.
answer_once now takes both timestamps from /proc/uptime, which the
kernel prints with two decimals whatever date the image ships. The
sampler already reads /proc/net/udp, so it was Linux-only before. The
resolution is 10 ms; the same probe reports 10 to 20 ms with uutils
date on PATH.
The new test puts a date on PATH that does not understand %3N; main's
sampler dies on it, the fixed one does not call date for the latency.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A ping proved the tunnels answer, not that a session lands on the right host
or that full-size TCP segments cross them. Each pair sharing a mesh now also
opens an SSH session to the peer's mesh address and must land on that peer,
then pulls 8 MiB through it and compares the checksum with the one taken at
the source. Each pair that shares no mesh must fail the SSH connection itself,
never on authentication, since every host trusts every other host's key.
WireGuard must refuse a packet forced into a mesh that holds no peer for it
(Required key not available), so the isolation is shown to come from the
cryptokey table rather than from a missing route. The two hosts must still
reach each other on the underlay, so the isolation is the mesh's doing.
The SSH options are the platform's own, mesh_ssh_args over the repository's
ssh_args, the ones the swarm transport switches Ansible onto; the addresses
come from mesh_address. Every probe runs through one check helper, so a probe
is a predicate rather than another copy of the pass and fail bookkeeping.
Verified in the dev stack: all 20 checks passed, and a mutant that inverted
every expectation (unreachable pairs, a wrong identity, a different file, a
denied key, the peer's own address forced through the cryptokey check) failed
all 9 of its checks.
evangelostsak
Prageeth Panicker
kevinveenbirkenbach
