Commit Graph
98 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 b2e594073e fix(ca): give the CA to a service that only calls the self-signed endpoint
ca_injected asked three questions, the first being whether the application
has a domain. That decides whether it is SERVED over the self-signed CA. An
application that only CALLS one has no domain of its own, so the predicate
answered false, sys-svc-compose-ca skipped its stack, and the container ran
with an empty trust store.

svc-ai-agent-broker is such a caller: it authorizes every request against
Keycloak group paths over the public HTTPS base. Without the CA every
outbound call died in the TLS handshake, the broker answered 503 "access
check unavailable", Open WebUI turned that into HTTP 400, and the spec
failed on a model it had just listed. Its container was the only one in the
rescue artifact without SSL_CERT_FILE, while litellm and open-webui carried
it.

A service may now declare ca_client: true, and the predicate then answers on
the deployment's TLS mode alone, because the domain question is meaningless
for a client. The three served-side conditions are untouched, so no
application that already received the CA stops receiving it.

The alternative of pointing the broker at an internal Keycloak URL was
rejected: the role declares no keycloak service, so the URL would render
without a route.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-16 22:02:56 +02:00
kevinveenbirkenbachandClaude Opus 5.5 8e7dd508c9 style(tests): apply the formatter to the network_served tests
The autoformat step of the gate wrapped the long assertions that
660ad2ff16 committed unformatted. No behaviour change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 21:40:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 660ad2ff16 fix(web-app-seaweedfs): keep the frontend on every node that serves clearnet
The frontend and its sso, logout, dashboard and prometheus consumers were
gated on 'svc-net-tor' not in group_names, written when a Tor node served
onion only. On a multi node that removed the clearnet filer vhost, so the
storage specs of every SeaweedFS consumer failed on a clearnet pick with
ERR_SSL_UNRECOGNIZED_NAME_ALERT on filer.<domain>.

lookup('network_served', <network>, *role_ids) answers whether the node's
resolved NETWORK_MODE serves a network and every named role is deployed.
The gates use it instead of restating the mode resolution and the provider
check, so the frontend now drops only on a tor-only node. is_explicit_truth
registers the lookup in GATED_LOOKUPS so the static closure still reads
those flags as dynamic, and the dynamic-flag and universal-logout checks ask
is_explicit_truth instead of matching "in group_names" themselves.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 13:35:29 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1ece531adb fix(lookup): keep the onion apex redirect for a homepage with an onion sibling
PR #932 emits the node onion apex only for an onion DOMAIN_HOMEPAGE. On a multi node the homepage is the clearnet dashboard on purpose and the redirect template sends the apex to its onion sibling, so the merge dropped the apex redirect there. The apex is now kept when the homepage is an onion host or network_siblings lists an onion sibling this node serves; a target without one, such as a software domain off the primary, still gets none. The #932 test reads the primary domain instead of writing it out.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 17:27:30 +02:00
kevinveenbirkenbach 1da2cef95e Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving
# Conflicts:
#	plugins/lookup/email.py
#	tests/unit/python/roles/web-app-prometheus/test_blackbox_targets.py
2026-10-15 16:45:18 +02:00
kevinveenbirkenbachandClaude Opus 5.5 4397f4f57f fix(lookup): serve assets from the CDN sibling of the vhost's network
The asset host took the canonical CDN domain and only guessed its scheme from the suffix, so pages on the onion sibling loaded their libraries from the clearnet CDN, which their CSP and Tor reach do not allow. It now takes web-svc-cdn's url.base, which follows the network of the vhost being rendered; without the internal CDN it stays jsDelivr.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 16:35:09 +02:00
kevinveenbirkenbachandClaude Opus 5.5 bb1394517c fix(csp): let onion pages frame the single SSO issuer
Keycloak runs once on clearnet, yet onion pages such as the dashboard embed it for the silent SSO check, and its frame-ancestors listed only clearnet sources, so the browser refused the frame. On the issuer's clearnet vhost each translatable ancestor now also appears on the node onion. Whether a token moves to the node onion is decided by its host, so the issuer keeps its one URL on onion vhosts wherever it is listed, not only through the logout source.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 16:29:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e967c3017d test(lookup): keep the onion apex redirect to a clearnet homepage on a multi node
PR #932 emits the node onion apex only when DOMAIN_HOMEPAGE ends in .onion. On a multi node the homepage is the clearnet dashboard on purpose and the redirect template maps the target to its onion sibling, so merging main would drop the apex redirect without a textual conflict. This test goes red on that merge; relax the guard to a target that is onion or has an onion sibling rather than weakening the test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 16:25:26 +02:00
kevinveenbirkenbachandClaude Opus 5.5 5a202218c9 feat(web-app-keycloak): run one oauth2-proxy per network a gated application is served on
One oauth2-proxy applies one cookie-secure flag to every host it answers, so a secure clearnet instance drops the CSRF cookie of its plaintext onion sibling and the login loops. proxy_instances now yields one instance per network with its own compose service, sso or sso_onion host port and config file; each config takes cookie_secure, redirect_url and cookie_domains from its own network, and every vhost routes /oauth2/ to the instance of its network. The 29 gated roles gain a sso_onion port, and Bluesky heals the bind mount of every instance.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 15:57:38 +02:00
kevinveenbirkenbach 06a74e7658 Merge pull request #869 from infinito-nexus/fix/web-app-stalwart/relay-rule/793
fix(web-app-stalwart): authenticate bots with an app password under SSO
2026-10-15 15:42:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 08fba6b3ef feat(networks): look up a same-node URL in the network a vhost or consumer needs
network_url moves a URL into the network of the vhost being rendered, network_urls lists it once per network sibling with that sibling's scheme, and network_redirects fans a redirect source out to its siblings while the target stays canonical. Consumers that hard-code the canonical origin can now emit the sibling a request actually arrives on.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 13:21:23 +02:00
kevinveenbirkenbachandClaude Opus 5.5 857639db2b fix(test-e2e-playwright): bypass the Tor proxy for every clearnet host the node serves
The onion pick only bypassed the SSO issuer, so a clearnet CDN, asset or dashboard host an onion page loads went through Tor, which the lab network cannot exit. browser_proxy_bypass derives the list from the deployed applications' domains plus the issuer; it is empty on a clearnet pick and on a tor-only node. Specs that passed their own proxy to newContext dropped the bypass and now inherit use.proxy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 12:44:38 +02:00
kevinveenbirkenbachandClaude Opus 5.5 45732329b6 fix(lookup): keep the onion apex from redirecting to a clearnet homepage
On a Tor node current_play_redirect_domains schedules the bare node
onion as a redirect source targeting DOMAIN_HOMEPAGE. Without
web-app-dashboard, DOMAIN_HOMEPAGE falls back to SOFTWARE_DOMAIN
(infinito.nexus), so web-opt-rdr-domains rendered a 301 from
http://<node>.onion/ to http://infinito.nexus/: an onion visitor sent
to a clearnet host over plain HTTP.

The onion apex entry is now added only when DOMAIN_HOMEPAGE is an
onion host. Otherwise the onion apex gets no redirect and the default
server answers.

Closes #924

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 13:35:29 +02:00
kevinveenbirkenbachandClaude Opus 5.5 eefa14c125 Merge PR #414 (feature/web-app-wazuh/add-role) into feature/network/multi-network-serving
Brings web-app-wazuh to the reviewed state of #414. This branch was cut
from an earlier #414 head, which has since been rebased and gained its
review fixes, so both sides added the role and every role file conflicted
as add/add.

Conflicts, decided per file:

- roles/web-app-wazuh: #414's side. It is the reviewed version of the
  same role, and this branch's own Wazuh changes are already part of it
  (drop the Tor dual-stack workarounds, drop the first commit's
  leftovers, ports 8096/8097, heap via service_memory_mb).
- roles/web-app-wazuh/templates/opensearch_dashboards.yml.j2: #414's file
  with this branch's trust_dynamic_headers instead of base_redirect_url,
  so the OIDC redirect follows the network sibling the request came in
  on rather than pinning the clearnet URL.
- plugins/filter/bcrypt_hash.py and its test: #414's stable salt, which
  the role's vars now pass.
- test-e2e-playwright keycloak.js: #414's leave-group helper, which fails
  loudly on a missing user or group instead of returning silently.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 01:43:38 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7fa3400cc6 Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving
Conflicts resolved by keeping both sides' intent:

- call-test-deploy.yml: main's re-enabled deploy concurrency group, keyed
  on the network axis instead of the removed tor axis.
- sys-svc-proxy: main's shared headers/forwarded.conf.j2 now carries the
  branch's per-vhost semantics (X-Forwarded-Host, TLS port and state read
  from the vhost domain), so html, ws, upload and the Prometheus probes
  all answer per network sibling.
- plugins/lookup/email.py: main's provider-agnostic mail lookup asks the
  network registry instead of testing the .onion suffix, as the branch
  does everywhere; container_extra_hosts.py likewise. The provider host's
  NETWORK_MODE now decides whether it is served onion-first, since a Tor
  node defaults to multi and keeps clearnet canonical.
- workspace common.sh: main's INFINITO_DOMAIN fallback without the
  duplicated no-op line.
- README.md and requirement 035: both sides' rows (Wazuh plus main's
  Pi-hole and Stalwart entries); adapter_candidate count 39.

Adapted to main where the branch's rules now reach main's code:

- six tests from main read the primary domain from default.env instead
  of writing it out, as the hardcoded-domain lint requires;
- the chutney worker-check test passes ONION_SUFFIX from the network
  registry, which the branch's svc-net-tor test script requires;
- web-app-wazuh moves its local http ports to 8096/8097 (8094/8095 now
  belong to web-app-stalwart) and sizes the indexer heap through
  service_memory_mb, the same fixes #414 carries;
- web-app-wazuh's README cosmos follows Stalwart as the mail provider.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 00:14:42 +02:00
kevinveenbirkenbachandClaude Opus 5.5 24d7872fde Merge remote-tracking branch 'origin/main' into HEAD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:54:31 +02:00
kevinveenbirkenbachandClaude Opus 5.5 eda6249d7b fix(sys-svc-compose): probe provider networks by their exact name
The network probe used community.docker.docker_network_info, whose
get_network() scans GET /networks and returns the first entry whose Name
equals the requested name OR whose Id starts with it, in a list order that
changes between calls. In main run 38038541631 (Zammad#1, swarm) the
central redis overlay got the ID n8nwzb1q39m3u4k7awwwmt6yg, the probe for
the not yet created provider network "n8n" returned it as existing, the
create step was skipped, docker stack deploy attached hermes to the redis
overlay as "n8n", and the oauth2-proxy then reached the password-protected
central redis by its bare host name: "NOAUTH Authentication required".
The same scan can also return the ID-prefix twin of a network that does
exist, so comparing the returned name alone would turn that network into
"missing" and fail the create with "already exists".

Both probes now run `container network ls --filter name=<entity> --format
'{{json .}}'`, whose name filter never matches an ID. The new lookup
network_driver picks the entry named exactly after the entity and returns
its driver, '' when there is none; the drift checks, their messages and the
create steps all read that one entry. No docker module is left in the file,
so its two docker-module-uses-container-cli suppressions go too.

Verified: on the host daemon GET /networks/<id prefix> returns a network of
another name and docker network ls --filter name=<id prefix> returns
nothing. In ansible-playbook against real docker network ls output the
lookup returned the exact network's driver in 20 of 20 probes beside an
ID-prefix twin, an empty driver (create path) for a missing network and for
a name only contained in a sibling, and the drift path for a foreign
driver. Unit tests cover the same cases; with a substring match instead of
the exact name the two sibling cases fail.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:35:22 +02:00
Prageeth Panicker f5b552a667 Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role 2026-10-09 08:34:37 +00:00
kevinveenbirkenbach 799b835799 Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/relay-rule/793 2026-10-09 00:53:26 +02:00
kevinveenbirkenbachandClaude Opus 5.5 64677d4790 chore: regenerate role docs and apply autoformat after #881 and #873
`make quality-high` rewrote these files on every branch: the SuiteCRM
README and the root role table lacked the Redis service #881 added, the
session path test was not formatted, and an OpenTalk secret description
from #873 was not wrapped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 22:27:12 +02:00
kevinveenbirkenbach 3874f66ac3 Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/relay-rule/793 2026-10-08 19:38:56 +02:00
kevinveenbirkenbach 6e9b4e23dc Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/relay-rule/793 2026-10-08 19:27:27 +02:00
kevinveenbirkenbach badc2583eb Merge remote-tracking branch 'origin/main' into fix/web-app-suitecrm/redis-sessions/879 2026-10-08 19:25:24 +02:00
kevinveenbirkenbachandClaude Opus 5.5 3cefb01d71 fix(web-app-suitecrm): keep sessions in Redis so replicas share them
With three swarm replicas the PHP session files on the shared NFS volume
were read as empty under concurrent requests of one browser, so SuiteCRM
answered 403 on /api/graphql and dropped the session. Sessions now live in
Redis with session locking that waits like flock did; the phpredis version
is pinned in services.yml with its own update source, and the engine lookup
renders the save path.

Refs #879

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 18:31:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 08ef66d8de fix(web-app-stalwart): authenticate bots with an app password under SSO
Under SSO the role tried to let applications relay without SMTP AUTH by
widening the relay rule to the private networks. The rule compared the
peer address to the text of a CIDR, which never matches, so an
application could reach local mailboxes but got "550 5.1.2 Relay not
allowed" for every external recipient. Repairing the comparison would
have opened a relay, because published ports, Tor and the swarm mesh all
hand outside clients an address inside those networks.

Stalwart checks app passwords before it asks the external directory, so
a bot can authenticate while the directory is OIDC. With SSO enabled the
role now creates one app password per bot, acting for that account as
the recovery admin, and stores it in the token store the SMTP password
is already read from. A bot gets a new one when its entry is deleted in
Stalwart or its token is removed.

The relay rule is back to authenticated sessions only. The lookup loses
its SSO relay branch and the role its submission_via_relay flag, so an
SSO deployment submits on 465 with AUTH like every other one.

The role's CLI test now fails when an unauthenticated session is given
an external recipient, or when the bot credential is refused one.

Closes #793

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 08:57:22 +02:00
kevinveenbirkenbach b3563f4ea3 Merge remote-tracking branch 'origin/main' into fix/web-app-prometheus/blackbox-targets/792 2026-10-08 08:57:13 +02:00
kevinveenbirkenbachandClaude Opus 5 50b9f60c43 fix(prometheus): resolve the native-metrics scrape target per deployment mode
`native_metrics_target` was a Jinja filter, so it could not read
DEPLOYMENT_MODE: filters see no Ansible variables. It returned
`<services.<key>.name>:<port>` unconditionally, which is `matrix-synapse:9000`
for matrix. Compose serves that name; swarm does not, because `docker stack
deploy` ignores `container_name` and the alias never exists. This repository
says as much itself, through `lookup('compose_only', 'container_name', ...)`.
The target was therefore scraped and refused, and `playwright.spec.js:274`
reported `web-app-matrix: 1 matching target(s) reporting up != 1`, meaning one
matching target whose `up` value is not 1.

It becomes a lookup that delegates host resolution to `scrape_target`, which
was already mode-aware, so exactly one plugin knows the difference between the
two modes. The port has to be handed over explicitly: matrix exposes metrics on
9000 while `services.synapse.ports.internal.http` is 8008, so the existing
internal-port lookup cannot serve this case on its own.

`scrape_target` now refuses `port` together with an explicit `port_kind`.
Before, `port` won silently and the error text still named the
`ports.internal.<port_kind>` path that the call had never consulted.

`native_metrics_apps` drops an app that pins itself to compose through
`compose_mode_force` while the cluster runs swarm, because the scrape target
would then name the wrong object. Only the literal case: matrix declares that
override as a Jinja expression over its flavour, which this lookup cannot
resolve from the scraping role's variable scope, so variant 0 on swarm stays a
latent mismatch.

The two comment lines claiming `container_name:port` went with the change;
after it they state the opposite of what the code does.

Verified by mutation: with the swarm branch of `scrape_target` disabled, the
new test emits `matrix-synapse:9000`, the exact string CI reported.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-08 08:57:13 +02:00
Prageeth Panicker 94aa75720a Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role
# Conflicts:
#	README.md
2026-10-07 12:55:54 +00:00
Prageeth Panicker d92de14192 fix(web-app-wazuh): use a stable salt for the Wazuh password hashes
bcrypt_hash takes an optional salt_seed. Without a seed it keeps the random salt. With a seed the salt is derived from the seed and the secret, so the same password always gives the same hash. Wazuh passes a fixed seed for the indexer admin and the dashboard service hashes, so internal_users.yml no longer changes on every run and no longer triggers a restart.
2026-10-07 08:28:15 +00:00
kevinveenbirkenbach d287e55964 Merge remote-tracking branch 'origin/main' into fix/code-scanning-alerts/807 2026-10-07 01:51:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 45d956e5ea fix(sidecars): cap an engine's sidecar at its provider's size
A Redis sidecar whose consumer declares no services.redis.mem_limit
runs in a container that inherits the consumer's primary entity limit.
Since the lookup replaced the old filter, maxmemory followed that limit
and rose from 204 MiB to between 819 and 3276 MiB in 17 roles.

lookup('service_memory_mb', ..., provider=<role>) caps the result at
the same share of the provider's own limit when the application does
not declare one for that service. A declared limit is not capped, and a
container smaller than the provider keeps its own share. The Redis
sidecar passes provider='svc-db-redis': those 17 roles get 409 MiB, the
seven with a 256 MiB container stay at 204.

The lookup now rejects unknown options and a provider that is no role
or declares no limit for the service, so a misspelled option fails
instead of silently lifting the cap.

The memcached sidecar's literal -m 64 takes the same lookup and keeps
its value.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 01:30:01 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e203d6f7ab feat(plugins): size memory parameters from the container limit with one lookup
lookup('service_memory_mb', application_id, service_name, share) returns
a share of a service's mem_limit in whole MiB. It resolves the limit the
way sys-svc-container resolves the container's own: the named service,
then the role's primary entity, then RESOURCE_MEM_LIMIT.

It replaces the three filters of memory_filters, each with its own unit
parser and its own idea of the limit: Redis maxmemory at 0.8 of the
limit, and the Confluence and Jira heap at 0.7 and 0.35.

The central Redis (409 MiB), Confluence (5734/2867) and Jira (4300/2150)
keep their values. A Redis sidecar without a mem_limit of its own does
not: the filter fell back to a fixed 256m, which gave 204 MiB, while the
lookup takes the limit that container really runs with, the primary
entity's. That moves 25 roles to between 409 and 3276 MiB.

The heap clamps of the old filters are gone: the 1024 MiB floor, limit
minus 1024, the 12288 cap and Xms bounded by mem_reservation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 23:05:26 +02:00
kevinveenbirkenbachandClaude Opus 5.5 3610e7b9ef fix(code-scanning): resolve the alerts that describe real code
Of the 76 open code scanning alerts, 39 point at code that can change.

svc-ai-mcp-adapter:
- A path argument that is empty or made only of dots is refused.
  Percent-encoding keeps a separator inside its segment, but "." and ".."
  need none, so an upstream that resolves dot segments could answer from a
  path the contract does not name.
- parse_duration splits on the units. The pattern it used before rescanned
  a long run of digits from every start position.

Tests and the README generators:
- Every Jinja environment built with autoescape=False and a "noqa: S701"
  uses select_autoescape now, as six test modules and
  utils/templating/jinja.py already did. It escapes templates named
  *.html, *.htm and *.xml, which none of these call sites loads.
- Dead code goes: a doubled "rel" block in the email lint, an unread
  _SKIP_DIRS, and a module that _resolve returned to no caller.
- Two "except ...: pass" blocks become contextlib.suppress and
  assertRaises.
- Nine test modules import unittest.mock one way instead of two, and one
  lookup test imports its module once.

Closes #807

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 09:18:00 +02:00
Prageeth Panicker bf37d5e7b3 fix(web-app-wazuh): address PR review feedback
Tor: use the shared tor block (bond 0.5, enabled when svc-net-tor is in the node's groups, tor pinned on in both variants) so a node serves one network, onion on a Tor node and clearnet otherwise.

SPOT and sizing: WAZUH_INDEXER_URL replaces four hand-built copies of the indexer URL, and the indexer heap is half of mem_limit instead of a hardcoded 1g (unchanged for the current 2g limit).

Fixes: bcrypt_hash rejects secrets over 72 bytes; the Keycloak group-removal helper uses the same resolver as the add helper and checks every response (the realm is no longer URL-encoded, matching the add helper), and a failed cleanup fails an otherwise passing test; the swarm certs generator removes a stale container before each attempt; vm.max_map_count is set before the early handler flush; securityadmin failed_when now needs rc 0 and the success marker.

The regenerated role README also picks up generator drift (mailu to stalwart).
2026-10-06 05:15:19 +00:00
kevinveenbirkenbachandClaude Opus 5.5 e6fbb4b3f4 chore: drop two leftovers of the Wazuh role's first commit
That commit added test_bcrypt_hash.py with an Exception comment and left a
trailing blank line in group_vars/all/11_oidc.yml. Both go.

What the comment said still holds: the test hashes with the raw bcrypt package
because Ansible's password_hash('bcrypt') routes through passlib, whose bcrypt
self-test can misreport a password well under 72 bytes as too long when
passlib and the installed bcrypt disagree on the version.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 05:15:17 +00:00
Prageeth Panicker 87c8199012 feat(web-app-wazuh): add Wazuh SIEM role with OIDC/LDAP RBAC
Add single-node Wazuh stack (manager, indexer, dashboard) as a new
Ansible role, integrated with the platform's Keycloak realm via native
OIDC and secondary LDAP authorization, with three-tier RBAC.

- Pins wazuh/wazuh-{manager,indexer,dashboard}:4.14.6 and
  wazuh-certs-generator:0.0.4 (one-shot cert bootstrap job)
- No external database: wazuh.indexer (OpenSearch) is self-contained
- Auth: native OIDC against Keycloak, working end-to-end after two
  infra fixes (JVM cacerts trust for the indexer's outbound calls to
  Keycloak, front-proxy vhost wiring)
- LDAP wired as a secondary, authentication-independent authorization
  backend (group-based backend_roles enrichment only)
- RBAC: three roles (administrator, security-analyst, readonly-auditor)
  via the platform's standard meta/rbac.yml + rbac_group_path pipeline
- Documents a genuine upstream Wazuh bug: wz-home as the default
  landing route throws an uncaught client exception for any
  non-all_access role; default route moved to OpenSearch Dashboards'
  own /app/home

Addressed two full rounds of review feedback (Kevin, Alejandro):
WEB_PROTOCOL usage, service-account naming, comment cleanup, shared
Keycloak-admin-helper extraction, a real TOCTOU race in RBAC test
setup, a Mermaid auth-flow diagram, meta/services.yml placement for
the certs-generator, CA-trust-path SPOT usage, CSP tightening
(including a genuine worker-src: blob: requirement found via live
testing), and a loud (non-silent) dev-bucket fallback warning.

Rebased onto main after a 3-month gap; brought the role in line with
~30 platform-convention changes introduced in that window (swarm
compose-template rules, Tor/onion contract, Playwright timeout/goto
wrappers, file layout, task numbering, comment policy, and more).

Live-deploy verification (9 iterations) found and fixed 6 further
real bugs invisible to lint/unit tests alone:
- credentials.* / meta/schema.yml -> secrets.credentials.* /
  meta/secrets.yml (another platform-drift casualty)
- notify: "compose up" -> "compose-up" (day-one handler-name typo)
- local.subnet collision with web-app-mailu's /24
- missing primary: false alongside tor.exclusive: false, which let
  the onion domain shadow the clearnet vhost entirely
- a second front-proxy include for the onion domain, later reverted
  (see below)
- shared sys-stk-front-proxy/sys-front-tls TLS-lookup keying bug
  (app-id-keyed vs. domain-keyed), which silently applied TLS to
  Wazuh's onion vhost

The last two are related: enabling genuine dual-stack (clearnet +
onion) reachability for tor.exclusive: false surfaced a real bug in
two shared platform roles. That fix was verified safe against all 103
apps currently in the repo, but is deliberately NOT included here -
shared-role changes deserve their own PR and review, not to ride in
on a single app's feature branch. Wazuh's second front-proxy include
was reverted accordingly; the resulting onion-reachability gap is
documented in the README as a known, platform-scoped limitation.

make test: 5508 passed, 3 failed (all three trace to untracked local
scratch files - Dockerfile.bak, scripts/lab-*.sh - not present in any
CI checkout).
2026-10-06 05:15:00 +00:00
kevinveenbirkenbachandClaude Opus 5.5 52fe3c9c2c style: apply make autoformat to the merged Stalwart and Tor files
Formatter output only: the parsed YAML and the Python AST of every touched file equal their previous versions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 19:57:30 +02:00
kevinveenbirkenbach d90f1e8ed6 Merge pull request #756 from kevinveenbirkenbach/fix/svc-db-postgres/pgdg-bullseye-archived
fix(svc-db-postgres): build on postgres 18 since PGDG archived bullseye
2026-10-02 17:41:28 +02:00
kevinveenbirkenbach 30417d2515 fix(svc-db-postgres): build on the official multi-arch postgres 18 image
postgis/postgis publishes amd64 only, so every postgres consumer failed
on arm64. The official postgres image serves both architectures and
already ships the contrib extensions bloom, pg_trgm and unaccent.

PostGIS moves into the postgres_libraries registry and is installed from
PGDG (postgresql-18-postgis-3) only for a consumer that declares it as
an extension, which is mobilizon alone. apt update reads every source
list again, because the PostGIS package depends on GDAL from Debian
main, which a PGDG-only update left without an install candidate.

Postgres 18 keeps its cluster in /var/lib/postgresql/18/docker and its
entrypoint treats a volume at the former /var/lib/postgresql/data as an
old database, so the central and the dedicated data volume both mount at
/var/lib/postgresql. Existing 17 clusters are not migrated: their data
directory trips the entrypoint's old-data check and the container
refuses to start.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f78ebe46a4)
2026-10-01 22:19:09 +02:00
Alejandro Roman b160d07008 fix(lint): annotate ClassVar in test_email and reorder pixelfed playwright hooks 2026-09-26 19:27:31 +04:00
Alejandro Roman fadf36e0a6 fix(swarm-stalwart): fix CI issues 2026-09-26 18:05:56 +04:00
kevinveenbirkenbachandClaude Opus 5.5 b77d7cbedb chore: drop two leftovers of the Wazuh role's first commit
That commit added test_bcrypt_hash.py with an Exception comment and left a
trailing blank line in group_vars/all/11_oidc.yml. Both go.

What the comment said still holds: the test hashes with the raw bcrypt package
because Ansible's password_hash('bcrypt') routes through passlib, whose bcrypt
self-test can misreport a password well under 72 bytes as too long when
passlib and the installed bcrypt disagree on the version.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 22:59:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e2beab9f1c test(domains): read the primary domain from default.env in every test
The hardcoded-primary-domain lint exempted tests/ and scripts/tests/, so test
fixtures kept their own copy of the domain and would drift from
INFINITO_DOMAIN the moment it changes. The lint now scans both trees, and a
unit test pins its scope: only default.env, Markdown, inventories/, .claude/
and build/ may name the domain.

Python tests take the value from default_domain_primary(), and the workspace
helper builds its dashboard and Matomo URLs from INFINITO_DOMAIN once the repo
.env is loaded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 22:53:54 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e8cb94e246 test(csp): a role without Tor keeps clearnet sources on a Tor node
build_csp_header moves sources to onion only when the vhost, or without one
the app's first domain, is on tor. The new case gives an app only a clearnet
domain on a node that runs Tor and renders its header without a vhost: frame-src
and connect-src stay clearnet and the header names no onion host.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 22:49:05 +02:00
kevinveenbirkenbachandClaude Opus 5.5 66c8df6729 feat(network): serve roles on clearnet and onion from one registry
A node runs in NETWORK_MODE clearnet, tor or multi; empty derives multi when svc-net-tor is deployed. meta/networks.yml reachability narrows a role (modes, single_mode, each justified by a network-reachability nocheck), and utils/networks/reachability.py is the one registry every network decision goes through. services.tor keeps only the dependency bond; exclusive and primary are gone and Matrix is single_mode.

Every canonical domain gains one sibling per effective network, clearnet first. sys-stk-front-proxy renders one vhost per sibling, and TLS, CSP, injected snippets, redirects, the web health probe and the CSP crawler are decided per vhost. Cross-app URLs follow the network of the vhost being rendered, the SSO issuer stays single, clearnet vhosts of multi roles send Onion-Location, and the shared location forwards X-Forwarded-Host, which lets Wazuh derive its OIDC redirect per request again. The to_onion_url filter is removed; its two consumers render per vhost instead.

Measured before commit: on a clearnet node the domain map of all 108 apps and every CSP header are identical to HEAD; on a multi node every role keeps its clearnet domains first and only gains onion siblings, and no CSP source crosses networks except the SSO issuer.

Refs: docs/requirements/037-multi-network-serving.md

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-24 19:42:05 +02:00
Prageeth Panicker 0e5288d2c5 feat(web-app-wazuh): add Wazuh SIEM role with OIDC/LDAP RBAC
Add single-node Wazuh stack (manager, indexer, dashboard) as a new
Ansible role, integrated with the platform's Keycloak realm via native
OIDC and secondary LDAP authorization, with three-tier RBAC.

- Pins wazuh/wazuh-{manager,indexer,dashboard}:4.14.6 and
  wazuh-certs-generator:0.0.4 (one-shot cert bootstrap job)
- No external database: wazuh.indexer (OpenSearch) is self-contained
- Auth: native OIDC against Keycloak, working end-to-end after two
  infra fixes (JVM cacerts trust for the indexer's outbound calls to
  Keycloak, front-proxy vhost wiring)
- LDAP wired as a secondary, authentication-independent authorization
  backend (group-based backend_roles enrichment only)
- RBAC: three roles (administrator, security-analyst, readonly-auditor)
  via the platform's standard meta/rbac.yml + rbac_group_path pipeline
- Documents a genuine upstream Wazuh bug: wz-home as the default
  landing route throws an uncaught client exception for any
  non-all_access role; default route moved to OpenSearch Dashboards'
  own /app/home

Addressed two full rounds of review feedback (Kevin, Alejandro):
WEB_PROTOCOL usage, service-account naming, comment cleanup, shared
Keycloak-admin-helper extraction, a real TOCTOU race in RBAC test
setup, a Mermaid auth-flow diagram, meta/services.yml placement for
the certs-generator, CA-trust-path SPOT usage, CSP tightening
(including a genuine worker-src: blob: requirement found via live
testing), and a loud (non-silent) dev-bucket fallback warning.

Rebased onto main after a 3-month gap; brought the role in line with
~30 platform-convention changes introduced in that window (swarm
compose-template rules, Tor/onion contract, Playwright timeout/goto
wrappers, file layout, task numbering, comment policy, and more).

Live-deploy verification (9 iterations) found and fixed 6 further
real bugs invisible to lint/unit tests alone:
- credentials.* / meta/schema.yml -> secrets.credentials.* /
  meta/secrets.yml (another platform-drift casualty)
- notify: "compose up" -> "compose-up" (day-one handler-name typo)
- local.subnet collision with web-app-mailu's /24
- missing primary: false alongside tor.exclusive: false, which let
  the onion domain shadow the clearnet vhost entirely
- a second front-proxy include for the onion domain, later reverted
  (see below)
- shared sys-stk-front-proxy/sys-front-tls TLS-lookup keying bug
  (app-id-keyed vs. domain-keyed), which silently applied TLS to
  Wazuh's onion vhost

The last two are related: enabling genuine dual-stack (clearnet +
onion) reachability for tor.exclusive: false surfaced a real bug in
two shared platform roles. That fix was verified safe against all 103
apps currently in the repo, but is deliberately NOT included here -
shared-role changes deserve their own PR and review, not to ride in
on a single app's feature branch. Wazuh's second front-proxy include
was reverted accordingly; the resulting onion-reachability gap is
documented in the README as a known, platform-scoped limitation.

make test: 5508 passed, 3 failed (all three trace to untracked local
scratch files - Dockerfile.bak, scripts/lab-*.sh - not present in any
CI checkout).
2026-09-23 20:55:25 +00:00
Alejandro Roman 866775924e fix(swarm-stalwart): Tor port conflicts and NFS localhost vs host issues 2026-09-23 13:30:07 +04:00
Alejandro Roman c8894b2dd2 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-22 22:33:13 +04:00
evangelostsak 603233f894 test(prometheus): pin that an unscraped role contributes no alert rules
Stubs native_metrics_apps and asserts the intersection, including the case the
refactor exists for: a role that ships a fragment and is deployed but is not
scraped must not reach the rule file.
2026-09-21 17:45:26 +02:00
Alejandro Roman 4b9a834546 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-21 03:25:20 +04:00