Commit Graph
112 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 eb7056dafd fix(storage): keep the NFS client off the ports the platform publishes
Main CI run 40976399687, socialhome#0 (swarm, tor, centos): the CLI tests
of svc-net-tor (onion:25 "socks CONNECT failed with reply 0x01") and
web-app-stalwart (probe_mail relay: ConnectionRefusedError on port 25)
failed. dockerd on the manager had aborted the whole ingress program of
the mail stack:

  Failed to add ingress: failed to program ingress ports:
  failed to bind host port 0.0.0.0:993/tcp: address already in use

/proc/net/tcp named the squatter: 192.168.244.10:993 -> :2049, the
node's own NFS client. The mount carries no noresvport, so the kernel
drew a source port from the 665-1023 reserved window and landed on
imaps. None of 25/465/993/995/4190/8095 got a DNAT rule or a
docker-proxy listener while every other published port did, and
stalwart still reported healthy because its healthcheck runs inside the
container. 993 and 995 are the only published ports inside that window,
so this is luck of the draw on any NFS-backed swarm host that runs the
mail role.

mount_opts now emits noresvport. insecure on the kernel-flavor export is
the other half: kernel nfsd defaults to secure and answers a mount from
an unprivileged source port with EACCES. It reaches the kernel flavor
only; ganesha already accepts unprivileged clients.

What it costs: the next deploy of every existing NFS client sees a
pending opts change and remounts the shared state; a host-side job
holding that path at that instant fails loudly. A plain remount may keep
a running mount on its old reserved port until the next real mount.

Port of f46c4d0bab from feature/store-i18n, applied identically so the
later store-i18n -> main merge stays conflict-free.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 23:02:41 +01:00
kevinveenbirkenbach 989eb7c0ea Merge pull request #1256 from infinito-nexus/fix/sys-svc-proxy/per-app-security-headers/1123
feat(proxy): per-app nosniff, Referrer-Policy, Permissions-Policy and HSTS directives
2026-10-29 18:14:28 +01:00
kevinveenbirkenbachandClaude Opus 5.5 d721e887a5 fix(web-app-dashboard): run inline scripts by hash instead of 'unsafe-inline'
The dashboard sent script-src 'unsafe-inline' from two sources. Its own
meta/csp.yml granted it to script-src-attr, and build_csp_header adds it
to script-src-elem and script-src-attr for every app with
services.logout.enabled. Both reach script-src through the family union,
and the token in script-src-elem also closes the hash gate, so the
hashes that sys-front-inj-javascript, -dashboard, -logout and -matomo
register for their inline snippets were discarded.

build_csp_header now skips the logout grant on a directive whose
meta/csp.yml sets unsafe-inline: false. Only web-app-dashboard sets it:
on script-src and script-src-elem. Its injected snippets now run by hash
and the logout one-liner is hashed like the others. A render of
build_csp_header for every role, variant, logout on/off and tor on/off
(1712 headers) differs from origin/main only in the 8 web-app-dashboard
headers; the 86 other roles that declare a logout service render
byte-identical policies.

script-src-attr keeps 'unsafe-inline': port-ui 2.2.2 still renders
inline event handlers (menu onclick entries, openDynamicPopup,
closeAllModals, the card image onerror). It goes once the port-ui
release without inline handlers is pinned and header.yml.j2 moves from
onclick to action.

The dashboard Playwright spec asserts that script-src and script-src-elem
carry no 'unsafe-inline' on the landing page, next to its existing check
for CSP violations.

Refs #1122

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 00:27:49 +01:00
kevinveenbirkenbachandClaude Opus 5.5 1f8fc336e6 feat(proxy): per-app nosniff, Referrer-Policy, Permissions-Policy and HSTS directives
#1115 raised HSTS to one year but set no X-Content-Type-Options,
Referrer-Policy, Permissions-Policy or HSTS includeSubDomains/preload,
because none of them is safe globally: nosniff lets ORB block no-cors
responses typed text/html, includeSubDomains on the apex covers hosts
the platform does not serve, and a server-level add_header is dropped
in every location that sets its own.

WEBSERVER_SECURITY_HEADERS (group_vars/all/04_webserver.yml) now holds
the platform values, all opt-in: nosniff false, no Referrer-Policy, no
Permissions-Policy, HSTS max-age one year without includeSubDomains or
preload. A role overrides any key under security_headers in its
meta/server.yml, an inventory under
applications.<role>.server.security_headers.

The new security_headers filter merges both, rejects unknown keys,
wrong types, unknown Referrer-Policy tokens, non-printable header values
and preload without includeSubDomains or below a one-year max-age, and
composes the HSTS value.

- nosniff, Referrer-Policy and Permissions-Policy are set in the Lua
  header filter of content_security_policy.conf.j2, next to CSP, so a
  location's own add_header cannot drop them. Referrer-Policy is only
  set when the upstream sends none; Permissions-Policy only on HTML
  documents.
- HSTS stays the single server-level add_header in nginx_ssl_header.j2,
  now with the per-app value. A second emitter in Lua would duplicate
  the header. It stays inside the TLS branch, which onion vhosts never
  enter, so an onion vhost gets no HSTS even when its app opts in.

No role opts in yet, so rendered vhosts are unchanged.

Closes #1123

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 21:38:38 +01:00
kevinveenbirkenbach b8846d9942 Merge branch 'main' into fix/web-app-seaweedfs/anonymous-deny 2026-10-27 17:45:23 +01:00
kevinveenbirkenbachandClaude Opus 5.5 d18feb9dae fix(web-app-seaweedfs): converge the anonymous-read deny on every provider deploy
Follow-up to the bucket-policy deny, closing holes two reviews found:

- Scope: the policy was only applied during a consumer's own deploy, so a
  provider-only or partial deploy left other buckets without it. Every
  provider deploy now converges all existing consumer buckets, and a
  consumer's provisioning applies it right after creating the bucket.
- Ownership: our statement is identified by Sid DenyAnonymousRead; a
  public bucket loses only that statement, operator statements survive.
- Errors: only a 404 NoSuchBucketPolicy means "no policy"; any other
  status fails the read loudly.
- Credentials: curl expands the admin credential itself via --variable
  instead of --user "$VAR", which put it on curl's argv and in the
  host's ps.
- Curl runs through the repository's curl filter with timeouts, the
  read-only tasks run in check mode, and bucket names are parsed exactly
  instead of by substring.
- README: corrected embedded-mode authentication and documented the
  convergence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 17:45:22 +01:00
kevinveenbirkenbachandClaude Opus 5.5 3b4ac40df5 fix(web-app-seaweedfs): deny unsigned reads on non-public consumer buckets
SeaweedFS 4.49+ honours a public-read object or bucket ACL on an unsigned
GET/HEAD/list even in a bucket the platform treats as private
(seaweedfs/seaweedfs#11605). Since consumers hold WriteAcp, an app could
expose objects through the public S3 API domain just by sending
x-amz-acl: public-read.

Put a bucket policy on every non-public consumer bucket that denies
s3:Get* and s3:List* to principal "*" when aws:username is null. SeaweedFS
evaluates that deny before any AllUsers grant; signed and presigned
requests carry an IAM user ARN, so aws:username is set and they are not
affected. Public buckets keep their anonymous Read grant, and a bucket
that turns public gets our deny lifted again. Embedded sidecars publish
no port and are left as they are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 13:46:17 +01:00
kevinveenbirkenbachandClaude Opus 5.5 ec7d4c9042 fix(web-app-seaweedfs): grant consumers WriteAcp on their bucket
SeaweedFS 4.49 (seaweedfs/seaweedfs#11592) authorizes a PutObject that
carries x-amz-acl as s3:PutObjectAcl as well. The consumer identities
only held Read/Write/List/Tagging, so every app that sends an ACL on
upload (Nextcloud, Fider and Pixelfed with private, Mastodon, PeerTube,
BookWyrm and Baserow with public-read) got AccessDenied after the 4.48 -> 4.51
bump in #1226. Main run 40401200083 failed Fider#0, BookWyrm#0,
Baserow#0, Pixelfed#0 and Nextcloud#0/#4/#8 on it.

Add WriteAcp to the s3.json identities and to the sidecar s3.configure
grant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 09:28:11 +01:00
kevinveenbirkenbach eeb3305e52 Merge remote-tracking branch 'origin/main' into fix/security/swarm-ingress-guard/1089 2026-10-22 19:17:39 +02:00
kevinveenbirkenbachandClaude Opus 5.5 b83b361ea9 fix(web-app-bigbluebutton): run BBB v3.0.39 with the shared-notes server
The version-source update in #1146 moved the bigbluebutton/docker ref from
v3.0.23 to v3.0.39. That tag changes three things the role did not
handle, and it exposed a fourth defect in compose_file_args.

1. generate-compose renders etherpad and bbb-pads only behind
   USE_INSECURE_ETHERPAD and otherwise emits bbb-shared-notes-server with
   a `build:` block and no `image:` (upstream adds the image only after
   the tag, in 9a6a9a9a4c). compose_mods drops `build` only where an image
   is set, so the guard "Fail if any service still builds locally" stopped
   every BigBlueButton deploy (main run 39749899667, Nextcloud#11:
   `rc: 0`, `stdout: '    build:'`, bootstrap_compose_project.yml:35).
   meta/services.yml now pins alangecker/bbb-shared-notes-server:v3.0.39,
   compose_mods takes `pinned` (BBB_PINNED_SERVICES, every service the
   role config pins an image for) and drops their `build`, and the
   etherpad leftovers go (ep_redis_publisher patch, etherpad postgres
   entrypoint and initdb hook, etherpad database tasks, ETHERPAD_* env,
   etherpad_api_key secret).

2. The notes server connects to postgres once at start and exits with 2
   when it is not up yet; upstream gives it no restart policy. The
   override sets the role's restart policy, so it retries until postgres
   accepts connections.

3. Greenlight moves from v3.8.0 to v3.9.0.1, which sets
   `config.hosts = URL_HOST`, and the template passes
   `URL_HOST: https://${DOMAIN}`. Rails compares hostnames, so every
   request was blocked (`Blocked hosts: bbb.talk.infinito.test`, 403).
   Greenlight documents URL_HOST as the bare hostname; the override sets
   it to the role domain.

4. compose_file_args decided whether a role ships a compose override by
   templating "{{ application_id | abs_role_path_by_application_id }}"
   through the templar. ansible-core 2.21 does not render an untrusted
   string, so the check saw the literal and always returned False: the CA
   override generation and the pull handler never read BBB's
   compose.override.yml. That was harmless while compose.yml alone was
   valid; with the notes-server image living in the override, `compose
   config` failed ("has neither an image nor a build context"). The check
   now resolves the role path for the application id it is given. BBB is
   the only role that ships an override, so no other role's file list
   changes.

Existing installations drop the etherpad and bbb-pads containers on the
next `compose up --remove-orphans`; the old `etherpad` database stays in
PostgreSQL untouched.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 16:56:48 +02:00
kevinveenbirkenbachandClaude Opus 5.5 788eabffbf fix(svc-swarm-node): drop outside traffic to non-public routing-mesh ports
In swarm mode `docker stack deploy` ignores the host IP of a short port
mapping, so every `127.0.0.1:<port>:<port>` that container_ports renders for
a `ports.local` entry is published by the ingress routing mesh on every
node, via docker-proxy on 0.0.0.0 and [::] plus DNAT in nat PREROUTING.
Databases, LDAP and application backends answered from outside the cluster.

svc-swarm-node now loads an nftables table through svc-net-firewall. Its
prerouting filter chain (priority mangle, ahead of Docker's dstnat) drops
packets addressed to the node for any ingress-published port that no
application declares under `ports.public`. Loopback, docker0,
docker_gwbridge, the user bridges and the swarm nodes' own addresses stay
admitted. Host-side provisioning over 127.0.0.1, containers that reach a
host port through host-gateway, and node-to-node consumers such as the
registry-cache mirror keep working.

Both sets are read live on the manager (`docker service inspect` for
`.Endpoint.Ports`, `docker node inspect` for `.Status.Addr`), so ephemeral
ingress ports and nodes outside `--limit` are covered. The table is loaded
on a node before it joins, reloaded on every node after the join, and
reloaded after each stack deploy by the sys-svc-compose swarm handler.

Refs #1089

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 20:56:29 +02:00
kevinveenbirkenbachandClaude Opus 5.5 6e776a5780 refactor(python): clear the open CodeQL code-quality alerts
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
  is imported by its modules (163 packages), so the constant counts as
  exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
  imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
  re-exports included, instead of silencing them with `noqa: F401`; the
  list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
  rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
  BMP-only positive class; it matches exactly the same 2111 code points
  as the old negated class across U+0000..U+10FFFF.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 21:40:58 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d48f8e9897 fix(bcrypt_hash): derive the seeded salt from the seed alone
The seeded salt was the first 16 bytes of sha256(seed + secret). bcrypt
stores its salt in clear inside the hash, so every seeded hash published
a fast sha256 digest of the password under a seed that is public in the
repository (`web-app-wazuh:indexer-admin`). Anyone holding the hash could
test password guesses at sha256 speed instead of paying bcrypt's cost.

The salt is now sha256(seed)[:16]. The hash stays deterministic for one
password, still changes when the password rotates, and its salt carries
nothing computed from the secret. Existing Wazuh deployments get a new
hash once; `securityadmin.sh` re-applies `internal_users.yml` on every
deploy and the hash still verifies the unchanged password.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 21:35:32 +02:00
kevinveenbirkenbach d0698daa5e Merge pull request #414 from pragepani/feature/web-app-wazuh/add-role
feat(web-app-wazuh): add Wazuh SIEM role with OIDC/LDAP RBAC
2026-10-19 14:35:24 +02:00
kevinveenbirkenbachandClaude Opus 5.5 474849a39c fix(svc-bkp-volume-2-local): read the baudolo version from pyproject.toml
Two bots bumped the one baudolo version independently. The pip pin
updater rewrites the literal package_name pin in
roles/svc-bkp-volume-2-local/tasks/01_install.yml and opens a PR scoped
to roles/<entry>/, and dependabot bumps the same package in
pyproject.toml. Each PR alone splits the pins that
tests/lint/repository/dependencies/test_baudolo_pin.py holds equal; #995
merged such a split and turned main's test-lint red until #996, and the
next upstream release (7.1.2 is tagged) would repeat it.

The version lookup now takes an optional distribution name and returns
its exact == pin from [project.dependencies]. The install task asks it
for backup-docker-to-local, so pyproject.toml is the only place the
version lives, dependabot is the only bot that moves it, and the pip
updater no longer finds a literal to bump. The lint now forbids a
literal pin in the task and requires the lookup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-18 21:44:40 +02:00
evangelostsak e62ed8c56f Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-17 09:34:00 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d36e203171 fix(cache): render the applications carrier without the caller's application_id
`lookup('tls', ...)` aligns a cross-app domain to the consumer named by
`application_id` (utils/tls_common.py `align_domain_to_consumer`): a
clearnet-primary consumer gets the clearnet sibling of an onion-primary
target. The carrier key does not cover `application_id`, so a park that
renders inside a role hands that role's alignment to every later role
whose key matches.

The constructor and `tasks/utils/setup/role.yml` park at play scope,
where `application_id` is undefined. The re-park in
`roles/sys-token-store/tasks/write.yml` (67f6c8d7e8) and the nested-load
re-park render inside the caller: `svc-prx-openresty` at the preload
loader, `web-app-stalwart` or a provider role on the NFS host, the web
app itself under `sys-stk-front-base`. In a tor deploy that turned onion
targets into clearnet domains for onion-primary roles served from such a
park, and baked the caller's group into Nextcloud's sociallogin
`groupMapping`.

Drop `application_id` from the render's variables and from the
templar's for the duration of the render, so every park carries the
play-scope result the constructor parks have always carried. Rename the
nested re-park task so its log lines do not read as a role boundary.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 21:25:43 +02:00
evangelostsak 6f24461720 refactor: drop rationale and narration from tests/unit/python/plugins/lookup/test_mesh_ranges.py 2026-10-16 19:10:01 +02:00
evangelostsak d480c2785e refactor: drop rationale and narration from tests/unit/python/plugins/lookup/test_mesh_address.py 2026-10-16 19:10:01 +02:00
Prageeth Panicker 5e39433ed1 Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role 2026-10-16 11:40:30 +00:00
kevinveenbirkenbachandClaude Opus 5.5 67f6c8d7e8 fix(cache): keep the applications carrier valid across token store writes
Since 778c18fb23 the carrier key holds the tokens file's mtime and size.
sys-token-store changed that stat on every inclusion (00_core.yml
touched the file) and on every persisted token, and nothing re-parked a
carrier afterwards. Every later task's fresh worker then rendered the
applications SPOT cold, 30 to 80 s each. On main fea6f31d9f (run
38812964308) seven heavy jobs logged 194 to 287 such tasks and ran out
of the 20400 s budget; the same jobs before 778c18fb23 logged 65 to 83,
all of them real waits.

- 00_core.yml touches the store with modification_time and access_time
  preserved, so it only creates a missing file. In swarm the controller
  sees the store over NFS with attribute caching, and the touch alone
  made the second pass go cold without any write.
- write.yml re-parks the carrier when the persist or the controller
  mirror changed the store. Both run once, so every host that ran the
  include gets the same verdict.
- tasks/utils/setup/role.yml re-parks after every role on every host,
  so hosts that did not run the writing role recover too. Where the
  key still matches this serves the carried payload; measured 0.1 s per
  role in a local compose deploy.
- The tokens file enters the key as the SHA-256 of its content,
  memoised per inode, mtime and size, so a touch no longer changes the
  key. A stat that still matches serves the memo (1.67 us against
  1.41 us for the old stat signature), and a stale stat keeps key and
  YAML cache on the same old content instead of pairing new content
  with an old key.
- applications_carrier opens the tokens file before it computes the
  key, so on the NFS-mounted controller the parked key reflects the
  revalidated attributes rather than a cached stat that changes a
  minute later.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 11:11:47 +02:00
Prageeth Panicker 8e4ba0f8a9 Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role 2026-10-15 21:58:47 +00:00
evangelostsak 1c5ba164f2 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard
No conflicts across 121 incoming commits. The one file both sides changed is
tasks/stages/01_constructor.yml: upstream parks an applications carrier for
the swarm bootstrap roles after the add_host tasks, and keys the carrier on
the tokens file as well. This branch's mesh bootstrap opens and drops its own
block-scoped carrier before that point, so both keep working as written.

Upstream also brings ruff 0.17.0, distro-derived env keys, a scoped act outer
teardown, the stalwart relay rule, the onion apex redirect and image and
version updates; ruff 0.17 reformats nothing on this branch.
2026-10-15 21:27:11 +02:00
kevinveenbirkenbach 06a74e7658 Merge pull request #869 from infinito-nexus/fix/web-app-stalwart/relay-rule/793
fix(web-app-stalwart): authenticate bots with an app password under SSO
2026-10-15 15:42:19 +02:00
evangelostsak 01e8c1a75f test(lookup): mesh_address returns only the host's own primary address 2026-10-14 15:03:41 +02:00
kevinveenbirkenbachandClaude Opus 5.5 45732329b6 fix(lookup): keep the onion apex from redirecting to a clearnet homepage
On a Tor node current_play_redirect_domains schedules the bare node
onion as a redirect source targeting DOMAIN_HOMEPAGE. Without
web-app-dashboard, DOMAIN_HOMEPAGE falls back to SOFTWARE_DOMAIN
(infinito.nexus), so web-opt-rdr-domains rendered a 301 from
http://<node>.onion/ to http://infinito.nexus/: an onion visitor sent
to a clearnet host over plain HTTP.

The onion apex entry is now added only when DOMAIN_HOMEPAGE is an
onion host. Otherwise the onion apex gets no redirect and the default
server answers.

Closes #924

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 13:35:29 +02:00
evangelostsak 762513ec32 test(lookup): hold mesh_ranges to a rendered pool
The declared template must come back rendered, and one that stays a template
must fail rather than reach the export list.
2026-10-14 11:23:23 +02:00
Prageeth Panicker 06f0b42f12 Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role 2026-10-14 09:00:39 +00:00
evangelostsak 75fd3e9433 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-13 01:08:20 +02:00
kevinveenbirkenbachandClaude Opus 5.5 24d7872fde Merge remote-tracking branch 'origin/main' into HEAD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:54:31 +02:00
kevinveenbirkenbachandClaude Opus 5.5 eda6249d7b fix(sys-svc-compose): probe provider networks by their exact name
The network probe used community.docker.docker_network_info, whose
get_network() scans GET /networks and returns the first entry whose Name
equals the requested name OR whose Id starts with it, in a list order that
changes between calls. In main run 38038541631 (Zammad#1, swarm) the
central redis overlay got the ID n8nwzb1q39m3u4k7awwwmt6yg, the probe for
the not yet created provider network "n8n" returned it as existing, the
create step was skipped, docker stack deploy attached hermes to the redis
overlay as "n8n", and the oauth2-proxy then reached the password-protected
central redis by its bare host name: "NOAUTH Authentication required".
The same scan can also return the ID-prefix twin of a network that does
exist, so comparing the returned name alone would turn that network into
"missing" and fail the create with "already exists".

Both probes now run `container network ls --filter name=<entity> --format
'{{json .}}'`, whose name filter never matches an ID. The new lookup
network_driver picks the entry named exactly after the entity and returns
its driver, '' when there is none; the drift checks, their messages and the
create steps all read that one entry. No docker module is left in the file,
so its two docker-module-uses-container-cli suppressions go too.

Verified: on the host daemon GET /networks/<id prefix> returns a network of
another name and docker network ls --filter name=<id prefix> returns
nothing. In ansible-playbook against real docker network ls output the
lookup returned the exact network's driver in 20 of 20 probes beside an
ID-prefix twin, an empty driver (create path) for a missing network and for
a name only contained in a sibling, and the drift path for a foreign
driver. Unit tests cover the same cases; with a substring match instead of
the exact name the two sibling cases fail.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:35:22 +02:00
evangelostsak 5a83785b53 Merge remote-tracking branch 'origin/main' into feature/svc-net-wireguard 2026-10-09 12:00:07 +02:00
Prageeth Panicker f5b552a667 Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role 2026-10-09 08:34:37 +00:00
kevinveenbirkenbach 799b835799 Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/relay-rule/793 2026-10-09 00:53:26 +02:00
kevinveenbirkenbachandClaude Opus 5.5 64677d4790 chore: regenerate role docs and apply autoformat after #881 and #873
`make quality-high` rewrote these files on every branch: the SuiteCRM
README and the root role table lacked the Redis service #881 added, the
session path test was not formatted, and an OpenTalk secret description
from #873 was not wrapped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 22:27:12 +02:00
kevinveenbirkenbach 3874f66ac3 Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/relay-rule/793 2026-10-08 19:38:56 +02:00
kevinveenbirkenbach 6e9b4e23dc Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/relay-rule/793 2026-10-08 19:27:27 +02:00
kevinveenbirkenbach badc2583eb Merge remote-tracking branch 'origin/main' into fix/web-app-suitecrm/redis-sessions/879 2026-10-08 19:25:24 +02:00
kevinveenbirkenbachandClaude Opus 5.5 3cefb01d71 fix(web-app-suitecrm): keep sessions in Redis so replicas share them
With three swarm replicas the PHP session files on the shared NFS volume
were read as empty under concurrent requests of one browser, so SuiteCRM
answered 403 on /api/graphql and dropped the session. Sessions now live in
Redis with session locking that waits like flock did; the phpredis version
is pinned in services.yml with its own update source, and the engine lookup
renders the save path.

Refs #879

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 18:31:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 08ef66d8de fix(web-app-stalwart): authenticate bots with an app password under SSO
Under SSO the role tried to let applications relay without SMTP AUTH by
widening the relay rule to the private networks. The rule compared the
peer address to the text of a CIDR, which never matches, so an
application could reach local mailboxes but got "550 5.1.2 Relay not
allowed" for every external recipient. Repairing the comparison would
have opened a relay, because published ports, Tor and the swarm mesh all
hand outside clients an address inside those networks.

Stalwart checks app passwords before it asks the external directory, so
a bot can authenticate while the directory is OIDC. With SSO enabled the
role now creates one app password per bot, acting for that account as
the recovery admin, and stores it in the token store the SMTP password
is already read from. A bot gets a new one when its entry is deleted in
Stalwart or its token is removed.

The relay rule is back to authenticated sessions only. The lookup loses
its SSO relay branch and the role its submission_via_relay flag, so an
SSO deployment submits on 465 with AUTH like every other one.

The role's CLI test now fails when an unauthenticated session is given
an external recipient, or when the bot credential is refused one.

Closes #793

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 08:57:22 +02:00
kevinveenbirkenbach b3563f4ea3 Merge remote-tracking branch 'origin/main' into fix/web-app-prometheus/blackbox-targets/792 2026-10-08 08:57:13 +02:00
kevinveenbirkenbachandClaude Opus 5 50b9f60c43 fix(prometheus): resolve the native-metrics scrape target per deployment mode
`native_metrics_target` was a Jinja filter, so it could not read
DEPLOYMENT_MODE: filters see no Ansible variables. It returned
`<services.<key>.name>:<port>` unconditionally, which is `matrix-synapse:9000`
for matrix. Compose serves that name; swarm does not, because `docker stack
deploy` ignores `container_name` and the alias never exists. This repository
says as much itself, through `lookup('compose_only', 'container_name', ...)`.
The target was therefore scraped and refused, and `playwright.spec.js:274`
reported `web-app-matrix: 1 matching target(s) reporting up != 1`, meaning one
matching target whose `up` value is not 1.

It becomes a lookup that delegates host resolution to `scrape_target`, which
was already mode-aware, so exactly one plugin knows the difference between the
two modes. The port has to be handed over explicitly: matrix exposes metrics on
9000 while `services.synapse.ports.internal.http` is 8008, so the existing
internal-port lookup cannot serve this case on its own.

`scrape_target` now refuses `port` together with an explicit `port_kind`.
Before, `port` won silently and the error text still named the
`ports.internal.<port_kind>` path that the call had never consulted.

`native_metrics_apps` drops an app that pins itself to compose through
`compose_mode_force` while the cluster runs swarm, because the scrape target
would then name the wrong object. Only the literal case: matrix declares that
override as a Jinja expression over its flavour, which this lookup cannot
resolve from the scraping role's variable scope, so variant 0 on swarm stays a
latent mismatch.

The two comment lines claiming `container_name:port` went with the change;
after it they state the opposite of what the code does.

Verified by mutation: with the swarm branch of `scrape_target` disabled, the
new test emits `matrix-synapse:9000`, the exact string CI reported.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-08 08:57:13 +02:00
Prageeth Panicker 94aa75720a Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role
# Conflicts:
#	README.md
2026-10-07 12:55:54 +00:00
Prageeth Panicker d92de14192 fix(web-app-wazuh): use a stable salt for the Wazuh password hashes
bcrypt_hash takes an optional salt_seed. Without a seed it keeps the random salt. With a seed the salt is derived from the seed and the secret, so the same password always gives the same hash. Wazuh passes a fixed seed for the indexer admin and the dashboard service hashes, so internal_users.yml no longer changes on every run and no longer triggers a restart.
2026-10-07 08:28:15 +00:00
evangelostsak e9929d07c3 Merge remote-tracking branch 'origin/main' into feature/svc-net-wireguard
One conflict, in the generated role overview table of README.md.

Both sides had edited the same block. Upstream dropped Redis from the pihole
row; this branch still listed it. Upstream also listed svc-net-wireguard-core,
-plain and -firewalled, which this branch removed in e706170a0b as superseded
by the unified svc-net-wireguard role.

Resolved by taking upstream's block and then running `make readme-index`, so
the table is generated from the roles that actually exist rather than
hand-merged: Redis is gone from pihole, the three legacy WireGuard roles stay
gone, and svc-net-wireguard is listed once. `make readme-index check=true`
reports the table up to date.

Git kept the role deletions silently and correctly - upstream has not touched
those three directories since the merge base, so there was no modify/delete
conflict to resolve.
2026-10-07 10:04:31 +02:00
kevinveenbirkenbach d287e55964 Merge remote-tracking branch 'origin/main' into fix/code-scanning-alerts/807 2026-10-07 01:51:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 45d956e5ea fix(sidecars): cap an engine's sidecar at its provider's size
A Redis sidecar whose consumer declares no services.redis.mem_limit
runs in a container that inherits the consumer's primary entity limit.
Since the lookup replaced the old filter, maxmemory followed that limit
and rose from 204 MiB to between 819 and 3276 MiB in 17 roles.

lookup('service_memory_mb', ..., provider=<role>) caps the result at
the same share of the provider's own limit when the application does
not declare one for that service. A declared limit is not capped, and a
container smaller than the provider keeps its own share. The Redis
sidecar passes provider='svc-db-redis': those 17 roles get 409 MiB, the
seven with a 256 MiB container stay at 204.

The lookup now rejects unknown options and a provider that is no role
or declares no limit for the service, so a misspelled option fails
instead of silently lifting the cap.

The memcached sidecar's literal -m 64 takes the same lookup and keeps
its value.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 01:30:01 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e203d6f7ab feat(plugins): size memory parameters from the container limit with one lookup
lookup('service_memory_mb', application_id, service_name, share) returns
a share of a service's mem_limit in whole MiB. It resolves the limit the
way sys-svc-container resolves the container's own: the named service,
then the role's primary entity, then RESOURCE_MEM_LIMIT.

It replaces the three filters of memory_filters, each with its own unit
parser and its own idea of the limit: Redis maxmemory at 0.8 of the
limit, and the Confluence and Jira heap at 0.7 and 0.35.

The central Redis (409 MiB), Confluence (5734/2867) and Jira (4300/2150)
keep their values. A Redis sidecar without a mem_limit of its own does
not: the filter fell back to a fixed 256m, which gave 204 MiB, while the
lookup takes the limit that container really runs with, the primary
entity's. That moves 25 roles to between 409 and 3276 MiB.

The heap clamps of the old filters are gone: the 1024 MiB floor, limit
minus 1024, the 12288 cap and Xms bounded by mem_reservation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 23:05:26 +02:00
kevinveenbirkenbachandClaude Opus 5.5 3610e7b9ef fix(code-scanning): resolve the alerts that describe real code
Of the 76 open code scanning alerts, 39 point at code that can change.

svc-ai-mcp-adapter:
- A path argument that is empty or made only of dots is refused.
  Percent-encoding keeps a separator inside its segment, but "." and ".."
  need none, so an upstream that resolves dot segments could answer from a
  path the contract does not name.
- parse_duration splits on the units. The pattern it used before rescanned
  a long run of digits from every start position.

Tests and the README generators:
- Every Jinja environment built with autoescape=False and a "noqa: S701"
  uses select_autoescape now, as six test modules and
  utils/templating/jinja.py already did. It escapes templates named
  *.html, *.htm and *.xml, which none of these call sites loads.
- Dead code goes: a doubled "rel" block in the email lint, an unread
  _SKIP_DIRS, and a module that _resolve returned to no caller.
- Two "except ...: pass" blocks become contextlib.suppress and
  assertRaises.
- Nine test modules import unittest.mock one way instead of two, and one
  lookup test imports its module once.

Closes #807

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 09:18:00 +02:00
Prageeth Panicker bf37d5e7b3 fix(web-app-wazuh): address PR review feedback
Tor: use the shared tor block (bond 0.5, enabled when svc-net-tor is in the node's groups, tor pinned on in both variants) so a node serves one network, onion on a Tor node and clearnet otherwise.

SPOT and sizing: WAZUH_INDEXER_URL replaces four hand-built copies of the indexer URL, and the indexer heap is half of mem_limit instead of a hardcoded 1g (unchanged for the current 2g limit).

Fixes: bcrypt_hash rejects secrets over 72 bytes; the Keycloak group-removal helper uses the same resolver as the add helper and checks every response (the realm is no longer URL-encoded, matching the add helper), and a failed cleanup fails an otherwise passing test; the swarm certs generator removes a stale container before each attempt; vm.max_map_count is set before the early handler flush; securityadmin failed_when now needs rc 0 and the success marker.

The regenerated role README also picks up generator drift (mailu to stalwart).
2026-10-06 05:15:19 +00:00