Two bots bumped the one baudolo version independently. The pip pin
updater rewrites the literal package_name pin in
roles/svc-bkp-volume-2-local/tasks/01_install.yml and opens a PR scoped
to roles/<entry>/, and dependabot bumps the same package in
pyproject.toml. Each PR alone splits the pins that
tests/lint/repository/dependencies/test_baudolo_pin.py holds equal; #995
merged such a split and turned main's test-lint red until #996, and the
next upstream release (7.1.2 is tagged) would repeat it.
The version lookup now takes an optional distribution name and returns
its exact == pin from [project.dependencies]. The install task asks it
for backup-docker-to-local, so pyproject.toml is the only place the
version lives, dependabot is the only bot that moves it, and the pip
updater no longer finds a literal to bump. The lint now forbids a
literal pin in the task and requires the lookup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`lookup('tls', ...)` aligns a cross-app domain to the consumer named by
`application_id` (utils/tls_common.py `align_domain_to_consumer`): a
clearnet-primary consumer gets the clearnet sibling of an onion-primary
target. The carrier key does not cover `application_id`, so a park that
renders inside a role hands that role's alignment to every later role
whose key matches.
The constructor and `tasks/utils/setup/role.yml` park at play scope,
where `application_id` is undefined. The re-park in
`roles/sys-token-store/tasks/write.yml` (67f6c8d7e8) and the nested-load
re-park render inside the caller: `svc-prx-openresty` at the preload
loader, `web-app-stalwart` or a provider role on the NFS host, the web
app itself under `sys-stk-front-base`. In a tor deploy that turned onion
targets into clearnet domains for onion-primary roles served from such a
park, and baked the caller's group into Nextcloud's sociallogin
`groupMapping`.
Drop `application_id` from the render's variables and from the
templar's for the duration of the render, so every park carries the
play-scope result the constructor parks have always carried. Rename the
nested re-park task so its log lines do not read as a role boundary.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Job 116975298820 (Nextcloud#5, swarm, arch, btrfs, main 463df06202) hit
its budget (rc=124 after 20136 s). The app phase loads every service as a
nested `tasks/utils/load/app.yml` include under sys-service-loader, so
`tasks/utils/setup/role.yml` parks no carrier between `svc-virt-kata`
(L6073) and `update` (L38213). The only re-park in that stretch is the one
in `roles/sys-token-store/tasks/write.yml`, and Stalwart's user management
reaches it on the manager alone. After its last token write (L23574) the
manager carried the new key while both workers kept the old one and
rendered the full tree on every task: 85 non-wait tasks of 30 s or more, 63 min
from Keycloak (L25641) to the carry past `update` (L38237).
Park the carrier after every nested load on every host of the include.
A re-park whose key still matches serves the carried payload without
rendering.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Since 778c18fb23 the carrier key holds the tokens file's mtime and size.
sys-token-store changed that stat on every inclusion (00_core.yml
touched the file) and on every persisted token, and nothing re-parked a
carrier afterwards. Every later task's fresh worker then rendered the
applications SPOT cold, 30 to 80 s each. On main fea6f31d9f (run
38812964308) seven heavy jobs logged 194 to 287 such tasks and ran out
of the 20400 s budget; the same jobs before 778c18fb23 logged 65 to 83,
all of them real waits.
- 00_core.yml touches the store with modification_time and access_time
preserved, so it only creates a missing file. In swarm the controller
sees the store over NFS with attribute caching, and the touch alone
made the second pass go cold without any write.
- write.yml re-parks the carrier when the persist or the controller
mirror changed the store. Both run once, so every host that ran the
include gets the same verdict.
- tasks/utils/setup/role.yml re-parks after every role on every host,
so hosts that did not run the writing role recover too. Where the
key still matches this serves the carried payload; measured 0.1 s per
role in a local compose deploy.
- The tokens file enters the key as the SHA-256 of its content,
memoised per inode, mtime and size, so a touch no longer changes the
key. A stat that still matches serves the memo (1.67 us against
1.41 us for the old stat signature), and a stale stat keeps key and
YAML cache on the same old content instead of pairing new content
with an old key.
- applications_carrier opens the tokens file before it computes the
key, so on the NFS-mounted controller the parked key reflects the
revalidated attributes rather than a cached stat that changes a
minute later.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
No conflicts across 121 incoming commits. The one file both sides changed is
tasks/stages/01_constructor.yml: upstream parks an applications carrier for
the swarm bootstrap roles after the add_host tasks, and keys the carrier on
the tokens file as well. This branch's mesh bootstrap opens and drops its own
block-scoped carrier before that point, so both keep working as written.
Upstream also brings ruff 0.17.0, distro-derived env keys, a scoped act outer
teardown, the stalwart relay rule, the onion apex redirect and image and
version updates; ruff 0.17 reformats nothing on this branch.
The constructor parked no applications carrier between the merge block
and the token store, so every lookup in the svc-swarm-node, svc-storage,
svc-registry and svc-virt bootstrap rendered the SPOT cold, 25 to 30 s
each on a loaded host.
A carrier there was unsafe: in swarm, svc-storage-nfs-server mounts the
NFS export over the controller's DIR_VAR_LIB, so the tokens file the
render reads can appear in the middle of that section, and the cache key
did not notice. merged_applications_cache_key now includes the tokens
file's path, mtime and size, as the users cache key already did, so a
carrier rejects itself once the tokens change. A carrier parked after
the add_host tasks then serves the bootstrap roles. It is parked only in
swarm mode: in compose the section runs in about a second, so a carrier
there would add a render without saving one.
In a swarm drill of web-app-prometheus up to the known NFS probe-write
failure, "Resolve the NFS server backend flavor" went from 29.6 s to
0.07 s, "Probe-mount the NFS export" from 26.4 s to 0.4 s, "Mount
cluster-shared infinito state on controller" from 23.7 s to 0.5 s and
"Ensure Docker-In-Docker DNS works" from 30.0 s to 3.6 s; the run to
that point took 768 s instead of 1735 s.
Closes#925
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On a Tor node current_play_redirect_domains schedules the bare node
onion as a redirect source targeting DOMAIN_HOMEPAGE. Without
web-app-dashboard, DOMAIN_HOMEPAGE falls back to SOFTWARE_DOMAIN
(infinito.nexus), so web-opt-rdr-domains rendered a 301 from
http://<node>.onion/ to http://infinito.nexus/: an onion visitor sent
to a clearnet host over plain HTTP.
The onion apex entry is now added only when DOMAIN_HOMEPAGE is an
onion host. Otherwise the onion apex gets no redirect and the default
server answers.
Closes#924
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A lookup is handed the play vars as declared, so WIREGUARD_SUBNET_POOLS
arrived as the literal '{{ [WIREGUARD_SUBNET_POOL] }}'. The NFS server put
that string into its export list, ganesha refused the config with
'Unexpected character ([)' and every meshed swarm pass failed at the restart.
The value is now templated, and one that is still a template afterwards fails
the lookup instead of reaching a config file.
The guarantees were asserted against the declared swarm and data pair,
so they proved two meshes work rather than that the planner works. A
deployment declares its own, and the review asks for any number of them with
the nodes arranged however the client wants.
The same invariants now run over generated topologies: one to eight meshes,
three arrangements each, hosts dealt out so some belong to several meshes and
some to none. 2247 subtests, four seconds, and the planner needed no change to
pass them.
The network probe used community.docker.docker_network_info, whose
get_network() scans GET /networks and returns the first entry whose Name
equals the requested name OR whose Id starts with it, in a list order that
changes between calls. In main run 38038541631 (Zammad#1, swarm) the
central redis overlay got the ID n8nwzb1q39m3u4k7awwwmt6yg, the probe for
the not yet created provider network "n8n" returned it as existing, the
create step was skipped, docker stack deploy attached hermes to the redis
overlay as "n8n", and the oauth2-proxy then reached the password-protected
central redis by its bare host name: "NOAUTH Authentication required".
The same scan can also return the ID-prefix twin of a network that does
exist, so comparing the returned name alone would turn that network into
"missing" and fail the create with "already exists".
Both probes now run `container network ls --filter name=<entity> --format
'{{json .}}'`, whose name filter never matches an ID. The new lookup
network_driver picks the entry named exactly after the entity and returns
its driver, '' when there is none; the drift checks, their messages and the
create steps all read that one entry. No docker module is left in the file,
so its two docker-module-uses-container-cli suppressions go too.
Verified: on the host daemon GET /networks/<id prefix> returns a network of
another name and docker network ls --filter name=<id prefix> returns
nothing. In ansible-playbook against real docker network ls output the
lookup returned the exact network's driver in 20 of 20 probes beside an
ID-prefix twin, an empty driver (create path) for a missing network and for
a name only contained in a sibling, and the drift path for a foreign
driver. Unit tests cover the same cases; with a substring match instead of
the exact name the two sibling cases fail.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With three swarm replicas the PHP session files on the shared NFS volume
were read as empty under concurrent requests of one browser, so SuiteCRM
answered 403 on /api/graphql and dropped the session. Sessions now live in
Redis with session locking that waits like flock did; the phpredis version
is pinned in services.yml with its own update source, and the engine lookup
renders the save path.
Refs #879
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Under SSO the role tried to let applications relay without SMTP AUTH by
widening the relay rule to the private networks. The rule compared the
peer address to the text of a CIDR, which never matches, so an
application could reach local mailboxes but got "550 5.1.2 Relay not
allowed" for every external recipient. Repairing the comparison would
have opened a relay, because published ports, Tor and the swarm mesh all
hand outside clients an address inside those networks.
Stalwart checks app passwords before it asks the external directory, so
a bot can authenticate while the directory is OIDC. With SSO enabled the
role now creates one app password per bot, acting for that account as
the recovery admin, and stores it in the token store the SMTP password
is already read from. A bot gets a new one when its entry is deleted in
Stalwart or its token is removed.
The relay rule is back to authenticated sessions only. The lookup loses
its SSO relay branch and the role its submission_via_relay flag, so an
SSO deployment submits on 465 with AUTH like every other one.
The role's CLI test now fails when an unauthenticated session is given
an external recipient, or when the bot credential is refused one.
Closes#793
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`native_metrics_target` was a Jinja filter, so it could not read
DEPLOYMENT_MODE: filters see no Ansible variables. It returned
`<services.<key>.name>:<port>` unconditionally, which is `matrix-synapse:9000`
for matrix. Compose serves that name; swarm does not, because `docker stack
deploy` ignores `container_name` and the alias never exists. This repository
says as much itself, through `lookup('compose_only', 'container_name', ...)`.
The target was therefore scraped and refused, and `playwright.spec.js:274`
reported `web-app-matrix: 1 matching target(s) reporting up != 1`, meaning one
matching target whose `up` value is not 1.
It becomes a lookup that delegates host resolution to `scrape_target`, which
was already mode-aware, so exactly one plugin knows the difference between the
two modes. The port has to be handed over explicitly: matrix exposes metrics on
9000 while `services.synapse.ports.internal.http` is 8008, so the existing
internal-port lookup cannot serve this case on its own.
`scrape_target` now refuses `port` together with an explicit `port_kind`.
Before, `port` won silently and the error text still named the
`ports.internal.<port_kind>` path that the call had never consulted.
`native_metrics_apps` drops an app that pins itself to compose through
`compose_mode_force` while the cluster runs swarm, because the scrape target
would then name the wrong object. Only the literal case: matrix declares that
override as a Jinja expression over its flavour, which this lookup cannot
resolve from the scraping role's variable scope, so variant 0 on swarm stays a
latent mismatch.
The two comment lines claiming `container_name:port` went with the change;
after it they state the opposite of what the code does.
Verified by mutation: with the swarm branch of `scrape_target` disabled, the
new test emits `matrix-synapse:9000`, the exact string CI reported.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
bcrypt_hash takes an optional salt_seed. Without a seed it keeps the random salt. With a seed the salt is derived from the seed and the secret, so the same password always gives the same hash. Wazuh passes a fixed seed for the indexer admin and the dashboard service hashes, so internal_users.yml no longer changes on every run and no longer triggers a restart.
One conflict, in the generated role overview table of README.md.
Both sides had edited the same block. Upstream dropped Redis from the pihole
row; this branch still listed it. Upstream also listed svc-net-wireguard-core,
-plain and -firewalled, which this branch removed in e706170a0b as superseded
by the unified svc-net-wireguard role.
Resolved by taking upstream's block and then running `make readme-index`, so
the table is generated from the roles that actually exist rather than
hand-merged: Redis is gone from pihole, the three legacy WireGuard roles stay
gone, and svc-net-wireguard is listed once. `make readme-index check=true`
reports the table up to date.
Git kept the role deletions silently and correctly - upstream has not touched
those three directories since the merge base, so there was no modify/delete
conflict to resolve.
A Redis sidecar whose consumer declares no services.redis.mem_limit
runs in a container that inherits the consumer's primary entity limit.
Since the lookup replaced the old filter, maxmemory followed that limit
and rose from 204 MiB to between 819 and 3276 MiB in 17 roles.
lookup('service_memory_mb', ..., provider=<role>) caps the result at
the same share of the provider's own limit when the application does
not declare one for that service. A declared limit is not capped, and a
container smaller than the provider keeps its own share. The Redis
sidecar passes provider='svc-db-redis': those 17 roles get 409 MiB, the
seven with a 256 MiB container stay at 204.
The lookup now rejects unknown options and a provider that is no role
or declares no limit for the service, so a misspelled option fails
instead of silently lifting the cap.
The memcached sidecar's literal -m 64 takes the same lookup and keeps
its value.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
lookup('service_memory_mb', application_id, service_name, share) returns
a share of a service's mem_limit in whole MiB. It resolves the limit the
way sys-svc-container resolves the container's own: the named service,
then the role's primary entity, then RESOURCE_MEM_LIMIT.
It replaces the three filters of memory_filters, each with its own unit
parser and its own idea of the limit: Redis maxmemory at 0.8 of the
limit, and the Confluence and Jira heap at 0.7 and 0.35.
The central Redis (409 MiB), Confluence (5734/2867) and Jira (4300/2150)
keep their values. A Redis sidecar without a mem_limit of its own does
not: the filter fell back to a fixed 256m, which gave 204 MiB, while the
lookup takes the limit that container really runs with, the primary
entity's. That moves 25 roles to between 409 and 3276 MiB.
The heap clamps of the old filters are gone: the 1024 MiB floor, limit
minus 1024, the 12288 cap and Xms bounded by mem_reservation.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Tor: use the shared tor block (bond 0.5, enabled when svc-net-tor is in the node's groups, tor pinned on in both variants) so a node serves one network, onion on a Tor node and clearnet otherwise.
SPOT and sizing: WAZUH_INDEXER_URL replaces four hand-built copies of the indexer URL, and the indexer heap is half of mem_limit instead of a hardcoded 1g (unchanged for the current 2g limit).
Fixes: bcrypt_hash rejects secrets over 72 bytes; the Keycloak group-removal helper uses the same resolver as the add helper and checks every response (the realm is no longer URL-encoded, matching the add helper), and a failed cleanup fails an otherwise passing test; the swarm certs generator removes a stale container before each attempt; vm.max_map_count is set before the early handler flush; securityadmin failed_when now needs rc 0 and the success marker.
The regenerated role README also picks up generator drift (mailu to stalwart).
Add single-node Wazuh stack (manager, indexer, dashboard) as a new
Ansible role, integrated with the platform's Keycloak realm via native
OIDC and secondary LDAP authorization, with three-tier RBAC.
- Pins wazuh/wazuh-{manager,indexer,dashboard}:4.14.6 and
wazuh-certs-generator:0.0.4 (one-shot cert bootstrap job)
- No external database: wazuh.indexer (OpenSearch) is self-contained
- Auth: native OIDC against Keycloak, working end-to-end after two
infra fixes (JVM cacerts trust for the indexer's outbound calls to
Keycloak, front-proxy vhost wiring)
- LDAP wired as a secondary, authentication-independent authorization
backend (group-based backend_roles enrichment only)
- RBAC: three roles (administrator, security-analyst, readonly-auditor)
via the platform's standard meta/rbac.yml + rbac_group_path pipeline
- Documents a genuine upstream Wazuh bug: wz-home as the default
landing route throws an uncaught client exception for any
non-all_access role; default route moved to OpenSearch Dashboards'
own /app/home
Addressed two full rounds of review feedback (Kevin, Alejandro):
WEB_PROTOCOL usage, service-account naming, comment cleanup, shared
Keycloak-admin-helper extraction, a real TOCTOU race in RBAC test
setup, a Mermaid auth-flow diagram, meta/services.yml placement for
the certs-generator, CA-trust-path SPOT usage, CSP tightening
(including a genuine worker-src: blob: requirement found via live
testing), and a loud (non-silent) dev-bucket fallback warning.
Rebased onto main after a 3-month gap; brought the role in line with
~30 platform-convention changes introduced in that window (swarm
compose-template rules, Tor/onion contract, Playwright timeout/goto
wrappers, file layout, task numbering, comment policy, and more).
Live-deploy verification (9 iterations) found and fixed 6 further
real bugs invisible to lint/unit tests alone:
- credentials.* / meta/schema.yml -> secrets.credentials.* /
meta/secrets.yml (another platform-drift casualty)
- notify: "compose up" -> "compose-up" (day-one handler-name typo)
- local.subnet collision with web-app-mailu's /24
- missing primary: false alongside tor.exclusive: false, which let
the onion domain shadow the clearnet vhost entirely
- a second front-proxy include for the onion domain, later reverted
(see below)
- shared sys-stk-front-proxy/sys-front-tls TLS-lookup keying bug
(app-id-keyed vs. domain-keyed), which silently applied TLS to
Wazuh's onion vhost
The last two are related: enabling genuine dual-stack (clearnet +
onion) reachability for tor.exclusive: false surfaced a real bug in
two shared platform roles. That fix was verified safe against all 103
apps currently in the repo, but is deliberately NOT included here -
shared-role changes deserve their own PR and review, not to ride in
on a single app's feature branch. Wazuh's second front-proxy include
was reverted accordingly; the resulting onion-reachability gap is
documented in the README as a known, platform-scoped limitation.
make test: 5508 passed, 3 failed (all three trace to untracked local
scratch files - Dockerfile.bak, scripts/lab-*.sh - not present in any
CI checkout).
One conflict, in the generated role table of README.md. Both sides changed
the same rows: upstream swapped WordPress's Mailu dependency for Stalwart,
while this branch replaced the three svc-net-wireguard-{core,plain,
firewalled} rows with the single svc-net-wireguard it deploys.
Resolved by regenerating rather than by hand, since make readme-index owns
that table and gates it with --check. The regenerated table carries both
sides and the check reports it up to date.
README.md was the only file both sides touched, and nothing in the tree
still names the three replaced roles.
Formatter output only: the parsed YAML and the Python AST of every touched file equal their previous versions.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
One conflict, in the MCP exclusion list of requirement 035. Both sides
edited the same line: main added svc-net-tor-smtp, this branch replaced the
three svc-net-wireguard-{core,plain,firewalled} entries with the single
svc-net-wireguard it deploys. Resolved to the union of both — 24 entries —
which is what the merged tree actually holds.
Everything else merged clean and was checked for overlap rather than
assumed: no reference anywhere still names the three replaced roles;
matrix.py kept main's PRIMARY_APPS alongside the reordered round, and the
bootstrap pass does not need it, since only one template reads it and
group_vars defaults it empty; and main's ansible.cfg addition sits in
[defaults], so the ssh_args the mesh tool derives its own connection
settings from are untouched.
postgis/postgis publishes amd64 only, so every postgres consumer failed
on arm64. The official postgres image serves both architectures and
already ships the contrib extensions bloom, pg_trgm and unaccent.
PostGIS moves into the postgres_libraries registry and is installed from
PGDG (postgresql-18-postgis-3) only for a consumer that declares it as
an extension, which is mobilizon alone. apt update reads every source
list again, because the PostGIS package depends on GDAL from Debian
main, which a PGDG-only update left without an install candidate.
Postgres 18 keeps its cluster in /var/lib/postgresql/18/docker and its
entrypoint treats a volume at the former /var/lib/postgresql/data as an
old database, so the central and the dedicated data volume both mount at
/var/lib/postgresql. Existing 17 clusters are not migrated: their data
directory trips the entrypoint's old-data check and the container
refuses to start.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit f78ebe46a4)
Brings in 225 commits, including the merged web-app-openbao work and the
LMStudio loopback fix.
Seven conflicts:
- The legacy svc-net-wireguard-core/plain/firewalled roles this branch deleted
in e706170a0 were modified upstream. The deletion stands, which is the whole
premise of the branch.
- The MCP audit takes upstream's list plus svc-net-firewall, with the three
legacy wireguard entries collapsed into the single svc-net-wireguard: 23.
- web-app-openbao/README.md takes main's reviewed version wholesale.
- matrix.py takes both sides: upstream's app_id argument to _reset_credentials
and this branch's _write_mesh call after it.
The merge pushed matrix.py to 511 lines against the 500 cap, so the step
runner moved to utils/tests/swarm/run.py, which is self-contained and leaves
matrix.py at 427. Its test moved with it.
