SeaweedFS 4.49 (seaweedfs/seaweedfs#11592) authorizes a PutObject that
carries x-amz-acl as s3:PutObjectAcl as well. The consumer identities
only held Read/Write/List/Tagging, so every app that sends an ACL on
upload (Nextcloud, Fider and Pixelfed with private, Mastodon, PeerTube,
BookWyrm and Baserow with public-read) got AccessDenied after the 4.48 -> 4.51
bump in #1226. Main run 40401200083 failed Fider#0, BookWyrm#0,
Baserow#0, Pixelfed#0 and Nextcloud#0/#4/#8 on it.
Add WriteAcp to the s3.json identities and to the sidecar s3.configure
grant.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every database consumer called baudolo-seed through the seed task, 37 times and always 'changed' in a full production run (54.4 s), although databases.csv rarely changes. The first seed of a play now reads the file once (05_read-seeded-databases.yml) into svc_bkp_volume_2_local_seeded; baudolo_seed_current compares the consumer's row with it and baudolo_seed_apply advances the cached state the way baudolo-seed writes the file, so the command runs only for a missing or differing row. A file baudolo did not write byte for byte (hand-edited, CRLF, short rows) falls back to one call per consumer; 02_reset forgets the cache with the file.
Replay harness old vs new: databases.csv byte-identical in fresh, empty, converged, drift and reset-mid-play scenarios; converged seeds 37 -> 0, drift 38 -> 2; the negative control without reset invalidation diverges. About 44 s per unchanged redeploy.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dashboard card of every app opened its primary domain, so Matrix led to the Synapse status page and Stalwart to its admin UI. A role can now declare services.dashboard.card: <key of meta/domains.yml canonical>; docker_cards builds the card URL with lookup('canonical_url', app, key), the Apps menu reuses it, and roles_with_service('dashboard') hands the same domain to the tile e2e spec. Matrix sets card: element, Stalwart card: webmail. Without card nothing changes, and the prometheus/matomo target lists are untouched. A lint requires the key to resolve in the role's own domains.canonical.
The Matrix ansible flavor also sets Synapse's web_client_location to Element through matrix_synapse_configuration_extension_yaml, as the compose flavor already does, so matrix.<domain>/ redirects there. Merge note: feature/iam-profile-rbac defines the same extension variable inside the OIDC block; the next merge must combine both into one block (agreed with that session).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
mcp_servers narrowed the MCP providers to application_closure(deployment.whitelist), which on a persistent host drops every provider an earlier run deployed and a partial run does not touch. sys-svc-mcp-reconcile then converges each client present on the host against that narrowed set: on echoserver the --id web-opt-rdr-domains run rdr-3 removed the Prometheus registration from Flowise.
deployment now reports ephemeral (runtime act or github), and the closure narrows only there, where a whitelist is all a run deploys; a persistent host narrows nothing, as a full deploy already did with its empty whitelist, so its clients keep every provider it serves. CI keeps its narrowing unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Host-wide redirects such as a legacy zone came from the top-level inventory variable redirect_domain_mappings, which only the role default declared. Once the role input moved to web_opt_rdr_domains_mappings nothing declared it any more and the inventory validation refused the deploy, because it does not see the lookup's vars_ read. The redirects now live where the role's other settings are: services.rdr-domains.redirects in meta/services.yml, empty by default and set per inventory under applications.web-opt-rdr-domains, and current_play_redirect_domains reads them through the config lookup. They still win over the application aliases, and a caller that includes the role without its own web_opt_rdr_domains_mappings still renders the merged CURRENT_PLAY_REDIRECT_DOMAINS.
Inventories move their redirect_domain_mappings list unchanged to applications.web-opt-rdr-domains.services.rdr-domains.redirects.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ssl_credentials.j2 resolved the certificate by application_id, so every vhost of an application got the lineage of its primary domain. echoserver serves webmail.infinito.nexus the CN=academy.infinito.nexus certificate that names mail.infinito.nexus but not webmail, and browsers refuse it with a hostname mismatch. The template and the letsencrypt existence check now resolve by the vhost's domain, so a secondary vhost points at its own live/<domain> link and a missing link triggers the issuance instead of a mismatch. Primary vhosts resolve to the same files as before in every TLS mode and flavor.
The LDAP stream passes its primary domain explicitly. Onion domains stay on the application lineage, and the app-scoped self-signed path always names the application leaf, because neither has a per-domain certificate.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The lookup assembled '.<version>.<software domain>' itself, a second copy of the infix the unlock and prune phases now read from the SPOT. It reads SYS_SERVICE_VERSION_SUFFIX instead and refuses one that does not start with a dot or carries an empty segment. The direct version-plugin fallback goes with it: it existed because the plugin rendered the untrusted literal "{{ lookup('version') }}", while the SPOT arrives as a trusted group_vars string. Every unit name rendered from group_vars is byte-identical before and after.
The timer description names the versioned service it starts instead of the legacy unversioned name.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
redirect_domain_mappings now renders its source and target, so an inventory can map *.{{ OLD_DOMAIN }} and the apex onto a target; keep_path: false drops the request URI from the 301. Wildcards get no www. variant, sys-dns-wildcards adds the *.<zone> A/AAAA records for every zone with a wildcard source next to DOMAIN_PRIMARY, and the CSP health check skips wildcard names.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The cert lookup handed an alias or redirect source the live directory of its app's canonical domain or of its target, so social.cymais.cloud got the infinito.nexus lineage and failed with 'Hostname mismatch'. With the san and dedicated flavors a domain term now resolves to its own live/<domain>, which cert_link_domains points at the lineage that covers it; application terms keep the canonical path. resolve_term follows redirect chains and fails on a cycle.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each redirect source went through the full TLS layer on its own: cert check,
CA environment cleanup, re-check, HTTP handler and vhost template, about
nine seconds per source. echoserver has 85 sources, so the role took
thirteen minutes of every deploy.
The role now resolves the tls state and renders both files of every source
on the controller, finds the certificates of all sources in one find, and
includes sys-front-tls only for a source whose certificate is missing, so
issuance still works in every mode. library/stack_host_contents.py writes
all files in one module call, atomically and only where the content
differs, and keeps one file per source under http/ and https/. Its action
plugin carries the IS_STACK_HOST gate like stack_host_template.
The name differs from the stack_host_files module of feature/store-i18n on
purpose: that one syncs a single directory of YAML documents and prunes
what it does not write, which would delete every other vhost here.
HTTPS_REDIRECT_ENABLED is gone: nothing set it, it was true everywhere,
and the redirect role could no longer see the sys-front-tls default it
lived in. Whether a domain gets its HTTP to HTTPS handler now follows from
lookup('tls', domain, 'enabled') alone, in sys-front-tls and here.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
meta/domains.yml lands on the top-level `domains` key of the merged
application, yet three readers still looked under `server.domains`,
which no role fills any more:
- current_play_redirect_domains built every mapping from an empty
config, so each app redirected to itself and was dropped. Neither
`domains.aliases` nor AUTO_BUILD_ALIASES produced a single redirect,
and echoserver's update check listed nextcloud.infinito.nexus and the
other entity domains as dropped by the upgrade.
- csp_skip_domains never found the canonical of a disabled service.
- the logout filter's fallback returned no domains.
The unit tests fed the old layout and passed regardless. They now use
the merged one, and a new test builds the redirect from keycloak's real
role defaults.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pull.yml pulls, on the target host, every pinned image of the applications the host deploys, the same closure the update check judges, and lists the images a role builds itself instead. It never touches a container, retries each pull a bounded number of times and fails loudly with the image and the registry error. The update_check filter moves next to the new update_pull filter into plugins/filter/update/, and both read the code pins from one helper.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The version-source update in #1146 moved the bigbluebutton/docker ref from
v3.0.23 to v3.0.39. That tag changes three things the role did not
handle, and it exposed a fourth defect in compose_file_args.
1. generate-compose renders etherpad and bbb-pads only behind
USE_INSECURE_ETHERPAD and otherwise emits bbb-shared-notes-server with
a `build:` block and no `image:` (upstream adds the image only after
the tag, in 9a6a9a9a4c). compose_mods drops `build` only where an image
is set, so the guard "Fail if any service still builds locally" stopped
every BigBlueButton deploy (main run 39749899667, Nextcloud#11:
`rc: 0`, `stdout: ' build:'`, bootstrap_compose_project.yml:35).
meta/services.yml now pins alangecker/bbb-shared-notes-server:v3.0.39,
compose_mods takes `pinned` (BBB_PINNED_SERVICES, every service the
role config pins an image for) and drops their `build`, and the
etherpad leftovers go (ep_redis_publisher patch, etherpad postgres
entrypoint and initdb hook, etherpad database tasks, ETHERPAD_* env,
etherpad_api_key secret).
2. The notes server connects to postgres once at start and exits with 2
when it is not up yet; upstream gives it no restart policy. The
override sets the role's restart policy, so it retries until postgres
accepts connections.
3. Greenlight moves from v3.8.0 to v3.9.0.1, which sets
`config.hosts = URL_HOST`, and the template passes
`URL_HOST: https://${DOMAIN}`. Rails compares hostnames, so every
request was blocked (`Blocked hosts: bbb.talk.infinito.test`, 403).
Greenlight documents URL_HOST as the bare hostname; the override sets
it to the role domain.
4. compose_file_args decided whether a role ships a compose override by
templating "{{ application_id | abs_role_path_by_application_id }}"
through the templar. ansible-core 2.21 does not render an untrusted
string, so the check saw the literal and always returned False: the CA
override generation and the pull handler never read BBB's
compose.override.yml. That was harmless while compose.yml alone was
valid; with the notes-server image living in the override, `compose
config` failed ("has neither an image nor a build context"). The check
now resolves the role path for the application id it is given. BBB is
the only role that ships an override, so no other role's file list
changes.
Existing installations drop the etherpad and bbb-pads containers on the
next `compose up --remove-orphans`; the old `etherpad` database stays in
PostgreSQL untouched.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
report.yml only dumped a host's state, so every comparison with the code
of the checkout was done by hand. It now also runs state.sh on the host,
which emits tab-separated records (volumes with their resolved storage
path, containers with their port bindings, listening sockets with the
container that owns them, docker subnets, data versions, server_name
lines, certificates, failed units, newest backups, disk usage), and the
update_check filter compares them with what lookup('applications'),
application_closure, compose_volumes, current_play_redirect_domains and
the nginx include directories derive from the code and the inventory.
The verdict lists every finding as OK, WARN or FAIL per area (inventory,
volumes, data, images, ports, networks, webserver, certificates,
operations), prints the warnings and failures, keeps the full list in
/tmp/infinito-check-<host>.txt and fails the play on a red check. The
host still needs no Python.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In swarm mode `docker stack deploy` ignores the host IP of a short port
mapping, so every `127.0.0.1:<port>:<port>` that container_ports renders for
a `ports.local` entry is published by the ingress routing mesh on every
node, via docker-proxy on 0.0.0.0 and [::] plus DNAT in nat PREROUTING.
Databases, LDAP and application backends answered from outside the cluster.
svc-swarm-node now loads an nftables table through svc-net-firewall. Its
prerouting filter chain (priority mangle, ahead of Docker's dstnat) drops
packets addressed to the node for any ingress-published port that no
application declares under `ports.public`. Loopback, docker0,
docker_gwbridge, the user bridges and the swarm nodes' own addresses stay
admitted. Host-side provisioning over 127.0.0.1, containers that reach a
host port through host-gateway, and node-to-node consumers such as the
registry-cache mirror keep working.
Both sets are read live on the manager (`docker service inspect` for
`.Endpoint.Ports`, `docker node inspect` for `.Status.Addr`), so ephemeral
ingress ports and nodes outside `--limit` are covered. The table is loaded
on a node before it joins, reloaded on every node after the join, and
reloaded after each stack deploy by the sys-svc-compose swarm handler.
Refs #1089
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The seeded salt was the first 16 bytes of sha256(seed + secret). bcrypt
stores its salt in clear inside the hash, so every seeded hash published
a fast sha256 digest of the password under a seed that is public in the
repository (`web-app-wazuh:indexer-admin`). Anyone holding the hash could
test password guesses at sha256 speed instead of paying bcrypt's cost.
The salt is now sha256(seed)[:16]. The hash stays deterministic for one
password, still changes when the password rotates, and its salt carries
nothing computed from the secret. Existing Wazuh deployments get a new
hash once; `securityadmin.sh` re-applies `internal_users.yml` on every
deploy and the hash still verifies the unchanged password.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two bots bumped the one baudolo version independently. The pip pin
updater rewrites the literal package_name pin in
roles/svc-bkp-volume-2-local/tasks/01_install.yml and opens a PR scoped
to roles/<entry>/, and dependabot bumps the same package in
pyproject.toml. Each PR alone splits the pins that
tests/lint/repository/dependencies/test_baudolo_pin.py holds equal; #995
merged such a split and turned main's test-lint red until #996, and the
next upstream release (7.1.2 is tagged) would repeat it.
The version lookup now takes an optional distribution name and returns
its exact == pin from [project.dependencies]. The install task asks it
for backup-docker-to-local, so pyproject.toml is the only place the
version lives, dependabot is the only bot that moves it, and the pip
updater no longer finds a literal to bump. The lint now forbids a
literal pin in the task and requires the lookup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`lookup('tls', ...)` aligns a cross-app domain to the consumer named by
`application_id` (utils/tls_common.py `align_domain_to_consumer`): a
clearnet-primary consumer gets the clearnet sibling of an onion-primary
target. The carrier key does not cover `application_id`, so a park that
renders inside a role hands that role's alignment to every later role
whose key matches.
The constructor and `tasks/utils/setup/role.yml` park at play scope,
where `application_id` is undefined. The re-park in
`roles/sys-token-store/tasks/write.yml` (67f6c8d7e8) and the nested-load
re-park render inside the caller: `svc-prx-openresty` at the preload
loader, `web-app-stalwart` or a provider role on the NFS host, the web
app itself under `sys-stk-front-base`. In a tor deploy that turned onion
targets into clearnet domains for onion-primary roles served from such a
park, and baked the caller's group into Nextcloud's sociallogin
`groupMapping`.
Drop `application_id` from the render's variables and from the
templar's for the duration of the render, so every park carries the
play-scope result the constructor parks have always carried. Rename the
nested re-park task so its log lines do not read as a role boundary.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Since 778c18fb23 the carrier key holds the tokens file's mtime and size.
sys-token-store changed that stat on every inclusion (00_core.yml
touched the file) and on every persisted token, and nothing re-parked a
carrier afterwards. Every later task's fresh worker then rendered the
applications SPOT cold, 30 to 80 s each. On main fea6f31d9f (run
38812964308) seven heavy jobs logged 194 to 287 such tasks and ran out
of the 20400 s budget; the same jobs before 778c18fb23 logged 65 to 83,
all of them real waits.
- 00_core.yml touches the store with modification_time and access_time
preserved, so it only creates a missing file. In swarm the controller
sees the store over NFS with attribute caching, and the touch alone
made the second pass go cold without any write.
- write.yml re-parks the carrier when the persist or the controller
mirror changed the store. Both run once, so every host that ran the
include gets the same verdict.
- tasks/utils/setup/role.yml re-parks after every role on every host,
so hosts that did not run the writing role recover too. Where the
key still matches this serves the carried payload; measured 0.1 s per
role in a local compose deploy.
- The tokens file enters the key as the SHA-256 of its content,
memoised per inode, mtime and size, so a touch no longer changes the
key. A stat that still matches serves the memo (1.67 us against
1.41 us for the old stat signature), and a stale stat keeps key and
YAML cache on the same old content instead of pairing new content
with an old key.
- applications_carrier opens the tokens file before it computes the
key, so on the NFS-mounted controller the parked key reflects the
revalidated attributes rather than a cached stat that changes a
minute later.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
No conflicts across 121 incoming commits. The one file both sides changed is
tasks/stages/01_constructor.yml: upstream parks an applications carrier for
the swarm bootstrap roles after the add_host tasks, and keys the carrier on
the tokens file as well. This branch's mesh bootstrap opens and drops its own
block-scoped carrier before that point, so both keep working as written.
Upstream also brings ruff 0.17.0, distro-derived env keys, a scoped act outer
teardown, the stalwart relay rule, the onion apex redirect and image and
version updates; ruff 0.17 reformats nothing on this branch.
On a Tor node current_play_redirect_domains schedules the bare node
onion as a redirect source targeting DOMAIN_HOMEPAGE. Without
web-app-dashboard, DOMAIN_HOMEPAGE falls back to SOFTWARE_DOMAIN
(infinito.nexus), so web-opt-rdr-domains rendered a 301 from
http://<node>.onion/ to http://infinito.nexus/: an onion visitor sent
to a clearnet host over plain HTTP.
The onion apex entry is now added only when DOMAIN_HOMEPAGE is an
onion host. Otherwise the onion apex gets no redirect and the default
server answers.
Closes#924
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The network probe used community.docker.docker_network_info, whose
get_network() scans GET /networks and returns the first entry whose Name
equals the requested name OR whose Id starts with it, in a list order that
changes between calls. In main run 38038541631 (Zammad#1, swarm) the
central redis overlay got the ID n8nwzb1q39m3u4k7awwwmt6yg, the probe for
the not yet created provider network "n8n" returned it as existing, the
create step was skipped, docker stack deploy attached hermes to the redis
overlay as "n8n", and the oauth2-proxy then reached the password-protected
central redis by its bare host name: "NOAUTH Authentication required".
The same scan can also return the ID-prefix twin of a network that does
exist, so comparing the returned name alone would turn that network into
"missing" and fail the create with "already exists".
Both probes now run `container network ls --filter name=<entity> --format
'{{json .}}'`, whose name filter never matches an ID. The new lookup
network_driver picks the entry named exactly after the entity and returns
its driver, '' when there is none; the drift checks, their messages and the
create steps all read that one entry. No docker module is left in the file,
so its two docker-module-uses-container-cli suppressions go too.
Verified: on the host daemon GET /networks/<id prefix> returns a network of
another name and docker network ls --filter name=<id prefix> returns
nothing. In ansible-playbook against real docker network ls output the
lookup returned the exact network's driver in 20 of 20 probes beside an
ID-prefix twin, an empty driver (create path) for a missing network and for
a name only contained in a sibling, and the drift path for a foreign
driver. Unit tests cover the same cases; with a substring match instead of
the exact name the two sibling cases fail.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`make quality-high` rewrote these files on every branch: the SuiteCRM
README and the root role table lacked the Redis service #881 added, the
session path test was not formatted, and an OpenTalk secret description
from #873 was not wrapped.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With three swarm replicas the PHP session files on the shared NFS volume
were read as empty under concurrent requests of one browser, so SuiteCRM
answered 403 on /api/graphql and dropped the session. Sessions now live in
Redis with session locking that waits like flock did; the phpredis version
is pinned in services.yml with its own update source, and the engine lookup
renders the save path.
Refs #879
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Under SSO the role tried to let applications relay without SMTP AUTH by
widening the relay rule to the private networks. The rule compared the
peer address to the text of a CIDR, which never matches, so an
application could reach local mailboxes but got "550 5.1.2 Relay not
allowed" for every external recipient. Repairing the comparison would
have opened a relay, because published ports, Tor and the swarm mesh all
hand outside clients an address inside those networks.
Stalwart checks app passwords before it asks the external directory, so
a bot can authenticate while the directory is OIDC. With SSO enabled the
role now creates one app password per bot, acting for that account as
the recovery admin, and stores it in the token store the SMTP password
is already read from. A bot gets a new one when its entry is deleted in
Stalwart or its token is removed.
The relay rule is back to authenticated sessions only. The lookup loses
its SSO relay branch and the role its submission_via_relay flag, so an
SSO deployment submits on 465 with AUTH like every other one.
The role's CLI test now fails when an unauthenticated session is given
an external recipient, or when the bot credential is refused one.
Closes#793
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`native_metrics_target` was a Jinja filter, so it could not read
DEPLOYMENT_MODE: filters see no Ansible variables. It returned
`<services.<key>.name>:<port>` unconditionally, which is `matrix-synapse:9000`
for matrix. Compose serves that name; swarm does not, because `docker stack
deploy` ignores `container_name` and the alias never exists. This repository
says as much itself, through `lookup('compose_only', 'container_name', ...)`.
The target was therefore scraped and refused, and `playwright.spec.js:274`
reported `web-app-matrix: 1 matching target(s) reporting up != 1`, meaning one
matching target whose `up` value is not 1.
It becomes a lookup that delegates host resolution to `scrape_target`, which
was already mode-aware, so exactly one plugin knows the difference between the
two modes. The port has to be handed over explicitly: matrix exposes metrics on
9000 while `services.synapse.ports.internal.http` is 8008, so the existing
internal-port lookup cannot serve this case on its own.
`scrape_target` now refuses `port` together with an explicit `port_kind`.
Before, `port` won silently and the error text still named the
`ports.internal.<port_kind>` path that the call had never consulted.
`native_metrics_apps` drops an app that pins itself to compose through
`compose_mode_force` while the cluster runs swarm, because the scrape target
would then name the wrong object. Only the literal case: matrix declares that
override as a Jinja expression over its flavour, which this lookup cannot
resolve from the scraping role's variable scope, so variant 0 on swarm stays a
latent mismatch.
The two comment lines claiming `container_name:port` went with the change;
after it they state the opposite of what the code does.
Verified by mutation: with the swarm branch of `scrape_target` disabled, the
new test emits `matrix-synapse:9000`, the exact string CI reported.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
