Commit Graph
2780 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.8 456ff59ca6 feat(lookups): tor_socks endpoint SPOT + tor_ports exposed-port forwarding
tor_socks returns <host>:<port> from svc-net-tor services.tor.ports.local.socks (single spot; default host 127.0.0.1, host overridable for 0.0.0.0/host.docker.internal). tor_ports gains variant-aware collect_exposed_ports: the loopback ports.local of exposed:true services become per-variant onion HiddenServicePort targets (public/TLS ports excluded).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:18 +02:00
kevinveenbirkenbachandClaude Opus 4.8 c4e64656be test(tor): let onion-surfaceless domain roles opt out of the tor contract
test_tor_contract gains a '# nocheck: tor-contract' opt-out (it previously only excluded svc-net-tor by name). web-opt-rdr-domains, a redirect-only role with no onion surface, drops its tor block and marks the exemption; its variant-coverage stub is removed with it. The new rule is registered in docs/contributing/actions/testing/suppression.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 23:53:17 +02:00
kevinveenbirkenbachandClaude Fable 5 1072ee27cc test(lint): catch argv-form container exec + mark legitimate bare-name uses
The container-exec-resolver lint only saw the inline shell form, so the
`command: argv: [container, exec, <name>]` list form (two separate lines)
slipped through, which is how checkmk's bare name reached swarm. Extend the
scanner to parse argv sequences via yaml.compose_all with node line numbers,
and annotate the nine legitimate bare-name execs (mariadb recovery container,
matomo compose-only debug, probe-gated nextcloud partner provisioning, pixelfed
runtime-resolved CID, wordpress launcher-managed discourse) with
# nocheck: container-exec-resolver.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 20:35:44 +02:00
kevinveenbirkenbachandClaude Opus 4.8 945fbccc1e refactor(swarm): move service_replicas into services.yml
Replace the per-role service_replicas Jinja variable with a per-service
replicas: field in meta/services.yml. deploy.yml.j2 resolves it via
resource_filter (with entity fallback), the same path already used for
cpus/mem, so services.yml is the single source of truth for replica
overrides.

- 13 web roles migrated to per-service (or entity-level) replicas: 1
- matomo bootstrap -> replicas: 0, mastodon migrate -> replicas: 1
- minio/seaweedfs provider entries -> replicas: 1, which also pins the
  central single-writer object-store providers instead of relying on the
  topology default
- service_replicas removed entirely; dropped its dead reference from the
  compose-structure lint allowlist and updated requirement 023 wording

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1b774e410c feat(update): support every registry via a generic v2 client
Add a generic Docker Registry v2 client (WWW-Authenticate bearer-token
challenge) so tag listing and reachability work for any registry (Docker
Hub, GHCR, MCR, Quay, GitLab/opencode), and route the version updater
through it so quay.io and opencode.de images auto-update like the rest.
Add an external test that hard-fails when a pinned image:version tag is
unreachable (404) and warns on incomplete image/version pairs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0f63cecc27 fix(swarm-nfs): mount NFSv4 volumes with local_lock=flock
Checkmk's OMD site crashes at boot with fcntl.flock raising OSError
[Errno 9] Bad file descriptor: NFSv4 ignores the nolock mount option and
routes flock() to the server. Force flock() client-local on v4 so
single-writer apps (checkmk, openldap) boot; POSIX locks stay
server-side. NFSv3 keeps nolock (statd-free, flock already local).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f5412682b0 refactor(container-port): derive healthcheck port, drop redundant defs
The sys-svc-container healthcheck templates (curl/wget/nc/tcp/http) now
derive container_port from services.<service_name>.ports.internal.http,
so a role no longer needs to set container_port only to feed the shared
healthcheck.

Add the container-port-set-unused lint that fails a role setting
container_port without using it elsewhere, then resolve every flagged
role: drop the redundant assignment where the derivation reproduces the
value, or mark it with nocheck where the port is non-http, a constant, or
keyed on a service the derivation cannot reach.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 05:05:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 cbe3555d94 feat(svc-net-tor): expose deployed roles' public TCP ports over the node onion
Add a `tor_ports` lookup that collects every single-int `ports.public` entry
from the deployed roles' meta/services.yml and returns HiddenServicePort
mappings (`127.0.0.1:<port>`). UDP-only categories (relay, media) are excluded
since Tor forwards TCP streams only. Wire it into TOR_ONION_EXTRA_PORTS
(19_tor.yml) alongside the SSH port so mail (25/143/465/587/993), gitea/gitlab
SSH, ldaps, matrix federation and STUN/TURN control ports are reachable at
<node>.onion. Register Mailu's mail endpoints under ports.public and add the
matching PORT_BANDS entries.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 03:26:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 23c102263b fix(cleanup): honor INFINITO_DOMAIN in matrix-round vhost purge
The between-rounds vhost purge computed which `servers/{http,https}/<domain>.conf`
files to remove from the domain, falling back to the hardcoded default
`infinito.example` when the DOMAIN env was unset. On a node whose
DOMAIN_PRIMARY differs (e.g. a .onion node, where DOMAIN_PRIMARY is driven
by INFINITO_DOMAIN), the purge computed non-existent filenames and removed
nothing, leaving stale vhosts. The next matrix-deploy round's health probe
then hit those orphaned vhosts as HTTP 502.

Fall back to INFINITO_DOMAIN (the dev harness' domain SPOT) before the
default so the purge targets the real vhost files on any node. Precedence:
explicit arg > DOMAIN > INFINITO_DOMAIN > default. Covered by 4 new tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 03:26:02 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f2af680a2f test(lint): require container_port to read the internal-port config lookup
Add a lint that rejects hardcoded container_port literals and the
host-published ports.local port; container_port is the port the container
listens on, so it must resolve from
lookup('config', application_id, 'services.<svc>.ports.internal.<name>')
or a constant built that way. Bring 42 roles into compliance: move each
literal into services.<svc>.ports.internal.<name> and reference it via the
lookup, using entity_name/service_name where they equal the service key.
Values are preserved (mattermost/prometheus pointed at ports.local with the
same value as internal). Drop the dead container_port from fider and
prometheus, and the narrative comments the edits pulled into the comment
lint's changed-file scope.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 00:20:10 +02:00
kevinveenbirkenbachandClaude Opus 4.8 44bd1ffbcd test(lint): forbid config/secret mounts onto a directory target
Swarm config and secret objects are single files, so 'docker stack deploy'
aborts with "is a directory" when a 'type: config'/'type: secret' mount
targets a path ending in '/' (the nextcloud infinito_conf bug). Add a lint
that scans meta volumes files and flags such mounts, with a
'config-secret-directory-target' nocheck escape.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 21:41:43 +02:00
kevinveenbirkenbachandClaude Opus 4.8 266a37d965 feat(svc-net-tor): onion mail/SSH ports + TLS-off compat (CA CN, HSTS, msmtp)
- svc-net-tor: expose SSH (22) + Mailu SMTP/submission/IMAP (25/465/587/143/993)
  as HiddenServicePorts on the node onion (TOR_ONION_SSH_ENABLED, 19_tor.yml).
- email lookup: relay plaintext (tls off, port 25) for .onion mail domains —
  no public TLS cert for a .onion; onion transport is already authenticated.
- CA root CN: drop the " Root CA" suffix when a 62-char v3 onion + suffix would
  overflow the X.509 commonName 64-char cap (02_tls.yml).
- Playwright onion compat: HSTS is HTTPS-only (RFC 6797), absent on the http
  onion vhost. New shared helper expectHstsWhenTls() gates the assertion on the
  reached scheme; wired into web-svc-cdn spec.
- rerun-spec.sh: route the inner-loop browser through the Tor SOCKS proxy when
  the staged env targets a .onion, mirroring the deploy-time runner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 16:39:40 +02:00
kevinveenbirkenbachandClaude Fable 5 1aba5707f8 chore: quality fixups for the run-28721118458 fix series
Drop the now-unused TAIGA_INSTANCE_DIRECTORY (its only consumer left
with the orphaned compose-inits render), mark the two discourse
launcher execs with the container-exec-resolver nocheck (the launcher
container is named literally in both modes), and purge narration
comments from the dotenv quote tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 16:20:19 +02:00
kevinveenbirkenbachandClaude Fable 5 97b1ffddb4 fix(dotenv): quote whitespace values in the swarm env passthrough
The swarm branch passed values through raw, so a value with interior
whitespace (MAIL_FROM_NAME='Infinito.Nexus by ...') landed unquoted in
the rendered .env and broke the env parser: the snipe-it service
crashlooped at 0/3 until the converge wait failed in every variant.

Quote (and escape backslash/double-quote) only when the value contains
space/tab/newline; everything else stays raw, so the swarm invariants
hold: no compose $$-doubling, raw $ and double quotes preserved,
and the playwright --env-file path (docker CLI keeps quotes literal)
is unaffected because no playwright env value can contain whitespace.
Compose quoting is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 15:34:28 +02:00
kevinveenbirkenbachandClaude Fable 5 b0f3e6e1ff fix(web-app-bigbluebutton): map the keycloak domain to host-gateway in the bbb compose project
Greenlight's server-side OIDC discovery failed with 'Connection refused'
against auth.<domain>: inside the isolated bbb-net the domain resolves
to 127.0.0.1, i.e. the greenlight container itself, while openresty
publishes 443 in host mode on the node. Playwright's OIDC login specs
timed out on greenlight's 404 page as a result.

compose_mods now takes an extra_hosts kwarg and injects the mapping into
every service except host-networked ones (docker rejects the combination
there). Existing list or dict extra_hosts entries are merged, not
replaced. The bootstrap task passes '<keycloak domain>:host-gateway'
when web-app-keycloak is in the play, so both compose (DinD host) and
swarm (manager node) route the discovery through the local openresty.

Verified live: curl from bbb-net with the mapping reaches the issuer
(200), and the full bbb swarm roundtrip including the OIDC specs passed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 15:34:28 +02:00
kevinveenbirkenbachandClaude Opus 4.8 e6538e4f61 feat(svc-net-tor): Tor onion-service provider (full onion node)
Turn a NAT/edge node into a full Tor v3 onion node: DOMAIN_PRIMARY
becomes the node's .onion, so the whole stack (web, SSO, LDAP, CA)
resolves onion domains and is reachable over Tor without a public IP,
port forwarding, VPS or DynDNS.

- utils/tor_onion.py: offline v3 key mint (address + HiddenServiceDir
  key files), verified against a known key->address test vector.
- cli .../inventory/onion: `init-env` pre-mints the node onion, writes
  INFINITO_DOMAIN(+_RE) into .env and authoritative key files into
  .onion-identity/hs (idempotent). The onion is a deploy-time input;
  no in-deploy random minting (that would not match the pre-set domain).
- roles/svc-net-tor: Tor sidecar (Debian-built, network_mode: host),
  installs the minted keys, torrc with one node HiddenService + a SOCKS
  proxy restricted to loopback/DiD ranges. files/test.sh probes every
  served onion domain over the Tor SOCKS.
- tls: .onion -> TLS off (onion v3 crypto is the transport auth; LE
  cannot sign .onion) + app-level flavor override.
- harness/DNS: coredns resolves the onion zone via INFINITO_DOMAIN;
  skip the clearnet DNS sanity check for onion; raise nginx
  server_names_hash_bucket_size to 128 for long onion server_names.
- test-e2e-playwright: gated browser SOCKS proxy (PLAYWRIGHT_PROXY) +
  navigation/action timeouts so specs reach .onion over Tor and fail
  fast; the onion node points it at the Tor SOCKS.
- docs/requirements/031: full design.

Verified: dashboard + SSO (Keycloak) + 9 apps reachable over Tor
(HTTP 200 through the node hidden service). make test green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 12:38:11 +02:00
kevinveenbirkenbachandClaude Fable 5 65e34746fc feat(lint): forbid role_path in shell/command task bodies
role_path expands to the controller's checkout path while shell/command
strings execute on the (possibly delegated) target host, so
{{ role_path }}/files/x references break on hosts without the checkout.
lookup()/query() expressions and ansible.builtin.script are exempt: they
evaluate on the controller respectively copy the file themselves.

Migrate the one real offender, the sys-service-terminator docker
diagnostics, from a shell wrapper to ansible.builtin.script with the
exports moved into the task environment.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:00:49 +02:00
kevinveenbirkenbachandClaude Fable 5 1371d01a4f fix(service-lookup): resolve out-of-play keys via the static role registry
The sys-service-loader iterates the role-wide static service order, so it
asks the service lookup about keys (e.g. the css alias of cdn) that the
play-local registry built from applications does not contain. The lookup
raised and killed the bootstrap play on every distro.

Fall back to the registry read from roles_dir, the same source the
loader iterates, so every loader-visible key resolves with its provider
role and play-computed flags (false when no consumer is deployed).
Canonical aliases aggregate their primary key's consumer flags. A term
unknown to both registries still raises.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:00:49 +02:00
kevinveenbirkenbachandClaude Fable 5 9c84ae2240 feat(engine): shared-aware container field + SPOT container names via services.<x>.name
Container-name vars derived their value inconsistently (hard-coded literals, entity_name composites) instead of the role-owned SPOT. Add a lint requiring every *_CONTAINER var to resolve through lookup('config', app, 'services.<x>.name'), the bare entity_name form, or the engine lookup; give each multi-container worker a name: in meta/services.yml (with realistic resource limits) and switch its var to the lookup. Extend the engine lookup with a shared-aware 'container' field (central provider container when shared, embedded <entity>-<engine> otherwise) and gate zammad's embedded memcached sidecar on not-shared so its container_name never collides with the central memcached.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:00:49 +02:00
kevinveenbirkenbachandClaude Fable 5 ba6ecba64b feat(lint): flag redundant compose_handlers_flush=false + manual flush
sys-svc-compose already flushes compose handlers (and runs post-deploy) when the including role passes compose_handlers_flush: true. Roles that set it false and then immediately hand-roll their own meta: flush_handlers just reimplement that flush. Add an adjacency-scoped lint that catches the back-to-back form (a deferred flush with real work between the include and the flush stays legitimate and keeps false), and collapse the five offenders (checkmk, chess, jellyfin, pihole, semaphore) to compose_handlers_flush: true.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:00:49 +02:00
kevinveenbirkenbachandClaude Fable 5 94e8671d61 refactor(container): route docker ops through the container CLI
community.docker.docker_* module calls in role tasks bypassed the CA-aware container wrapper and split lifecycle handling across two mechanisms. Migrate lifecycle sites (rm/stop/start/run/build, network create) to the container CLI, keep read-only *_info modules with an inline nocheck, and add a lint that rejects new raw module usage. The mariadb recovery flow gains a stale-container pre-remove so an aborted reset stays re-runnable, discourse teardown keeps its graceful stop, and removals guard on 'No such container' instead of masking every failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:00:48 +02:00
kevinveenbirkenbach 2b8edf9011 ruff 2026-07-04 18:24:31 +02:00
kevinveenbirkenbach 9ee4cd0fb9 Merge branch 'main' of github.com:infinito-nexus/core into feature/swarm-nfs
# Conflicts:
#	docs/agents/action/iteration/compose.md
#	roles/svc-db-openldap/vars/main.yml
#	roles/svc-runner/defaults/main.yml
#	roles/svc-runner/tasks/02_cleanup_instance.yml
#	roles/web-app-discourse/meta/server.yml
#	roles/web-app-mailu/tasks/01_core.yml
#	roles/web-app-odoo/tasks/main.yml
2026-07-04 18:16:12 +02:00
kevinveenbirkenbachandClaude Fable 5 63328b488f fix(swarm-tests): carry DOMAIN_PRIMARY into the inventory extras
extend_inventory adds the nfs-server host without host_vars, so the
extras file is its only var source; without the cluster domain it fell
back to the group_vars default and rendered unresolvable
*.infinito.localhost endpoints.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:57 +02:00
kevinveenbirkenbachandClaude Fable 5 f18d909727 fix(service-registry): build the ordered loader list from static role metadata
Per-host rendered applications flip dynamic shared flags with
group_names, so hosts computed divergent ordered lists and Ansible's
result-arrival include registration could deploy consumers before
their engines. The ordered mode now reads the static roles dir; the
per-host when gate in load_service.yml keeps deciding what loads.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:57 +02:00
kevinveenbirkenbachandClaude Fable 5 7d5775ff96 fix(matrix): force compose rendering for the node-local ansible flavor
The ansible flavor deploys matrix as node-local compose even inside a
swarm cluster, but compose_only stripped container_name and friends
because DEPLOYMENT_MODE said swarm, so follow-up exec tasks could not
find the containers. compose_mode_force pins the rendering to compose;
the lookup honors it ahead of the cluster mode.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:57 +02:00
kevinveenbirkenbachandClaude Fable 5 0e644f8599 fix(joomla): stream plugin dir via exec stdin; lint against container cp
container cp needs its source on the node running the docker CLI, so
role_path sources break on swarm workers. The joomla plugin dir now
ships as a controller-side tarball through container exec -i. A new
repository lint forbids docker/container cp in tracked .yml and .sh
files; the six legitimate same-node uses carry justified nochecks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:56 +02:00
kevinveenbirkenbachandClaude Fable 5 15a8d26ea4 fix(keycloak): move kcadm module and helper to global library paths
Role-local library/ and module_utils/ resolve only inside the owning
role's task scope, so kcadm update tasks invoked from other contexts
in the swarm flow could not find keycloak_kcadm_update. ansible.cfg
already declares ./library and ./utils as the global paths; the filter
plugin and unit tests follow the move. Helper narrows its parse guard
to ValueError.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:56 +02:00
kevinveenbirkenbachandClaude Fable 5 de1ac9947b fix(decidim): reach seaweedfs s3 through the public endpoint
Active Storage signs URLs against AWS_ENDPOINT, so it must be the
browser-reachable public endpoint instead of the internal service url.
Compose reaches the canonical s3 api domain via extra_hosts and the
host gateway; swarm resolves it through a new proxy_aliases beacon
field that puts only the canonical domain on the front proxy, keeping
the ambiguous seaweedfs entity alias off shared overlays.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:56 +02:00
kevinveenbirkenbachandClaude Fable 5 30afef6bbe fix(resources): entity fallback, short compose service names + SPOT vars
resource_filter now falls back service_name -> entity_name -> hard
default, so entity-level limits keep applying to per-service compose
entries. Compose service names drop the entity prefix (kix, jitsi,
zammad, bluesky); the docker container_name keeps it via compose_only.
A new lint forbids '<entity>-'-prefixed service names.

Service names and internal ports are declared once: vars/main.yml
holds <ROLE>_<SERVICE>_SERVICE constants (container names derived via
entity_name), meta/services.yml declares ports.internal consumed
through config lookups. sso-proxy upstreams stack-qualify the origin
host in swarm because short aliases are ambiguous on shared overlays.
Stale proxy.service_key entries fixed for erpnext/shopware/nextcloud.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:56 +02:00
kevinveenbirkenbachandClaude Fable 5 eef3ec316c fix(config): missing-key sentinel must not swallow genuine False leaves
get() signalled a missing key in non-strict mode by returning False, so
every genuinely-False leaf was coerced to the caller's default and
lookup('config', ..., true) could never return False. Moodle's LDAP gate
therefore fired in the ldap-disabled variant round and crashed templating
against never-generated openldap provider credentials. Use a unique
_MISSING sentinel for traversal, keep the default semantics for genuinely
absent keys, and make the moodle gate strict since services.ldap.enabled
is always defined.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:56 +02:00
kevinveenbirkenbachandClaude Fable 5 6d7bcdf705 fix(swarm-test): mirror the force-shared DB view in the credential closure
derive_includes applies the same shared:true flip as force_shared_db, so
variant rounds that pin a consumer's DB to shared:false keep the central
provider (and its generated credentials) in the include set the deploy
will actually load. extend_inventory derives its manager-placed groups
from the same variant-aware closure instead of a base-config walk, which
re-added pruned provider groups without credentials. matrix.py passes the
round's variant map to the extend step.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 17:05:55 +02:00
kevinveenbirkenbachandClaude Opus 4.8 2f9a54cea7 fix(deploy): honor variants in include resolution
Inventory include sets ignored the active variant: swarm
derive_includes walked the variant-free base config (dynamic service
flags count as enabled) and the compose CombinedResolver followed
run_after as an include edge. Providers a variant pins to false
(e.g. mobilizon variant 1 sso) were still deployed.

- derive_includes: swap every app listed in INFINITO_APP_VARIANTS to
  its variant config, index 0 included (nextcloud variant 0 disables
  11 partner services)
- CombinedResolver: follow_run_after flag; include resolution passes
  False so run_after stays a pure ordering hint
- drop dead deps.py chain (resolve_deploy_ids_for_*) and unused
  resolve_with_edges; rewiring them would resurrect variant-blind
  run_after-following includes
- inventory.md: document the real include pipeline and
  is_explicit_truth semantics

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 03:21:56 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d4ee9a85ba refactor(compose): align templates to the service_name skeleton
Every compose.yml.j2 now declares services as '{{ service_name }}:' with
its '{% set %}' knobs on two-space lines above the key, includes at four
spaces and if/for tags aligned to the content they wrap. The layout
renders correctly because stack_host_template enables lstrip_blocks for
compose.yml.j2; indent(4, True) in the container base and '{%+' guards
in whitespace-sensitive scalars keep the shared fragments identical
under both render modes. A new lint test enforces the skeleton.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 02:36:58 +02:00
kevinveenbirkenbachandClaude Opus 4.8 11efe2cfdf feat(sys-ca-selfsigned): shared java-with-ca-trust wrapper for JVM roles
One entrypoint wrapper imports the project CA into the JVM truststore for penpot and jenkins: env-driven (CA_TRUST_CERT from sys-svc-compose-ca, set in both modes), fail-open without a cert (production), fail-loud with one; writable cacerts are patched in place, read-only layouts fall back to /tmp + JAVA_TOOL_OPTIONS. Replaces penpot's inline compose command and jenkins' role-local entrypoint, which silently skipped the import when cacerts were not writable. ca_cert_container becomes a plain constant since its only Jinja consumer moved into the wrapper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:21 +02:00
kevinveenbirkenbachandClaude Opus 4.8 af6f2bdcdf feat(filter): ca_trust_paths as the SPOT for the root-CA cert locations
ca_cert_host derives the host cert path from SOFTWARE_DOMAIN; ca_cert_container carries the fixed in-container bind-mount target. CA_ROOT.cert_host and the test fixtures consume the filter; the standalone runtime scripts (compose_ca.py, container.py) cannot import it and are pinned to the SPOT by the unit test instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 3f60a0422f feat(web-app-gitea): classify auth-source idempotency errors via gitea_auth_noop
'login source already exists' was matched inline in both the LDAP and the OIDC setup; the filter is the single marker source and normalizes case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ea33483b15 feat(svc-db-openldap): classify ldapmodify idempotency errors via ldap_modify_noop
The two handler tasks carried the identical four-marker stderr chain inline and matched case-sensitively; the filter is the single marker source and normalizes case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 61b75f7505 feat(web-app-discourse): classify idempotent network errors via docker_network_noop
The five docker network connect/disconnect tolerances were inline stderr substring chains, inconsistent between sites (the handler variants matched case-sensitively without a default). The docker_network_noop filter is the single marker source per operation; disconnect additionally tolerates absent networks/containers, which is the proven swarm failure ('No such container: postgres' when the central DB is a swarm service). Drop the now-orphaned DISCOURSE_ERROR_NOT_CONNECTED.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b1b24b9014 feat(networks): external-network lookup + probe-before-create with driver-drift fail
compute_external_network_roles() + compose_external_networks lookup expose which foreign provider networks a compose project references as external, so the swarm deploy path can pre-create them. network/create.yml probes for an existing network instead of unconditional state:present (the module's delete+recreate on attached networks crashed redeploys) and hard-fails on driver drift instead of silently skipping it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 22:18:20 +02:00
kevinveenbirkenbachandClaude Opus 4.8 1adc01e0ae fix(web-app-keycloak): provision realm users + RBAC groups on the LDAP-disabled path
Without LDAP the fallback only seeded the administrator user and no RBAC groups, so oauth2-proxy's allowed_groups check 403'd every group-gated app (prometheus/seaweedfs, matomo/css downstream) and non-admin logins hit user_not_found (biber). 08_users_realm now seeds every non-reserved user with a usable password (parity with the LDAP path, which federates the same set); new 08b_rbac_groups_realm materialises the /roles/<app>/<role> group tree + memberships directly in the realm: build_realm_rbac_groups mirrors build_ldap_role_entries and restricts members to the users 08 actually seeds (by username attribute); ensure_group_path_members.sh walks the path via kcadm (children endpoint, exact user lookup, idempotent membership PUT) and hard-fails on any missed group or membership instead of masking. Both routines are gated to KEYCLOAK_LDAP_ENABLED false; LDAP-enabled deploys are untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:57:31 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8332357377 fix(email): resolve host to localhost when not external, even under TLS
email.py's environment folds TLS_ENABLED into its "external" base, so a non-external relay (no mailu deployed) under TLS resolved host to the mailu domain while auth/tls/from stayed in localhost mode: an unreachable, inconsistent msmtprc that aborted the sys-ctl-hlth-msmtp unit and the deploy (chess/bridgy-fed/hugo/legal). Gate host on external directly, like tls/auth/from, so a mailu-less deploy targets the loopback postfix that sys-svc-mail-smtp deploys under the same email.external gate. Adds a guarding unit test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:57:30 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9e7f28db20 test(lint): enforce swarm config objects are world-readable + widen 18 roles to 0444
Swarm config objects mount root:root; a 0440 mode is unreadable by a non-root container app user and crashloops the service (jenkins JCasC AccessDeniedException, exit 5). docker compose never hits this since there the file is a host bind-mount inheriting host perms.

New lint test_config_object_world_readable flags type:config modes lacking o+r (type:secret exempt, per-line # nocheck escape). Widen the 22 offending 0440 configs across 18 roles to 0444: adds read only, cannot break a reader, and fixes latent non-root cases (e.g. synapse, pixelfed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:57:30 +02:00
kevinveenbirkenbachandClaude Opus 4.8 96aa7784b6 feat(sys-svc-compose): scale all task timeouts via TIMEOUT_FACTOR lookup
Add a 'timeout' lookup plugin that returns round(base * TIMEOUT_FACTOR)
(group_vars default 1, so value-identical to the old literals). One knob
in group_vars stretches every wait in the stack for a slow uplink:
TIMEOUT_FACTOR: 2 doubles them all.

- plugins/lookup/timeout.py + unit tests (identity, factor scaling,
  string coercion, error paths).
- group_vars/all/17_resource.yml: TIMEOUT_FACTOR: 1.
- Convert all 34 literal 'timeout:' task keywords / module args across 25
  task+handler files to "{{ lookup('timeout', <base>) | int }}".
- tests/lint/repository/yaml/test_timeout_uses_lookup.py enforces the
  lookup in every task/handler timeout (opt-out: # nocheck: timeout-lookup).
- swarm.yml pre-deploy build handler: add the missing timeout (same
  network-build hang vector as compose.yml's Build compose).
- Drop 13 pre-existing narration comments in the touched files (comment
  lint enforces whole-file on any edited file).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 13:57:30 +02:00
Alejandro Roman 48c49b8c7c Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail 2026-07-03 07:26:30 +02:00
kevinveenbirkenbachandClaude Opus 4.8 7fce11867c fix(compose_volumes): drop orphan top-level config/secret when all mounts are gated off
compose_volumes emitted a top-level configs:/secrets: entry for every declared
config/secret, but container_volumes already gates each per-service reference on
its mount's when:. So a config whose only mount was gated off (e.g. LDAP
disabled) became an orphan: a top-level object no service references, whose file:
source points at a file the role never renders (same flag). Under docker stack
deploy that source is read at deploy time and the deploy fails; under compose it
was a harmless unused entry.

Gate the top-level emission on the same mount_when_passes container_volumes uses,
so a top-level config/secret exists iff a service actually references it. To share
the predicate, promote the private _mount_when_passes from container_volumes into
utils/roles/applications/mounts and import it in both filters; the bare except
moves with it (inline noqa), so container_volumes no longer needs its BLE001
per-file-ignore.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 00:26:06 +02:00
kevinveenbirkenbachandClaude Opus 4.8 6c87cbbf2f fix(web-app-yourls): render env as dotenv KEY=value + lint the format
The env file was written in YAML colon syntax; docker compose's lenient parser
tolerated it but docker stack deploy's kvfile reader rejects the colon lines and
aborts the swarm deploy. Convert to canonical KEY=value (dotenv_quote on the two
passwords) and add a repo-wide lint (env-file-dotenv-syntax) so no env.j2
regresses to YAML mapping syntax.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-03 00:26:05 +02:00
Alejandro Roman 500e5fef34 fix playwright tests 2026-07-02 23:08:03 +02:00
kevinveenbirkenbachandClaude Opus 4.8 674d26de3a feat(lint/role-readme): derive Credits author from meta author SPOT
Role README Credits now name galaxy_info.author (plain bold or wrapped
in a Markdown link), matched byte-for-byte against the role's own
meta/main.yml, instead of a hard-coded maintainer name. Regenerate all
245 role README Credits blocks to the new 'Implemented by ...' format.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 16:47:34 +02:00
kevinveenbirkenbach 506da992d7 Merge branch 'main' into feature/web-app-semaphore 2026-07-02 13:10:49 +02:00