Commit Graph
232 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 bab08063f3 feat(plugins): ask for a container-network origin instead of assembling one
A sibling container is reached at http://<entity name>:<internal port>,
and that string was written out per call site: a literal protocol, an
entity_name interpolation for the host, and a lookup('config', ...,
'services.<key>.ports.internal.http') for the port. Nine of them across
seven roles, each needing its own `# nocheck: literal-protocol-lookup`,
because the sibling lint rightly objects to a protocol glued in front of
a lookup - the protocol is supposed to come from wherever the TLS
posture is decided.

A container network has no TLS posture. It speaks plaintext by
construction, which is a property of the network rather than of any
caller, so stating it nine times stated it eight times too often.
lookup('container_url', application_id, service_key) holds all three
facts, and because no literal protocol is left at the call site the
suppression markers go with them.

The port is deliberately the internal one. A sibling that goes out to
the published port leaves the container network and comes back through
the ingress, which the dns-search suffix turns into the public FQDN.
The lookup refuses a service that declares no internal http port rather
than falling back to anything.

test_container_url_lookup.py keeps the shape from coming back: a
protocol, an interpolation mentioning entity_name, a colon, a second
interpolation. Nothing else matches that, and `# nocheck:
container-url-lookup` covers an origin this lookup genuinely cannot
express - a port that is not ports.internal.http, or a host that is not
the application's entity.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-18 02:13:05 +02:00
evangelostsak e62ed8c56f Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-17 09:34:00 +02:00
kevinveenbirkenbachandClaude Opus 5 bf5aa1abd8 Merge branch 'feature/designer' into feature/store-i18n
Two conflicts, both resolved by combining rather than by picking a side.

`roles/web-app-socialhome/tasks/02_design.yml`: designer restructured the pass,
moving the token request and the theme write into the new
`02_design/converge.yml` and adding a stat of the name-state file. HEAD still
carried the converge block inline. Kept designer's stat, dropped HEAD's inline
block: the merged body below the conflict already includes `converge.yml` and
reads `socialhome_name_marker`, which only designer defines, so keeping HEAD's
side would have duplicated the token request and left that variable undefined.
HEAD's `retries: 0` on the token request is deliberately not carried over; the
default of 30 retries covers a service that is not yet accepting connections,
which matters more here than failing fast on bad credentials.

`roles/web-app-fider/files/playwright/test-design.js`: the import line takes
both sides, since the merged body uses designer's `apiGetOnion` and HEAD's
`requireDotenvValue`. The env block keeps HEAD's `requireDotenvValue` form,
because designer's `|| ""` defaults are what
`tests/lint/ansible/.../test_unguarded_env_default.py` exists to reject, and
drops HEAD's module-scope `title` and `logoEnabled`: the merged body defines
`title` locally and reads `DESIGN_LOGO_ENABLED` directly, so both were dead and
would fail `no-unused-vars`.

Checked after resolution: no conflict markers (`git diff --check`), no `nocheck`
line dropped by the merge, and no duplicated YAML key in the seven merged files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-17 00:13:02 +02:00
kevinveenbirkenbach f8311d0780 Merge branch 'main' into feature/network/multi-network-serving 2026-10-16 22:36:07 +02:00
kevinveenbirkenbachandClaude Opus 5.5 69aee158f6 fix(ca_injected): answer a ca_client on the issuer it calls, and let Matrix opt in
Matrix is the only single_mode role: on a multi node it is served on onion
only, so it has no TLS of its own, yet Synapse calls the clearnet Keycloak
issuer over self-signed TLS. ca_injected returned false, Synapse got no CA,
and OIDC discovery failed with CERTIFICATE_VERIFY_FAILED, which Synapse
reports as "504 Timeout waiting for response" and Element as "This login
provider is unavailable right now".

An application with a domain and self-signed TLS of its own still gets the
CA. Otherwise a ca_client service now gets it when the SSO issuer
application resolves TLS enabled and self-signed. The cherry-picked
store-i18n rule answered a domain-less client on the global TLS_MODE; the
issuer's own resolved mode honours per-app overrides on both sides,
answers false on a tor-only node, and true when TLS_ENABLED is off but the
issuer keeps an override, or when the client serves letsencrypt itself.
Matrix declares ca_client on its matrix service.

A later merge of feature/store-i18n conflicts in ca_injected.py and its
test; keep this side.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 22:35:41 +02:00
kevinveenbirkenbachandClaude Opus 5 df0e98f146 fix(ca): pass the CA override to the deployment that was granted it
sys-svc-compose writes compose.ca.override.yml for exactly lookup('ca_injected', application_id), which since 1254e27c2d also answers true for a service that declares ca_client and only calls the self-signed endpoint. compose_file_args rebuilt that predicate instead of asking it, and its copy still demanded a domain, so for a domainless ca_client the file was written and never reached --compose-file. Swarm takes its file list from that lookup; compose finds the file on disk by itself, which is why only the swarm jobs of run 36064589578 failed while the compose job passed.

The broker's container in that run carries no SSL_CERT_FILE and no CA mount, while openwebui, same job and same node, carries seven CA env vars and four binds. Every outbound call to Keycloak then fails the handshake, which the CLI test reports as 503 and Open WebUI relays as HTTP 400.

Two corrections in ca_injected itself. Its templar.template() call was dead: as_str strips the TrustedAsTemplate tag and ansible-core 2.19+ refuses to render an untrusted string, so the line could not resolve what it advertised, and tls.py and cert.py never template TLS_MODE either. And the ca_client branch short-circuited on the global TLS_MODE, skipping resolve_mode's per-app server.tls.mode override; it now applies only where there is no domain and so no per-app answer to get.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-16 22:02:57 +02:00
kevinveenbirkenbachandClaude Opus 5 b2e594073e fix(ca): give the CA to a service that only calls the self-signed endpoint
ca_injected asked three questions, the first being whether the application
has a domain. That decides whether it is SERVED over the self-signed CA. An
application that only CALLS one has no domain of its own, so the predicate
answered false, sys-svc-compose-ca skipped its stack, and the container ran
with an empty trust store.

svc-ai-agent-broker is such a caller: it authorizes every request against
Keycloak group paths over the public HTTPS base. Without the CA every
outbound call died in the TLS handshake, the broker answered 503 "access
check unavailable", Open WebUI turned that into HTTP 400, and the spec
failed on a model it had just listed. Its container was the only one in the
rescue artifact without SSL_CERT_FILE, while litellm and open-webui carried
it.

A service may now declare ca_client: true, and the predicate then answers on
the deployment's TLS mode alone, because the domain question is meaningless
for a client. The three served-side conditions are untouched, so no
application that already received the CA stops receiving it.

The alternative of pointing the broker at an internal Keycloak URL was
rejected: the role declares no keycloak service, so the URL would render
without a route.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-16 22:02:56 +02:00
kevinveenbirkenbachandClaude Opus 5.5 8e7dd508c9 style(tests): apply the formatter to the network_served tests
The autoformat step of the gate wrapped the long assertions that
660ad2ff16 committed unformatted. No behaviour change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 21:40:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d36e203171 fix(cache): render the applications carrier without the caller's application_id
`lookup('tls', ...)` aligns a cross-app domain to the consumer named by
`application_id` (utils/tls_common.py `align_domain_to_consumer`): a
clearnet-primary consumer gets the clearnet sibling of an onion-primary
target. The carrier key does not cover `application_id`, so a park that
renders inside a role hands that role's alignment to every later role
whose key matches.

The constructor and `tasks/utils/setup/role.yml` park at play scope,
where `application_id` is undefined. The re-park in
`roles/sys-token-store/tasks/write.yml` (67f6c8d7e8) and the nested-load
re-park render inside the caller: `svc-prx-openresty` at the preload
loader, `web-app-stalwart` or a provider role on the NFS host, the web
app itself under `sys-stk-front-base`. In a tor deploy that turned onion
targets into clearnet domains for onion-primary roles served from such a
park, and baked the caller's group into Nextcloud's sociallogin
`groupMapping`.

Drop `application_id` from the render's variables and from the
templar's for the duration of the render, so every park carries the
play-scope result the constructor parks have always carried. Rename the
nested re-park task so its log lines do not read as a role boundary.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 21:25:43 +02:00
evangelostsak 6f24461720 refactor: drop rationale and narration from tests/unit/python/plugins/lookup/test_mesh_ranges.py 2026-10-16 19:10:01 +02:00
evangelostsak d480c2785e refactor: drop rationale and narration from tests/unit/python/plugins/lookup/test_mesh_address.py 2026-10-16 19:10:01 +02:00
Prageeth Panicker 5e39433ed1 Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role 2026-10-16 11:40:30 +00:00
kevinveenbirkenbachandClaude Opus 5.5 660ad2ff16 fix(web-app-seaweedfs): keep the frontend on every node that serves clearnet
The frontend and its sso, logout, dashboard and prometheus consumers were
gated on 'svc-net-tor' not in group_names, written when a Tor node served
onion only. On a multi node that removed the clearnet filer vhost, so the
storage specs of every SeaweedFS consumer failed on a clearnet pick with
ERR_SSL_UNRECOGNIZED_NAME_ALERT on filer.<domain>.

lookup('network_served', <network>, *role_ids) answers whether the node's
resolved NETWORK_MODE serves a network and every named role is deployed.
The gates use it instead of restating the mode resolution and the provider
check, so the frontend now drops only on a tor-only node. is_explicit_truth
registers the lookup in GATED_LOOKUPS so the static closure still reads
those flags as dynamic, and the dynamic-flag and universal-logout checks ask
is_explicit_truth instead of matching "in group_names" themselves.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 13:35:29 +02:00
kevinveenbirkenbach 338a2ffa7f Merge branch 'feature/designer' into feature/store-i18n 2026-10-16 13:19:15 +02:00
kevinveenbirkenbachandClaude Opus 5 adacac7f29 fix(mcp): present the account's login in basic auth, not the users-dict key
The WordPress MCP endpoint answered 401 to every authenticated call, and
web-app-hermes and web-app-openclaw failed their CLI tests as cascades of the
same refusal. The WordPress container's own Apache log settles it: 121 POST,
100 GET and 21 HEAD on the MCP path, each with %u = mcp-web-app-wordpress and
status 401. A populated %u rules out a stripped Authorization header, and the
body is infinito_mcp_unauthorized, which only the mu-plugin's rest_pre_dispatch
guard emits - a wrong password for an existing user would surface WP core's
incorrect_password instead. So WordPress never resolved a user, because the
deploy sent the users-dict key while the login the deploy creates is
mcp-wordpress.

7916c0177e introduced the split when it capped usernames, deliberately keeping
the key because meta/mcp.yml's owner and the token store wire to it.
web-app-wordpress is the only role with auth: basic_auth - every other MCP
provider uses bearer, where the owner never travels - which is why only this
one broke.

resolve_credential now also returns the login, mcp_servers emits it on every
discovered entry, and the filter encodes login:token. It raises when no login
resolves rather than authenticating as the empty user, so the failure is loud
instead of another 401. The new mcp_client_login lookup shares
resolve_provider_credential with mcp_client_bearer so a second resolver cannot
drift from the first, and a test pins the two to the same function object.
test.env.j2 is the one place that hand-builds the server dict instead of
reading MCP_DISCOVERED_SERVERS, so it passes login as well.

The lint closes the class: auth: basic_auth requires credential.owner to be a
key some role's meta/users.yml declares. Without it a future basic-auth
provider sourced from credentials would hit the same fallback silently. It
imports BASIC_AUTHS from the filter so the rule and the renderer cannot drift
onto different spellings.

Tool names were verified against the adapter version the addon actually
installs, v0.7.0, not the v0.5.0 recorded in meta/mcp.yml: the sanitizer turns
the three registered abilities into infinito-get-post,
infinito-list-categories and infinito-search-posts, matching the rendered
allowlist exactly. Adding login to a discovered entry changes nothing for the
five consumers - three of them do not list basic_auth in supported_auths and
never see this server at all, and the other two read named keys only.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-16 12:37:30 +02:00
kevinveenbirkenbachandClaude Opus 5.5 67f6c8d7e8 fix(cache): keep the applications carrier valid across token store writes
Since 778c18fb23 the carrier key holds the tokens file's mtime and size.
sys-token-store changed that stat on every inclusion (00_core.yml
touched the file) and on every persisted token, and nothing re-parked a
carrier afterwards. Every later task's fresh worker then rendered the
applications SPOT cold, 30 to 80 s each. On main fea6f31d9f (run
38812964308) seven heavy jobs logged 194 to 287 such tasks and ran out
of the 20400 s budget; the same jobs before 778c18fb23 logged 65 to 83,
all of them real waits.

- 00_core.yml touches the store with modification_time and access_time
  preserved, so it only creates a missing file. In swarm the controller
  sees the store over NFS with attribute caching, and the touch alone
  made the second pass go cold without any write.
- write.yml re-parks the carrier when the persist or the controller
  mirror changed the store. Both run once, so every host that ran the
  include gets the same verdict.
- tasks/utils/setup/role.yml re-parks after every role on every host,
  so hosts that did not run the writing role recover too. Where the
  key still matches this serves the carried payload; measured 0.1 s per
  role in a local compose deploy.
- The tokens file enters the key as the SHA-256 of its content,
  memoised per inode, mtime and size, so a touch no longer changes the
  key. A stat that still matches serves the memo (1.67 us against
  1.41 us for the old stat signature), and a stale stat keeps key and
  YAML cache on the same old content instead of pairing new content
  with an old key.
- applications_carrier opens the tokens file before it computes the
  key, so on the NFS-mounted controller the parked key reflects the
  revalidated attributes rather than a cached stat that changes a
  minute later.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 11:11:47 +02:00
Prageeth Panicker 8e4ba0f8a9 Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role 2026-10-15 21:58:47 +00:00
kevinveenbirkenbachandClaude Opus 5 5b69b5c7c6 Merge branch 'feature/designer' into feature/store-i18n
Six conflicts, all from git pairing store-i18n's renamed roles with their designer originals.

web-app-meta/meta/services.yml and web-app-sphinx/vars/main.yml: store-i18n renamed web-app-sphinx to web-app-meta and dropped its custom build, so the role keeps the prebuilt ghcr.io/infinito-nexus/meta image and the old sphinx role stays deleted.

svc-ai-tika/templates/compose.yml.j2: a bogus rename pairing dropped sphinx build lines into the tika service. Rejected; the template is byte-identical to its pre-merge state again.

web-app-docs/templates/Dockerfile.j2: the designer file renders sphinx under the design title and reads SPHINX_IMAGE, SPHINX_OUTPUT_DIR and SPHINX_EXEC_DIR, none of which exist in web-app-docs, which builds from files/Dockerfile. Rejected.

web-app-docs/templates/playwright.env.j2 and files/playwright/test-design.js: both sides add independently, so both are kept. Of the designer's three new env reads only designTitle survives; iconUrl and faviconUrl are shadowed inside the test by stricter requireDotenvValue reads and were dead at module scope.

plugins/lookup/email.py merges clean toward designer, which is intended rather than a loss: 08ef66d8de (2026-10-08) replaces the submission_via_relay port-25 rule of 233f15f3e0 (2026-07-01) with per-bot app passwords under SSO, and carries the matching 04_manage_user/app_password tasks.

Checked after resolving: no duplicate YAML keys across the 17 merged YAML files, and no nocheck line dropped above a renamed key.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-15 22:21:50 +02:00
kevinveenbirkenbachandClaude Opus 5 1e61debad2 fix(objstore): sign funkwhale's media URLs against the host the browser dials
funkwhale has two S3 surfaces sharing one private bucket. Audio goes out as X-Accel-Redirect and the front nginx fetches it server-side, but attachments (instance logo, avatars, covers) land in the API JSON as absolute URLs the BROWSER dials. AWS_S3_ENDPOINT_URL drives the boto3 client, so with the internal endpoint django-storages signed URLs for http://seaweedfs:8333, a host no browser can reach.

Point the client at the new bucket-less public_endpoint key. Measured against the live gateway: a URL signed for seaweedfs:8333 and sent to the public host answers 403 SignatureDoesNotMatch, while one signed for the public host reaches bucket lookup — query-string SigV4 binds the Host, so the signing host and the fetch host must be the same string. AWS_S3_CUSTOM_DOMAIN cannot serve this: storages/backends/s3.py:689 returns UNSIGNED without a CloudFront signer, and the bucket also holds the music library that music/views.py exists to gate.

AWS_CA_BUNDLE follows, because the public endpoint is TLS on the local CA and botocore reads its trust store only from that variable (configprovider.py:59) — sys-svc-compose-ca injects SSL_CERT_FILE, REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE and NODE_EXTRA_CA_CERTS, never this one. It takes the bundle rather than the bare root cert, since AWS_CA_BUNDLE replaces the default store.

AWS_QUERYSTRING_AUTH=True fixes a pre-existing break: upstream defaults it to `not PROXY_MEDIA`, which produced unsigned URLs against a private bucket. The new front nginx.conf.j2 serves /_protected/media, which the stock template resolves with `alias ${MEDIA_ROOT}/$1;` and therefore 404s for every S3-backed listen.

web-svc-seaweedfs keeps no onion vhost. Its tor-contract reason now states the mechanical cause instead of the old claim that the role is never a Tor-browser surface, which decidim and funkwhale have made false: the node hidden service publishes http only while sys-front-tls keys listen/ssl on application_id, so an onion vhost would listen on 443 against a port-80 hidden service. Attachment fetches therefore still fail on an onion pick, and the spec is left to fail there rather than record the miss and pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-15 22:14:38 +02:00
evangelostsak 1c5ba164f2 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard
No conflicts across 121 incoming commits. The one file both sides changed is
tasks/stages/01_constructor.yml: upstream parks an applications carrier for
the swarm bootstrap roles after the add_host tasks, and keys the carrier on
the tokens file as well. This branch's mesh bootstrap opens and drops its own
block-scoped carrier before that point, so both keep working as written.

Upstream also brings ruff 0.17.0, distro-derived env keys, a scoped act outer
teardown, the stalwart relay rule, the onion apex redirect and image and
version updates; ruff 0.17 reformats nothing on this branch.
2026-10-15 21:27:11 +02:00
kevinveenbirkenbachandClaude Opus 5.5 1ece531adb fix(lookup): keep the onion apex redirect for a homepage with an onion sibling
PR #932 emits the node onion apex only for an onion DOMAIN_HOMEPAGE. On a multi node the homepage is the clearnet dashboard on purpose and the redirect template sends the apex to its onion sibling, so the merge dropped the apex redirect there. The apex is now kept when the homepage is an onion host or network_siblings lists an onion sibling this node serves; a target without one, such as a software domain off the primary, still gets none. The #932 test reads the primary domain instead of writing it out.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 17:27:30 +02:00
kevinveenbirkenbach 1da2cef95e Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving
# Conflicts:
#	plugins/lookup/email.py
#	tests/unit/python/roles/web-app-prometheus/test_blackbox_targets.py
2026-10-15 16:45:18 +02:00
kevinveenbirkenbachandClaude Opus 5.5 4397f4f57f fix(lookup): serve assets from the CDN sibling of the vhost's network
The asset host took the canonical CDN domain and only guessed its scheme from the suffix, so pages on the onion sibling loaded their libraries from the clearnet CDN, which their CSP and Tor reach do not allow. It now takes web-svc-cdn's url.base, which follows the network of the vhost being rendered; without the internal CDN it stays jsDelivr.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 16:35:09 +02:00
kevinveenbirkenbachandClaude Opus 5.5 bb1394517c fix(csp): let onion pages frame the single SSO issuer
Keycloak runs once on clearnet, yet onion pages such as the dashboard embed it for the silent SSO check, and its frame-ancestors listed only clearnet sources, so the browser refused the frame. On the issuer's clearnet vhost each translatable ancestor now also appears on the node onion. Whether a token moves to the node onion is decided by its host, so the issuer keeps its one URL on onion vhosts wherever it is listed, not only through the logout source.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 16:29:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 e967c3017d test(lookup): keep the onion apex redirect to a clearnet homepage on a multi node
PR #932 emits the node onion apex only when DOMAIN_HOMEPAGE ends in .onion. On a multi node the homepage is the clearnet dashboard on purpose and the redirect template maps the target to its onion sibling, so merging main would drop the apex redirect without a textual conflict. This test goes red on that merge; relax the guard to a target that is onion or has an onion sibling rather than weakening the test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 16:25:26 +02:00
kevinveenbirkenbachandClaude Opus 5.5 5a202218c9 feat(web-app-keycloak): run one oauth2-proxy per network a gated application is served on
One oauth2-proxy applies one cookie-secure flag to every host it answers, so a secure clearnet instance drops the CSRF cookie of its plaintext onion sibling and the login loops. proxy_instances now yields one instance per network with its own compose service, sso or sso_onion host port and config file; each config takes cookie_secure, redirect_url and cookie_domains from its own network, and every vhost routes /oauth2/ to the instance of its network. The 29 gated roles gain a sso_onion port, and Bluesky heals the bind mount of every instance.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 15:57:38 +02:00
kevinveenbirkenbach 06a74e7658 Merge pull request #869 from infinito-nexus/fix/web-app-stalwart/relay-rule/793
fix(web-app-stalwart): authenticate bots with an app password under SSO
2026-10-15 15:42:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 08fba6b3ef feat(networks): look up a same-node URL in the network a vhost or consumer needs
network_url moves a URL into the network of the vhost being rendered, network_urls lists it once per network sibling with that sibling's scheme, and network_redirects fans a redirect source out to its siblings while the target stays canonical. Consumers that hard-code the canonical origin can now emit the sibling a request actually arrives on.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 13:21:23 +02:00
kevinveenbirkenbachandClaude Opus 5.5 857639db2b fix(test-e2e-playwright): bypass the Tor proxy for every clearnet host the node serves
The onion pick only bypassed the SSO issuer, so a clearnet CDN, asset or dashboard host an onion page loads went through Tor, which the lab network cannot exit. browser_proxy_bypass derives the list from the deployed applications' domains plus the issuer; it is empty on a clearnet pick and on a tor-only node. Specs that passed their own proxy to newContext dropped the bypass and now inherit use.proxy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 12:44:38 +02:00
evangelostsak 01e8c1a75f test(lookup): mesh_address returns only the host's own primary address 2026-10-14 15:03:41 +02:00
kevinveenbirkenbachandClaude Opus 5.5 45732329b6 fix(lookup): keep the onion apex from redirecting to a clearnet homepage
On a Tor node current_play_redirect_domains schedules the bare node
onion as a redirect source targeting DOMAIN_HOMEPAGE. Without
web-app-dashboard, DOMAIN_HOMEPAGE falls back to SOFTWARE_DOMAIN
(infinito.nexus), so web-opt-rdr-domains rendered a 301 from
http://<node>.onion/ to http://infinito.nexus/: an onion visitor sent
to a clearnet host over plain HTTP.

The onion apex entry is now added only when DOMAIN_HOMEPAGE is an
onion host. Otherwise the onion apex gets no redirect and the default
server answers.

Closes #924

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 13:35:29 +02:00
evangelostsak 762513ec32 test(lookup): hold mesh_ranges to a rendered pool
The declared template must come back rendered, and one that stays a template
must fail rather than reach the export list.
2026-10-14 11:23:23 +02:00
Prageeth Panicker 06f0b42f12 Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role 2026-10-14 09:00:39 +00:00
kevinveenbirkenbachandClaude Opus 5 7c13e96c96 Merge branch 'main' into feature/store-i18n
Twelve files conflicted; most resolutions keep both sides rather than pick
one.

cli env.py and compose.py: both branches extend the same spots. The tools
override is appended before write_override, because that call filters the
mirror override to the compose files it is handed. compose.py keeps this
branch's CACHE_ROLE_SHELL_DIR, since the cache scripts moved into a role and
main's literal scripts/docker/cache path is their old home, and takes main's
mirrored init image alongside it.

utils/docker/image/discovery.py: this branch's version_variants expansion
plus main's whole runner-image block, which runner_mirror imports.

sys-svc-compose network create: main's side wins on substance and this
branch's on structure. main carries eda6249d7b, the fix for a probe that
matched an id prefix and let a sibling shadow the exact name; this branch had
refactored the task into create/overlay.yml and create/bridge.yml but brought
back the docker_network_info probe that fix replaced. The subfiles stay and
now probe through the exact-name lookup. main's half also read
SWARM_CLUSTER_AVAILABLE, which this branch removed in favour of
IS_SWARM_MODE, as the merge base confirms, so the gates follow this branch.
The lookup's import follows this branch's get_entity_name to entity_name
rename.

web-app-pixelfed: the custom entrypoints stay removed, since the new image
brings its own and the variables behind them no longer exist, and main's
host_alias=true comes along.

web-app-suitecrm: main's session.yaml and its config/packages path, because
only that version wires framework.session and only that directory is read by
Symfony.

web-app-odoo: main's onionOptions import is required by the test.use below
it; this branch's test-seaweedfs registration stays.

Four defects surfaced that no conflict marker shows.

The auto-merge of web-app-suitecrm/vars/main.yml duplicated
SUITECRM_SESSION_CONFIG_HOST, which ansible-lint rejects. Removing it exposed
the second: SUITECRM_SESSION_CONFIG_DOCKER still named config/session.yaml
while the mount had moved to config/packages, so the freshness comparison in
tasks/01_manager_ops.yml would have tested a path nothing writes.

svc-db-elasticsearch carried environment: on two include_tasks, which Ansible
accepts on include_role only. ELASTIC_PW now sits on the five tasks that read
it, which is also the tighter scope for a credential.

The openclaw templates render NETWORK_TRUSTED_PROXY_CIDRS while their test
renderer passes only its explicit kwargs, so seven tests raised Undefined.
That predates the merge: the template line is on this branch and not on main,
and both sides share the same test file.

test_runner_mirror neutralises four environment variables under clear=False
and missed three, so the gpu and tools overrides leaked in from the container
and the assertion on the layered list broke. It now neutralises all seven.

make quality-high passes both phases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-13 10:57:22 +02:00
kevinveenbirkenbachandClaude Opus 5.5 eefa14c125 Merge PR #414 (feature/web-app-wazuh/add-role) into feature/network/multi-network-serving
Brings web-app-wazuh to the reviewed state of #414. This branch was cut
from an earlier #414 head, which has since been rebased and gained its
review fixes, so both sides added the role and every role file conflicted
as add/add.

Conflicts, decided per file:

- roles/web-app-wazuh: #414's side. It is the reviewed version of the
  same role, and this branch's own Wazuh changes are already part of it
  (drop the Tor dual-stack workarounds, drop the first commit's
  leftovers, ports 8096/8097, heap via service_memory_mb).
- roles/web-app-wazuh/templates/opensearch_dashboards.yml.j2: #414's file
  with this branch's trust_dynamic_headers instead of base_redirect_url,
  so the OIDC redirect follows the network sibling the request came in
  on rather than pinning the clearnet URL.
- plugins/filter/bcrypt_hash.py and its test: #414's stable salt, which
  the role's vars now pass.
- test-e2e-playwright keycloak.js: #414's leave-group helper, which fails
  loudly on a missing user or group instead of returning silently.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 01:43:38 +02:00
evangelostsak 75fd3e9433 Merge remote-tracking branch 'upstream/main' into feature/svc-net-wireguard 2026-10-13 01:08:20 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7fa3400cc6 Merge remote-tracking branch 'origin/main' into feature/network/multi-network-serving
Conflicts resolved by keeping both sides' intent:

- call-test-deploy.yml: main's re-enabled deploy concurrency group, keyed
  on the network axis instead of the removed tor axis.
- sys-svc-proxy: main's shared headers/forwarded.conf.j2 now carries the
  branch's per-vhost semantics (X-Forwarded-Host, TLS port and state read
  from the vhost domain), so html, ws, upload and the Prometheus probes
  all answer per network sibling.
- plugins/lookup/email.py: main's provider-agnostic mail lookup asks the
  network registry instead of testing the .onion suffix, as the branch
  does everywhere; container_extra_hosts.py likewise. The provider host's
  NETWORK_MODE now decides whether it is served onion-first, since a Tor
  node defaults to multi and keeps clearnet canonical.
- workspace common.sh: main's INFINITO_DOMAIN fallback without the
  duplicated no-op line.
- README.md and requirement 035: both sides' rows (Wazuh plus main's
  Pi-hole and Stalwart entries); adapter_candidate count 39.

Adapted to main where the branch's rules now reach main's code:

- six tests from main read the primary domain from default.env instead
  of writing it out, as the hardcoded-domain lint requires;
- the chutney worker-check test passes ONION_SUFFIX from the network
  registry, which the branch's svc-net-tor test script requires;
- web-app-wazuh moves its local http ports to 8096/8097 (8094/8095 now
  belong to web-app-stalwart) and sizes the indexer heap through
  service_memory_mb, the same fixes #414 carries;
- web-app-wazuh's README cosmos follows Stalwart as the mail provider.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 00:14:42 +02:00
kevinveenbirkenbachandClaude Opus 5.5 24d7872fde Merge remote-tracking branch 'origin/main' into HEAD
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:54:31 +02:00
kevinveenbirkenbachandClaude Opus 5.5 eda6249d7b fix(sys-svc-compose): probe provider networks by their exact name
The network probe used community.docker.docker_network_info, whose
get_network() scans GET /networks and returns the first entry whose Name
equals the requested name OR whose Id starts with it, in a list order that
changes between calls. In main run 38038541631 (Zammad#1, swarm) the
central redis overlay got the ID n8nwzb1q39m3u4k7awwwmt6yg, the probe for
the not yet created provider network "n8n" returned it as existing, the
create step was skipped, docker stack deploy attached hermes to the redis
overlay as "n8n", and the oauth2-proxy then reached the password-protected
central redis by its bare host name: "NOAUTH Authentication required".
The same scan can also return the ID-prefix twin of a network that does
exist, so comparing the returned name alone would turn that network into
"missing" and fail the create with "already exists".

Both probes now run `container network ls --filter name=<entity> --format
'{{json .}}'`, whose name filter never matches an ID. The new lookup
network_driver picks the entry named exactly after the entity and returns
its driver, '' when there is none; the drift checks, their messages and the
create steps all read that one entry. No docker module is left in the file,
so its two docker-module-uses-container-cli suppressions go too.

Verified: on the host daemon GET /networks/<id prefix> returns a network of
another name and docker network ls --filter name=<id prefix> returns
nothing. In ansible-playbook against real docker network ls output the
lookup returned the exact network's driver in 20 of 20 probes beside an
ID-prefix twin, an empty driver (create path) for a missing network and for
a name only contained in a sibling, and the drift path for a foreign
driver. Unit tests cover the same cases; with a substring match instead of
the exact name the two sibling cases fail.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 18:35:22 +02:00
kevinveenbirkenbachandClaude Opus 5 c7238adb0a fix(web-app-keycloak): absorb a rotated admin password instead of locking out
Keycloak stores the permanent admin's password in its own database. When the
inventory value rotates, the next deploy renders a credential that
authenticates against nothing, and the role cannot reach the instance it just
restarted. The bootstrap account is no way back either: Keycloak writes
KC_BOOTSTRAP_ADMIN_* once, on an empty database, and upstream states it is
meant to be removed once a permanent admin exists.

The role now keeps the credential its predecessor used. Before sys-stk-full
re-renders the environment file with force: true, utils/capture_previous_admin.yml
reads KEYCLOAK_PERMANENT_ADMIN_PASSWORD out of the copy still on disk. When the
login after the restart is refused, utils/converge_previous_admin.yml signs in
with that captured value and moves the account onto the newly rendered one,
which the restarted container already carries in its environment. The bootstrap
fallback stays what its name says: the path for an instance that has never been
deployed. A host with no previous file captures nothing and goes straight there.

Reading the file back needs the inverse of dotenv_quote, which did not exist.
dotenv_unquote switches on the same DEPLOYMENT_MODE as the forward direction, so
a value written in compose is unescaped and a value written in swarm is taken
verbatim; dotenv_value pulls one assignment's raw right-hand side out of the
rendered text. The password never reaches argv: the convergence login passes it
through container exec -e plus the task's environment:.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-12 16:56:51 +02:00
kevinveenbirkenbachandClaude Opus 5 9e7c62a119 fix(network): let every proxied role reach its object store directly
A role that exports an outbound proxy for one purpose applies it to every
HTTP call the app makes, so an internal dependency reached by name is proxied
too. web-app-pixelfed exported the Tor SOCKS proxy container-wide for its
onion OIDC issuer and its S3 client then failed with

    cURL error 97: cannot complete SOCKS5 connection to seaweedfs

for six minutes per attempt, until sixty retries had consumed the whole
suite budget and the job was killed. web-app-espocrm ships a byte-identical
proxy block and configures its S3 endpoint from the same lookup.
web-app-nextcloud sets the proxy globally in proxy.config.php and listed two
internal hosts in proxyexclude, neither of them the object store it reaches
by hostname.

9d91fbea83 had tried to retire the per-app proxies in favour of onion routing
over DNS, and 20ddda8c1c restored them for the OIDC issuer, so a broad proxy
is the current design and the exemption list is load-bearing.

lookup('proxy_excluded_hosts', application_id) is now that list: the loopback
forms plus the object store host when the role stores objects. It is a lookup
and not a group_vars value because no group_vars file resolves
application_id, so a global would have depended on which role happened to be
running. Nextcloud keeps its hand-written list of public hosts; only the
internal half is derived.

The lint gained a second rule for the class rather than the symptom: the
nextcloud list named no loopback literal at all, so the literal rule could
never have fired on it. A variable whose name marks it a proxy exclusion list
must render the lookup, unless it only composes other exclusion variables, and
a ..._PUBLIC_... list is exempt because it names no internal service.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-09 17:12:24 +02:00
evangelostsak 5a83785b53 Merge remote-tracking branch 'origin/main' into feature/svc-net-wireguard 2026-10-09 12:00:07 +02:00
Prageeth Panicker f5b552a667 Merge remote-tracking branch 'origin/main' into feature/web-app-wazuh/add-role 2026-10-09 08:34:37 +00:00
kevinveenbirkenbach 799b835799 Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/relay-rule/793 2026-10-09 00:53:26 +02:00
kevinveenbirkenbachandClaude Opus 5.5 64677d4790 chore: regenerate role docs and apply autoformat after #881 and #873
`make quality-high` rewrote these files on every branch: the SuiteCRM
README and the root role table lacked the Redis service #881 added, the
session path test was not formatted, and an OpenTalk secret description
from #873 was not wrapped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 22:27:12 +02:00
kevinveenbirkenbachandClaude Opus 5 fde90cd413 style(tests): wrap the redis session path call the merge brought in
Autoformat output on the test PR #881 added: the redis_session_path call
exceeded the line width, so its arguments move onto their own line. The
token sequence is unchanged, so the assertion still proves what it did
before, that a password carrying &, =, ?, # and a space survives the
connection URL intact instead of splitting it into extra parameters.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-08 20:05:22 +02:00
kevinveenbirkenbach cb74bb73ec Merge branch 'main' into feature/store-i18n 2026-10-08 19:52:53 +02:00
kevinveenbirkenbach 3874f66ac3 Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/relay-rule/793 2026-10-08 19:38:56 +02:00
kevinveenbirkenbach 6e9b4e23dc Merge remote-tracking branch 'origin/main' into fix/web-app-stalwart/relay-rule/793 2026-10-08 19:27:27 +02:00
kevinveenbirkenbach badc2583eb Merge remote-tracking branch 'origin/main' into fix/web-app-suitecrm/redis-sessions/879 2026-10-08 19:25:24 +02:00