Commit Graph
536 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 cfb2f9c70b perf(web-opt-rdr-domains): check every certificate in one task and write every file in one call
Each redirect source went through the full TLS layer on its own: cert check,
CA environment cleanup, re-check, HTTP handler and vhost template, about
nine seconds per source. echoserver has 85 sources, so the role took
thirteen minutes of every deploy.

The role now resolves the tls state and renders both files of every source
on the controller, finds the certificates of all sources in one find, and
includes sys-front-tls only for a source whose certificate is missing, so
issuance still works in every mode. library/stack_host_contents.py writes
all files in one module call, atomically and only where the content
differs, and keeps one file per source under http/ and https/. Its action
plugin carries the IS_STACK_HOST gate like stack_host_template.

The name differs from the stack_host_files module of feature/store-i18n on
purpose: that one syncs a single directory of YAML documents and prunes
what it does not write, which would delete every other vhost here.

HTTPS_REDIRECT_ENABLED is gone: nothing set it, it was true everywhere,
and the redirect role could no longer see the sys-front-tls default it
lived in. Whether a domain gets its HTTP to HTTPS handler now follows from
lookup('tls', domain, 'enabled') alone, in sys-front-tls and here.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-23 16:35:45 +02:00
kevinveenbirkenbachandClaude Opus 5.5 285ce910d2 feat(inventory): reject unread keys and a missing Cloudflare token before the deploy
The Keycloak stage on echoserver ran ten minutes before sys-svc-dns stopped
it with "API.cloudflare.api_token must be set": the inventory still kept the
token under the flat name CLOUDFLARE_API_TOKEN, which nothing reads any
more. The validator that runs before every deploy now catches that class.

- api.py reports a host that selects DNS_PROVIDER: cloudflare without
  API.cloudflare.api_token, with group_vars/all merged under the host_vars
- variables.py reports every top-level key of host_vars and group_vars that
  core does not read: not declared in group_vars or a role's defaults or
  vars, not tested with "is defined" or "| default(", not read by name in a
  plugin, and not a SYSTEM_EMAIL_<KEY> the email lookup builds; a flat
  credential name such as CLOUDFLARE_API_TOKEN names its API.<provider>.<key>
  path; ansible_* keys are exempt

A key that is read in a way the scan cannot see takes
"# nocheck: inventory-variable", documented in suppression.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-23 16:18:12 +02:00
kevinveenbirkenbachandClaude Opus 5.5 c728b4427b fix(tls): resolve a redirect source through its target
With AUTO_BUILD_ALIASES enabled, current_play_redirect_domains adds
<entity>.<DOMAIN_PRIMARY> as a redirect source for every application whose
canonical domain differs. web-opt-rdr-domains then asks the tls and cert
lookups about each source, but resolve_term only knew the domains mapping
and applications[*].domains, so the Keycloak stage on echoserver stopped
with "tls: domain 'funkwhale.infinito.nexus' not found".

redirect_targets() turns CURRENT_PLAY_REDIRECT_DOMAINS into a source to
target map, and resolve_term falls back to it, one hop, only where it
raised before, so every term that resolved keeps its answer. The tls and
cert lookups pass the map.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-23 16:08:18 +02:00
kevinveenbirkenbachandClaude Opus 5.5 a32f241051 fix(sys-svc-certs): link each domain to the certbundle lineage that covers it
certbundle names its lineages certbundle-<hash>-<n>, while the cert lookup
reads live/<domain>/fullchain.pem. Since 16023dbc9b removed cert_folder_find,
nothing joined the two, so every letsencrypt deploy with the SAN flavor
stopped at "Let's Encrypt certificate files not found after issuance".

library/cert_link_domains.py points live/<domain> at the newest lineage
whose SAN covers the domain, using the same CertUtils lookup as
cert_check_exists, and swaps the link atomically. A real lineage directory
is left alone, and a domain without a certificate is only reported, so the
letsencrypt re-check still fails loudly on it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-23 16:05:27 +02:00
kevinveenbirkenbachandClaude Opus 5 3cc0214c28 fix(deploy): stop the dedicated deploy from git-cleaning its own checkout
`MODE_CLEANUP` says what to tidy on the deployed system: prune unused Docker
resources, clear Mastodon caches, switch an LDAP user list from `present` to
`exact`. Eight roles read it that way. The dedicated runner read the same flag
and ran `make clean` in the control repository instead, which is
`git clean -fdX` over every ignored path there.

`inventories/production/` is one of those paths. The published host install
guide provisions into it and then deploys against `$INVENTORY/.password`, so
the deploy deleted the vault password it was about to use:

    Removing inventories/production/
    [ERROR] The vault password file .../inventories/production/.password was
    not found

Reproduced in CI on two unrelated rows, arm64/zfs/debian and
amd64/btrfs/centos, so it is neither architecture nor filesystem specific.

The coupling was a name collision, not a decision. 7e5990aa16 generated the
MODE_* flags from `01_modes.yml` and removed the legacy skip flags, and the
repository clean - which had had a flag of its own - was attached to the
generated `MODE_CLEANUP` because one now existed by that name. No other deploy
path does this; `container`, `development` and `swarm` never call `make` at
all.

`make clean` keeps its own entry points, `make setup-clean`, `make clean-sudo`
and `infinito --git-clean`, where a caller asks for it deliberately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-23 10:59:05 +02:00
kevinveenbirkenbach e4ac05dfb6 Merge remote-tracking branch 'origin/main' into feature/updater 2026-10-23 10:46:59 +02:00
kevinveenbirkenbach e53751402f Merge pull request #1114 from infinito-nexus/fix/security/swarm-ingress-guard/1089
fix(svc-swarm-node): drop outside traffic to non-public routing-mesh ports
2026-10-23 01:03:04 +02:00
kevinveenbirkenbachandClaude Opus 5.5 bfdf88fae0 fix(domains): read an application's domains where the cache puts them
meta/domains.yml lands on the top-level `domains` key of the merged
application, yet three readers still looked under `server.domains`,
which no role fills any more:

- current_play_redirect_domains built every mapping from an empty
  config, so each app redirected to itself and was dropped. Neither
  `domains.aliases` nor AUTO_BUILD_ALIASES produced a single redirect,
  and echoserver's update check listed nextcloud.infinito.nexus and the
  other entity domains as dropped by the upgrade.
- csp_skip_domains never found the canonical of a disabled service.
- the logout filter's fallback returned no domains.

The unit tests fed the old layout and passed regardless. They now use
the merged one, and a new test builds the redirect from keycloak's real
role defaults.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 22:18:07 +02:00
kevinveenbirkenbachandClaude Opus 5.5 96fe8bd7e6 feat(playbooks/update): pull the images of the deploy before the upgrade window
pull.yml pulls, on the target host, every pinned image of the applications the host deploys, the same closure the update check judges, and lists the images a role builds itself instead. It never touches a container, retries each pull a bounded number of times and fails loudly with the image and the registry error. The update_check filter moves next to the new update_pull filter into plugins/filter/update/, and both read the code pins from one helper.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 21:30:18 +02:00
kevinveenbirkenbach 3cf840c72b Merge remote-tracking branch 'origin/main' into feature/updater 2026-10-22 21:14:49 +02:00
kevinveenbirkenbach 4a86376aeb Merge remote-tracking branch 'origin/main' into feature/updater 2026-10-22 21:08:26 +02:00
kevinveenbirkenbachandClaude Opus 5.5 8567cb6992 fix(svc-opt-ssd-hdd): recognise database images behind registry, namespace and custom builds
is_database() compared the whole repository part of the image reference
against postgres, mariadb, redis, memcached and mongo. Every reference
with a registry or namespace prefix, every derivative image and every
locally built `<engine>_custom` image therefore counted as application
data and went to the mass storage. On a local stack the central
databases report `postgres_custom:18` and `openldap_custom:13.7-slim`,
both classified False and moved to the HDD.

The check now takes the last path component of the reference, drops
digest, tag and a `_custom` suffix, and compares it against the engines
the svc-db-* providers and the app-local databases run: elasticsearch,
mariadb, memcached, mongo, mysql, openldap, opensearch, postgis,
postgres, qdrant, rabbitmq, redis, typesense, valkey.

Volumes the script already symlinked stay where they are; a database
volume misplaced on the HDD has to be moved back by hand.

Fixes #1163

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 21:02:06 +02:00
kevinveenbirkenbachandClaude Opus 5.5 558784e6ca feat(web-app-erpnext): set up the service provider's company and its accounting basics
The wizard bypass left ERPNext without a Company, so no app could create quotes, orders or invoices. The deploy now creates the Company from the service provider's name and country, with the currency from ERPNext's country data and an abbreviation that defaults to the name's initials, plus a fiscal year, five tax accounts, a cash account and the default company. It never rewrites an existing Company and fails loudly on a country ERPNext does not know. The README generator marks roles that read the service provider country, and the CI replay fills the placeholder.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 20:52:30 +02:00
kevinveenbirkenbachandClaude Opus 5.5 286004349a fix(web-app-suitecrm): upgrade the code in the data volume to the pinned release
The entrypoint stamped an installed volume with the image's version and kept the old tree, so a newer pin never reached it and the CreateUser patch stayed out. On boot it now runs SuiteCRM's own suitecrm:app:upgrade and upgrade-finalize with the image's patched tree as the package, resumes an interrupted upgrade, stamps the version the tree really has, and refuses a tree newer than the pin.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 20:44:13 +02:00
kevinveenbirkenbach b40c05dbbf Merge pull request #1177 from infinito-nexus/fix/install/shfmt-test-docstring
fix(install): drop the shfmt docstrings that misstate ensure()
2026-10-22 20:41:38 +02:00
kevinveenbirkenbachandClaude Opus 5.5 091c58307d fix(install): drop the shfmt docstrings that misstate ensure()
Three docstrings from #1173 claimed that ensure() leaves only the pinned
release on PATH or replaces whatever version PATH finds. It does neither:
it installs the release into /usr/local/bin and fails unless PATH then
resolves to it, so a distribution copy such as /usr/bin/shfmt stays on
disk behind it.

The test module and ensure() lose their docstrings; the module docstring
of the installer returns to the form its siblings use ("Install hadolint
via GitHub-release prebuilt binary.").

Reported by CodeRabbit on #1173.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 20:37:01 +02:00
kevinveenbirkenbachandClaude Opus 5.5 628e2587b8 fix(database_query): keep psql's command tags out of the returned rows
With `-tA` psql drops headers and the row-count footer but still prints
the command tag of every write statement. A guarded
`UPDATE … RETURNING` that matched nothing therefore came back as
`rows: [['UPDATE 0']]`, and web-app-peertube's oidc/enable.yml, which
judges its change by the row count, reported `changed` and restarted
PeerTube on every deploy after the first.

The PostgreSQL rows flags are now `-qtA`. Probed against a running
postgres container:

    -tA  -> CREATE TABLE / INSERT 0 1 / UPDATE 0 / 3 / UPDATE 1
    -qtA -> 3

SELECT output and the MariaDB path (`--batch --skip-column-names`, no
status line) are unchanged.

Fixes #1164

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 20:36:32 +02:00
kevinveenbirkenbach eeb3305e52 Merge remote-tracking branch 'origin/main' into fix/security/swarm-ingress-guard/1089 2026-10-22 19:17:39 +02:00
kevinveenbirkenbachandClaude Opus 5.5 a9a12da75c fix(install): install the pinned shfmt release on every distro
`shfmt.ensure()` returned as soon as any `shfmt` was on PATH and tried
the distribution package before the release pinned in
roles/sys-lint/meta/services.yml. The Debian dev container therefore
formats with its packaged 3.8.0 (/usr/bin/shfmt), Fedora with its packaged
3.7.0, the CentOS one with the pinned v3.14.1 (/usr/local/bin/shfmt), and
they disagree on
scripts/tests/deploy/swarm/utils/unmount/nfs_mounts.sh: every gate in a
CentOS stack moved the `then` of the heredoc `if` below the terminator.

ensure() now follows ruff's installer: it reads `shfmt --version`, keeps
the binary only when it reports the pinned release, and otherwise installs
the release binary into /usr/local/bin. It fails when PATH still resolves
another version afterwards instead of formatting with it. The system
package path is gone.

nfs_mounts.sh is committed in the pinned release's format (byte-identical
to the v3.14.1 output from the CentOS container), so the first gate on any
distro leaves the tree clean.

The install-lint stamp now also depends on roles/sys-lint/meta/services.yml,
the pin file of actionlint, hadolint and shfmt, so a bumped pin reaches
environments whose stamp predates it; until now only pyproject.toml and
scripts/install/lint.sh invalidated it.

Fixes #1155

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 19:16:22 +02:00
kevinveenbirkenbach 1ad4f0ac39 Merge branch 'main' into feature/updater 2026-10-22 18:59:45 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d2ef619faa fix(web-app-matomo): install the image's release into the data volume on upgrade
The upstream entrypoint copies the release only into an empty volume, so a newer pinned image kept running the old code from matomo_data. The deploy now copies the release over the volume, prunes the files the release dropped from its own trees, runs core:update, and recreates the security files after a code change. A volume newer than the image stops the deploy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 18:51:32 +02:00
kevinveenbirkenbachandClaude Opus 5.5 b83b361ea9 fix(web-app-bigbluebutton): run BBB v3.0.39 with the shared-notes server
The version-source update in #1146 moved the bigbluebutton/docker ref from
v3.0.23 to v3.0.39. That tag changes three things the role did not
handle, and it exposed a fourth defect in compose_file_args.

1. generate-compose renders etherpad and bbb-pads only behind
   USE_INSECURE_ETHERPAD and otherwise emits bbb-shared-notes-server with
   a `build:` block and no `image:` (upstream adds the image only after
   the tag, in 9a6a9a9a4c). compose_mods drops `build` only where an image
   is set, so the guard "Fail if any service still builds locally" stopped
   every BigBlueButton deploy (main run 39749899667, Nextcloud#11:
   `rc: 0`, `stdout: '    build:'`, bootstrap_compose_project.yml:35).
   meta/services.yml now pins alangecker/bbb-shared-notes-server:v3.0.39,
   compose_mods takes `pinned` (BBB_PINNED_SERVICES, every service the
   role config pins an image for) and drops their `build`, and the
   etherpad leftovers go (ep_redis_publisher patch, etherpad postgres
   entrypoint and initdb hook, etherpad database tasks, ETHERPAD_* env,
   etherpad_api_key secret).

2. The notes server connects to postgres once at start and exits with 2
   when it is not up yet; upstream gives it no restart policy. The
   override sets the role's restart policy, so it retries until postgres
   accepts connections.

3. Greenlight moves from v3.8.0 to v3.9.0.1, which sets
   `config.hosts = URL_HOST`, and the template passes
   `URL_HOST: https://${DOMAIN}`. Rails compares hostnames, so every
   request was blocked (`Blocked hosts: bbb.talk.infinito.test`, 403).
   Greenlight documents URL_HOST as the bare hostname; the override sets
   it to the role domain.

4. compose_file_args decided whether a role ships a compose override by
   templating "{{ application_id | abs_role_path_by_application_id }}"
   through the templar. ansible-core 2.21 does not render an untrusted
   string, so the check saw the literal and always returned False: the CA
   override generation and the pull handler never read BBB's
   compose.override.yml. That was harmless while compose.yml alone was
   valid; with the notes-server image living in the override, `compose
   config` failed ("has neither an image nor a build context"). The check
   now resolves the role path for the application id it is given. BBB is
   the only role that ships an override, so no other role's file list
   changes.

Existing installations drop the etherpad and bbb-pads containers on the
next `compose up --remove-orphans`; the old `etherpad` database stays in
PostgreSQL untouched.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 16:56:48 +02:00
kevinveenbirkenbachandClaude Opus 5.5 3838114696 feat(playbooks/update): judge a host and its inventory against the current code
report.yml only dumped a host's state, so every comparison with the code
of the checkout was done by hand. It now also runs state.sh on the host,
which emits tab-separated records (volumes with their resolved storage
path, containers with their port bindings, listening sockets with the
container that owns them, docker subnets, data versions, server_name
lines, certificates, failed units, newest backups, disk usage), and the
update_check filter compares them with what lookup('applications'),
application_closure, compose_volumes, current_play_redirect_domains and
the nginx include directories derive from the code and the inventory.

The verdict lists every finding as OK, WARN or FAIL per area (inventory,
volumes, data, images, ports, networks, webserver, certificates,
operations), prints the warnings and failures, keeps the full list in
/tmp/infinito-check-<host>.txt and fails the play on a red check. The
host still needs no Python.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 12:50:03 +02:00
kevinveenbirkenbachandClaude Opus 5.5 77349c73e1 style(web-app-stalwart, web-app-joomla): apply the formatter and its lint rules
The upgrade work only ran make test, never make autoformat, so ruff now
reformats the new Stalwart and Joomla code and reports five rules it
cannot fix itself: Path.readlink instead of os.readlink (with Path moved
into the type-checking block), list.extend for a collecting loop, no
manual "Z" replacement before datetime.fromisoformat, and an S310 opt-out
for the two requests to the fixed internal JMAP origin.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 11:40:43 +02:00
kevinveenbirkenbach 2d7f842613 Merge branch 'main' of github.com:infinito-nexus/core into feature/updater 2026-10-22 11:27:47 +02:00
kevinveenbirkenbach cbb29f898d Merge remote-tracking branch 'origin/main' into fix/security/swarm-ingress-guard/1089 2026-10-22 11:22:34 +02:00
kevinveenbirkenbachandClaude Opus 5.5 05135a7cee feat(svc-net-wireguard): let a compose deployment name a hub with roaming spokes
Without a swarm manager the mesh planner only built full meshes, whose
handshake gate waits for every peer, so laptops behind NAT could not join.
An inventory may now name a hub in networks.hub: the planner builds every
declared mesh around it, the hub forwards between its spokes, a spoke marked
networks.roaming gets no endpoint in the hub's peer list and dials the hub
with PersistentKeepalive, and the hub's gate skips roaming spokes while a
roaming spoke gates only on the hub. The mesh CLI takes the hub's public
address per host (--endpoint) and a vault password per host
(--host-vault-password-file). Full meshes and the swarm manager hub plan
exactly as before; NETWORK_MESH_POOL keeps the operator's subnet.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 10:56:50 +02:00
kevinveenbirkenbachandClaude Opus 5.5 8701ddb769 feat(web-app-stalwart): import every Mailu mailbox and provision aliases and forwards
The Mailu import covered only inventory users and logged in with their
passwords. It now reads every <address>/ maildir of the legacy volume
(files/python/mailu_maildir.py), logs in as the recovery administrator
acting for each account, creates missing accounts under the name the OIDC
directory resolves, keeps folders, flags, keywords and internal dates,
imports and activates each mailbox's Sieve script, merges mailboxes the
inventory maps onto another account (migration.mailbox_map), skips and
reports mailboxes of domains Stalwart does not serve, and imports nothing
twice on a re-run. The recovery credentials reach the script through a
0600 file, not argv. The import task allows 6 hours like the other long
data migrations, because a production mailbox of 157k messages does not
fit into one.

services.stalwart.aliases becomes Stalwart mailing lists, with @<domain>
as the domain's catch-all and destinations outside the served domains
allowed; services.stalwart.forwards becomes a redirect :copy Sieve script
chained with the user's own. Both are provisioned on every deploy and only
touch what the role created.

The test harness groups its probes under files/test/probe/ and its
scenarios under files/test/verify/.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 10:53:11 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d06fc7bd28 feat(web-app-mediawiki): refresh the extensions volume when MediaWiki changes version
Docker seeds mediawiki_extensions from the image only once, so after an
upgrade the bundled extensions of the old release kept running against the
new core. On every deploy a one-shot container of the pinned image runs
files/shell/refresh_extensions.sh against the volume: it adopts a volume
whose content matches the image, replaces the content with the image's
extensions when the recorded version differs, records the version in a
marker, and refuses when the volume records a newer version than the image.
After a refresh the role reinstalls its add-on extensions and runs
update.php.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 10:46:19 +02:00
kevinveenbirkenbachandClaude Opus 5.5 eddbccd63c feat(web-app-confluence, web-app-jira): refuse a pinned image older than the data
Atlassian applications refuse to start on data a newer build wrote, after
the old container is gone. Before the stack starts, each role compares
the build number of the data with the build of the pinned image
(files/python/build_guard.py) and stops the deploy with both builds named
when the data is newer. Confluence reads the build from the home in its
volume; Jira reads it from the database (files/sql/data_build.sql), so its
guard needs the database reachable before the stack starts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 10:38:11 +02:00
kevinveenbirkenbachandClaude Opus 5.5 c65c5ce48c fix(update): pin only image tags the registry carries
7e82ba6d71 moved web-app-bigbluebutton's bbb-pads to v1.5.11, the tag
https://raw.githubusercontent.com/bigbluebutton/docker/develop/repos/tags
names for the bbb-pads source. alangecker/bbb-docker-pads has no such
image (Docker Hub: newest v1.5.8), so every BigBlueButton deploy pulls a
missing tag and test-external fails "These pinned Docker images do not
exist in their registry (HTTP 404): web-app-bigbluebutton/bbb-pads".

bbb-pads goes back to v1.5.8. The version-source updater now knows the
image a `version` pin tags (pulled directly or as the base of a custom
build) and withholds a bump when the registry answers 404 for the new
tag; an indeterminate answer does not block it, the same line
test-external draws. Run against main, the updater now reports no
update for bbb-pads instead of v1.5.11.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 10:30:10 +02:00
kevinveenbirkenbachandClaude Opus 5.5 93525d4f29 feat(web-app-joomla): update an older core in the code volume up to the image's core
The Joomla image never touches the core that sits in joomla_data, so an
old site kept its old core and the SSO task failed on a missing
extension:enable. The role now reads Version.php of the code volume and of
the image tree plus the version the database was finalised for, plans the
steps through the declared bridge releases (5.4.9, checksum pinned, with
its compat plugin states), lays each full package over the code volume
without its installation/ folder, runs the database and schema update with
files/php/core_update.php, and asserts one core at or above the image's.
A newer core than the image is left alone; a database that records another
version than the code volume stops the deploy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 10:27:23 +02:00
kevinveenbirkenbachandClaude Opus 5.5 8075c68c92 feat(web-app-moodle): upgrade an old code volume through the releases Moodle requires
Moodle 5.3 upgrades no site older than 4.4. The image now also ships the
4.5.15 transit release (tracked by the version updater) and records which
release each shipped tree accepts as its oldest source. On start, the
entrypoint reads the release in the code volume, migrates a Bitnami-layout
volume to this image's layout, runs admin/cli/upgrade.php with the transit
tree first when the site is older than the pinned release's minimum source,
then with the pinned tree, and refuses to start over an empty database.

The entrypoint and its release library move to files/build/moodle/, the
core installers to files/build/install/moodle/, next to the plugin
installers, and their unit tests follow.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 10:24:09 +02:00
kevinveenbirkenbachandClaude Opus 5.5 cd28286ae6 feat(web-app-openproject): pass an older schema through every skipped major
OpenProject 17 refuses a 15.x database with IncompleteMigrationsError;
upstream requires each major in between. Before the pinned image is built
or started, the compose deploy reads schema_migrations, derives the major
that last migrated the schema from its aggregated migrations, and for each
major strictly between that and the pin stops the old containers, runs the
seeder of openproject/openproject:<major> once against the stack's
database and data volume, and works off the GoodJob queue in the same image
before GoodJob 4 takes over. A schema newer than the pin stops the deploy;
an empty database skips everything.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 10:20:22 +02:00
kevinveenbirkenbachandClaude Opus 5.5 678011cca0 feat(web-app-peertube): run the release migration scripts and refresh the client on upgrade
PeerTube 8.0, 8.1 and 8.3 ship scripts that must run after the server
migrations, and the image-seeded peertube_assets volume keeps the old
client forever. Before the pinned image starts, the role reads the release
the data was written by from a marker file and application.migrationVersion
(table of every release's last migration from 6.0.0 to 8.3.1) and plans the
upgrade; a downgrade or data older than the table stops the deploy. After
the server is healthy it asserts the pinned migration version, runs every
dist/scripts/migrations/peertube-X.Y.js above the data's release in order,
empties /app/client/dist so Docker reseeds it from the pinned image, and
records the pinned release in the marker. The health wait allows 30
minutes while existing data migrates.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 10:18:09 +02:00
kevinveenbirkenbachandClaude Opus 5.5 f6fffd13e1 feat(web-app-nextcloud): step existing data through every major up to the pin
Nextcloud refuses to skip a major. Before the pinned image starts, the
role reads $OC_Version from version.php in the code volume, plans one tag
per major between that version and the pin, and runs the image of each
step once against the data (occ upgrade through the entrypoint, with the
debug log lines filtered out of the output). Then the regular manager ops
run on the pinned image and assert that occ status reports the pinned
major. A fresh volume plans no step; swarm stops when data needs one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 09:52:36 +02:00
kevinveenbirkenbachandClaude Opus 5.5 7ac49234ca test(inventory): read the width test's tempdir file without the cache
The file is written by the test itself, so the path-keyed read cache does
not fit; opt the read out of the uncached-read lint instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 09:47:59 +02:00
kevinveenbirkenbachandClaude Opus 5.5 849dd92afb feat(svc-db-postgres): upgrade an older cluster with pg_upgrade --link on deploy
Before the server starts, the role finds the cluster in postgres_data,
whether it sits in the volume root, in pgdata/ or in <major>/docker, and
upgrades an older one major by major into the PGDATA of the pinned image:

- builds an upgrade image from the pinned PostgreSQL image with the binaries
  of every older major on the path and the extension packages the
  consumers in the host's groups declare
- keeps a reflink copy of the whole volume next to it, or stops until
  services.postgres.upgrade.backup_confirmed is true on a filesystem
  without reflinks
- runs pg_upgrade --check, then --link, with the old cluster's superuser,
  encoding, locale and checksum setting, and carries pg_hba.conf,
  pg_ident.conf and postgresql.auto.conf over
- after the start, applies the extension updates pg_upgrade wrote,
  collects statistics, reports databases whose collation version changed
  and removes the retired clusters, never the reflink copy

A cluster newer than the pin, a pinned major outside PGDATA, or a retired
cluster without a newer one stops the deploy. Swarm mode only detects and
stops on an older cluster.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 09:46:24 +02:00
kevinveenbirkenbachandClaude Opus 5.5 46f32392b9 fix(inventory): stop the inventory writers from folding long lines
ruamel folds every scalar and flow collection past 80 columns, so each
run of the provision, credentials or mesh CLI broke SSH keys and alias
lists in host_vars over several lines. round_trip_yaml() now sets an
unlimited width, and the credentials CLI and the services disabler use it
instead of building their own handler.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-22 09:41:11 +02:00
kevinveenbirkenbachandClaude Opus 5.5 788eabffbf fix(svc-swarm-node): drop outside traffic to non-public routing-mesh ports
In swarm mode `docker stack deploy` ignores the host IP of a short port
mapping, so every `127.0.0.1:<port>:<port>` that container_ports renders for
a `ports.local` entry is published by the ingress routing mesh on every
node, via docker-proxy on 0.0.0.0 and [::] plus DNAT in nat PREROUTING.
Databases, LDAP and application backends answered from outside the cluster.

svc-swarm-node now loads an nftables table through svc-net-firewall. Its
prerouting filter chain (priority mangle, ahead of Docker's dstnat) drops
packets addressed to the node for any ingress-published port that no
application declares under `ports.public`. Loopback, docker0,
docker_gwbridge, the user bridges and the swarm nodes' own addresses stay
admitted. Host-side provisioning over 127.0.0.1, containers that reach a
host port through host-gateway, and node-to-node consumers such as the
registry-cache mirror keep working.

Both sets are read live on the manager (`docker service inspect` for
`.Endpoint.Ports`, `docker node inspect` for `.Status.Addr`), so ephemeral
ingress ports and nodes outside `--limit` are covered. The table is loaded
on a node before it joins, reloaded on every node after the join, and
reloaded after each stack deploy by the sys-svc-compose swarm handler.

Refs #1089

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 20:56:29 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d4b3609488 fix(install): refuse the upstream Node.js install on non-Linux hosts
`tarball_name()` picked the archive from `platform.machine()` alone, so
an end-of-life Node.js on Intel macOS would have unpacked a Linux build
into /usr/local, Homebrew's prefix there. The upstream install now
supports Linux only and stops with a message to install a supported
Node.js through the package manager everywhere else.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 16:52:56 +02:00
kevinveenbirkenbachandClaude Opus 5.5 30268cf552 fix(install): replace an end-of-life distro Node.js with the current LTS
Debian trixie ships nodejs 20.19, whose release line reached its end of
life on 2026-04-30. `@mermaid-js/mermaid-cli` 12 and its puppeteer
declare `node >=22.13.0`, so `make install-lint` failed in every fresh
Debian stack container with `All providers failed for
chrome-headless-shell` (#1088).

`ensure_node_present()` still installs the distro package first, then
reads the release line of `node` on PATH and looks it up in the
nodejs/Release schedule. A line past its `end` date is replaced by the
newest LTS from nodejs.org/dist/index.json: SHASUMS256.txt is verified
with gpgv against the active Node.js release keys from
nodejs/release-keys, the tarball against its SHA-256 entry, and the
release is unpacked into /usr/local, after which `node` on PATH must be
that release. Nothing pins a major: the next LTS line is picked up the
same way. Ubuntu 26.04's Node.js 22, supported until 2027-04-30, stays.

`ensure_npm_present()` calls it first, so a container that already holds
the end-of-life node is upgraded too rather than skipped because npm
exists. The call runs once per process.

Closes #1088

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 16:02:52 +02:00
kevinveenbirkenbachandClaude Opus 5.5 6e776a5780 refactor(python): clear the open CodeQL code-quality alerts
- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
  is imported by its modules (163 packages), so the constant counts as
  exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
  imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
  re-exports included, instead of silencing them with `noqa: F401`; the
  list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
  rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
  BMP-only positive class; it matches exactly the same 2111 code points
  as the old negated class across U+0000..U+10FFFF.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 21:40:58 +02:00
kevinveenbirkenbachandClaude Opus 5.5 d48f8e9897 fix(bcrypt_hash): derive the seeded salt from the seed alone
The seeded salt was the first 16 bytes of sha256(seed + secret). bcrypt
stores its salt in clear inside the hash, so every seeded hash published
a fast sha256 digest of the password under a seed that is public in the
repository (`web-app-wazuh:indexer-admin`). Anyone holding the hash could
test password guesses at sha256 speed instead of paying bcrypt's cost.

The salt is now sha256(seed)[:16]. The hash stays deterministic for one
password, still changes when the password rotates, and its salt carries
nothing computed from the secret. Existing Wazuh deployments get a new
hash once; `securityadmin.sh` re-applies `internal_users.yml` on every
deploy and the hash still verifies the unchanged password.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 21:35:32 +02:00
kevinveenbirkenbachandClaude Opus 5.5 21eba1e88f fix(backup): carry hard links, ACLs and xattrs on every backup and recover hop
baudolo 7.1.x writes volume generations with `rsync -aHAX`, so the
producing host keeps `user.*` extended attributes (OpenCloud keeps its
storage metadata in `user.oc.*`). Every later copy dropped them again,
because none of the other rsync calls passed `-X` (nor `-A`/`-H`):

- NFS export snapshots (svc-bkp-nfs-2-local `script.sh`),
- the backup host's pull (svc-bkp-remote-2-local `pull_specific_host.py`),
- the device sync (svc-bkp-local-2-device `script.py`),
- every recover into a live target (`utils/recovery/base.py`, used by the
  volume, NFS and device recover scripts),
- the `recover full` and `recover remote` tree pulls.

A disaster recovery through the documented chain therefore restored
OpenCloud without `user.oc.*`, silently: the copies succeeded and the
dirval stamp does not cover attributes.

All of these now run `rsync -aHAX` (`-aHAXP` for the device sync). A
target filesystem without xattr support makes rsync exit non-zero as soon
as a source file carries attributes, so the hop fails loudly instead of
dropping them; there is no per-device opt-out. The single-image pull in
`recover remote` keeps `-a`. The provider's ssh-wrapper needs no change:
its glob accepts any server flags and its character guard allows them.

Two tests write a `user.oc.id` attribute and assert it survives the NFS
snapshot and the recover; with the old `-a` both fail with
`OSError: [Errno 61] No data available`.

Closes #1021

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 18:20:46 +02:00
kevinveenbirkenbachandClaude Opus 5.5 310663ae8c feat(compose): recreate the stack network without losing volumes or worktree stacks
When the IPAM config of a compose project's network changes (#999 added
`ip_range`), compose recreates the network on the next `up`. With a
foreign endpoint attached, as every worktree stack is on the primary's
network, compose first stops the project's own containers, then fails
with `error while removing network: ... has active endpoints`, and a
repeated `up` or `up --force-recreate` does not recover. On 2026-10-19
this took the shared registry/package cache and coredns down for every
local session.

`make compose-network-recreate` (alias `i8conere`) runs the sequence that
recovered it: detach every running container of another project from the
network, `docker compose down --remove-orphans` without `-v` so the
caches keep their volumes, the normal `up`, then attach the detached
containers again, which draw addresses from the dynamic pool. If `up`
fails, the still-detached containers are listed instead of reattached.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 13:45:37 +02:00
kevinveenbirkenbachandClaude Opus 5.5 a6c3c1cc91 fix(apparmor): keep docker-default loaded when tearing AppArmor down
`security-apparmor-teardown` ran `aa-teardown`, which unloads every
profile including dockerd's in-memory `docker-default`. dockerd only loads
that profile when it starts, so every container start afterwards that
requests it fails until dockerd restarts. Since #1067 runs the teardown on
the GitHub runners, BuildKit failed each RUN of the swarm node image three
times (run 39332343490, 39 of 40 swarm jobs):

    runc run failed: unable to start container process: error during
    container init: unable to apply apparmor profile: apparmor failed to
    apply profile: write .../attr/apparmor/exec: no such file or directory

The teardown now removes each loaded profile by name through securityfs
and skips `docker-default` and child profiles, which leave with their
parent. Dev hosts gain the same: no dockerd restart is needed after
`make environment-bootstrap`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-20 12:11:14 +02:00
kevinveenbirkenbachandClaude Opus 5.5 b055eece05 feat(ci): keep GitHub-hosted runner labels current through the updater
The workflows pin `runs-on: ubuntu-NN.NN`, so no job moves to a new Ubuntu
release behind our back, but nothing raised the pin either. A sixth job of
`🔄 Update versions` closes that: `utils/update/runner.py` reads the image
table of actions/runner-images, takes the newest x64 Ubuntu label whose
image carries no preview or beta badge, and raises every older bare
`runs-on: ubuntu-NN.NN` to it; arch-suffixed labels stay untouched. The job
opens the change as one PR over `.github/workflows`, so its CI run tests
the new image before it reaches main.

This commit carries the updater's first result: every job now runs on
ubuntu-26.04 (49 `runs-on`).

actionlint 1.7.12, the newest release, does not know `ubuntu-26.04` yet
(rhysd/actionlint#682, fix proposed in rhysd/actionlint#743), so
`.github/actionlint.yaml` declares the label until a release knows it.

act maps each `ubuntu-NN.NN` label the workflows use to the local platform
image, derived from `.github/workflows`, instead of a fixed list that did
not know ubuntu-26.04.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 16:56:52 +02:00
kevinveenbirkenbach d0698daa5e Merge pull request #414 from pragepani/feature/web-app-wazuh/add-role
feat(web-app-wazuh): add Wazuh SIEM role with OIDC/LDAP RBAC
2026-10-19 14:35:24 +02:00
kevinveenbirkenbach 19914777c4 Merge remote-tracking branch 'origin/main' into fix/compose/primary-net-ip-range 2026-10-19 09:46:19 +02:00