Commit Graph
7 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 e3d2a24bcd fix(sys-svc-compose-php): scope memory_limit -1 to unpatched images
The interim CVE-2026-109618 mitigation lived in the shared hardening.ini,
so every PHP role ran without a memory limit until the last image was
patched, and no role's service_memory_mb share applied.

Registry metadata of the images pinned on main shows that 16 of the 20
consumers already ship PHP 8.2.35, 8.3.36, 8.4.27 or 8.5.12 (or the
Alpine/Debian build of it). Four predate the fix:
markoshust/magento-php:8.4-fpm (8.4.21), ghcr.io/heviat/webmail:2024.06
(the Mailu OIDC flavor, built 2025-04-04), phpldapadmin/phpldapadmin:2.3.11
(built 2026-05-15) and ghcr.io/shopware/docker-base:8.3 (8.3.25).

- Move memory_limit = -1 into its own content-addressed ini
  (FILE_PHP_MEMORY_UNLIMITED_INI) and mount it only in magento, mailu,
  phpldapadmin and shopware.
- hardening.ini keeps expose_php, session.upload_progress.enabled and
  fatal_error_backtraces as permanent hardening.
- Joomla's php-upload.ini loads as conf.d/uploads.ini again: no -1 has
  to be outranked anymore.
- The root TODO names the four images that still need the override.

Refs #1126

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 22:39:07 +01:00
kevinveenbirkenbachandClaude Opus 5.5 d89c74feab feat(security): harden PHP services and proxy response headers
Proxy:
- server_tokens off, X-Powered-By stripped from upstream responses,
  HSTS max-age raised to one year.
- No global X-Content-Type-Options: under nosniff, ORB (step 3.4)
  blocks no-cors cross-origin responses typed text/html, e.g. tracking
  beacons answered with an empty 204.

PHP (CVE-2026-109618, CVE-2026-109619):
- New role sys-svc-compose-php installs a shared ini (expose_php Off,
  session.upload_progress.enabled 0, fatal_error_backtraces 0 for
  PHP 8.5, interim memory_limit -1) and a PHP-FPM [www] drop-in
  (pm.max_requests = PHP_FPM_MAX_REQUESTS) once per stack host,
  before 04_files renders any compose file.
- The host file names carry a sha1 prefix of the controller-side
  source, so a content change alters every PHP app's compose.yml and
  recreates it, also on remote deploys where compose_volumes cannot
  read the stack host file and hashes the path instead.
- 20 PHP roles mount the ini as a compose/swarm config into their
  image's scan dir; moodle, nextcloud and shopware also mount the FPM
  drop-in. Magento keeps its own pool: its php-fpm.conf has no
  include= and is rendered without notify.
- Joomla's php-upload.ini now loads as zzz-joomla-uploads.ini so its
  own memory_limit wins: the media API compares Content-Length with
  toBytes(memory_limit) and rejects every upload at -1.

TODOs record the memory_limit revert condition and accepted
trade-offs, Akaunting on PHP 8.1, FusionDirectory on PHP 7.4 and the
public Keycloak admin console.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-21 21:03:38 +02:00
kevinveenbirkenbachandClaude Opus 5 750476b26a feat(lint): give every mount target one spot in meta/volumes.yml
A volume that reaches several services repeated its container path once per
mount. Written out each time it has no single point of truth: a rename that
misses one line still parses, still deploys, and mounts one service somewhere
nobody intended. Nothing fails until that service looks for content which is
not there.

YAML already solves this, and 62722a6ab used it here first -- web-app-openproject
anchors await_migrations_target and aliases it for the second service. Nineteen
roles now follow, and tests/lint/repository/spot/test_volumes_target_single_spot.py
fails on any target spelled out twice. An alias carries no path, so the rule
cannot see it, which is the point.

The conversion was mechanical and is provable rather than reviewed: every file
was rewritten and then reloaded, and any file whose yaml.safe_load result
differed from the original was discarded instead of kept. Nineteen written,
none discarded. Anchors and aliases are a serialisation form; the loader
resolves them before any consumer sees a document.

validate_volumes_meta over all 81 meta/volumes.yml reports no violation.
make test-lint: 328 passed.

web-app-prometheus carries two unrelated deletions: comments that predate this
change and fell under the comment lint once the file was touched. They explained
a 0444 mode and a compose-only bind; both facts are visible in the entries
themselves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 02:02:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 fe5d395a9f feat(web-app-mailu): remap to central engines + swarm hardening
Consume central redis and unbound (drop embedded sidecars). Swarm: resolve central mariadb/redis overlay VIPs into admin extra_hosts so the DNSSEC resolver does not NXDOMAIN internal names; pre-create nginx/certs bind dirs plus a placeholder cert before stack-up; recreate the API token when a fresh cluster lacks it; wait for antispam via container exec; disable OIDC SSL verify under self-signed TLS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 19:29:39 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5477b56e00 feat(volumes): file-bind → config so single-file mounts work in swarm
Adds a strict lint that flags every `type: bind` entry in
roles/<role>/meta/volumes.yml whose source or any mount target points
to a single FILE (path ends in a recognised file extension or a
Jinja var with a file-suffix name like *_CONF/*_FILE). Single-file
binds are swarm-fragile: the source path is only present on the node
where the file was rendered, so swarm rejects the task as soon as it
lands on a different node.

Fix is `type: config`: the manager loads the file at `stack deploy`
time and swarm distributes it through raft to every node.
compose-mode honours the same canonical entry (the compose_volumes
filter renders it as a top-level `configs:` block; container_volumes
emits the per-service reference), so the role stays mode-agnostic.

Per-line opt-out: `# nocheck: meta-volume-file-bind-should-be-config`
with a short `# Reason:` for legitimate exceptions (sockets, kernel
interfaces, runtime-rotated files, multi-MB blobs, host writebacks).

This commit migrates 23 file-bind entries across 16 roles to
type:config: web-app-flowise, friendica, kix (2), listmonk, mailu,
matrix (2), mediawiki, mobilizon, nextcloud, pgadmin, pixelfed,
prometheus (4), shopware (2), suitecrm, taiga (2), zammad. All gain a
default mode:"0440"; mount-level read_only:true is dropped (configs
are read-only by default and the schema forbids the field for
type:config).

Updates the prometheus role-wiring test to accept both type:bind and
type:config for the alert_rules mount probe.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 6e49fe74fb feat(volumes): canonical dict-of-dicts meta schema + lookup('volume')
Migrate every role's meta/volumes.yml from ad-hoc dict-of-named-volumes
to a canonical dict-of-dicts schema where the YAML key is the semantic
short name and the entry body carries type / name / source / mounts /
nfs / mode / read_only. The optional 'name:' field is the docker volume
name (defaults to the key); the previous 'docker_name:' field is gone.

What's in this commit (210 paths):

Infrastructure (23)
- New filter plugins/filter/{compose,container}_volumes.py — render the
  top-level volumes:/configs:/secrets: section and per-service mount
  blocks from the canonical meta. container_volumes uses the SEMANTIC
  name as per-service mount source so it resolves against compose_volumes'
  top-level dict key (docker_name as source produced 'undefined volume'
  on swarm).
- New lookup plugins/lookup/{compose,container,volume}_volumes.py.
  volume(application_id, name) returns a normalised single entry;
  volume(application_id) returns the whole canonical dict.
- utils/cache/applications.py keeps a sibling registry
  _CANONICAL_VOLUMES_BY_ROLE outside the merged applications payload so
  raw Jinja source: strings never enter the templar walk (root-cause of
  the 200x cache-miss + recursive get_merged_applications storm).
- utils/roles/applications/mounts.py — schema validator + normaliser;
  list-shape is rejected.
- plugins/lookup/nginx.py: _volume_path_by_name reads canonical directly.
- scripts/tests/deploy/swarm/_context.sh: NFS_VOLUMES extraction handles
  both legacy dict and canonical dict shapes.
- roles/svc-prx-openresty/handlers/restart_and_validate.yml: gate
  compose-only validate+reload behind a single when-block; swarm uses
  service-update instead (bind-mounts of a FILE pin source inode at
  task-create time, so atomic write-tmp+rename leaves the running task
  on a deleted inode).
- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch to
  lookup('volume', application_id) | dict2items so NFS subdir
  pre-creation works under the new cache split.

Tests (12)
- New strict lints flagging inline mount sections in compose templates,
  single-file bind mounts, and legacy lookup('config', ..., 'volumes.X.Y')
  call sites — all volume-mount surfaces MUST flow through container_volumes
  / compose_volumes / volume lookups.
- Integration: test_volumes_compose_chain + test_volume_constants_are_lookups
  accept both the legacy and the new lookup('volume', app, 'X').name forms
  during the migration window.
- Unit tests for the new lookup, filter, mounts helper, plus updated
  nginx lookup test.
- Prometheus role-wiring test reads canonical meta for the
  alert_rules.yml bind mount.

Role migrations (175)
- 62 meta/volumes.yml converted to dict-of-dicts canonical shape
- 63 templates/compose*.yml.j2 replace inline volumes:/configs:/secrets:
  with lookup('container_volumes', application_id, '<service>') |
  indent(N) per service + a single trailing lookup('compose_volumes',
  application_id) for the top-level block
- 50 vars/main.yml drop dead volume constants and refactor 23 remaining
  legacy lookup('config', ..., 'volumes.X.Y') to lookup('volume',
  application_id, 'X').Y across 9 roles

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:46 +02:00
kevinveenbirkenbachandClaude Opus 4.7 5adf3011fe refactor(swarm-nfs): generalize swarm+NFS chain across all roles
Migrate the mediawiki-specific swarm patterns into shared sys-svc-compose
helpers so every role inherits swarm + NFS portability:

- meta/volumes.yml: dict schema with mandatory name|path + optional nfs
  block (uid/gid/mode); 47 roles migrated, 48 lookup consumers updated to
  volumes.<key>.{name,path}
- sys-svc-compose helpers: finalize.yml (flush + post_deploy),
  post_deploy.yml (swarm-resolve + DB-healthy-wait), nfs_prep.yml
  (pre-create NFS subdirs), swarm/resolve_self.yml (canonical
  <ENTITY>_CONTAINER fact), render_replicated_templates.yml +
  _render_one_template.yml (template render + swarm-peer replication,
  per-entry notify, fileglob support)
- 15 roles migrated to render_replicated_templates.yml helper
  (mediawiki, prometheus, kix, friendica, dashboard, nextcloud,
  suitecrm, matrix, flowise, openldap)
- 5 new lint tests pin the architecture: volumes schema, compose chain,
  volume-constants-are-lookups, template-render-centralization,
  meta-no-role-var-refs
- ROLE_FILE_TEMPL_COMPOSE SPOT constant + 7 tests migrated to use it
- Handler test scanner extended to detect nested notify in
  vars: role_templates: lists

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-02 18:57:35 +02:00