Commit Graph
35 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 2348d48110 fix(web-app-shopware): move PHP base to docker-base 8.3.36-caddy
The bare ghcr.io/shopware/docker-base:8.3 tag is frozen. Its digest
sha256:14826eb9 is the same as 8.3.25, built 2025-09-05, and carries
PHP_VERSION=8.3.25. Upstream has published only flavoured tags since
8.3.26. 8.3.36-caddy (sha256:b1fdca6e, built 2026-10-30, also the
current 8.3-caddy) carries PHP_VERSION=8.3.36. It has the same
supervisord entrypoint, www-data user and exposed ports 8000/9000 as
the bare tag. That clears the 8.3.36 security floor that hlth-runtime
reports for shopware-web, -worker and -scheduler.

The flavoured tag needs two follow-ups:

- SHOPWARE_COMPOSER_PLATFORM_PHP drops the "-<flavor>" suffix before it
  pads major.minor. Without that, composer gets platform.php
  "8.3.36-caddy". It now renders 8.3.36, and 8.3 still renders 8.3.0.
- shopware/docker 6ae1717 ("feat: remove default env vars") removed
  SHOPWARE_ES_ENABLED=0, SHOPWARE_ES_INDEXING_ENABLED=0 and
  SHOPWARE_ES_INDEX_PREFIX= from the image. Shopware 6.7.13.0 reads
  them through %env()% with no container default, so env.j2 now sets
  the values the old image shipped. SHOPWARE_HTTP_DEFAULT_TTL keeps its
  7200 default from core services.xml. Nothing in Shopware reads
  DATABASE_PORT, and the role uses DATABASE_URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-31 00:45:19 +01:00
kevinveenbirkenbachandClaude Opus 5.5 462bc4ccf0 fix(web-app-shopware): keep the init memory limit in the role's env file
The init service received INIT_PHP_MEMORY_LIMIT through an inline
environment block in the compose template. It moves to templates/env.j2,
which the container base hands to every Shopware container as env_file.
Only files/shell/init.sh reads the variable, behind a `:?` guard, so the
other containers seeing it changes nothing.

The lookup names 'init' instead of service_name, which is bound in the
compose template but not in env.j2. The value stays 1024M.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 01:37:17 +02:00
kevinveenbirkenbachandClaude Opus 5 dcb921c260 fix(web-app-shopware): address the OIDC provider by its service name
container_address resolves a container id for exec and emits a $(...) subshell in swarm; setup/oidc.yml passes the value through shlex.quote, so the substitution never ran and client.php wrote the literal text into token_endpoint and userinfo_endpoint. The deploy stayed green and every admin SSO login failed at the code-to-token exchange. container_service is the maintained lookup for the addressable form and yields keycloak in compose, keycloak_keycloak in swarm.

init.sh no longer waits on the S3 endpoint: c5a149b11 unmounted the config it guarded, while a hard fail there still took web, worker and scheduler with it. The admin-bundle check now runs unconditionally — it was skipped exactly when objstore was on, i.e. in the only variant with SSO, although the bundles are local either way. The two objstore env vars lost their last consumer with it.

no_log now sits on the two tasks that carry the admin password and the OIDC client secret; handing it through include_tasks apply: does not censor them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:30 +02:00
kevinveenbirkenbachandClaude Opus 4.8 15e7647d2f refactor(env): central TZ+LANG base included by every role env.j2
New roles/sys-svc-container/templates/env.j2 sets TZ={{ HOST_TIMEZONE }} and LANG=C.UTF-8. Every role env.j2 includes it at the top; it renders via stack_host_template, the same engine that resolves the includes in compose.yml.j2, so it works in both compose and swarm. TZ/LANG are removed from the individual role env.j2 so each value stays single-source.

New lint tests/lint/ansible/test_env_base_include.py verifies every role env.j2 includes the base and never re-sets a base-owned key (keys derived from the base file).

mattermost and gitea meta/volumes.yml drop the /etc/timezone host-path bind: swarm hard-rejects a task whose bind source is missing on the node (Debian-trixie nodes have no /etc/timezone). TZ now comes from the base env. /etc/localtime bind also dropped; the container timezone follows TZ.

Non-directive comments were removed from the touched env.j2 (banners, restatements, reference URLs, narration) so the changed files pass the comment-lint; only tool directives remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 d5ae126bec fix(shopware): shared sessions via the native phpredis handler
The earlier attempt routed sessions through framework.session.handler_id with a redis DSN, which lazy-loads symfony/cache's RedisProxy whose dump() declaration is incompatible with the image's php-redis - a PHP fatal in every php container under swarm. Route sessions below Symfony instead: bake a conf.d ini registering phpredis's native session save handler with session.save_path taken from the REDIS_SESSION_TARGET env at runtime (php.ini env substitution, no secret in the image), emit that env with phpredis auth[user]/auth[pass] url-encoded query syntax from the engine lookup, and set framework.session.handler_id to null so Symfony defers to php.ini. Sessions are shared across replicas; cache and messenger paths stay untouched. All Dockerfile prose comments dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 02:37:27 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f4774ec145 refactor(apps): remap consumers to central engines
About 38 web-app/web-svc roles now consume the central svc-db-* and svc-dns engines via lookup('engine', ...) instead of embedded sidecars: meta/services flags, variant matrices, and env/conf hosts point at the shared services. Lets these apps run unpinned and NFS-shareable in swarm.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 19:29:39 +02:00
kevinveenbirkenbachandClaude Opus 4.8 51c3d41ede Merge origin/main into feature/swarm-nfs
Reconcile main's object-store feature and pre-swarm deploy tasks with the branch's compose/networks/volumes SPOT and swarm lints.

objstore (seaweedfs/minio): standalone compose + embedded service templates migrated to the container_image/container_volumes/container_networks/compose_volumes/compose_networks/depends_on lookups; meta/volumes.yml on the canonical dict schema with s3.json and nginx.conf as type:config; embedded services made swarm-compatible via compose_only/compose_restart.

deploy tasks (erpnext, opentalk, seaweedfs, shopware, snipe-it, wordpress, yourls): container exec routed through the container_address resolver and gated on IS_STACK_HOST, shell module plus stack_host_template, no chdir or compose-verb forms.

Also removed orphaned vars, restored MAILU_INSTANCE_DIR, freed the seaweedfs subnet (192.168.206.0/24), stripped literal quotes from the nextcloud/bookwyrm env templates, and routed main's objstore compose_volumes tests through dir_var_lib.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 21:08:49 +02:00
kevinveenbirkenbachandClaude Opus 4.8 2cbf9d6ad5 fix(web-app-shopware): wait for the S3 object store and retry asset/theme builds in init.sh so a concurrent SeaweedFS hard-restart no longer fails the deploy (DiD-reproduced)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 02:42:48 +02:00
kevinveenbirkenbachandClaude Opus 4.7 c24252bd33 refactor(env-templates): strip literal double quotes from env-file values
Across 23 web-app env.j2 templates, drop the surrounding "..." from
plain KEY="value" entries. compose silently strips them at env_file
parse time, but docker stack deploy preserves them literally, so the
container then sees KEY="value" with the quotes embedded and parsers
that call int(os.environ['KEY']) or split URLs/DSNs break under swarm.

The change is mechanical: every removed line was `KEY="value"` and
every added line is the same `KEY=value` without surrounding quotes.
Lines that use `| dotenv_quote` are untouched because that filter
deliberately produces escaped values for env_file consumers; the
existing password-quote integration test continues to require it for
sensitive variables.

Affected: bigbluebutton, bluesky, bookwyrm, bridgy-fed, chess,
confluence, decidim, espocrm, friendica, funkwhale, gitea, jira,
magento, mailu, mobilizon, openproject, phpmyadmin, pixelfed, pretix,
shopware, snipe-it, suitecrm, xwiki.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 19:08:45 +02:00
kevinveenbirkenbach ab1c4c2ebb fix(web-app-shopware): skip local public/bundles check when the S3 object-store asset backend is active (DiD root-caused) 2026-06-14 18:07:32 +02:00
kevinveenbirkenbachandClaude Opus 4.7 177f039b11 refactor(web-app-zammad): rename basic_auth_password → api_bot_password + SPOTs
- Rename credential basic_auth_password → api_bot_password and var
  ZAMMAD_ADMINISTRATOR_PASSWORD → ZAMMAD_API_BOT_PASSWORD to reflect
  that the value is the role-internal bot's local password, not the
  administrator's (reused for the wizard-bypass admin only because both
  need the same shell-safe alphanumeric form).
- Introduce ORGANIZATION constant (defaults to SOFTWARE_NAME) and migrate
  organization-string consumers (mattermost team, shopware shop title,
  dashboard login/logout descriptions, simpleicons "provided by",
  user-administrator description/lastname).
- Add HOST_LL_CC_TAG (BCP-47 form, e.g. en-gb) and use in auto_wizard.
- Add NETWORK_TRUSTED_PROXY_CIDRS SPOT and consume from suitecrm,
  shopware and zammad reverse-proxy configs.
- Extract ZAMMAD_WEB_PROTOCOL via lookup('tls', …, 'protocols.web')
  instead of inline 'https' if … else 'http'.
- Use database_query for CREATEDB grant so it works on shared+non-shared
  databases; back the SQL with files/sql/grant_createdb.sql SPOT.
- Read administrator firstname/lastname from users lookup in
  auto_wizard.json.j2 instead of hardcoding.
- Drop redundant elasticsearch: block from web-app-zammad services.yml
  (no consumer reads it).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 00:03:15 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2dbb2aaa1a feat(email): introduce lookup('email') plugin, migrate all consumers
Moves the SYSTEM_EMAIL_* resolution logic out of group_vars/all/03_email.yml
into a dedicated Ansible lookup plugin, so every role template reads outgoing
mail settings via lookup('email') (global) or lookup('email', application_id)
(per-app). Per-app compose.services.email is merged on top of the resolved
defaults; SYSTEM_EMAIL_* variables remain usable as an inventory escape hatch.

Also splits the email docs: administrator-facing configuration stays under
docs/administration/configuration/, plugin contract and extension rules move
to docs/contributing/design/services/email.md. services.md is relocated to
services/base.md so the services/ subtree can host per-service design pages.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 22:47:40 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbach 3bb8c0ce30 refactor(tls): rename tls_resolve lookup to tls
The TLS lookup has been renamed from `tls_resolve` to `tls` to better
reflect its responsibility as a strict TLS state resolver.

This refactor updates all usages across the codebase, including:
- lookup plugins and internal helpers
- roles, vars, tasks, and templates
- tests, stubs, and error messages
- documentation and changelog references

Behavior, lookup API, and semantics remain unchanged.

https://chatgpt.com/share/6985e635-5a94-800f-b794-937e566ee308
2026-02-06 14:01:32 +01:00
kevinveenbirkenbach 5e33e70892 refactor(database lookup usage): switch conditionals to enabled flag
Replace indirect type!='' checks with explicit database.enabled lookups
to improve semantics, readability, and alignment with strict lookup API.

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:25:37 +01:00
kevinveenbirkenbach 8fe6b92e63 refactor(lookups): migrate nginx_paths, tls_resolve and cert_plan to strict positional want-path API
- Replace legacy want= keyword usage with mandatory positional want-path
- Align all nginx_paths, tls_resolve and cert_plan lookups to strict API
- Keep behavior unchanged while enforcing consistent lookup semantics

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:06:09 +01:00
kevinveenbirkenbach f6f96a11ac Refactor: use database lookup helper everywhere
https://chatgpt.com/share/6983bcf6-85bc-800f-a0fc-5684b9078469
2026-02-04 22:52:08 +01:00
kevinveenbirkenbach 8b1ae03e77 fix(env): prevent dotenv_quote double-quoting in env.j2 templates
- Replace `"{{ ... | dotenv_quote }}"` with `{{ ... | dotenv_quote }}` across roles
- Update integration test to enforce pure Jinja + dotenv_quote and forbid wrapped quotes

https://chatgpt.com/share/6974e6c6-51b8-800f-80a1-a76f885cf57e
2026-01-24 16:35:27 +01:00
kevinveenbirkenbach fdb2683ddb test(env): enforce dotenv_quote + double quotes for password vars in env.j2 templates
https://chatgpt.com/share/6974cbac-023c-800f-8198-fd85df2aa7bc
2026-01-24 15:32:33 +01:00
kevinveenbirkenbach aa7d0c642c feat(dotenv): add dotenv_quote filter and enforce quoted password vars in env.j2
https://chatgpt.com/share/6974cbac-023c-800f-8198-fd85df2aa7bc
2026-01-24 15:22:19 +01:00
kevinveenbirkenbach 0b9f71463c refactor(tls): replace legacy tls filters with strict tls_resolve lookup
- Remove deprecated filter_plugins/tls.py and all dependent filters
- Introduce strict lookup plugin lookup_plugins/tls_resolve.py as single TLS authority
- Migrate all roles, templates, vars, and tasks to use tls_resolve (protocols, ports, URLs, cert paths)
- Enforce explicit TLS context (domains, applications, TLS_ENABLED, TLS_MODE)
- Support letsencrypt and self_signed with SAN handling and per-app overrides
- Update TLS provider roles to assert resolver state instead of setting facts
- Add comprehensive unit tests for tls_resolve lookup

https://chatgpt.com/share/6972b49b-b074-800f-a323-dd9fe69d02eb
2026-01-23 00:36:46 +01:00
kevinveenbirkenbach 9fdb0cddd0 Replace global WEB_PROTOCOL / WEB_PORT / WEBSOCKET_PROTOCOL logic
with explicit per-application TLS resolution.

Key changes:
- Introduce sys-front-tls-layer as the single TLS abstraction
- Centralize certificate paths via TLS_CERT_FILE / TLS_KEY_FILE / TLS_CA_FILE
- Replace sys-svc-letsencrypt SSL templates with generic TLS layer includes
- Add tls_* Jinja filters (enabled, mode, protocol, port, websocket)
- Resolve TLS mode per application (letsencrypt | self_signed | off)
- Remove implicit defaults and WEB_* globals to fail fast on misconfiguration
- Support self-signed certificates with SAN awareness
- Make TLS behavior explicit and consistent across all roles

This decouples application logic from certificate providers and
allows switching TLS flavor without touching app roles.

https://chatgpt.com/share/69727c61-8e68-800f-a267-473a7e627978
2026-01-22 20:36:49 +01:00
kevinveenbirkenbach 72929e83fc feat(shopware,credentials): add strong OpenSearch admin password generation
- add strong_password generator with special characters
- introduce opensearch_initial_admin_password credential
- wire OpenSearch admin password via env template
- decouple OpenSearch admin password from Shopware admin user
- minor formatting cleanup in service resolver
2026-01-18 23:33:19 +01:00
kevinveenbirkenbach 8d3e8d0b73 feat(email): unify SYSTEM_EMAIL_* vars and improve msmtp/postfix local delivery
- Replace DEFAULT_SYSTEM_EMAIL merge with flat SYSTEM_EMAIL_* variables
- Update templates across apps to new variable names
- Fix msmtp health check to use -t and local username fallback
- Improve postfix localhost config (localdomain, mydestination, timeouts)
- Extend sys-hostname to ensure /etc/hosts mapping once

https://chatgpt.com/share/695e8f39-0694-800f-b91c-a74f4b78d7c8
2026-01-07 17:52:04 +01:00
kevinveenbirkenbach 0f7add0f1a Unify Mailu token handling via users[*].tokens and clean up inventory validation
- Migrate Mailu credentials from legacy mailu_token to tokens['web-app-mailu']
- Update all affected application templates to use the new token structure
- Adjust inventory validation to no longer special-case mailu_token
- Minor typing and formatting cleanups in inventory creation and tests

https://chatgpt.com/share/69554915-87c8-800f-afd5-12118dbb1107
2025-12-31 17:02:20 +01:00
kevinveenbirkenbach 26fa760847 Added system email enabled switch 2025-12-30 12:39:28 +01:00
kevinveenbirkenbach 6d6b0fdea6 Implemented CLI installer draft and replaced OIDC by OAuth2 Proxy for SuiteCRM - See https://chatgpt.com/share/69274924-9dec-800f-8653-345aa2c25664 2025-11-27 01:10:11 +01:00
kevinveenbirkenbach 493d5bbbda refactor(web-app-shopware): externalize trusted proxy and host configuration with mounted framework.yaml
- added new file roles/web-app-shopware/files/framework.yaml defining trusted_proxies and trusted_headers for Symfony
 - mounted framework.yaml into /var/www/html/config/packages/ in docker-compose
 - exposed new role vars SHOPWARE_FRAMEWORK_HOST/DOCKER for mounting path
 - rendered framework.yaml via Ansible copy task with proper permissions
 - adjusted env.j2 to set TRUSTED_PROXIES and TRUSTED_HOSTS dynamically from domains and networks
 - added SHOPWARE_DOMAIN var to vars/main.yml
 - removed inline framework.yaml creation from Dockerfile (now managed via mount)
 - updated proxy template (html.conf.j2) to include X-Forwarded-Ssl header
 - improved init.sh permission handling for shared volumes

See ChatGPT conversation for implementation details and rationale:
https://chatgpt.com/share/690d4fe7-2830-800f-8b6d-b868e7fe0e97
2025-11-07 02:48:49 +01:00
kevinveenbirkenbach d66ad37c5d enh(shopware): improve healthchecks and proxy configuration
Removed obsolete EXPOSE/healthcheck from Dockerfile and added robust service-specific healthchecks:

- web: HTTP robots.txt check

- worker/scheduler: php -v runtime check

- opensearch: cluster health API check

Added TRUSTED_PROXIES=* for proxy-aware headers and centralized OPENSEARCH_PORT in vars.

Context: discussed implementation details in ChatGPT conversation on 2025-11-06 — https://chatgpt.com/share/690c9fb3-79f4-800f-bbdf-ea370c8f142c
2025-11-06 14:17:00 +01:00
kevinveenbirkenbach 48557b06e3 refactor(web-app-shopware): make init script idempotent and handle admin via Ansible
- moved init.sh from template to files/ for direct copying and bind mounting
- removed hardcoded user creation from init process
- added database emptiness check before running system:install
- added new task 03_admin.yml to ensure admin user exists and update password/email via Ansible
- switched docker exec shell from bash to sh for Alpine compatibility
- updated Dockerfile and docker-compose.yml accordingly for mount-based init script
2025-11-03 03:36:13 +01:00
kevinveenbirkenbach 1cff5778d3 Activated debugging 2025-11-03 02:42:52 +01:00
kevinveenbirkenbach f5efbce205 feat(shopware): migrate to single Shopware base image and split services (web/worker/scheduler/init)
• Introduce init container and runtime-ready Dockerfile (Alpine) installing php83-gd/intl/pdo_mysql
• Disable composer scripts in builder and ignore build-time ext reqs
• New docker-compose template (web/worker/scheduler/opensearch) + persistent volumes
• Use TRUSTED_PROXIES env; fix APP_URL formatting; set OPENSEARCH_HOST=opensearch
• Replace SHOPWARE_PHP_CONTAINER refs with SHOPWARE_WEB_CONTAINER in tasks
• Render and copy init.sh via volumes path
• Remove old nginx/php split and legacy DB env task
• Fix svc-db-postgres var: database_type now uses entity_name
https://chatgpt.com/share/6907fc58-7c28-800f-a993-c207f28859c9
2025-11-03 01:51:38 +01:00
kevinveenbirkenbach d6f3618d70 Add reusable HTTP healthcheck template and integrate into Shopware and Taiga roles 2025-11-02 22:26:42 +01:00
kevinveenbirkenbach 7bc9f7abd9 Refactor Shopware role to use dedicated OpenSearch service and improved environment handling.
Changes include:
- Added OpenSearch configuration and variable definitions (image, version, heap, memory limits)
- Replaced legacy search/elasticsearch logic with OpenSearch integration
- Updated docker-compose template for OpenSearch with proper JVM heap and ulimits
- Ensured both OPENSEARCH_URL and ELASTICSEARCH_URL are set for compatibility

Reference: https://chatgpt.com/share/6907b0d4-ab14-800f-b576-62c0d26c8ad1
2025-11-02 21:05:52 +01:00
kevinveenbirkenbach d1ccfd9cdd Add new Shopware 6 role with OIDC/LDAP plugin integration and Docker-based deployment configuration.
Includes:
- New role: web-app-shopware (Docker, MariaDB, Redis, OpenSearch)
- Updated networks and ports configuration
- Automated install, migration, and admin creation
- Optional IAM integration via OIDC/LDAP plugins

Reference: https://chatgpt.com/share/6907b0d4-ab14-800f-b576-62c0d26c8ad1
2025-11-02 20:29:13 +01:00