Commit Graph
21 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5 cc50e92b2d feat(baserow): serve MCP and address containers by node in swarm
Provision the native MCP server surface: create the endpoint, its owner
and its workspace, probe it on the internal origin, and refuse an
endpoint that never serves the stream. Baserow addresses a session
through the URL path, so meta/services.yml declares the endpoint key and
suffix the discovery layer composes the client URL from.

Split the role into a run-once guarded core. Registering the MCP surface
makes baserow a shared provider, and the variant-aware loader checks the
run-once flag before loading; without the flag as the first task a second
load request can include the role again.

Address the container by its hosting node. Migration and bootstrap ran
`container exec` on the manager while the replica sits on a worker, so
the resolved id existed nowhere the command could reach it. Re-resolve
after the post-migration restart as well: `service update --force`
replaces the container, and the bootstrap was still holding the id of the
one it replaced.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-02 22:06:11 +02:00
kevinveenbirkenbachandClaude Opus 4.8 15e7647d2f refactor(env): central TZ+LANG base included by every role env.j2
New roles/sys-svc-container/templates/env.j2 sets TZ={{ HOST_TIMEZONE }} and LANG=C.UTF-8. Every role env.j2 includes it at the top; it renders via stack_host_template, the same engine that resolves the includes in compose.yml.j2, so it works in both compose and swarm. TZ/LANG are removed from the individual role env.j2 so each value stays single-source.

New lint tests/lint/ansible/test_env_base_include.py verifies every role env.j2 includes the base and never re-sets a base-owned key (keys derived from the base file).

mattermost and gitea meta/volumes.yml drop the /etc/timezone host-path bind: swarm hard-rejects a task whose bind source is missing on the node (Debian-trixie nodes have no /etc/timezone). TZ now comes from the base env. /etc/localtime bind also dropped; the container timezone follows TZ.

Non-directive comments were removed from the touched env.j2 (banners, restatements, reference URLs, narration) so the changed files pass the comment-lint; only tool directives remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 22:13:57 +02:00
kevinveenbirkenbachandClaude Opus 4.8 27e4ccf89d fix(baserow): external locked_migrate as sole migrator + timebox
The all-in-one image runs supervisord with the startup migrate and celery-beat concurrently. With MIGRATE_ON_STARTUP defaulting true, the image's startup migrate races celery-beat (which crash-loops on the un-migrated schema) and deadlocks on the locked_migrate advisory lock; the schema never completes and the external locked_migrate task then blocks forever on that same lock. Set MIGRATE_ON_STARTUP=false so the external task is the sole migrator, and add timeout: 600 to the migrate and bootstrap exec tasks so a stuck exec fails cleanly instead of hanging (retries stay 60; timeout bounds the whole task).

Verified in dev container (deploy-4): locked_migrate now completes in ~6s (was a 30min+ hang); deploy reached ok=1879 end-to-end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 00:44:37 +02:00
kevinveenbirkenbachandClaude Opus 4.8 b48ab523c7 feat(web-app-baserow): trusted-header SSO bridge, drop LDAP
Baserow's native SSO is Enterprise-only, so gate the community image with the shared Keycloak oauth2-proxy and patch urls.py (via a custom Dockerfile) with a trusted-header bridge. The bridge provisions/repairs a local Baserow user from the oauth2-proxy identity headers and mints native refresh/access/user_session tokens, mirroring Baserow's own SSO redirect contract (?token=&user_session=).

proxy.conf.j2 performs the nginx handoff: an authenticated browser without a jwt_token cookie is 302'd to /api/infinito/sso/login/, which redirects back to /?token=&user_session= for the frontend middleware to consume. bootstrap_admin marks the superuser email_verified + completed_onboarding to skip onboarding.

Replace the LDAP service/variant with proxy-gating and rewrite the seaweedfs + persona Playwright tests to drive Baserow through its API using the SSO-minted JWT.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 12:47:35 +02:00
kevinveenbirkenbachandClaude Opus 4.8 0ae22d7c59 feat(objstore): consume the object store across 14 web-app roles
Switch penpot, nextcloud, gitea, mattermost, openwebui, funkwhale, baserow, openproject, pixelfed, mastodon, bookwyrm, listmonk, peertube and opentalk to the engine-agnostic objstore lookup: the minio service entries are pinned off, a seaweedfs service (gated on the provider being in group_names, public for the media apps) replaces them, and the matrix variants pin the new flag. OpenTalk keeps object storage mandatory through an embedded fallback and provisions its bucket via weed shell.

Add objstore lookup SPOT keys consumed by these roles: endpoint (host:port), public_domain, public_bucket (public S3 host plus bucket) and public_url (scheme plus public_bucket, scheme derived from the provider TLS state). This removes the repeated lookup('config', lookup('objstore', ...), 'server.domains.canonical.api') / host:port concatenations from the consumer templates. Listmonk settings render their JSON values via the to_json filter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 21:00:13 +02:00
kevinveenbirkenbachandClaude Opus 4.7 bc2bc5400b fix(web-app-baserow): defensive env so variant-1 backend actually binds :8000
In CI run 26005522256, baserow Round 2 (variant 1, fresh sidecar
postgres) responded with status 500 + "connect ECONNREFUSED
127.0.0.1:8000" from baserow's internal Caddy. Container was up,
sidecar postgres + redis were Healthy, migrations succeeded (auth_user
existed, bootstrap admin worked), but gunicorn never came online on
:8000.

Cross-referenced baserow's official docs and the all-in-one image's
startup scripts:
- deploy/all-in-one/baserow.sh + supervisor/start.sh: embedded-postgres
  trigger is DATABASE_HOST==localhost AND DATABASE_URL empty. Our
  DATABASE_HOST=database avoids that, but DISABLE_EMBEDDED_PSQL/REDIS
  removes the path-of-least-resistance fallback inside supervisor.
- backend/docker/docker-entrypoint.sh: wait_for_postgres tries
  BASEROW_POSTGRES_STARTUP_CHECK_ATTEMPTS×2s before giving up (default
  5×2s = 10s, too tight for a fresh sidecar's first migration round).
- supervisor.conf: [program:backend] has startsecs=30, startretries=3.
  A long template-import pass (SYNC_TEMPLATES_ON_STARTUP=true is the
  default) after migrations can starve the gunicorn bind on :8000 and
  leave Caddy returning ECONNREFUSED.

Changes:
- vars/main.yml: introduce BASEROW_DATABASE_USER/NAME/HOST/PORT/PASSWORD
  + BASEROW_DATABASE_URL and BASEROW_REDIS_HOST/PORT/URL as single
  SPOTs derived from lookup('database', ...). env.j2 renders all of
  them from these vars instead of duplicating the lookup calls.
- templates/env.j2: emit discrete DATABASE_* AND DATABASE_URL together
  (docs note URL completely overrides discrete, but the all-in-one
  startup script also reads DATABASE_HOST directly to decide on
  embedded mode, so both must be present). Same shape for REDIS_*.
- templates/env.j2: add DISABLE_EMBEDDED_PSQL=yes, DISABLE_EMBEDDED_
  REDIS=yes, SYNC_TEMPLATES_ON_STARTUP=false, BASEROW_TRIGGER_SYNC_
  TEMPLATES_AFTER_MIGRATION=false, BASEROW_DISABLE_PLUGIN_INSTALL_ON_
  STARTUP=yes, BASEROW_POSTGRES_STARTUP_CHECK_ATTEMPTS=30.

Verified each name and semantics against
baserow/docs/installation/configuration.md and the actual source code
in baserow/deploy/all-in-one/. Notable: the docs name the variable
POSTGRES_STARTUP_CHECK_ATTEMPTS at line 108 without the BASEROW_
prefix, but the source uses BASEROW_POSTGRES_STARTUP_CHECK_ATTEMPTS —
source wins.

Caveat: EMAIL_SMTP is intentionally NOT emitted in the disabled
branch. The docs (line 335) state any non-empty value enables SMTP,
so an "EMAIL_SMTP=FALSE" defensive default would have inverted the
intent. Leaving the variable unset is the documented way to fall
back to console-only email logging.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 13:43:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 2dbb2aaa1a feat(email): introduce lookup('email') plugin, migrate all consumers
Moves the SYSTEM_EMAIL_* resolution logic out of group_vars/all/03_email.yml
into a dedicated Ansible lookup plugin, so every role template reads outgoing
mail settings via lookup('email') (global) or lookup('email', application_id)
(per-app). Per-app compose.services.email is merged on top of the resolved
defaults; SYSTEM_EMAIL_* variables remain usable as an inventory escape hatch.

Also splits the email docs: administrator-facing configuration stays under
docs/administration/configuration/, plugin contract and extension rules move
to docs/contributing/design/services/email.md. services.md is relocated to
services/base.md so the services/ subtree can host per-service design pages.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-21 22:47:40 +02:00
kevinveenbirkenbachandClaude Opus 4.6 b70a783423 refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup
plugins and runtime hydration. Introduces `add_csp_hash` filter,
integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak
one-shot realm import into `05a_realm_import_run.yml`, and fixes the
Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter,
`sys-token-store` Jinja filter, and empty CLI setup modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-15 23:44:19 +02:00
kevinveenbirkenbach 5e33e70892 refactor(database lookup usage): switch conditionals to enabled flag
Replace indirect type!='' checks with explicit database.enabled lookups
to improve semantics, readability, and alignment with strict lookup API.

https://chatgpt.com/share/6985d642-b40c-800f-9cdf-ed54fc484a92
2026-02-06 13:25:37 +01:00
kevinveenbirkenbach f6f96a11ac Refactor: use database lookup helper everywhere
https://chatgpt.com/share/6983bcf6-85bc-800f-a0fc-5684b9078469
2026-02-04 22:52:08 +01:00
kevinveenbirkenbach 6096890998 feat(web-app-baserow): add Django admin bootstrap and SECRET_KEY handling
- add optional idempotent bootstrap of Django superuser via script module
- execute admin bootstrap through docker compose exec with env injection
- avoid shell/quoting issues by passing credentials via environment
- introduce BASEROW_SECRET_KEY and wire it into container env
- ensure container recreation when DB credentials change
- document admin bootstrap and SECRET_KEY requirements in README

https://chatgpt.com/share/6975538c-a6d0-800f-b2ea-fbab963ef142
2026-01-25 00:19:31 +01:00
kevinveenbirkenbach 8b1ae03e77 fix(env): prevent dotenv_quote double-quoting in env.j2 templates
- Replace `"{{ ... | dotenv_quote }}"` with `{{ ... | dotenv_quote }}` across roles
- Update integration test to enforce pure Jinja + dotenv_quote and forbid wrapped quotes

https://chatgpt.com/share/6974e6c6-51b8-800f-80a1-a76f885cf57e
2026-01-24 16:35:27 +01:00
kevinveenbirkenbach fdb2683ddb test(env): enforce dotenv_quote + double quotes for password vars in env.j2 templates
https://chatgpt.com/share/6974cbac-023c-800f-8198-fd85df2aa7bc
2026-01-24 15:32:33 +01:00
kevinveenbirkenbach aa7d0c642c feat(dotenv): add dotenv_quote filter and enforce quoted password vars in env.j2
https://chatgpt.com/share/6974cbac-023c-800f-8198-fd85df2aa7bc
2026-01-24 15:22:19 +01:00
kevinveenbirkenbach 8d3e8d0b73 feat(email): unify SYSTEM_EMAIL_* vars and improve msmtp/postfix local delivery
- Replace DEFAULT_SYSTEM_EMAIL merge with flat SYSTEM_EMAIL_* variables
- Update templates across apps to new variable names
- Fix msmtp health check to use -t and local username fallback
- Improve postfix localhost config (localdomain, mydestination, timeouts)
- Extend sys-hostname to ensure /etc/hosts mapping once

https://chatgpt.com/share/695e8f39-0694-800f-b91c-a74f4b78d7c8
2026-01-07 17:52:04 +01:00
kevinveenbirkenbach 0f7add0f1a Unify Mailu token handling via users[*].tokens and clean up inventory validation
- Migrate Mailu credentials from legacy mailu_token to tokens['web-app-mailu']
- Update all affected application templates to use the new token structure
- Adjust inventory validation to no longer special-case mailu_token
- Minor typing and formatting cleanups in inventory creation and tests

https://chatgpt.com/share/69554915-87c8-800f-afd5-12118dbb1107
2025-12-31 17:02:20 +01:00
kevinveenbirkenbach 26fa760847 Added system email enabled switch 2025-12-30 12:39:28 +01:00
kevinveenbirkenbach 942e8c9c12 Updated baserow CSP adn variables for new Infinito.Nexus structure 2025-08-26 22:20:31 +02:00
kevinveenbirkenbach d3cc187c3b Made System Email Variables UPPER 2025-08-19 09:34:18 +02:00
kevinveenbirkenbach 4c7ee0441e Solved baserow variable bugs 2025-08-12 11:23:56 +02:00
kevinveenbirkenbach 563d5fd528 Huge role refactoring/cleanup. Other commits will propably follow. Because some bugs will exist. Still important for longrun and also for auto docs/help/slideshow generation 2025-07-08 23:43:13 +02:00