Commit Graph
22 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 d3ca143343 docs(roles): regenerate READMEs for Stalwart as the mail provider
MAIL_PROVIDER moved to web-app-stalwart without a make docs run, so every generated README still drew Mailu as the email dependency. This is the generator output for the current tree: the root role table and each role cosmos now name Stalwart, Mailu loses its dependents, and Stalwart gains its swarm marker, the Tor dependencies and the 0..0 edges.

The hand-written cosmos of svc-net-tor-smtp gives way to the generated one, which no longer shows the SOCKS5 hop through svc-net-tor because the role does not declare that dependency.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 19:57:30 +02:00
Alejandro Roman 4b9a834546 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-21 03:25:20 +04:00
kevinveenbirkenbachandClaude Opus 5 4986e80dd4 feat(readme): ask the reader for a domain the deployment can resolve
The Production block never set one. inventory provision writes only
ansible_connection for a local host, so DOMAIN_PRIMARY stayed at the
group_vars default of infinito.localhost and the deployment served a zone its
own resolver knows nothing about. CoreDNS templates answers for
INFINITO_DOMAIN_RE and nothing else, so every lookup of the deployment's own
name came back NXDOMAIN.

Mastodon is where that surfaced. tootctl accounts create validates the address
through EmailMxValidator, which resolves MX and then, for the domain itself,
A and AAAA. All six queries returned NXDOMAIN in run 33574056480, the
validator recorded email :unreachable, and the admin account was never created.
Only the guide rows were affected: the matrix rows write
DOMAIN_PRIMARY: "{{ lookup('env', 'INFINITO_DOMAIN') }}" into their host_vars
and resolve fine.

The template now carries DOMAIN=<your-domain> in both the host and the
container form and threads it into --vars, with the container form passing it
through the docker run boundary as well. The two guide runners substitute it
the way they already substitute the SSH key, so the replay deploys under
INFINITO_DOMAIN like every other CI deploy. compose_deploy.sh moves its env
load ahead of the substitutions to have the value; host_deploy.sh had no env
load at all.

A production reader is asked for a domain now instead of silently inheriting
infinito.localhost, which no real deployment wants either. The shipped default
keeps its value: sys-svc-nss documents that *.localhost resolves through the
myhostname NSS module without any zone, and host-side tasks lean on it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 18:30:25 +02:00
Alejandro Roman 333079cc2a feat(UI): fix migration, errors happening in local deploy and swarm integraton + merged main 2026-08-30 21:39:52 +04:00
kevinveenbirkenbachandClaude Opus 5 c55f508e05 feat(readme): draw mandatory and disabled service relations
The cosmos generator linked a consumer to its provider only through an
'<role>' in group_names reference or an explicit provides: name. A hard
dependency has neither -- pinning the provider on unconditionally is
exactly what removes the Jinja reference the generator was reading -- so
web-svc-asset's bond to web-svc-file vanished from the diagram the moment
it became binding. Match a service key against the provider's own primary
key from the registry, in both directions.

Services the role turns off drew a solid 1:1 edge, claiming a mandatory
relationship for something that carries nothing at deploy time. They now
render as a red 0..0 edge; edge merging gained a real precedence so a
provider consumed through two keys stays live when only one is off.

Named colour, not hex: a '#' opens an entity code in the mermaid lexer and
turns the whole diagram into a parse error on GitHub.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:02:31 +02:00
kevinveenbirkenbachandClaude Opus 5 ef5ea7cb55 refactor(roles): name the credential file meta/secrets.yml
meta/schema.yml held credential definitions and their runtime values, but the
name said 'schema' and read like structural metadata next to services.yml and
volumes.yml. Nothing in the file is a schema in the sense the other meta files
use the word, and a contributor had to open it to learn it holds secrets.

62 role files renamed. ROLE_FILE_META_SCHEMA becomes ROLE_FILE_META_SECRETS and
keeps its role as the SPOT the path is built from.

Two readers bypassed that constant and spelled the name themselves:
utils/cache/applications.py built meta_dir / 'schema.yml' twice, and
plugins/lookup/application_gid.py carries the filename in the marker list it
detects application roles by. Left behind, the rename would not have raised —
credentials would have resolved empty and the role would have stopped counting
as an application.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 12:08:06 +02:00
kevinveenbirkenbachandClaude Opus 5 946b3769cb feat(svc-net-tor): let CI deploy the role in compose and swarm
modes.compose.enabled and modes.swarm.enabled were both false, and a
stack role is forced host=false, so all three test_<mode> columns were
false and the CI discovery filter (test_<mode> == true) dropped the role
from every matrix. It was resolved into the priority whitelist as a new
role and then silently filtered out again — svc-net-tor was only ever
exercised indirectly, as a dependency other roles pull in.

The flag is read solely by the discovery model and the README generator;
nothing in the deploy path consults it, so this has no runtime effect.

meta/categories.yml pins the whole net category to compose/swarm false,
which fits svc-net-wireguard-{core,plain,firewalled}: none of them ships
a compose template. svc-net-tor does, and is the only role in the
category that does, so it takes the category-modes exemption the lint
provides rather than bending the category default for three host roles.

Regenerated READMEs follow: svc-net-tor now carries its mode symbols in
every cosmos diagram that names it as a dependency.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 19:02:24 +02:00
kevinveenbirkenbachandClaude Fable 5 62a46e472f Merge branch 'feature/swarm-nfs' into feature/svc-net-tor
Resolves the runner Dockerfile to the lint-standard Acquire::Retries=3
and keeps the collabora TODO wording; hardens the merged tree for the
stricter lints (apt retries in the tor Dockerfile, shell-or-true
markers, comment cleanup).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 16:27:25 +02:00
kevinveenbirkenbachandClaude Fable 5 3125c5ac03 docs: regenerate role catalogs with the tor onion wiring
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-22 15:49:50 +02:00
Alejandro Roman 1915ec115b Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-22 12:16:40 +02:00
kevinveenbirkenbachandClaude Opus 4.8 ea9ebe0de3 refactor(inventory): rename prod inventory to production
Standardize the private inventory dir on the fully spelled "production" (matching inventories/development). Update the role README template (SPOT) and regenerate 153 role READMEs plus the admin deploy guide; add inventories/production/ to .gitignore, propagated to .dockerignore via make fix-dockerignore.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-21 19:08:08 +02:00
Alejandro Roman ac101d2efa Merge kevinveenbirkenbach/feature/swarm-nfs into stalwart integration branch 2026-07-21 10:00:17 +02:00
kevinveenbirkenbachandClaude Opus 4.8 9ad8fa2971 feat(guide): default self_signed and test the whole README block
The guide deploys to a .localhost target, where letsencrypt cannot issue (Cloudflare DNS-01 needs a real domain). Default TLS_MODE=self_signed in the generated Production block so the guide uses the working self-signed provider. Document every deploy parameter as a top-level shell var (APP, HOST, TLS_MODE, SSH_PUBLIC_KEY) instead of an inline placeholder.

The compose guide now extracts and runs the whole README block, sourcing params from it (no more -e injection); HOST is forced to localhost. Skip checkout/login under ACT so the guide workflow is iterable locally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 21:23:38 +02:00
kevinveenbirkenbachandClaude Opus 4.8 972596a758 fix(guide): document the administrator SSH key step and seed it in CI via --vars
The README Production block went provision -> deploy with no admin-key step, so user-administrator failed early (authorized_keys empty) for both the Guide CI jobs and any real user copying the README. Add a visible 'provision --vars' step that sets users.administrator.authorized_keys (merged into host_vars) in both Production variants of the role README template, regenerate all role READMEs, and replace the invisible CI payload splice with a visible substitution of the documented <your-ssh-public-key> placeholder.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-20 05:45:26 +02:00
kevinveenbirkenbachandClaude Fable 5 17cd53609a feat(readme): merge the Production deploy into one docker command
The role README Production section now runs provision and deploy in a
single docker call: APP and HOST are shell variables passed via -e,
INVENTORY is the in-container SPOT for the duplicated paths, and the
redundant --vars-file (optional, default None) and --id (the inventory
is already narrowed by --include) flags are gone. All 153 role READMEs
regenerated from the template; test-run.yml executes exactly this block.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 13:03:25 +02:00
kevinveenbirkenbachandClaude Opus 4.8 dd3d83148e feat(readme): show Ansible role dependencies in Cosmos with a gear marker
Roles declared under meta/main.yml dependencies now appear on both the
Dependencies and Dependents side, labelled with the glossary gear
symbol and connected by a fixed 1:1 edge; the legend explains it. All
role READMEs regenerated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:44:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 a9fd37907b feat(readme): cardinality labels and a symbol legend on Cosmos diagrams
Edges now carry their relation: solid 1:1 for fixed consumption, dashed
0..1 for conditional (group-membership gated) consumption. Below every
diagram the generator renders a reading key for the edge kinds and the
node markers, with the glyphs drawn from the shared symbol glossary.
All role READMEs regenerated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 14:44:34 +02:00
kevinveenbirkenbachandClaude Opus 4.8 90b3845998 feat(readme): role README generator with Cosmos diagrams, Quick Setup, and lint
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).

The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.

test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.

Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.

Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-17 14:44:33 +02:00
kevinveenbirkenbachandClaude Opus 4.8 674d26de3a feat(lint/role-readme): derive Credits author from meta author SPOT
Role README Credits now name galaxy_info.author (plain bold or wrapped
in a Markdown link), matched byte-for-byte against the role's own
meta/main.yml, instead of a hard-coded maintainer name. Regenerate all
245 role README Credits blocks to the new 'Implemented by ...' format.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 16:47:34 +02:00
Alejandro Roman 2f693f00e6 feat(stalwart):Created the role and the connection like mailu had it 2026-06-30 16:17:48 +02:00
kevinveenbirkenbachandClaude Opus 4.7 39c990a7ad feat(web-app-erpnext): ship ERPNext role with OIDC SSO, LDAP, outbound mail
Implements requirement docs/requirements/024-web-app-erpnext.md.

Stack (per frappe_docker v15 single-image multi-role pattern):
- configurator (one-shot bench set-config: db_host/port, three redis URLs
  split by DB number 0/1/2 for cache/queue/socketio, socketio_port, host_name
  HTTPS)
- backend (gunicorn), frontend (nginx-entrypoint.sh), websocket (socketio),
  scheduler, queue-short (--queue short,default), queue-long
  (--queue long,default,short)

Central-service consumer pattern (Decision #7):
- ERPNEXT_DB_ROOT_PASSWORD branches on services.mariadb.shared: central
  svc-db-mariadb root pw when true, per-role MariaDB consumer pw when false
- Redis via in-compose `redis` alias, 3-DB-number split for Frappe's logical
  buses
- Explicit erpnext_assets volume mount on every container (image's
  VOLUME ['sites/assets'] otherwise creates anonymous volumes that fail the
  hlth-volumes check)

Bench bootstrap (Decision #9):
- bench new-site --install-app erpnext with 30-min hard timeout
- Post-install restart of all long-lived Frappe containers (otherwise the
  cached app set serves HTTP 500 on /login)
- TCP-port readiness probe (HTTP would 404 pre-site)
- API-bot user seed + setup_complete=1 via apply_api_bot.py

OIDC (Decisions #4 + #5):
- redirect_uris filter auto-includes ERPNext in the shared Keycloak client
- Social Login Key wired via apply_oidc_settings.py with
  social_login_provider=Keycloak (delete+recreate due to set_only_once),
  custom_base_url=1, relative authorize/token/userinfo paths,
  auth_url_data={response_type:code,scope:openid}, login_via_keycloak
  redirect path
- Group-to-Frappe-role mapping persisted as a frappe.db default; per-login
  reconciliation hook deferred to follow-up (documented in README)

LDAP (V3): LDAP Settings doctype applied via apply_ldap_source.py. End-to-end
LDAP login (auto-create Frappe user from LDAP bind) deferred to follow-up;
LDAP-login Playwright specs intentionally absent in v1 (documented).

Email (Decision #8, outbound only): outbound Email Account applied via
apply_email_account.py using db_insert/db_update to bypass Frappe's
deploy-time SMTP socket validation. IMAP inbound deferred to follow-up.

Variant matrix (Decision #11): V1 sso+ldap, V2 all-false, V3 ldap-only.
Three variants verified end-to-end via
`make compose-deploy mode=reinstall apps=web-app-erpnext full_cycle=true
purge=true` — 6 PLAY RECAPs, all failed=0, all Playwright runs
passed-or-skipped with no failures.

Playwright specs: test-landing, test-csp-headers, test-login-administrator
(OIDC), test-login-biber (OIDC), test-login-native-administrator
(break-glass local login, always runs), test-guest-persona. _shared.js
exposes signInViaErpnextOidc + signInViaErpnextLocal.

Repo-wide lint addition: tests/lint/ansible/modules/test_task_name_length.py
caps Ansible task `name:` strings at 120 chars (opt-out via
`# nocheck: name-length`). Fixed two pre-existing violations in
web-app-keycloak/tasks/update/ldap/per_app_mapper_one.yml.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-03 09:53:56 +02:00
kevinveenbirkenbachandClaude Opus 4.7 014fa54ad3 feat(web-app-erpnext): ship ERPNext role with OIDC SSO, LDAP, outbound mail
Implements requirement docs/requirements/024-web-app-erpnext.md.

Stack (per frappe_docker v15 single-image multi-role pattern):
- configurator (one-shot bench set-config: db_host/port, three redis URLs
  split by DB number 0/1/2 for cache/queue/socketio, socketio_port, host_name
  HTTPS)
- backend (gunicorn), frontend (nginx-entrypoint.sh), websocket (socketio),
  scheduler, queue-short (--queue short,default), queue-long
  (--queue long,default,short)

Central-service consumer pattern (Decision #7):
- ERPNEXT_DB_ROOT_PASSWORD branches on services.mariadb.shared: central
  svc-db-mariadb root pw when true, per-role MariaDB consumer pw when false
- Redis via in-compose `redis` alias, 3-DB-number split for Frappe's logical
  buses
- Explicit erpnext_assets volume mount on every container (image's
  VOLUME ['sites/assets'] otherwise creates anonymous volumes that fail the
  hlth-volumes check)

Bench bootstrap (Decision #9):
- bench new-site --install-app erpnext with 30-min hard timeout
- Post-install restart of all long-lived Frappe containers (otherwise the
  cached app set serves HTTP 500 on /login)
- TCP-port readiness probe (HTTP would 404 pre-site)
- API-bot user seed + setup_complete=1 via apply_api_bot.py

OIDC (Decisions #4 + #5):
- redirect_uris filter auto-includes ERPNext in the shared Keycloak client
- Social Login Key wired via apply_oidc_settings.py with
  social_login_provider=Keycloak (delete+recreate due to set_only_once),
  custom_base_url=1, relative authorize/token/userinfo paths,
  auth_url_data={response_type:code,scope:openid}, login_via_keycloak
  redirect path
- Group-to-Frappe-role mapping persisted as a frappe.db default; per-login
  reconciliation hook deferred to follow-up (documented in README)

LDAP (V3): LDAP Settings doctype applied via apply_ldap_source.py. End-to-end
LDAP login (auto-create Frappe user from LDAP bind) deferred to follow-up;
LDAP-login Playwright specs intentionally absent in v1 (documented).

Email (Decision #8, outbound only): outbound Email Account applied via
apply_email_account.py using db_insert/db_update to bypass Frappe's
deploy-time SMTP socket validation. IMAP inbound deferred to follow-up.

Variant matrix (Decision #11): V1 sso+ldap, V2 all-false, V3 ldap-only.
Three variants verified end-to-end via
`make compose-deploy mode=reinstall apps=web-app-erpnext full_cycle=true
purge=true` — 6 PLAY RECAPs, all failed=0, all Playwright runs
passed-or-skipped with no failures.

Playwright specs: test-landing, test-csp-headers, test-login-administrator
(OIDC), test-login-biber (OIDC), test-login-native-administrator
(break-glass local login, always runs), test-guest-persona. _shared.js
exposes signInViaErpnextOidc + signInViaErpnextLocal.

Repo-wide lint addition: tests/lint/ansible/modules/test_task_name_length.py
caps Ansible task `name:` strings at 120 chars (opt-out via
`# nocheck: name-length`). Fixed two pre-existing violations in
web-app-keycloak/tasks/update/ldap/per_app_mapper_one.yml.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-02 21:01:38 +02:00