Standardize the private inventory dir on the fully spelled "production" (matching inventories/development). Update the role README template (SPOT) and regenerate 153 role READMEs plus the admin deploy guide; add inventories/production/ to .gitignore, propagated to .dockerignore via make fix-dockerignore.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A transient Debian/Ubuntu mirror fetch aborts a docker build layer with exit 100 (it killed the web-app-nextcloud swarm openldap build while the same image built fine elsewhere). New lint under tests/lint/docker/dockerfile/apt/ flags any RUN with apt-get update/install that omits Acquire::Retries; retrofit all 15 apt Dockerfiles with -o Acquire::Retries=3. Both apt lints move into the apt/ subpackage. dnf retries by default and apk has no equivalent flag, so both stay out of scope.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The guide deploys to a .localhost target, where letsencrypt cannot issue (Cloudflare DNS-01 needs a real domain). Default TLS_MODE=self_signed in the generated Production block so the guide uses the working self-signed provider. Document every deploy parameter as a top-level shell var (APP, HOST, TLS_MODE, SSH_PUBLIC_KEY) instead of an inline placeholder.
The compose guide now extracts and runs the whole README block, sourcing params from it (no more -e injection); HOST is forced to localhost. Skip checkout/login under ACT so the guide workflow is iterable locally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The README Production block went provision -> deploy with no admin-key step, so user-administrator failed early (authorized_keys empty) for both the Guide CI jobs and any real user copying the README. Add a visible 'provision --vars' step that sets users.administrator.authorized_keys (merged into host_vars) in both Production variants of the role README template, regenerate all role READMEs, and replace the invisible CI payload splice with a visible substitution of the documented <your-ssh-public-key> placeholder.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The role README Production section now runs provision and deploy in a
single docker call: APP and HOST are shell variables passed via -e,
INVENTORY is the in-container SPOT for the duplicated paths, and the
redundant --vars-file (optional, default None) and --id (the inventory
is already narrowed by --include) flags are gone. All 153 role READMEs
regenerated from the template; test-run.yml executes exactly this block.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rewrite all 254 role descriptions to a uniform noun-phrase schema
(function-first, no leading verb, no trailing period, <=120 chars) and
enforce it via the extended description lint. Rename 12 role README H1
titles for clarity (e.g. Backup Container to Local -> Backup Docker
Volumes, CSS Service -> Corporate Design, NGINX Legal -> Imprint) and
regenerate their Cosmos and Quick Setup sections. Overview table
regenerated.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add cli.build.readme.overview (make readme-index / make docs) rendering a
roles overview in the root README: Name, Lifecycle, Homepage, Video,
Description, Integrated with, Install. New info-media lint requires
homepage + video in every invokable role's meta/info.yml (opt out via
'# nocheck: info-media' in meta/main.yml); backfilled 147 roles with
researched, adversarially-verified homepages and official videos.
Retire the 'unsupported' lifecycle value: fold it into 'eol' as a valid
live end-of-life status (shipped but untested/unmaintained) across the
stage lists, symbol glossary, allow-list lint and docs. Retag confluence,
jira, minio, phpldapadmin to eol and magento to pre-alpha.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Roles declared under meta/main.yml dependencies now appear on both the
Dependencies and Dependents side, labelled with the glossary gear
symbol and connected by a fixed 1:1 edge; the legend explains it. All
role READMEs regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Edges now carry their relation: solid 1:1 for fixed consumption, dashed
0..1 for conditional (group-membership gated) consumption. Below every
diagram the generator renders a reading key for the edge kinds and the
node markers, with the glyphs drawn from the shared symbol glossary.
All role READMEs regenerated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add cli/build/readme: derives each role README's managed sections from
templates/roles/README.md.j2.tmpl (the Cosmos mermaid diagram from
meta/services.yml, an invokable-gated Quick Setup, and Credits).
The Cosmos diagram shows dependencies, every declared service, and
dependents (roles that consume this one, matched by role name and by
provides provision-name in both directions). Role nodes carry deploy-mode
emoji markers (compose/swarm/host); disabled and shared services get a
stop mark; dependents are capped at 12.
test_readme enforces the Quick Setup body is byte-exact with the template
render. `make readme-generate` and `make cosmos` regenerate; `make cosmos`
also runs inside `make quality`.
Refactor: utils/symbol_glossary.py is the single source of truth for the
word/emoji mapping (unique project-wide), consumed by the complexity
matrix and the Cosmos builder. utils/roles/deploy.py holds role_has_stack
and role_deploy_modes, shared by both so the logic never drifts.
Add `make lint-mermaid` and its CI workflow, rendering every diagram via
mermaid-cli; the render step provisions chrome-headless-shell and unzip
on demand so it works in the minimal container.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-role deploy-mode opt-out moves from meta/tests.yml 'skip:' to meta/services.yml <entity>.modes.{compose,swarm}.enabled on the primary entity. get_role_skip now derives from it; get_role_mode_enabled added. A new lint requires every role shipping templates/*compose*.yml.j2 to declare both mode flags. Migrates 107 services.yml and drops 11 skip-only tests.yml. Also removes the variant_status deployability gate so swarm runs one job per variant unconditionally (variant_status.py and deployable_variant_indices deleted, complexity model simplified). Adds the i8matix alias.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New roles/sys-svc-container/templates/env.j2 sets TZ={{ HOST_TIMEZONE }} and LANG=C.UTF-8. Every role env.j2 includes it at the top; it renders via stack_host_template, the same engine that resolves the includes in compose.yml.j2, so it works in both compose and swarm. TZ/LANG are removed from the individual role env.j2 so each value stays single-source.
New lint tests/lint/ansible/test_env_base_include.py verifies every role env.j2 includes the base and never re-sets a base-owned key (keys derived from the base file).
mattermost and gitea meta/volumes.yml drop the /etc/timezone host-path bind: swarm hard-rejects a task whose bind source is missing on the node (Debian-trixie nodes have no /etc/timezone). TZ now comes from the base env. /etc/localtime bind also dropped; the container timezone follows TZ.
Non-directive comments were removed from the touched env.j2 (banners, restatements, reference URLs, narration) so the changed files pass the comment-lint; only tool directives remain.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Over Tor the redirect chain behind a login/logout/OIDC click routinely exceeds Playwright's fixed 30s action timeout, so the click throws before the flow's scaled waits run. Sweep all role Playwright suites and the shared persona helpers: give every navigation/auth-transition click an onion-scaled timeout (.click({ timeout: resolveTimeout(30_000) })) and route the remaining raw millisecond literals (timeout options, waitForTimeout, sleep-promise setTimeout, positional wait args) through resolveTimeout. Local UI clicks (menus, toggles, modal dismissals, DOM clicks inside evaluate) stay untouched.
Two new lints keep it that way: test_no_raw_timeout_literals forbids raw numeric timeout literals in Playwright specs (opt-out: nocheck: raw-timeout), test_persona_clicks_scaled requires every click in the shared persona auth flows to carry a resolveTimeout timeout (opt-out: nocheck: unscaled-click).
Verified: node --check on all 133 changed JS files, all 13 playwright lints green, make autoformat clean, make test green (2745+196+163+4 tests OK).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The custom flag moves from 33 hardcoded template kwargs into the merged
service config: 'custom: true' names the shared <entity>_custom build,
a string names a second distinct '<string>_custom' image inside one app
(nextcloud whiteboard). The image lookup reads the declaration as a
fallback, so config consumers (e.g. backup image matching) resolve the
exact deployed reference from one declaration point. nextcloud cron now
runs the nextcloud_custom build like the app service instead of the
upstream image.
A new services lint couples the declaration to reality in both
directions: custom requires a role Dockerfile, and a role Dockerfile
requires a custom declaration. The eight legacy roles deploying local
builds under upstream tags or ad-hoc image= overrides are ratcheted via
'# nocheck: dockerfile-custom'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rename the role, its test tree, the primary entity key
(volume-2-local), the derived vars (BKP_VOLUME_2_LOCAL_*,
SYS_SERVICE_BACKUP_VOLUME_2_LOCAL), the run-once fact, and every
group_names consumer flag, include path, doc, and lint reference. The
container_backup service key and the on-disk backup-docker-to-local
tree keep their names: both are contracts with existing consumer
entries, snapshot trees, and the baudolo pull tooling.
Operators must rename the svc-bkp-container-2-local inventory group and
disable the previous versioned units on already-provisioned hosts; the
pre-deploy snapshot glob derives its base from the new role name and
will not find them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Variant 0 pins the backup consumer flag enabled so the matrix baseline exercises the in-group branch; every other variant pins it disabled (shared: false carries the static-override suppression since the base value stays literally true). The matrix and wordpress variant entries move their services mapping to the list-item head so the static-override line parser resolves the override paths; yaml semantics are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every invokable stack role that declares named volumes in meta/volumes.yml now carries the standard container_backup consumer entry: enabled resolves via 'svc-bkp-container-2-local' in group_names, shared stays literally true since there is no per-consumer backup instance. Playwright/dynamic-flag/bond-baseline suppressions ride on the block per the established idioms; stale explanatory comments in the touched files are dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New lint test_no_raw_goto.py forbids raw `page.goto(...)` in role playwright specs + persona flows (the gotoOnion wrapper in personas/utils is exempt) so navigation retries transient Tor-transport errors over onion. Migrates every raw call across ~75 roles (343 sites) to `gotoOnion(page, ...)` and teaches gotoOnion to detect relative onion URLs (`isOnionCanonical`). gotoOnion is a behaviour-identical drop-in on clearnet (single attempt, same return). Verified: no-raw-goto lint green, eslint 0 errors, make test green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the global TIMEOUT_FACTOR and a shared timeouts.js helper: resolveTimeout(base) scales by TIMEOUT_FACTOR and, over a .onion target, an extra multiplier, so Tor's per-request latency no longer trips hard-coded timeouts. Migrate every raw timeout in the specs (timeout:/timeout=/setTimeout/waitForTimeout) to resolveTimeout, stage the helper in all three staging paths, and add test_no_raw_timeouts to forbid raw numeric timeouts. Clearnet behaviour is unchanged (factor 1: resolveTimeout(N) === N).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The curl-timeout lint scanned tasks, templates and .sh scripts but not
Dockerfiles, so four build-time downloads (composer for mediawiki and
joomla, wp-cli for wordpress, the Docker GPG key for svc-runner) could
still hang a build against a silent mirror. Pin --connect-timeout and
--max-time inline (Dockerfiles cannot render the filter) and extend the
lint scope to roles/*/files/**/Dockerfile so future ones are caught.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The root-run auto-install leaves administrator/cache, administrator/logs, cache and tmp root-owned on the shared NFS docroot; apache (www-data) then wedges every request in a cache-write permission loop, each healthcheck probe hangs its full 10s timeout, and the swarm converge wait expires - the earlier start_period raise addressed a misdiagnosis and could not help. The role-side chown runs only after converge, a circular dependency. Chown the four dirs in the entrypoint leader branch before touching the ready marker; the role-side chown becomes an idempotent no-op in compose.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
make autoformat output across role meta files, a task file and 19_tor.yml (single->double quotes on mode values, Jinja line reflow). No semantic changes. Kept separate from the onion CSP / oauth2-proxy fixes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Indentation/whitespace normalization emitted by make autoformat across role meta/services.yml, meta/variants.yml and a task file. No semantic changes (verified: git diff --ignore-all-space is empty for every file). Kept separate from the Tor egress-router feature.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The stock install runs blocking in the entrypoint before apache serves, so no HTTP is answered during it and the only healthcheck lever is start_period. The entrypoint's follower-wait already tolerates a ~10min leader install (300x2s), but start_period was 5m, so in swarm the leader or a waiting follower was reaped mid-install, orphaning the mkdir lock and deadlocking the shared-NFS init. Raise start_period to 15m so it exceeds the follower-wait window and no task is reaped mid-install. Compose is unaffected (it does not reap on unhealthy).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Onion routing is transport-layer with no persona/UI surface, so tor cannot be gated from the per-persona Playwright specs. Mark the tor block with '# nocheck: playwright-service-flag' in the 60 domain roles that ship a playwright.env.j2, per test_env_services_match's documented exemption for services that legitimately cannot be Playwright-gated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each role's meta/variants.yml now pins the tor onion gate both ways so the matrix exercises it: variant 0 pins services.tor.enabled: true, every non-baseline variant pins enabled: false + shared: false (# nocheck: variants-static-override, since tor.shared is a static literal). This gives real test_coverage / test_bond_baseline coverage without the removed per-key code exemptions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Domain-serving roles reference the svc-net-tor provider via a tor: block. Consumers declare enabled (dynamic group_names gate) and shared: true (static — onion is always the shared node model, inline nocheck: dynamic-flag). The provider svc-net-tor additionally declares exclusive/primary defaults that consumers inherit via resolve_service_config unless overridden.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The sys-svc-container healthcheck templates (curl/wget/nc/tcp/http) now
derive container_port from services.<service_name>.ports.internal.http,
so a role no longer needs to set container_port only to feed the shared
healthcheck.
Add the container-port-set-unused lint that fails a role setting
container_port without using it elsewhere, then resolve every flagged
role: drop the redundant assignment where the derivation reproduces the
value, or mark it with nocheck where the port is non-http, a constant, or
keyed on a service the derivation cannot reach.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a lint that rejects hardcoded container_port literals and the
host-published ports.local port; container_port is the port the container
listens on, so it must resolve from
lookup('config', application_id, 'services.<svc>.ports.internal.<name>')
or a constant built that way. Bring 42 roles into compliance: move each
literal into services.<svc>.ports.internal.<name> and reference it via the
lookup, using entity_name/service_name where they equal the service key.
Values are preserved (mattermost/prometheus pointed at ports.local with the
same value as internal). Drop the dead container_port from fider and
prometheus, and the narrative comments the edits pulled into the comment
lint's changed-file scope.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The stock image copies /usr/src/joomla into /var/www/html and runs the
auto-installer on every boot. With more than one replica sharing the
NFS volume they corrupt the tree and race the install. Add a custom
entrypoint that takes an atomic mkdir lock on the shared volume so one
leader runs the setup while the others block on a ready marker, wired
through the same host-path build args as the odoo entrypoint. Verified
locally: 2 concurrent replicas on a shared volume copy and install
exactly once and both serve HTTP 200. Sessions are already stored in
the database and caching is off, so the replicas stay stateless.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The web SAPI ran at PHP's 128M default and OOMed during the first-boot
install tree scans over NFS, so the healthcheck flapped through kill
cycles until the swarm jobs hit the time budget. Raise memory_limit to
512M, declared in services.joomla.performance.php.memory_limit and
consumed via a vars constant like the nextcloud performance block, and
give the healthcheck a 5m start period since the observed first boot
already exceeds the 30s default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
container cp needs its source on the node running the docker CLI, so
role_path sources break on swarm workers. The joomla plugin dir now
ships as a controller-side tarball through container exec -i. A new
repository lint forbids docker/container cp in tracked .yml and .sh
files; the six legitimate same-node uses carry justified nochecks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every compose.yml.j2 now declares services as '{{ service_name }}:' with
its '{% set %}' knobs on two-space lines above the key, includes at four
spaces and if/for tags aligned to the content they wrap. The layout
renders correctly because stack_host_template enables lstrip_blocks for
compose.yml.j2; indent(4, True) in the container base and '{%+' guards
in whitespace-sensitive scalars keep the shared fragments identical
under both render modes. A new lint test enforces the skeleton.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swarm config objects mount root:root; a 0440 mode is unreadable by a non-root container app user and crashloops the service (jenkins JCasC AccessDeniedException, exit 5). docker compose never hits this since there the file is a host bind-mount inheriting host perms.
New lint test_config_object_world_readable flags type:config modes lacking o+r (type:secret exempt, per-line # nocheck escape). Widen the 22 offending 0440 configs across 18 roles to 0444: adds read only, cannot break a reader, and fixes latent non-root cases (e.g. synapse, pixelfed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Role README Credits now name galaxy_info.author (plain bold or wrapped
in a Markdown link), matched byte-for-byte against the role's own
meta/main.yml, instead of a hard-coded maintainer name. Regenerate all
245 role README Credits blocks to the new 'Implemented by ...' format.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three folded 'when:/failed_when: >- {{ ... }}' conditionals (mediawiki, joomla, sys-front-tls-selfsigned) triggered the ansible-core 'Conditionals should not be surrounded by templating delimiters' deprecation, which fails the deploy deprecation gate. Drop the delimiters (a folded conditional is already a bare Jinja expression).
Extend two lints so this cannot regress and does not misfire: test_no_templated_conditionals now catches folded/block scalars (was inline-only); test_vars_usage_in_yaml now scans failed_when/until/changed_when for var usage (was when/loop/with_* only), so bare-expression conditionals no longer make their vars look unused.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The test-deploy-compose "Fail on deprecation warnings" gate failed ~10 apps
while the deploys themselves succeeded. Sources cleared:
- svc-db-postgres: community.postgresql `db` -> `login_db` (the `db` param
alias is deprecated, removed in collection 5.0.0)
- assert_no_deprecation_warnings.sh: ignore ansible-core's legacy filter/test
PluginLoader-aliases deprecation (needs a collection migration with
meta/runtime.yml, not fixable in role YAML)
- joomla/wordpress: drop the `{{ }}` delimiters from task conditionals
(when/until/changed_when), per ansible-core 2.23
Also drop `upstream`/`compat`/`security`/`bug` from the comment-policy markers
and delete those comments from the joomla install tasks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
