mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 10:59:59 +00:00
fix/dev-python/python-upper-bound
20
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  kevinveenbirkenbachandClaude Opus 5.5
|
30d03287de |
fix(web-app-dashboard): wait only for branding assets the deployment serves
Without web-svc-asset in the inventory, services.asset.url falls back to the public https://file.infinito.nexus/assets. The branding-asset wait then matched the public logo and favicon and polled the project's public website before the first compose start. Since that host stopped answering on 2026-10-21, every deploy that disables the asset service failed after 60 retries, among them the workspace compose track on main (run 39684179395: ubuntu and centos, "Status code was -1 ... Errno 101 Network is unreachable" for file.infinito.nexus/assets/img/logo.png). The slurp, the URL collection and the wait now run in one block guarded by the dashboard's own services.asset.enabled, so the deploy waits only for assets the shared file service of this deployment serves. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 5
|
5137465c6f |
style: delete every comment the comments-valid policy rejects
3529 invalid comments in 528 files, grandfathered in while the linter only saw the working-tree diff. Removal ran through the linter's own collectors, so detection and deletion cannot drift: a comment whose opener line is blank before it and whose closer line is blank after it loses all its lines, anything else loses only the comment region, and a line left holding a bare delimiter goes with it. Five files keep their comments behind a nocheck marker instead. Four are upstream configs whose inline docs are upstream's own (mautrix, systemd journald, nextcloud-oidc-login, MediaWiki). In littlejs the commented-out entries ARE the deactivation mechanism for an example tile, so deleting them would delete configuration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
36ebf34b64 |
Merge branch 'main' into feature/swarm-nfs
Resolve 69 conflicts across ~70 files: union main's image-version bumps and new service fields (bond, resource limits) with the swarm branch's default_placement / NFS / IS_STACK_HOST config; retire the three per-deploy-type test workflows main still edited (consolidated into compose+swarm); adopt main's run-once 01_core dispatchers while re-applying each role's swarm guard. Fix the post-merge lint surfaced by main's stricter rules: bond + resource-budget keys, container-exec stack-host gating + resolver routing, leaky/dead proxy vars, suitecrm once-flag, xwiki variant nocheck, nextcloud oversized-block to gated include, matrix-mdad capped at 3g. make autoformat clean, make test green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.8
|
ffac55c1d5 |
fix(proxy): stop cross-role webserver_* var leak (config fields + task-scoping + front-proxy reset + lint)
A role's vars/main.yml is auto-loaded into play scope and persists, so proxy
override vars (webserver_*) leaked into the central front-proxy render of OTHER
roles (re-evaluated against the wrong application_id) — silently applying one
role's value elsewhere, or hard-failing when the lazy lookup hit a role lacking
the key (the seaweedfs<->matrix client_max_body_size crash).
- Templates (sys-svc-proxy html/upload/ws_generic, matrix synapse homeserver)
resolve client_max_body_size from the role's own server config
(lookup('config', application_id, 'server.client_max_body_size', '100m')).
- Move every role-level webserver_* out of vars/main.yml into task-scoped
include params (9 roles); they apply only to that role's proxy and can't leak.
- sys-stk-front-proxy resets webserver_* to safe defaults via task-scoped vars
on the render tasks, so a leaked value from any other source is neutralized;
a role's own include params (#20) still override the reset (precedence-tested,
direct + nested).
- Add test_no_leaky_proxy_role_vars lint forbidding webserver_* in vars/main.yml
(suppressable via '# nocheck: leaky-proxy-var').
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
0ffbb73d72 |
refactor(images): central container_image lookup + migrate all roles
Add plugins/lookup/container_image.py: returns the full `image: "<ref>"`
line so templates drop the redundant image: prefix and surrounding
quotes. Supports `custom=True` to auto-construct `<entity_name>_custom`
for locally-built images, plus explicit `image=`/`version=` overrides
for exotic cases (taiga back-auth/front-oidc variants, decidim infinito
suffix, etc.).
* 25 unit tests for the lookup cover compose vs swarm prefix, custom
flag, overrides, tag_only=True, missing version warnings.
* New strict lint (tests/lint/.../test_no_raw_image_in_compose_templates)
fails on any raw `image:` line in roles/*/templates/*.j2 unless
nocheck-suppressed. Legacy `image: "{{ lookup(...) }}"` form no
longer accepted.
* Migrate 159 image refs across ~80 roles to the standalone
`{{ lookup('container_image', application_id, '<service_key>') }}`
form. 26 roles use `custom=True` (mariadb, postgres, openldap,
jenkins, jira, gitlab, nextcloud, mediawiki, shopware, wordpress,
odoo, joomla, openproject, bookwyrm, espocrm, hugo, littlejs,
fediwall, roulette-wheel, openresty, confluence and more).
* Fix svc-db-redis application_id leak: service.yml.j2 is included
into consuming roles where application_id evaluates to the outer
role, so the image lookup needs the hardcoded 'svc-db-redis'.
* Integration tests (test_templates_require_image,
test_build_requires_image) recognise the standalone lookup as an
implicit `image:` at the same indent.
* Mediawiki swarm tasks: add explicit `:{{ MEDIAWIKI_VERSION }}` to
the registry-prefixed image string because the custom var is now
bare.
Only 3 legitimate exotic nocheck-suppressed image lines remain:
matrix-mdad runner (nested services.matrix.runner.image path),
matrix mautrix bridges (per-iteration dynamic) and svc-registry-docker
(chicken-and-egg self-reference).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
6e49fe74fb |
feat(volumes): canonical dict-of-dicts meta schema + lookup('volume')
Migrate every role's meta/volumes.yml from ad-hoc dict-of-named-volumes
to a canonical dict-of-dicts schema where the YAML key is the semantic
short name and the entry body carries type / name / source / mounts /
nfs / mode / read_only. The optional 'name:' field is the docker volume
name (defaults to the key); the previous 'docker_name:' field is gone.
What's in this commit (210 paths):
Infrastructure (23)
- New filter plugins/filter/{compose,container}_volumes.py — render the
top-level volumes:/configs:/secrets: section and per-service mount
blocks from the canonical meta. container_volumes uses the SEMANTIC
name as per-service mount source so it resolves against compose_volumes'
top-level dict key (docker_name as source produced 'undefined volume'
on swarm).
- New lookup plugins/lookup/{compose,container,volume}_volumes.py.
volume(application_id, name) returns a normalised single entry;
volume(application_id) returns the whole canonical dict.
- utils/cache/applications.py keeps a sibling registry
_CANONICAL_VOLUMES_BY_ROLE outside the merged applications payload so
raw Jinja source: strings never enter the templar walk (root-cause of
the 200x cache-miss + recursive get_merged_applications storm).
- utils/roles/applications/mounts.py — schema validator + normaliser;
list-shape is rejected.
- plugins/lookup/nginx.py: _volume_path_by_name reads canonical directly.
- scripts/tests/deploy/swarm/_context.sh: NFS_VOLUMES extraction handles
both legacy dict and canonical dict shapes.
- roles/svc-prx-openresty/handlers/restart_and_validate.yml: gate
compose-only validate+reload behind a single when-block; swarm uses
service-update instead (bind-mounts of a FILE pin source inode at
task-create time, so atomic write-tmp+rename leaves the running task
on a deleted inode).
- roles/sys-svc-compose/tasks/utils/nfs_prep.yml: switch to
lookup('volume', application_id) | dict2items so NFS subdir
pre-creation works under the new cache split.
Tests (12)
- New strict lints flagging inline mount sections in compose templates,
single-file bind mounts, and legacy lookup('config', ..., 'volumes.X.Y')
call sites — all volume-mount surfaces MUST flow through container_volumes
/ compose_volumes / volume lookups.
- Integration: test_volumes_compose_chain + test_volume_constants_are_lookups
accept both the legacy and the new lookup('volume', app, 'X').name forms
during the migration window.
- Unit tests for the new lookup, filter, mounts helper, plus updated
nginx lookup test.
- Prometheus role-wiring test reads canonical meta for the
alert_rules.yml bind mount.
Role migrations (175)
- 62 meta/volumes.yml converted to dict-of-dicts canonical shape
- 63 templates/compose*.yml.j2 replace inline volumes:/configs:/secrets:
with lookup('container_volumes', application_id, '<service>') |
indent(N) per service + a single trailing lookup('compose_volumes',
application_id) for the top-level block
- 50 vars/main.yml drop dead volume constants and refactor 23 remaining
legacy lookup('config', ..., 'volumes.X.Y') to lookup('volume',
application_id, 'X').Y across 9 roles
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
24d4ab8e4a |
refactor(sso)\!: collapse oauth2 + oidc into unified services.sso block (req 21)
Implements docs/requirements/021-sso-flavor-migration.md atomically:
* Schema: every role's meta/services.yml replaces the parallel
`oauth2:` / `oidc:` top-level blocks with a single `sso:` block
carrying `enabled`, `shared`, and a `flavor` discriminator
(`oidc` | `oauth2` | `saml`; default `oidc`). Flavor-specific
sub-keys live under `sso.<flavor>.*` (`sso.oauth2.{origin,acl,
allowed_groups}`, `sso.oidc.plugin`).
* Provider role: roles/web-app-oauth2-proxy/ is deleted in its
entirety; its 5 sidecar templates move under
roles/web-app-keycloak/templates/sso_proxy/, and the per-consumer
rendering task lives at roles/web-app-keycloak/tasks/sso_proxy.yml.
web-app-keycloak now declares `provides: sso`.
* Runtime: new resolver utils/roles/applications/services/sso.py
centralises the SSO predicates (`is_enabled`, `is_proxy_gated`,
`is_oidc_native`) and the oauth2 sub-values
(`oauth2_origin_{host,port}`, `oauth2_acl`, `oauth2_allowed_groups`).
Three lookup plugins consume the resolver: `sso` (single-app
predicate access), `sso_proxy_consumers` (enumerates oauth2-flavored
consumers), `sso_oidc_plugin` (renamed from `oidc_flavor`, Nextcloud
plugin selector). sys-stk-backend, sys-svc-compose, sys-svc-proxy
and plugins/filter/compose_volumes.py all gate on
`lookup('sso', ..., 'is_proxy_gated')` instead of manually combining
`enabled` and `flavor`.
* Identifier renames repo-wide: `services.<entity>.ports.local.oauth2`
→ `ports.local.sso`; `PORT_BANDS.local.oauth2` → `PORT_BANDS.local.sso`;
`OAUTH2_PROXY_*` env/Jinja vars → `SSO_PROXY_*`;
`oauth2_proxy_cookie_secret` → `sso_proxy_cookie_secret`;
`OAUTH2_SERVICE_ENABLED` + `OIDC_SERVICE_ENABLED` collapse into
`SSO_SERVICE_ENABLED`; Playwright gating helpers
(isServiceEnabled/skipUnlessServiceEnabled/requireService/safeIsEnabled/
safeSkipUnlessEnabled) move from `oidc`/`oauth2` to `sso`.
* Dual-block resolution: per req's Background-Table, web-app-bookwyrm
is `flavor: oauth2`; web-app-gitea and web-app-friendica keep
`flavor: oauth2` with their existing `oauth2.acl.blacklist`
(/user/login) and `oauth2.acl.whitelist` (federation + discovery
endpoints) — preserving the operative oauth2-proxy gating.
* Tests: the two mutual-exclusion guards
(tests/integration/iam/oauth2_oidc/test_mutual_exclusive.py and
test_acl_mutual_exclusive.py) are deleted per Decision #10;
test_oauth2_contract.py is renamed and rewritten to
test_sso_contract.py, asserting that `flavor: oauth2` roles carry
`oauth2.origin.{host,port}` plus `ports.local.sso`; test_sso.py
(web-app integration) now requires the unified `sso:` block with an
explicit flavor or a `# nocheck: sso` opt-out; new
tests/lint/repository/no_legacy_sso_paths/ guards the source tree
against re-introducing the legacy strings.
* Docs: req 21 itself has every AC checked off, and reqs 006, 008,
009, 013, 015, 016, 019, 020 plus 10+ role READMEs and
docs/contributing/actions/testing/suppression.md are updated to the
unified SSO naming.
Empirically validated end-to-end against the validation app set
(web-app-keycloak, web-app-nextcloud, web-app-bookwyrm, web-app-gitea,
web-app-friendica, web-app-dashboard) in matrix variants 0 and 1; v2
also passes the 4 target roles' Playwright after a documented
pre-existing KC_HOSTNAME-drift workaround (separate concern, not
caused by this refactor).
BREAKING CHANGE: every consumer of `services.oauth2.*`,
`services.oidc.*`, `OAUTH2_PROXY_*`, `OAUTH2_SERVICE_ENABLED`,
`OIDC_SERVICE_ENABLED`, `oauth2_proxy_cookie_secret`,
`oauth2_proxy_application_id`, `ports.local.oauth2`, the
`web-app-oauth2-proxy` role, or the legacy gating helper service names
(`oidc`/`oauth2`) must migrate to the unified `sso` shape documented
in docs/requirements/021-sso-flavor-migration.md.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
fd785f13d7 |
fix(web-app-dashboard): route Simple Icons card sources through the public TLS URL
Dashboard card icons were rendered with `http://172.17.0.1:8044/<slug>.svg`, the in-cluster sync URL that only the Ansible runner can reach. The browser saw `NS_ERROR_CONNECTION_REFUSED` for every Simple Icons card. The `add_simpleicon_source` filter now takes two URLs: it probes the local sync URL (plain HTTP, no TLS, no redirect chain — reachable from the deploy container's `requests` without a CA bundle) and writes the public `lookup('tls', 'web-svc-simpleicons', 'url.base')` URL into `icon.source`. Cards render `<img src="https://icon.<DOMAIN>/<slug>.svg">`, which is reachable from the browser and CSP-permitted (`img-src *`). Drops the dead local-cache scaffolding (static-subdir vars, volume mount, `Ensure directory` task, refresh-download step) since the browser now fetches Simple Icons directly. Renames the previously-static `templates/style.css.j2` to `files/style.css` (no Jinja markers, matches the new template-uses-jinja lint warning). Adds a Playwright spec `test-icons.js` that asserts the regression: no card may still reference `172.17.0.1`, at least one card must resolve under `icon.<DOMAIN>`, with a favicon fallback path when the Simple Icons service is disabled. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
fb89592c3c |
style(web-app-dashboard): wrap long DASHBOARD_PLATFORM_*_URL ternaries across two lines
YAML folds the line break inside the double-quoted scalar into a single space, so the resolved value is byte-identical to before. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
df1b8b8526 |
fix(web-app-dashboard): gate oidc.js on services.oidc.enabled to stop CSP frame-ancestors violation in no-OIDC variant
In variant-1 (services.oidc.{enabled,shared}=false) the dashboard role
still deployed oidc.js unconditionally via the hardcoded
DASHBOARD_JS_FILES list. The runtime loader then ran
keycloak.init({onLoad: "check-sso"}), which opens a silent-SSO iframe
at auth.infinito.example. With no Keycloak client registered for the
dashboard in this variant, Keycloak returned its error page directly
(no 302 to silentCheckSsoRedirectUri), and that page carries the
realm-level CSP "frame-ancestors 'self'" hardcoded in
roles/web-app-keycloak/templates/import/realm.json.j2:1457. Chrome
blocked the iframe, produced a console error, and
test-no-console-noise.js failed.
The dashboard's menu template (header.yml.j2:22) already gates the
Login/Account items on DASHBOARD_OIDC_ENABLED. The JS file list now
matches that gate, so oidc.js is no longer rendered, shipped, or
loaded when OIDC is off. iframe.js stays unconditional — it is a
pure local postMessage listener with no network or CSP surface.
Fixes the variant-1/sync failures in: web-app-dashboard, web-app-fediwall,
web-app-moodle and the five test-development matrix jobs (all distros).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
adb901c7ce |
refactor(web-app-dashboard/playwright): declutter _shared.js + split out console-cleanliness check
Console cleanliness moved to its own scenario --------------------------------------------- ``test-header-navbar-logos.js`` failed in CI when port-ui's Keycloak silent-check-sso iframe was blocked by frame-ancestor CSP — the ``expectNoUnexpectedDiagnostics`` call at the test's end caught a console.error logged by chrome for the blocked iframe (chrome-error:// chromewebdata + ``Framing 'https://auth.…'``) and failed every retry. That diagnostic was a generic "no surprises" tail, unrelated to the logo assertions the test is named after. - Drop the diagnostic-tail call from the logo test; it now does exactly what its name says. - New ``test-no-console-noise.js`` runs the diagnostic check as its own scenario, with two noise filters enabled (``ignoreMatomoConsoleNoise`` + ``ignoreOidcSilentCheckNoise``). - New ``isOidcSilentCheckNoise`` helper in ``_shared.js`` matches the ``chrome-error://chromewebdata`` URL plus the ``Framing '…'`` / ``silent-check-sso`` text patterns. These are test-env config artefacts (Keycloak ``frame-ancestors`` not whitelisted), not dashboard regressions. - ``expectNoUnexpectedDiagnostics`` learns a corresponding ``ignoreOidcSilentCheckNoise`` option. _shared.js no longer hides single-consumer code ----------------------------------------------- Audited every export of ``_shared.js`` against every ``test-*.js`` consumer; ten symbols had exactly one caller and were therefore not "shared". Moved each to the test file that owns it: - ``runGuestFlow`` / ``runBiberFlow`` / ``runAdminFlow`` → matching ``test-*-persona.js`` (now ``require("./personas")`` direct) - ``assertCspMetaParity`` / ``assertCspResponseHeader`` / ``expectNoCspViolations`` / ``installCspViolationObserver`` plus ``canonicalDomain`` → ``test-csp-canonical-domain.js`` (CSP observer install is now a local ``test.beforeEach`` in that file) - ``dashboardTargetRoles`` parser → ``test-tile-reachability.js`` - ``cdnBaseUrl`` + each prefix (sharedCss / roleCss / sharedJs) → ``test-shared-css.js`` / ``test-cdn-role-stylesheet.js`` / ``test-logout-js-injection.js`` - ``platformLogoUrl`` + ``platformFaviconUrl`` → ``test-header-navbar-logos.js`` - ``appBaseUrl`` / ``oidcIssuerUrl`` / ``loginUsername`` / ``loginPassword`` / ``expectedOidcAuthUrl`` → ``test-oidc-login.js`` Dropped completely as dead test code: ``companyLogoUrl`` (no spec ever read it) and its ``COMPANY_LOGO_URL`` env render in ``playwright.env.j2``. ``DASHBOARD_COMPANY_LOGO_URL`` in ``vars/main.yml`` stays — ``config.yaml.j2`` still consumes it for the port-ui company.logo.source field. ``_shared.js`` is down to genuinely-shared state: ``dashboardJsBaseUrl`` + ``matomoBaseUrl`` (each ≥ 2 callers), the diagnostic plumbing (``attachDiagnostics`` / ``expectNoUnexpectedDiagnostics`` / ``waitForDashboardReady`` / ``waitForResourceResponse``), the noise predicates, ``beforeEach``, and the ``skipUnlessServiceEnabled`` re-export. 116 LOC of state/exports collapse to 63. Hitchhikers ----------- - ``vars/main.yml`` loses an outdated docstring above the branding SPOTs (the SPOTs themselves stay; ``config.yaml.j2`` consumes them). - ``sys-svc-compose/defaults/main.yml``: linter re-joined the multi-line ``compose_repository_pull`` default onto one line; pure formatting, no semantic change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
e97b1095e6 |
refactor(web-app-dashboard): split playwright spec, SPOT-ify branding, bump port-ui 2.0.0
Playwright spec split
---------------------
The 747-line ``playwright.spec.js`` monolith is split into a thin
orchestrator + ``_shared.js`` (env, ``beforeEach``, common helpers) +
one ``test-*.js`` per scenario, matching the convention used by
web-app-fediwall/friendica/nextcloud. Each scenario file holds its own
specialised helpers (e.g. the auth/dropdown helpers live in
``test-oidc-login.js``, image-loaded assertions live in
``test-header-navbar-logos.js``); only helpers used by ≥ 2 tests stay
in ``_shared.js``. ``dashboard-card-flow.js`` becomes
``test-tile-reachability.js`` (git-detected rename) — the
``runDashboardCardScenario`` helper had a single consumer, so the
indirection is gone.
SPOT for branding URLs
----------------------
The ``'https:' ~ source if source.startswith('//') else source``
transformation lived inline in ``config.yaml.j2`` and was duplicated in
``playwright.env.j2`` when the logo assertion was added. Three new
SPOTs in ``vars/main.yml``:
- ``DASHBOARD_PLATFORM_LOGO_URL``
- ``DASHBOARD_PLATFORM_FAVICON_URL``
- ``DASHBOARD_COMPANY_LOGO_URL``
Both templates now reference these vars; the inline transformations
and the now-redundant ``{% set ... %}`` lines in ``config.yaml.j2``
are removed. The slurp + parse of the rendered config in
``tasks/01_core.yml`` stays because it covers the host-override case
(``DASHBOARD_CONFIG_INV_PATH`` template may use different branding),
but is now documented as the override-discovery path rather than the
default source of truth.
port-ui 2.0.0 + test alignment
------------------------------
- ``meta/services.yml``: port-ui pinned ``1.2.0 → 2.0.0`` (probe-first
asset resolver, see kevinveenbirkenbach/port-ui#2.0.0).
- ``test-header-navbar-logos.js``: drop the ``asset`` skip-gate (port-ui
>= 2.0.0 embeds the URL directly instead of producing the broken
``/static/<absolute-url>``), replace the shape regex with an
exact-match against the SPOT ``PLATFORM_LOGO_URL`` env var, and add
a favicon assertion that checks the rendered ``<link rel="icon">``
href against the SPOT ``PLATFORM_FAVICON_URL``.
- ``test-cdn-role-stylesheet.js``: keep the ``asset`` skip-gate — that
test exercises ``web-svc-cdn`` which depends on ``web-svc-asset`` as
its file backend, unrelated to the port-ui bug.
- ``playwright.env.j2``: rendered ``ASSET_SERVICE_ENABLED``,
``PLATFORM_LOGO_URL``, ``PLATFORM_FAVICON_URL``, ``COMPANY_LOGO_URL``.
- ``meta/services.yml``: drop the now-stale
``# nocheck: playwright-service-flag`` annotation on the ``asset``
service block.
.claude/settings.json
---------------------
Resort ``permissions.allow`` after the runtime appended
``WebFetch(domain:cloud.veen.world)`` at the end of the array during
the asset-resolver work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
2a023ca6a1 |
chore(lint): autoformat strict-failure semantics + claude.sh rename
* scripts/lint/autoformat.sh:
- rename run_sort_claude_settings worker to run_claude (matches the
renamed scripts/lint/claude.sh).
- drop the EXIT trap that wiped status_dir; cleanup happens explicitly
after `wait` returns, so a parallel worker that interrupts the
parent never races a vanished status dir.
- each worker now captures its tool's exit code and writes one of
OK / SKIP / FAIL. Tolerated `expected violations remain`-style exits
(markdownlint-cli2 rc≤1, ansible-lint rc≤2) still count as OK; only
real failures count as FAIL.
- all workers run to completion regardless of individual tool failures
(so the user always sees the full picture); after the summary, the
script exits 1 if any worker reported FAIL.
* scripts/lint/sort_claude_settings.sh -> scripts/lint/claude.sh
(history preserved via git mv); update test_claude_settings_sorted.py
fix-hint accordingly.
Plus working-tree drift across roles/, group_vars/, inventories/,
requirements/, and .pre-commit-config.yaml accumulated from the
make-test setup phase regenerating generated artefacts during recent
test runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
951f6d36d2 |
chore(lint): unify autoformat/galaxy parallelism into one PARALLEL env var
* scripts/lint/autoformat.sh: rename AUTOFORMAT_PARALLEL to PARALLEL, declare it at file start with default `true`; still env-overridable. Reword the worker comment so the literal `# shellcheck` token no longer appears at start-of-line and trips SC1072/SC1073. * scripts/lint/galaxy.sh: drop the GALAXY_FATAL / GALAXY_NAMESPACE / GALAXY_PARALLEL env vars (over-engineered for a single repo). Failure is always fatal. NAMESPACE is an internal constant. Add the same PARALLEL env var (default `true`, env-overridable); falsy demotes the xargs fan-out to one worker. * docs/contributing/environment/variables.md: collapse the four AUTOFORMAT_*/GALAXY_* rows into one `PARALLEL` row covering both consumers. * docs/contributing/tools/make.md: refer to `PARALLEL=0` everywhere; add the lint-galaxy parallelism note. Plus working-tree drift across roles/, group_vars/, inventories/, and tasks/ that accumulated from the make-test setup phase regenerating generated artefacts during recent test runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
kevinveenbirkenbachandClaude Opus 4.7
|
ed04dad5d5 |
refactor(meta): land req-008/009/010 role-meta layout/networks/ports/run_after migration
Single atomic landing for three back-to-back role-meta refactor requirements
(see docs/requirements/008-role-meta-layout.md, 009-per-role-networks-and-ports.md,
010-role-meta-runafter-lifecycle-migration.md).
req-008 — role meta layout
* Move every role's metadata under meta/<topic>.yml with file-root
convention (no wrapping key matching the basename):
- schema/main.yml + config/main.yml.credentials → meta/schema.yml
- users/main.yml → meta/users.yml
- config/main.yml.server → meta/server.yml
- config/main.yml.rbac → meta/rbac.yml
- config/main.yml.compose.services + every other top-level key
of config/main.yml inlined under <entity>.<key>
→ meta/services.yml
- config/main.yml.compose.volumes → meta/volumes.yml
* Drop the legacy schema/, users/, config/ directories from every role.
* meta/schema.yml: support nested credential keys, optional `default:`
field (Jinja literal, not rendered/validated/regenerated), implicit
`algorithm: plain`.
* Materialised paths drop the `compose.` prefix:
compose.services.<e>.<…> → services.<e>.<…>
compose.volumes.<k> → volumes.<k>
<former-top-level>.<…> → services.<entity_name>.<former-top-level>.<…>
* Update every Python consumer (utils/cache, utils/manager, utils/applications,
plugins/lookup, plugins/filter, cli/*) plus every Jinja template, vars/*.yml
and task file to the new paths.
* No legacy fallback. Repository-wide grep for `compose.services.` /
`compose.volumes.` / `schema/main.yml` / `users/main.yml` / `config/main.yml`
is empty outside docs/requirements/ and tasks/utils/migrate_meta_layout.py.
* Add tests/lint/repository/test_role_meta_layout.py guarding against the
old paths returning, and an explicit lint guard that meta/variants.yml
files do not carry a `compose:` wrapper.
req-009 — per-role networks/ports
* Move group_vars/all/08_networks.yml.defaults_networks.local.<role>.{subnet,
dns_resolver} into roles/<role>/meta/server.yml.networks.local.{subnet,
dns_resolver}.
* Delete group_vars/all/09_ports.yml entirely; ports now live on the
service entity that exposes them at
roles/<role>/meta/services.yml.<entity>.ports.{inter,local,public}.
* Rename + int-cast every former top-level `services.<entity>.port`
field to `services.<entity>.ports.inter` (single int).
* Always-category-keyed maps for ports.local / ports.public; relay
spans use {start, end} integers.
* Add PORT_BANDS map to group_vars/all/08_networks.yml — single source
of truth for the per-<scope>.<category> band ranges, consumed by both
the new CLI helpers and the lint test.
* New CLI helpers (cli meta ports suggest, cli meta networks suggest)
walk the live role tree gap-first then increment-fallback, with
deterministic output and non-zero exit on capacity overflow.
* cli create role consumes both helpers when scaffolding a new role.
* Lint extensions: bare-int port shape rejection, host-bound port
collision check (single ints + relay span integers, inter excluded),
band-membership check with documented allow-list for the legacy
web-app-bigbluebutton.bigbluebutton.local.http: 48087 entry.
req-010 — run_after / lifecycle migration
* Move galaxy_info.run_after and galaxy_info.lifecycle out of
meta/main.yml into meta/services.yml.<primary_entity>.{run_after,
lifecycle}; <primary_entity> is the value returned by
get_entity_name(role_name).
* For multi-entity roles whose primary entity has no compose service
(web-app-bluesky → bluesky, web-app-matrix → matrix, desk/dev/drv/
gen roles), create a dedicated metadata-only top-level entry under
<primary_entity>.
* Introduce utils/roles/meta_lookup.py with get_role_run_after /
get_role_lifecycle that resolve <primary_entity> once and read from
meta/services.yml; degrade gracefully when meta/services.yml is
absent OR the field is absent; raise MetaServicesShapeError when the
file exists but is malformed.
* Update every consumer (utils/service_registry, utils/invokable,
cli/meta/applications/resolution/run_after/__main__, cli/meta/roles/
lifecycle_filter/__main__, cli/meta/applications/resolution/combined,
cli/build/{tree,graph,role_include}, cli/deploy/development/{common,
init,deps,deploy}, plugins/filter/canonical_domains_map, etc.) to
use the helper.
* Lint guards: no run_after/lifecycle in meta/main.yml; lifecycle
values restricted to {planned,pre-alpha,alpha,beta,stable,deprecated};
run_after/lifecycle never on a non-primary entity; empty run_after
lists rejected.
Also folded into this change set
* services_disabler now writes `services.<svc>.{enabled,shared}` into
host_vars instead of the old `compose.services.<svc>.{...}` path; the
discover/find pair walks meta/services.yml and only treats entries
carrying enabled/shared/provides as provider declarations (so
metadata-only entries from req-010 do not create spurious provider
rows).
* service_registry skips `run_after` deps whose target was filtered
out via SERVICES_DISABLED instead of failing the whole load.
* cli/mirror/resolver and cli/create/inventory/mirror_overrides emit/
consume the new applications.<role>.services.<svc>.image shape
(mirror.md updated to match).
* utils/domains/list.build_applications_from_roles now reads
meta/server.yml directly (it had been silently returning {} since
config/main.yml stopped existing).
* utils/roles/resource_validation.filter_roles_by_min_storage reads
meta/services.yml.<entity>.min_storage instead of the legacy
compose.services path.
* web-app-bluesky pre-clones the social-app build context BEFORE
sys-svc-compose so the build context is on disk by the time
sys-stk-front-proxy/tasks/main.yml flushes the docker build handler.
* 4 meta/variants.yml files (svc-ai-ollama, web-app-phpmyadmin,
web-app-wordpress, web-app-keycloak) lose their `compose: services:`
wrapper so deep-merge over meta/services.yml lands on the new
file-root shape.
Validation
* Unit tests: 1571 passed.
* Lint tests: 50 passed.
* Integration tests: 161 passed (1 skipped).
* Full-cycle fresh-purged-apps deploy succeeded for the 8-app validation
set: web-app-{nextcloud,wordpress,yourls,bigbluebutton,bluesky,gitea,
discourse,mattermost}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.6
|
e2016cd320 |
fix(users-applications): unify runtime lookup of users/applications
Lookups for 'service', 'service_registry', and 'applications_current_play'
always re-derive the merged applications mapping from inventory+role defaults
instead of trusting the passed kwargs, which could be shadowed or left
unrendered by nested template evaluations.
Dashboard/keycloak tests now consume the administrator password via
lookup('users','administrator').password so the rendered Playwright .env
matches the realm user provisioned by Ansible. Remove the hardcoded
DASHBOARD_KEYCLOAK_TEST_LOGIN_PASSWORD / KEYCLOAK_LOCAL_REALM_ADMIN_PASSWORD
overrides that previously masked the mismatch.
run_one.yml: drop users/applications from the template vars block. Those
entries created a shadowing context that broke nested lookup('users', ...)
inside lookup('template', ...).
utils/runtime_lookup_data: decrypt Ansible 2.19 EncryptedString values at
the lookup boundary and try templar rendering before falling back to
aggregated defaults when override mappings arrive as placeholders.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
kevinveenbirkenbachandClaude Opus 4.6
|
b70a783423 |
refactor(lookups): migrate to runtime applications/users lookups
Replaces inline `applications`/`users` merging with dedicated lookup plugins and runtime hydration. Introduces `add_csp_hash` filter, integrates sort into `find_dock_val_by_bkp_entr`, splits the Keycloak one-shot realm import into `05a_realm_import_run.yml`, and fixes the Mastodon administrator task YAML. Removes obsolete `csp_hashes` filter, `sys-token-store` Jinja filter, and empty CLI setup modules. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
kevinveenbirkenbach
|
a98b72a083 |
feat(web-app-dashboard): use local cached Simple Icons assets
- switch icon.source from remote URLs to local static paths - add local Simple Icons directory creation on host - mount local static directory into dashboard container - introduce DASHBOARD_SIMPLEICONS_* variables for paths and sync base URL - download required SVGs into local static directory during refresh - update Playwright test to validate cached image paths and dimensions - enhance add_simpleicon_source filter: - return local static SVG paths instead of remote URLs - validate non-empty local_static_dir - update unit tests for new behavior and error handling - remove dependency on TLS lookup for Simple Icons base URL |
||
|
kevinveenbirkenbach
|
f7318feae5 |
feat(web-app-dashboard): switch dashboard deployment to ghcr.io/kevinveenbirkenbach/port-ui:1.0.0
- stop pulling/building from the port-ui repository - run dashboard via fixed image tag - mount generated config.yaml into /app/config.yaml (read-only) |
||
|
kevinveenbirkenbach
|
6efde55647 |
refactor(dashboard): rename web-app-desktop to web-app-dashboard and update service toggle
* replace compose.services.desktop with compose.services.dashboard across all roles * rename role web-app-desktop to web-app-dashboard * rename sys-front-inj-desktop to sys-front-inj-dashboard * update resolver mapping to include web-app-dashboard when dashboard is enabled * adjust CSP filters to reference dashboard for frame-ancestors handling * update lookup plugins, filter plugins, templates, vars and run_once flags * migrate all tests to dashboard naming and service key * update role dependencies and port mappings * remove legacy desktop role and variables BREAKING CHANGE: The service key compose.services.desktop has been renamed to compose.services.dashboard. All custom configs, inventories, or overrides must be updated accordingly. https://chatgpt.com/share/698da7ad-1978-800f-82b4-4755b17be026 |