mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-09 04:06:42 +00:00
feat(variants): give clearnet-only services a round of their own
services.tor is a consumer flag, not a capability: every role declares the same
"{{ 'svc-net-tor' in group_names }}" and joins the onion wherever svc-net-tor is
deployed. Only five roles pin it to a literal false, because onion routing
breaks real-time media transport: bigbluebutton, jitsi, opentalk, peertube and
coturn.
Seven matrix-deploy rounds dropped the onion for one of those five and took
everything else down with them. The services that rode along are onion-capable
and were therefore proven from a clearnet origin only, for no reason of their
own. Each round now carries either the media service or the rest, not both:
* wordpress: variant 2 carries bigbluebutton and peertube, variant 1 keeps
multisite and listmonk over the onion
* mediawiki: libretranslate moves to variant 2, leaving variant 1 to peertube;
litellm leaves variant 2, which would otherwise sit 1.7 GB over the cap
* moodle: variant 3 carries bigbluebutton and jitsi, variant 1 gets its four
partners and SSO back with the onion on
* discourse: variant 2 carries bigbluebutton and jitsi, variant 1 keeps the
mastodon, matrix and mattermost couplings over the onion
* nextcloud: variant 11 drops SSO, which bbb.spec.js never gated on, mirroring
the peertube round that already runs without it
Two rounds keep a marker because the collateral is not removable. matrix cannot
host either hookshot partner in an onion round: measured at 66 GB with n8n and
81 GB with gitlab against the 64 GB cap. mattermost pins services.javascript to
a literal true that no variant may override, and every addon spec gates on
skipUnlessServiceEnabled("sso").
Two lints hold the line from here. test_variant_tor_off_needs_a_reason rejects a
round that drops the onion without an incapable service in it, and its only
escape is a data change; it found web-app-opencloud, now fixed.
test_variant_tor_collateral rejects a round that takes an onion-capable frontend
down with it, and its marker sits on the variant's own tor pin because the
reason is one fact about the round rather than one per partner.
Splitting discourse also brought variant 1 from over the cap to 59.36 GB, so its
mem-limit-budget exemption is deleted and the cap guards the round again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kevinveenbirkenbachandClaude Opus 5 committed
1 parent
b3e0dce578
commit
24d0308995
12 files changed
+632
-25
No files matched your search
@@ -84,6 +84,7 @@ The placement rule is per check. The catalog "Position" column uses these labels
|
||||
| `variants-shared-false-once` | same or above | [test_variants_shared_false_once.py](../../../../tests/lint/ansible/services/test_variants_shared_false_once.py) | Exempts one `roles/*/meta/variants.yml` `services.<key>.shared: false` pin from the one-local-provider-variant limit for a service whose base `meta/services.yml` uses literal `enabled: true` and dynamic `shared`. Place on the extra `shared: false` line or service key line. Place above the service key in `meta/services.yml` to exempt the service. |
|
||||
| `variants-services-match` | same or above | [test_services_match.py](../../../../tests/integration/roles/meta/variants/test_services_match.py) | Allows a `roles/*/meta/variants.yml` `services.<key>` entry to reference a service key that is NOT declared as a top-level entry in `meta/services.yml`. Reserved for resolver-only matrix hooks (typically on invokable infrastructure roles) whose variant entries pin the round's companion topology via service-key flags the resolver maps to provider roles, with no real consumer contract in the base meta. |
|
||||
| `variants-auth-isolation` | same or above | [test_auth_isolation.py](../../../../tests/integration/roles/meta/variants/test_auth_isolation.py) | Exempts an auth-isolated variant entry in `roles/*/meta/variants.yml` (one that enables exactly one of LDAP / OIDC / OAuth2) from the rule that every non-auth service-key in the role's variant union must be pinned to literal `false`. Place on the leading `- services:` line of the variant or on the line above. Use only when the broader closure is intentional. |
|
||||
| `variant-tor-collateral` | same or above | [test_variant_tor_collateral.py](../../../../tests/lint/ansible/roles/meta/test_variant_tor_collateral.py) | Allows a `roles/*/meta/variants.yml` entry that pins `services.tor.enabled: false` to keep onion-capable frontend partners enabled. Place on the variant's own `tor` key line. The marker MUST name why those partners belong in a clearnet round; the usual reason is a real-time media partner in the same round whose transport the onion breaks. |
|
||||
| `lookup-config-path` | same or above | [config/](../../../../tests/integration/ansible/lookups/config/) | Skips a single `lookup('config', …)` call from path-validation passes. Use when the call resolves against runtime-only state, typically self-referential roles. |
|
||||
| `unused-var` | same or above | [test_role_and_group_vars_used.py](../../../../tests/lint/ansible/variables/test_role_and_group_vars_used.py) | Exempts a top-level key in `roles/<role>/{vars,defaults}/main.yml` or `group_vars/**/*.yml` from the "must be referenced" check. Use when the consumer is non-YAML/Jinja (Python plugins, external tools). |
|
||||
| `project-root-import` | same or above | [test_project_root_import.py](../../../../tests/lint/filesystem/python/test_project_root_import.py) | Permits a local `PROJECT_ROOT` / `parents[N]` / `os.pardir` chain. Reserved for `__main__.py` `sys.path` bootstrap and standalone scripts under `roles/<role>/files/`. The marker MUST carry an inline explanation. |
|
||||
|
||||
@@ -2,11 +2,10 @@
|
||||
# Matrix-deploy variant overrides for web-app-discourse.
|
||||
# See docs/contributing/artefact/files/role/variants.md for the file format.
|
||||
#
|
||||
# nocheck: sso-variant-tor — variant 1 is the bridge round: it pairs bigbluebutton
|
||||
# and jitsi, both clearnet-only, so it drops the onion. The five partners it turns on
|
||||
# (bigbluebutton, jitsi, mastodon, matrix, mattermost) are therefore only driven from
|
||||
# a clearnet forum origin; the forum's own onion surface is variant 0's job, which
|
||||
# pairs the onion with SSO.
|
||||
# nocheck: sso-variant-tor — variant 2 exists so bigbluebutton and jitsi, both
|
||||
# clearnet-only, have a round of their own. It carries nothing else; the mastodon,
|
||||
# matrix and mattermost couplings moved to variant 1, which keeps the onion, and
|
||||
# variant 0 pairs the onion with SSO.
|
||||
|
||||
- services:
|
||||
container_backup:
|
||||
@@ -59,7 +58,6 @@
|
||||
enabled: false
|
||||
shared: false
|
||||
- services:
|
||||
# nocheck: mem-limit-budget this round reaches web-app-matrix, whose variant 1 pins gitlab true, so the footprint transitively bills gitlab's webservice, migrations and gitaly, about 10 GB of mem_limit discourse never asks for. No placement of those pins, of web-app-mattermost's jitsi pin or of the bridge pins below keeps every variant under the cap; the best alternative only moves the overage to web-app-nextcloud v12 at 65.76 GB. mem_limit is a per-container ceiling, and this round's guaranteed mem_reservation stays at 30.28 of 32 GB.
|
||||
container_backup:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
@@ -94,13 +92,13 @@
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
tor:
|
||||
enabled: false
|
||||
enabled: true
|
||||
bigbluebutton:
|
||||
enabled: true
|
||||
shared: true
|
||||
enabled: false
|
||||
shared: false
|
||||
jitsi:
|
||||
enabled: true
|
||||
shared: true
|
||||
enabled: false
|
||||
shared: false
|
||||
mastodon:
|
||||
enabled: true
|
||||
shared: true
|
||||
@@ -110,3 +108,55 @@
|
||||
mattermost:
|
||||
enabled: true
|
||||
shared: true
|
||||
|
||||
- services:
|
||||
container_backup:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
asset:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
dashboard:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
email:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
logout:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
matomo:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
litellm:
|
||||
enabled: false
|
||||
sso:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
postgres:
|
||||
shared: true
|
||||
redis:
|
||||
shared: true
|
||||
prometheus:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
css:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
tor:
|
||||
enabled: false
|
||||
bigbluebutton:
|
||||
enabled: true
|
||||
shared: true
|
||||
jitsi:
|
||||
enabled: true
|
||||
shared: true
|
||||
mastodon:
|
||||
enabled: false
|
||||
shared: false
|
||||
matrix:
|
||||
enabled: false
|
||||
shared: false
|
||||
mattermost:
|
||||
enabled: false
|
||||
shared: false
|
||||
@@ -1,6 +1,16 @@
|
||||
---
|
||||
# Matrix-deploy variant overrides for web-app-mattermost.
|
||||
# See docs/contributing/artefact/files/role/variants.md for the file format.
|
||||
#
|
||||
# nocheck: sso-variant-tor — variant 3 exists so jitsi, which is clearnet-only, has
|
||||
# a round of its own. Every other round keeps the onion on.
|
||||
#
|
||||
# nocheck: variant-limit — folding the jitsi round into an existing variant was
|
||||
# measured and does not work: variant 2 is the only round pinning sso false, which
|
||||
# is the polarity variants-coverage needs, and every addon spec gates on
|
||||
# skipUnlessServiceEnabled("sso"), so a jitsi round without sso would deploy jitsi
|
||||
# and prove nothing. A fourth round is the only way hermes, jenkins, openclaw and
|
||||
# openwebui reach the onion.
|
||||
|
||||
# variant pinning every dynamic services.yml flag to true so the
|
||||
# matrix exercises the in-group_names branch of web-app-mattermost.
|
||||
@@ -48,6 +58,15 @@
|
||||
openclaw:
|
||||
enabled: false
|
||||
shared: false
|
||||
gitlab:
|
||||
enabled: true
|
||||
shared: true
|
||||
jenkins:
|
||||
enabled: false
|
||||
shared: false
|
||||
jitsi:
|
||||
enabled: false
|
||||
shared: false
|
||||
- services:
|
||||
container_backup:
|
||||
enabled: false
|
||||
@@ -93,6 +112,15 @@
|
||||
openclaw:
|
||||
enabled: true
|
||||
shared: true
|
||||
gitlab:
|
||||
enabled: false
|
||||
shared: false
|
||||
jenkins:
|
||||
enabled: true
|
||||
shared: true
|
||||
jitsi:
|
||||
enabled: false
|
||||
shared: false
|
||||
|
||||
- services:
|
||||
container_backup:
|
||||
@@ -139,3 +167,67 @@
|
||||
openclaw:
|
||||
enabled: false
|
||||
shared: false
|
||||
gitlab:
|
||||
enabled: false
|
||||
shared: false
|
||||
jenkins:
|
||||
enabled: false
|
||||
shared: false
|
||||
jitsi:
|
||||
enabled: false
|
||||
shared: false
|
||||
|
||||
- services:
|
||||
container_backup:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
dashboard:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
email:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
logout:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
matomo:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
sso:
|
||||
enabled: true
|
||||
shared: true
|
||||
postgres:
|
||||
shared: true
|
||||
redis:
|
||||
shared: true
|
||||
prometheus:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
seaweedfs:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
css:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
litellm:
|
||||
enabled: false
|
||||
tor: # nocheck: variant-tor-collateral jitsi refuses the onion and this round exists only for it; javascript is pinned statically true in meta/services.yml so no variant can drop it, and sso has to stay on because every addon spec gates on it - both run over the onion in variants 0 and 1
|
||||
enabled: false
|
||||
openwebui:
|
||||
enabled: false
|
||||
shared: false
|
||||
hermes:
|
||||
enabled: false
|
||||
shared: false
|
||||
openclaw:
|
||||
enabled: false
|
||||
shared: false
|
||||
gitlab:
|
||||
enabled: false
|
||||
shared: false
|
||||
jenkins:
|
||||
enabled: false
|
||||
shared: false
|
||||
jitsi:
|
||||
enabled: true
|
||||
shared: true
|
||||
@@ -2,10 +2,10 @@
|
||||
# Matrix-deploy variant overrides for web-app-mediawiki.
|
||||
# See docs/contributing/artefact/files/role/variants.md for the file format.
|
||||
#
|
||||
# nocheck: sso-variant-tor — variant 1 is the only round enabling peertube, which is
|
||||
# clearnet-only, so it drops the onion. The MachineTranslation extension it
|
||||
# exercises is consequently proven against a clearnet wiki origin only;
|
||||
# variants 0 and 2 keep the onion, and variant 0 pairs it with SSO.
|
||||
# nocheck: sso-variant-tor — variant 1 exists so peertube, which is clearnet-only,
|
||||
# has a round of its own. It carries nothing else; libretranslate moved to
|
||||
# variant 2, which keeps the onion, so the MachineTranslation extension is now
|
||||
# proven over it. Variant 0 pairs the onion with SSO.
|
||||
|
||||
- services:
|
||||
container_backup:
|
||||
@@ -66,8 +66,8 @@
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
libretranslate:
|
||||
enabled: true
|
||||
shared: true
|
||||
enabled: false
|
||||
shared: false
|
||||
peertube:
|
||||
enabled: true
|
||||
shared: true
|
||||
@@ -110,8 +110,8 @@
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
libretranslate:
|
||||
enabled: false
|
||||
shared: false
|
||||
enabled: true
|
||||
shared: true
|
||||
peertube:
|
||||
enabled: false
|
||||
shared: false
|
||||
@@ -124,7 +124,7 @@
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
litellm:
|
||||
enabled: true
|
||||
enabled: false
|
||||
sso:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
|
||||
@@ -1,6 +1,16 @@
|
||||
---
|
||||
# Matrix-deploy variant overrides for web-app-moodle.
|
||||
# See docs/contributing/artefact/files/role/variants.md for the file format.
|
||||
#
|
||||
# nocheck: sso-variant-tor — variant 3 exists so mod_bigbluebuttonbn and mod_jitsi,
|
||||
# whose partners pin tor.enabled to a literal false, have a round of their own. It
|
||||
# carries nothing else; every other round keeps the onion on.
|
||||
#
|
||||
# nocheck: variant-limit — folding the media round into an existing variant was
|
||||
# measured and does not fit: variant 2 already carries matrix, lam, ldap and
|
||||
# litellm, and adding the AI and cloud partners there puts the round at 69 GB
|
||||
# against the 64 GB cap. A fourth round is the only way repository_owncloud,
|
||||
# aiprovider_openwebui, hermes and openclaw reach the onion.
|
||||
|
||||
# variant pinning every dynamic services.yml flag to true so the
|
||||
# matrix exercises the in-group_names branch of web-app-moodle.
|
||||
@@ -43,6 +53,18 @@
|
||||
enabled: false
|
||||
tor:
|
||||
enabled: true
|
||||
bigbluebutton:
|
||||
enabled: false
|
||||
shared: false
|
||||
jitsi:
|
||||
enabled: false
|
||||
shared: false
|
||||
matrix:
|
||||
enabled: false
|
||||
shared: false
|
||||
nextcloud:
|
||||
enabled: false
|
||||
shared: false
|
||||
|
||||
openwebui:
|
||||
enabled: false
|
||||
@@ -90,6 +112,18 @@
|
||||
enabled: false
|
||||
tor:
|
||||
enabled: true
|
||||
bigbluebutton:
|
||||
enabled: false
|
||||
shared: false
|
||||
jitsi:
|
||||
enabled: false
|
||||
shared: false
|
||||
matrix:
|
||||
enabled: false
|
||||
shared: false
|
||||
nextcloud:
|
||||
enabled: true
|
||||
shared: true
|
||||
openwebui:
|
||||
enabled: true
|
||||
shared: true
|
||||
@@ -99,6 +133,7 @@
|
||||
openclaw:
|
||||
enabled: true
|
||||
shared: true
|
||||
|
||||
- services:
|
||||
container_backup:
|
||||
enabled: false
|
||||
@@ -137,6 +172,77 @@
|
||||
enabled: true
|
||||
tor:
|
||||
enabled: true
|
||||
bigbluebutton:
|
||||
enabled: false
|
||||
shared: false
|
||||
jitsi:
|
||||
enabled: false
|
||||
shared: false
|
||||
matrix:
|
||||
enabled: true
|
||||
shared: true
|
||||
nextcloud:
|
||||
enabled: false
|
||||
shared: false
|
||||
openwebui:
|
||||
enabled: false
|
||||
shared: false
|
||||
hermes:
|
||||
enabled: false
|
||||
shared: false
|
||||
openclaw:
|
||||
enabled: false
|
||||
shared: false
|
||||
|
||||
- services:
|
||||
container_backup:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
css:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
dashboard:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
email:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
ldap:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
logout:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
mariadb:
|
||||
shared: true
|
||||
matomo:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
sso:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
prometheus:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
seaweedfs:
|
||||
enabled: false
|
||||
shared: false
|
||||
litellm:
|
||||
enabled: false
|
||||
tor:
|
||||
enabled: false
|
||||
bigbluebutton:
|
||||
enabled: true
|
||||
shared: true
|
||||
jitsi:
|
||||
enabled: true
|
||||
shared: true
|
||||
matrix:
|
||||
enabled: false
|
||||
shared: false
|
||||
nextcloud:
|
||||
enabled: false
|
||||
shared: false
|
||||
openwebui:
|
||||
enabled: false
|
||||
shared: false
|
||||
|
||||
@@ -1103,8 +1103,8 @@
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
sso:
|
||||
enabled: true
|
||||
shared: true
|
||||
enabled: false
|
||||
shared: false
|
||||
coturn:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
|
||||
@@ -67,6 +67,7 @@ prometheus:
|
||||
shared: "{{ 'web-app-prometheus' in group_names }}"
|
||||
|
||||
# nocheck: playwright-service-flag
|
||||
# nocheck: variants-coverage this role pairs with no service that refuses the onion, so variant-tor-off-needs-a-reason keeps the onion on in every variant and the false polarity has nowhere to live
|
||||
tor:
|
||||
bond: 0.5
|
||||
enabled: "{{ 'svc-net-tor' in group_names }}"
|
||||
|
||||
@@ -68,7 +68,7 @@
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
tor:
|
||||
enabled: false
|
||||
enabled: true
|
||||
|
||||
- services:
|
||||
container_backup:
|
||||
@@ -99,4 +99,4 @@
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
tor:
|
||||
enabled: false
|
||||
enabled: true
|
||||
@@ -7,6 +7,11 @@
|
||||
# between rounds whenever the variant changes, so Single-Site state
|
||||
# cannot bleed into the Multisite run.
|
||||
# See docs/contributing/artefact/files/role/variants.md for the file format.
|
||||
#
|
||||
# nocheck: sso-variant-tor — variant 2 exists so bigbluebutton and peertube,
|
||||
# which pin tor.enabled to a literal false, have a round of their own. It
|
||||
# carries nothing else, so dropping the onion there costs no coverage; every
|
||||
# other round keeps it on.
|
||||
|
||||
# variant 0: canonical Single-Site deploy.
|
||||
|
||||
@@ -19,6 +24,15 @@
|
||||
discourse:
|
||||
enabled: true
|
||||
shared: true
|
||||
bigbluebutton:
|
||||
enabled: false
|
||||
shared: false
|
||||
listmonk:
|
||||
enabled: false
|
||||
shared: false
|
||||
peertube:
|
||||
enabled: false
|
||||
shared: false
|
||||
dashboard:
|
||||
enabled: true
|
||||
shared: true
|
||||
@@ -65,6 +79,15 @@
|
||||
discourse:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
bigbluebutton:
|
||||
enabled: false
|
||||
shared: false
|
||||
listmonk:
|
||||
enabled: true
|
||||
shared: true
|
||||
peertube:
|
||||
enabled: false
|
||||
shared: false
|
||||
dashboard:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override backup shared stays literally true
|
||||
@@ -98,7 +121,7 @@
|
||||
openclaw:
|
||||
enabled: false
|
||||
shared: false
|
||||
addons:
|
||||
addons: &wordpress_addons_off
|
||||
activitypub:
|
||||
enabled: false
|
||||
server:
|
||||
@@ -107,3 +130,57 @@
|
||||
- blog.{{ DOMAIN_PRIMARY }}
|
||||
- biberblog.{{ DOMAIN_PRIMARY }}
|
||||
- biber.blog.{{ DOMAIN_PRIMARY }}
|
||||
|
||||
- services:
|
||||
container_backup:
|
||||
enabled: false
|
||||
shared: false # nocheck: variants-static-override base shared is a static literal and this round joins no backup host
|
||||
css:
|
||||
enabled: false
|
||||
shared: false
|
||||
discourse:
|
||||
enabled: false
|
||||
shared: false
|
||||
bigbluebutton:
|
||||
enabled: true
|
||||
shared: true
|
||||
listmonk:
|
||||
enabled: false
|
||||
shared: false
|
||||
peertube:
|
||||
enabled: true
|
||||
shared: true
|
||||
dashboard:
|
||||
enabled: false
|
||||
shared: false
|
||||
email:
|
||||
enabled: false
|
||||
shared: false
|
||||
logout:
|
||||
enabled: false
|
||||
shared: false
|
||||
mariadb:
|
||||
shared: true
|
||||
matomo:
|
||||
enabled: false
|
||||
shared: false
|
||||
litellm:
|
||||
enabled: false
|
||||
sso:
|
||||
enabled: false
|
||||
shared: false
|
||||
prometheus:
|
||||
enabled: false
|
||||
shared: false
|
||||
seaweedfs:
|
||||
enabled: false
|
||||
shared: false
|
||||
tor:
|
||||
enabled: false
|
||||
hermes:
|
||||
enabled: false
|
||||
shared: false
|
||||
openclaw:
|
||||
enabled: false
|
||||
shared: false
|
||||
addons: *wordpress_addons_off
|
||||
@@ -1,3 +1,39 @@
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import yaml
|
||||
|
||||
from utils.cache.files import read_text
|
||||
from utils.cache.yaml import load_yaml_str
|
||||
|
||||
PROJECT_ROOT: Path = Path(__file__).resolve().parents[5]
|
||||
ROLES_DIR: Path = PROJECT_ROOT / "roles"
|
||||
|
||||
|
||||
def role_dirs() -> list[Path]:
|
||||
"""Every role directory under ``roles/``, sorted by name."""
|
||||
return sorted(path for path in ROLES_DIR.iterdir() if path.is_dir())
|
||||
|
||||
|
||||
def load_role_meta(path: Path) -> Any:
|
||||
"""Parse one role meta file.
|
||||
|
||||
Args:
|
||||
path: the meta file to read.
|
||||
|
||||
Returns:
|
||||
The parsed document, or ``None`` when the file is absent, empty,
|
||||
undecodable or not valid YAML.
|
||||
"""
|
||||
if not path.is_file():
|
||||
return None
|
||||
try:
|
||||
text = read_text(str(path))
|
||||
except UnicodeDecodeError:
|
||||
return None
|
||||
if not text.strip():
|
||||
return None
|
||||
try:
|
||||
return load_yaml_str(text)
|
||||
except yaml.YAMLError:
|
||||
return None
|
||||
@@ -0,0 +1,143 @@
|
||||
"""A clearnet round may not quietly take an onion-capable partner down with it.
|
||||
|
||||
A variant that pins ``tor.enabled: false`` serves every service it enables
|
||||
over clearnet only. That is unavoidable for the partner forcing the drop - the
|
||||
real-time media roles pin ``tor.enabled`` to a literal ``false`` because onion
|
||||
routing breaks their transport - and for the role the variant belongs to. It
|
||||
is not unavoidable for the others: ``web-app-gitlab`` carries the dynamic flag
|
||||
and serves fine over the onion, yet sat in a round that had dropped it, so the
|
||||
matrix-gitlab bridge was only ever proven from a clearnet origin.
|
||||
|
||||
A variant that takes such partners down with it therefore needs
|
||||
``# nocheck: variant-tor-collateral`` on its own ``tor`` pin, with text naming
|
||||
why those partners belong in a clearnet round. The marker sits on the pin that
|
||||
makes the decision rather than on each affected service, because the reason is
|
||||
one fact about the round, not one per partner.
|
||||
|
||||
The round is read from ``get_variants``, the merged view: a service the base
|
||||
``meta/services.yml`` enables counts even when the variant never names it.
|
||||
Reading the override file alone would miss those and under-report.
|
||||
|
||||
Backends are out of scope: they publish no surface, so the onion means nothing
|
||||
to them even though they carry the same consumer flag.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from utils.annotations.suppress import is_suppressed_at
|
||||
from utils.cache.applications import get_variants
|
||||
from utils.cache.files import read_text
|
||||
from utils.roles.applications.services.registry import (
|
||||
build_service_registry_from_roles_dir,
|
||||
)
|
||||
from utils.roles.mapping import ROLE_FILE_META_SERVICES, ROLE_FILE_META_VARIANTS
|
||||
|
||||
from . import ROLES_DIR, load_role_meta, role_dirs
|
||||
|
||||
RULE = "variant-tor-collateral"
|
||||
TOR_KEY = "tor"
|
||||
|
||||
|
||||
def _incapable_roles() -> set[str]:
|
||||
"""Roles whose own ``services.tor.enabled`` is a literal ``false``."""
|
||||
found: set[str] = set()
|
||||
for role_dir in role_dirs():
|
||||
services = load_role_meta(role_dir / ROLE_FILE_META_SERVICES)
|
||||
tor = services.get(TOR_KEY) if isinstance(services, dict) else None
|
||||
if isinstance(tor, dict) and tor.get("enabled") is False:
|
||||
found.add(role_dir.name)
|
||||
return found
|
||||
|
||||
|
||||
def _variant_spans(lines: list[str]) -> list[range]:
|
||||
"""The line range each top-level list entry occupies.
|
||||
|
||||
Args:
|
||||
lines: the variants file's lines.
|
||||
"""
|
||||
starts = [n for n, line in enumerate(lines, 1) if line.startswith("- ")]
|
||||
bounds = [*starts[1:], len(lines) + 1]
|
||||
return [range(start, end) for start, end in zip(starts, bounds, strict=True)]
|
||||
|
||||
|
||||
def _pin_line(lines: list[str], span: range, key: str) -> int | None:
|
||||
"""The line declaring *key* inside one variant entry.
|
||||
|
||||
Args:
|
||||
lines: the variants file's lines.
|
||||
span: the variant's line range.
|
||||
key: the service key to locate.
|
||||
"""
|
||||
for number in span:
|
||||
if lines[number - 1].strip().startswith(f"{key}:"):
|
||||
return number
|
||||
return None
|
||||
|
||||
|
||||
def _collateral(
|
||||
services: dict, role: str, registry: dict, incapable: set[str]
|
||||
) -> list[str]:
|
||||
"""Onion-capable frontend partners a clearnet round keeps enabled.
|
||||
|
||||
Args:
|
||||
services: the variant's merged ``services`` mapping.
|
||||
role: the role the variant belongs to.
|
||||
registry: service key to registry entry.
|
||||
incapable: roles that refuse the onion outright.
|
||||
"""
|
||||
return sorted(
|
||||
key
|
||||
for key, entry in services.items()
|
||||
if isinstance(entry, dict)
|
||||
and entry.get("enabled") is True
|
||||
and (registry.get(key) or {}).get("service_type") == "frontend"
|
||||
and (registry.get(key) or {}).get("role") not in incapable
|
||||
and (registry.get(key) or {}).get("role") != role
|
||||
)
|
||||
|
||||
|
||||
class TestVariantTorCollateral(unittest.TestCase):
|
||||
def test_every_onion_capable_frontend_in_a_clearnet_round_is_justified(
|
||||
self,
|
||||
) -> None:
|
||||
incapable = _incapable_roles()
|
||||
registry = build_service_registry_from_roles_dir(ROLES_DIR)
|
||||
offenders: list[str] = []
|
||||
|
||||
for role, variants in sorted(get_variants().items()):
|
||||
if role in incapable:
|
||||
continue
|
||||
path = ROLES_DIR / role / ROLE_FILE_META_VARIANTS
|
||||
if not path.is_file():
|
||||
continue
|
||||
lines = read_text(str(path)).splitlines()
|
||||
spans = _variant_spans(lines)
|
||||
for index, variant in enumerate(variants):
|
||||
services = variant.get("services") or {}
|
||||
if (services.get(TOR_KEY) or {}).get("enabled") is not False:
|
||||
continue
|
||||
if index >= len(spans):
|
||||
continue
|
||||
collateral = _collateral(services, role, registry, incapable)
|
||||
if not collateral:
|
||||
continue
|
||||
number = _pin_line(lines, spans[index], TOR_KEY)
|
||||
if number is None or not is_suppressed_at(lines, number, RULE):
|
||||
offenders.append(
|
||||
f"{role} variant {index}: drops the onion while pinning "
|
||||
f"{', '.join(collateral)} on, all of which serve over it"
|
||||
)
|
||||
|
||||
if offenders:
|
||||
self.fail(
|
||||
f"{len(offenders)} clearnet round(s) take an onion-capable partner "
|
||||
"down with them. Move the partner into a variant that keeps the "
|
||||
f"onion, or mark the tor pin '# nocheck: {RULE} <why they belong "
|
||||
"here>':\n" + "\n".join(offenders)
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,101 @@
|
||||
"""A variant may drop the onion only when something in it cannot carry one.
|
||||
|
||||
``services.tor`` is a consumer flag, not a capability: every role declares the
|
||||
same ``enabled: "{{ 'svc-net-tor' in group_names }}"`` and so joins the onion
|
||||
whenever ``svc-net-tor`` is deployed. A handful of roles pin it to a literal
|
||||
``false`` instead, because onion routing breaks their transport - the
|
||||
real-time media services. Those are the only legitimate reason for a variant
|
||||
to pin ``tor.enabled: false``.
|
||||
|
||||
A variant that drops the onion without one of them in scope removes onion
|
||||
coverage from every service it does enable, for nothing. ``web-app-gitlab`` is
|
||||
the case that motivated this guard: it carries the dynamic flag and works over
|
||||
the onion, yet sat in a round that had dropped it.
|
||||
|
||||
Scope: a variant is excused when its own role is one of the incapable ones, or
|
||||
when it pins a service whose provider role is. A role coupled only through
|
||||
``group_names`` is invisible here, which is why the failure message asks for a
|
||||
pin rather than assuming the author is wrong.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from utils.cache.applications import get_variants
|
||||
from utils.roles.applications.services.registry import (
|
||||
build_service_registry_from_roles_dir,
|
||||
)
|
||||
from utils.roles.mapping import ROLE_FILE_META_SERVICES
|
||||
|
||||
from . import ROLES_DIR, load_role_meta, role_dirs
|
||||
|
||||
TOR_KEY = "tor"
|
||||
|
||||
|
||||
def _incapable_roles() -> set[str]:
|
||||
"""Roles whose own ``services.tor.enabled`` is a literal ``false``."""
|
||||
found: set[str] = set()
|
||||
for role_dir in role_dirs():
|
||||
services = load_role_meta(role_dir / ROLE_FILE_META_SERVICES)
|
||||
tor = services.get(TOR_KEY) if isinstance(services, dict) else None
|
||||
if isinstance(tor, dict) and tor.get("enabled") is False:
|
||||
found.add(role_dir.name)
|
||||
return found
|
||||
|
||||
|
||||
def _service_providers() -> dict[str, str]:
|
||||
"""Map every service key to the role that provides it."""
|
||||
registry = build_service_registry_from_roles_dir(ROLES_DIR)
|
||||
return {
|
||||
key: entry["role"]
|
||||
for key, entry in registry.items()
|
||||
if isinstance(entry, dict) and entry.get("role")
|
||||
}
|
||||
|
||||
|
||||
class TestVariantTorOffNeedsAReason(unittest.TestCase):
|
||||
def test_no_variant_drops_the_onion_without_an_incapable_role(self) -> None:
|
||||
incapable = _incapable_roles()
|
||||
providers = _service_providers()
|
||||
offenders: list[str] = []
|
||||
|
||||
for role, variants in sorted(get_variants().items()):
|
||||
if role in incapable:
|
||||
continue
|
||||
for index, variant in enumerate(variants):
|
||||
services = variant.get("services") or {}
|
||||
if (services.get(TOR_KEY) or {}).get("enabled") is not False:
|
||||
continue
|
||||
excused = [
|
||||
key
|
||||
for key, entry in services.items()
|
||||
if isinstance(entry, dict)
|
||||
and entry.get("enabled") is True
|
||||
and providers.get(key) in incapable
|
||||
]
|
||||
if not excused:
|
||||
offenders.append(
|
||||
f"{role} variant {index}: pins tor.enabled false while "
|
||||
"enabling nothing that refuses the onion"
|
||||
)
|
||||
|
||||
if offenders:
|
||||
self.fail(
|
||||
f"{len(offenders)} variant(s) drop the onion for no reason. Either "
|
||||
"pin tor.enabled true so the round keeps onion coverage, or pin the "
|
||||
"service whose role refuses the onion so the drop is visible:\n"
|
||||
+ "\n".join(offenders)
|
||||
)
|
||||
|
||||
def test_at_least_one_role_refuses_the_onion(self) -> None:
|
||||
self.assertTrue(
|
||||
_incapable_roles(),
|
||||
"no role pins tor.enabled false any more, so every variant dropping "
|
||||
"the onion is unjustified and this guard needs rewriting rather than "
|
||||
"passing vacuously",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user