- Declare `__all__` in every package `__init__.py` whose `PROJECT_ROOT`
is imported by its modules (163 packages), so the constant counts as
exported; list every public name where the package defines more.
- Empty the 15 package `__init__.py` files whose `PROJECT_ROOT` no module
imports.
- Declare the public API of `utils.github.variant.axes` in `__all__`,
re-exports included, instead of silencing them with `noqa: F401`; the
list matches the one `feature/store-i18n` already carries.
- Use `import unittest.mock as mock` in `test_applications.py`, as the
rest of the suite does.
- Spell `_NON_PRINTABLE` in `utils/yaml_bootstrap.py` as the equivalent
BMP-only positive class; it matches exactly the same 2111 code points
as the old negated class across U+0000..U+10FFFF.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The workflows pin `runs-on: ubuntu-NN.NN`, so no job moves to a new Ubuntu
release behind our back, but nothing raised the pin either. A sixth job of
`🔄 Update versions` closes that: `utils/update/runner.py` reads the image
table of actions/runner-images, takes the newest x64 Ubuntu label whose
image carries no preview or beta badge, and raises every older bare
`runs-on: ubuntu-NN.NN` to it; arch-suffixed labels stay untouched. The job
opens the change as one PR over `.github/workflows`, so its CI run tests
the new image before it reaches main.
This commit carries the updater's first result: every job now runs on
ubuntu-26.04 (49 `runs-on`).
actionlint 1.7.12, the newest release, does not know `ubuntu-26.04` yet
(rhysd/actionlint#682, fix proposed in rhysd/actionlint#743), so
`.github/actionlint.yaml` declares the label until a release knows it.
act maps each `ubuntu-NN.NN` label the workflows use to the local platform
image, derived from `.github/workflows`, instead of a fixed list that did
not know ubuntu-26.04.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The scheduled mirror job has failed every day since 2026-09-25 on the same two
images: quay.io/minio/minio and quay.io/minio/mc answer `unauthorized` to
anonymous pulls ("2/197 failed", run 38352569503). Both belong to
web-app-minio, an eol role outside the lifecycle envelope that default.env
tests, so no CI deploy pulls them.
iter_role_images now skips every role that is_maintained() reports as
unmaintained, the check the version updaters already apply to the same list.
On the tree the discovery yields 187 images instead of 192; the five it drops
belong to the four eol roles web-app-confluence, web-app-jira, web-app-minio
and web-app-phpldapadmin. The mirror sync, resolver and wait read this list,
and their descriptions and mirror.md now say so.
The roles directory is resolved first: is_maintained reads a relative path as
a role name and would keep every role.
475ab9b386 on feature/store-i18n takes the same approach with a lifecycle
reader of its own; main already has is_maintained for it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The deploy decides what ADAPTER_PORT or OPENWEBUI_ADMIN_NAME is and
writes it into the environment. A default in the reader is a second place
that value lives, and it is the one nobody greps for. os.environ["X"]
fails loudly on a missing key, which is the behaviour a misconfiguration
deserves.
Empty values stay free, because they add no second source and only say
absent: "", None, False, 0, the empty containers, and the "[]"/"{}" a
json.loads reader hands an absent variable. A default that is itself an
environ.get is free too, since it reads the one source again under
another key; a literal ending such a chain is still reported. Detection
runs on the parsed tree, so a call split across lines reads the same.
Five sites were real duplication and lost their default: the adapter
port, openwebui's admin name (a role that runs as a bot user), odoo's
OIDC uid field, seaweedfs' public-primary flag, and the swarm hosts
entry IP. Nineteen carry a nocheck naming why the value is not the
deploy's: contributor tooling whose default is the only source, test
scaffolding that writes the file it names, and two readers of
CA_TRUST_CERT_EXTRA_HOST, which nothing in the repo sets.
utils/diagnostics/container.py resolves without a marker: RESCUE_DEPTH is
a recursion base case, and the integer 0 says what the string "0" said.
The file sits exactly at the 500-line cap, so a comment line was not
available.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twelve files conflicted; most resolutions keep both sides rather than pick
one.
cli env.py and compose.py: both branches extend the same spots. The tools
override is appended before write_override, because that call filters the
mirror override to the compose files it is handed. compose.py keeps this
branch's CACHE_ROLE_SHELL_DIR, since the cache scripts moved into a role and
main's literal scripts/docker/cache path is their old home, and takes main's
mirrored init image alongside it.
utils/docker/image/discovery.py: this branch's version_variants expansion
plus main's whole runner-image block, which runner_mirror imports.
sys-svc-compose network create: main's side wins on substance and this
branch's on structure. main carries eda6249d7b, the fix for a probe that
matched an id prefix and let a sibling shadow the exact name; this branch had
refactored the task into create/overlay.yml and create/bridge.yml but brought
back the docker_network_info probe that fix replaced. The subfiles stay and
now probe through the exact-name lookup. main's half also read
SWARM_CLUSTER_AVAILABLE, which this branch removed in favour of
IS_SWARM_MODE, as the merge base confirms, so the gates follow this branch.
The lookup's import follows this branch's get_entity_name to entity_name
rename.
web-app-pixelfed: the custom entrypoints stay removed, since the new image
brings its own and the variables behind them no longer exist, and main's
host_alias=true comes along.
web-app-suitecrm: main's session.yaml and its config/packages path, because
only that version wires framework.session and only that directory is read by
Symfony.
web-app-odoo: main's onionOptions import is required by the test.use below
it; this branch's test-seaweedfs registration stays.
Four defects surfaced that no conflict marker shows.
The auto-merge of web-app-suitecrm/vars/main.yml duplicated
SUITECRM_SESSION_CONFIG_HOST, which ansible-lint rejects. Removing it exposed
the second: SUITECRM_SESSION_CONFIG_DOCKER still named config/session.yaml
while the mount had moved to config/packages, so the freshness comparison in
tasks/01_manager_ops.yml would have tested a path nothing writes.
svc-db-elasticsearch carried environment: on two include_tasks, which Ansible
accepts on include_role only. ELASTIC_PW now sits on the five tasks that read
it, which is also the tighter scope for a credential.
The openclaw templates render NETWORK_TRUSTED_PROXY_CIDRS while their test
renderer passes only its explicit kwargs, so seven tests raised Undefined.
That predates the merge: the template line is on this branch and not on main,
and both sides share the same test file.
test_runner_mirror neutralises four environment variables under clear=False
and missed three, so the gpu and tools overrides leaked in from the container
and the assertion on the layered list broke. It now neutralises all seven.
make quality-high passes both phases.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The GHCR mirror covered role images only. The images the runner pulls on
its own went straight to Docker Hub: coredns from compose.yml, the cache
stack in compose/cache.override.yml and alpine:latest, the init image of
scripts/docker/cache/package-frontend/certs.sh. With the anonymous
githubactions quota spent, the guide step of Postmarks#0 (main run
37954203718) and Shopware#2 (#872) died on "toomanyrequests" while pulling
alpine:latest, after coredns had hit the same limit.
iter_runner_images (utils/docker/image/discovery.py) finds those images,
and the mirror sync and the fork-PR wait now cover them next to the role
images. On a real GitHub runner compose_file_args layers a generated
build/compose/runner-mirror.override.yml that points only the services of
the layered compose files at their mirror, and the certificate generator
gets the mirrored init image. Local runs and act are unchanged, and the
coredns literal stays in compose.yml for Dependabot. test-dns now waits for
images-mirror-missing like the deploy and workspace jobs, since it brings up
the stack too.
Verified with docker compose config on a simulated GitHub runner: the
service set is unchanged and only the runner images point at
ghcr.io/infinito-nexus/core/mirror/docker.io/..., and an existing role mirror
under that scheme pulls anonymously.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The README said a templated pin moves whenever the assembled value
changes. newer_version() additionally rejects an assembled value that
sorts below the current pin, which
test_an_unchanged_or_older_templated_tag_reports_nothing covers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An http_regex source read one document, so a tag whose parts upstream
keeps in two files could only carry the second part as a literal. Every
pin also fetched its document on its own: a failed request left one pin
of a release behind while the others moved.
url now takes a list whose bodies are searched as one text, and every
document is fetched once per run, so all pins that read it resolve against
the same snapshot. The HTTP reads move into utils/update/fetch.py.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A pin that names its own upstream in an update: block was still collected
by the image updater or, for a ref, by the repository updater. Those raise
it to the newest tag of its registry or repository, which for a component
of a release is not the release's value. A tag built from several upstream
values, such as <freeswitch>-<bigbluebutton>, could not be resolved at
all: the resolver takes one capture and keeps a pin on its suffix.
The image updater now skips a version and the repository updater a ref
that declares a source for itself. An http_regex source accepts a template
over the pattern's named groups, and a templated pin moves whenever the
assembled value changes, its suffix included.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A pin that names its own upstream in an update: block was still collected
by the image updater, which raises it to the newest tag of its registry.
For a component of a release bundle that tag is not the bundle's, so the
two updaters would propose different versions for the same line and such
a pin had to carry nocheck: docker-version.
The image updater now skips a version that declares a source for itself.
The source resolver alone bumps it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The update bot opened a version PR for web-app-jira (#821) although the role is end of life: none of the updaters looked at the lifecycle. utils.update.base.is_maintained now answers that question once, and the collectors for image versions, declared version sources, pip pins, repository refs and the addon check skip a role whose lifecycle is eol.
The lint that requires every semver pin to be watched shares that scope, otherwise it would report the pins of the eol roles as unwatched. Every other stage, deprecated included, stays in the update scope.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The version-source resolver orders upstream versions by semver. A pin that
is not one, such as `latest` or a branch name, has no depth, so no candidate
ever matches it and the declaration never moves anything. Nothing reported
that: the watched-pins lint only looks at values that already are a semver.
test_version_keys now fails on an `update:` block whose pinned key does not
hold a semver. A monitored addon the resolver does not read is held to the
same rule as soon as it pins a `version`, because the catalog check skips a
non-semver pin just as silently. An addon without a `version` keeps tracking
the app default and is left alone.
The Stalwart README no longer asks for `webui_version` to be bumped by hand
next to the image tag. Its `update:` block names the upstream tags, the
server itself follows `releases/latest` of the bundle, and the resolver
plans v1.0.5 to v1.0.11 for it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The version-source updater only read meta/services.yml. An addon pinned in
meta/addons/<id>.yml stayed where it was: the ONLYOFFICE connector for
Nextcloud sat on v10.1.2, which does not install on Nextcloud 35, until
someone bumped it by hand.
The updater now reads addon files too. An addon declares its source in the
`update:` block it already has, beside monitored, catalog and upstream_id.
A monitored addon of the github-releases catalog needs nothing more: its
upstream_id names the repository whose tags are its versions. The
update-version-sources job opens the PRs, one per role.
Every pinned addon is wired:
- nextcloud onlyoffice carries its own version pin now. Its archive URL
stays a literal that carries the pin, because
test_pinned_addon_archives_available reads it as written
- wordpress mcp-adapter, moodle auth_oidc and jellyfin sso-authentication
follow the tags of their repository, moodle within its v4.5 series
- the three xwiki extensions read their Maven metadata
- pretix-oidc reads the PyPI simple index. The JSON API answers 404 behind
the package cache the test stack routes pypi.org through
- joomla's plugin is built here and jellyfin's ldap-authentication is bound
to the md5 of its archive; both are opted out with that reason
A bump rewrites the pin and the config.archive line together, and an
archive that does not carry the pin fails test_version_keys, so the two
cannot drift apart again. test_version_pins_are_watched covers addon pins
as well: a new one without a source or a reasoned opt-out fails the lint.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Docker updater watches image plus version, the repository updater
repository plus ref. A dozen pins fell outside both - an app_version, a
release, a tag in a registry neither speaks - so nothing ever told anyone
they had gone stale.
Such a pin now names its upstream next to itself in an `update:` block:
git_tags, registry_tags, npm, http_regex or a role-local script, with
`match` selecting a tag flavour and `strip` removing a prefix the pin does
not carry. utils/update/source.py resolves them beside the two existing
updaters, `python -m cli.contributing.update.source` bumps them, the
update-version-sources job opens one PR per role, and
tests/external/update/source warns while a bump is pending.
Two lints hold the rule: every semver pin is watched by one of the three
updaters or opted out with a reason, and a semver only ever sits behind
`version`, `<what>_version` or `ref` - which renames moodle's release to
app_version and simpleicons' local_version to local_tag, the latter being
the tag of a locally built image rather than an upstream version. A
declaration naming a key the entity does not carry, or a type the resolver
does not know, fails the lint instead of resolving nothing in silence.
Picked from feature/store-i18n (94d607fd0a) onto current main. Two things
differ from the original: svc-ai-lmstudio no longer carries an app_version
pin, so its declaration is dropped, and web-app-stalwart's webui_version,
added since, declares the git tags of the webui repository.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
d6d7ff8c9e made iter_role_images yield one ref per version_variants suffix so
the mirror copies the extra tags. Two of its four consumers assumed one ref per
service. The resolver writes services[service] = {image, version}, so the last
ref won and the applications override carried v1.9.6-cuda; the engine role
appends -cuda itself and asked the mirror for v1.9.6-cuda-cuda, a tag nothing
ever pushed. The version updater builds one entry per ref and writes back into
meta/services.yml, so it would have replaced the declared version with a cuda
tag once this branch reaches the bot on main.
ImageRef now carries `derived`. The resolver and the updater skip those refs;
the mirror sync keeps iterating all of them, which is what the variants exist
for, and the mirror wait already keyed on the full ref including the version.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
autodoc documents whatever __all__ lists, including a name the module only
imported, so sphinx found two targets for every annotation: nine for
ImageRef and one for DevInventorySpec.
mirror/model is a five-line shim whose whole purpose is the local name, so
it drops __all__ entirely. The inventory package keeps its own, which also
declares private helpers, and loses only that one entry; the reason sits in
the module docstring so nobody re-adds it innocently.
Both names stay importable from their packages, because the import binds
them regardless of __all__. Every importer in the repository was enumerated
and none uses a wildcard.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The same failure class as 3dee332350, in the places that only became
visible once the louder messages were gone. Nine of them are class
docstrings, which the detector used to find the first round skipped
because it walked modules and functions only.
restart_condition carried a section title inside its docstring, where
docutils allows none, and the indented block below it held a second.
cert_utils indented the closing brace of its example shallower than the
body it closes.
Two are regressions of my own first round: filters lacked the blank line
between its new literal block and the paragraph after it, and
fediwall_active had the wrong one of its two colons corrected.
make i18n-extract domain=docs now finishes with zero warnings under -W.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The lint accepted a link that resolved to a file or a directory. myst
builds a document reference only when the target is_file(), so a link to
a directory without a README is a bare file listing on GitHub and a dead
cross-reference in the documentation build. It now reports which of the
two reasons applies.
The tightened rule found 41 links. Where a README backs the directory,
bd15eae9fa's resolver carries the link and it stays. Where none exists,
the link becomes a code span rather than an invented README: 15 of them
point at a role's meta/addons/, whose addon table already sits below the
sentence, and the rest at code directories such as utils/ or
cli/administration/deploy/runner/.
The changed sentences are msgids, so these entries return to the
translator on the next extract.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Docker updater watches image plus version, the repository updater
repository plus ref. Twelve pins fell outside both - an app_version, a
release, a tag in a registry neither speaks - so nothing ever told anyone
they had gone stale.
Such a pin now names its upstream next to itself in an `update:` block:
git_tags, registry_tags, npm, http_regex or a role-local script, with
`match` selecting a tag flavour and `strip` removing a prefix the pin does
not carry. utils/update/source.py resolves them beside the two existing
updaters, `python -m cli.contributing.update.source` bumps them, the
update-version-sources job opens one PR per role, and
tests/external/update/source warns while a bump is pending.
Two lints hold the rule: every semver pin is watched by one of the three
updaters or opted out with a reason, and a semver only ever sits behind
`version`, `<what>_version` or `ref` - which renames moodle's release to
app_version and simpleicons' local_version to local_tag, the latter being
the tag of a locally built image rather than an upstream version. A
declaration naming a key the entity does not carry, or a type the resolver
does not know, fails the lint instead of resolving nothing in silence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The nocheck markers were written as long single lines; ruff splits the
call across three lines and leaves the marker behind the closing paren.
Committing the formatter's own output keeps the tree and the history in
step.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gate flagged six of my own changes. INFINITO_ANSIBLE_COLLECTIONS_SOURCE
reached default.env without a handler, so the generator dropped it
silently; ansible.sh then carried a shell-side default for the same key,
a second source that drifts. Both are now one declaration, read as
required, so a missing key fails loudly instead of falling back.
File reads route through the session cache except where the same run
rewrites the file, which is the documented opt-out and now says so per
call. The updater reads yaml through utils.cache.yaml rather than calling
safe_load itself.
restore_collection_history.sh was committed without its executable bit,
and the new workflow's name was 31 characters where the dispatch menu
truncates at 25; the explanatory wording lives in run-name instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Which of the two sources is faster is not a constant. galaxy.ansible.com
degraded from 207s to 687s over one day and then served 502s outright,
while the pinned git fallback carried the same install in 31s; a week
later the order may be the other way round. The daily job times both and
swaps the default only when the preferred one's median reaches three
times the other's over twelve samples.
A failed install is recorded as None and ranks worse than any duration,
so an outage counts instead of dropping out of the statistic. Below a
full window nothing changes, which keeps a single bad day from flipping
the default.
History lives in an artifact rather than the Actions cache: a schedule
run resolves to the default branch and has read-only cache access there,
so it could not carry the window forward.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pinning the collections froze them, and nothing thaws a pin: dependabot
reads pyproject.toml, the docker updater reads image tags, and neither
sees a version inside requirements/. The new job resolves upstream
releases with git ls-remote --tags and opens a PR the same way the pip,
image and repository-ref updaters do.
Both requirement files are bumped together, because the git fallback has
to install what the Galaxy file declares; moving only one would break
that. A collection whose two files disagree, or whose version is not a
semver, is left alone -- that covers the branch ref and the commit pin,
which state an intent a bumper must not guess.
The job calls pr.sh rather than role_prs.sh: the latter splits per
sub-directory and would open no PR at all for flat paths like
requirements/*.yml.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The version cron updates docker image tags and repository refs, and tests/external/update covers docker, addons and repository. Neither sees a version inside an Ansible task variable, and dependabot reads pyproject.toml rather than roles/. A pin set once therefore froze and nothing thawed it: backup-docker-to-local has not moved since it was written, and the automtu pin one commit back would have gone the same way.
utils/update/pip mirrors the docker module: collect the pins, ask PyPI, rewrite in place. Only exact == pins are collected, because a range or a VCS reference states an intent an updater must not overrule. The external check warns rather than fails, like its docker sibling, so a slow PyPI cannot redden a run and a release the repository has deliberately not taken is not a regression.
The cron job is the docker job with one script swapped, so a changed pin arrives as its own PR under update/pip-pins.
Executed: the collector finds both pins in the repository; a synthetic file rewrites from 1.0.0 to 1.1.1 with the indent, quoting and neighbouring line untouched; automtu==1.0.0 would be reported while a bare name and a git+ reference are passed over.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The package-cache frontend terminates TLS for package traffic and signs
with its own CA: probed from inside the test container, pypi.org arrives
as subject CN=pypi.org issued by CN=Infinito Package-Cache Frontend CA.
package-frontend-ca.sh installs that CA into the OS store, which is why
apt, pip, curl and urllib work — but requests reads certifi, which has
never heard of it, so test-external failed its reachability precondition
with CERTIFICATE_VERIFY_FAILED. Upgrading certifi does not help; the
issuer is not a public root.
load.sh is where the runner puts the venv on PATH, so it is also where the
interpreter carrying the pip certifi is chosen; pointing REQUESTS_CA_BUNDLE
and SSL_CERT_FILE at the store the stack deliberately fills keeps one trust
decision instead of two. A caller-set value still wins.
The bundle location was about to become a fourth copy: dev-nix and the
network diagnose CLI each carried their own candidate list, and the two had
already drifted apart in their fourth entry. All of them now read
group_vars CA_BUNDLE_CANDIDATES — Ansible directly, the shell through a
generated env key, python through read_group_paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3529 invalid comments in 528 files, grandfathered in while the linter
only saw the working-tree diff. Removal ran through the linter's own
collectors, so detection and deletion cannot drift: a comment whose
opener line is blank before it and whose closer line is blank after it
loses all its lines, anything else loses only the comment region, and a
line left holding a bare delimiter goes with it.
Five files keep their comments behind a nocheck marker instead. Four are
upstream configs whose inline docs are upstream's own (mautrix, systemd
journald, nextcloud-oidc-login, MediaWiki). In littlejs the
commented-out entries ARE the deactivation mechanism for an example
tile, so deleting them would delete configuration.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The min_storage gate measures every declared pin on each run, and every measurement was an anonymous manifest request to Docker Hub, which meters those per IP. A full run exhausted the pull budget, so the following local deploy could no longer pull at all and the gate itself stopped measuring: 96 of 165 images answered indeterminately and only warned.
Read the GHCR mirror first and fall back to the upstream registry only for images it does not carry. The copy is byte-identical, so the measured size is unchanged. Persist measured sizes under the temporary directory; store positive results only, because a cached indeterminate answer would be remembered as fact and the gate would stop measuring for good.
Lift the mirror path formula out of the provider into utils so the writer and the reader share one definition, resolving the namespace from GITHUB_REPOSITORY, then remote.pushDefault, then origin. pushDefault has to win: origin may point at an upstream that hosts no mirror. Parse .git/config by hand because configparser rejects the duplicate keys git writes and the test container ships no git binary.
Measured over all 165 declared pins: no requests to Docker Hub, 277 to the mirror, 9 to registries it does not carry, and 164 of 165 images measured.
Leave manifest_exists on the upstream registry. It answers whether the declared pin is still pullable from where the deploy pulls it, which a mirrored copy cannot attest; it takes the cache only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Conflicts resolved:
- .claude/.gitignore: union of ignore patterns
- plugins/lookup/rbac_group_path.py: drop both unused conflicting imports
(body fetches apps via lookup_loader)
- roles/sys-svc-compose/handlers/main.yml: keep HEAD's refactor (compose-up
handlers live in compose.yml, included per DEPLOYMENT_MODE)
- utils/cache/base.py: keep HEAD's merge_with_defaults/FILE_TOKENS imports
Post-merge fixes for main's newly-enabled lints:
- pyproject.toml: grandfather 5 files under the BLE001 ratchet
- justify 6 pre-existing security # noqa markers (S108/S311/S701)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the csp, domains, and networks blocks out of roles/*/meta/server.yml
into dedicated meta files, each mapping to its own top-level applications key
via the recursive meta-walk:
csp: -> meta/csp.yml -> applications.<app>.csp
domains: -> meta/domains.yml -> applications.<app>.domains
networks: -> meta/networks.yml -> applications.<app>.networks
Residual nginx/proxy keys (status_codes, client_max_body_size, locations,
ip6_enabled, config_upstream_url) stay in server.yml; an emptied server.yml is
deleted. 98 server.yml -> 249 new files, 74 deleted.
- Consumers repointed applications.<app>.server.{csp,domains,networks} ->
applications.<app>.{csp,domains,networks} across templates, vars, tasks,
lookups and filters.
- Infra readers migrated: canonical_domains_map, application_domain_index,
build_ldap_role_entries, service-registry overlay discovery,
roles_with_service, and the file-based readers utils/domains/list.py +
utils/meta/scan.py (read the now-deleted server.yml directly).
- Detection + mapping: _META_TOPICS gains csp/domains/networks;
ROLE_FILE_META_{CSP,DOMAINS,NETWORKS} constants + ROLE_FILES entries.
- New self-expiring lint guard (test_server_meta_no_split_fields) fails if a
server.yml still carries a split field; auto-fails from 2026-12-28 as a
delete reminder.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve the 80 open CodeQL/code-scanning alerts on this branch.
Security-sensitive fixes:
- cli/administration/deploy/development: break the unsafe import cycle by
extracting the env/arg helpers into a leaf module `env.py`, giving the
load-time DAG compose -> common -> deps -> env.
- web-app-bluesky login-broker server.js: prototype-pollution guard on
cookie parsing, CR/LF + control-char log sanitisation, HTML-escaped
exception text, and a ReDoS-bounded handle regex.
- web-app-baserow SSO: harden the open-redirect `next` handling.
- network diagnose probes: require TLS >= 1.2.
- actionlint installer: validate tar members against path traversal.
- decidim/apt-purge lint regexes: rewrite to avoid catastrophic backtracking.
- 6x Playwright URL regexes anchored; crs-k/stale-branches pinned to a commit SHA.
Mechanical cleanups: unused imports/globals/locals, empty-except comments,
import-and-import-from, implicit-string-concat, multiple-definition,
unreachable/useless-assignment.
False positives are intentionally left in place (Django settings patches,
Odoo __manifest__, side-effect imports, cross-module symbols).
Recurrence guards:
- import-linter contract (pyproject + tests/lint) locks the dev-deploy
package acyclic; a new cycle now fails CI instead of only CodeQL.
- new lint test requires every bandit (S###) `noqa` to carry a justification.
- BLE001 (blind-except) added to ruff as a ratchet (tests/roles exempt,
current production offenders grandfathered).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a lint-packages target/workflow that parses the generated packaging metadata with each ecosystem's native parser (dpkg-parsechangelog, rpmspec, bash -n for the PKGBUILD), failing on any parser stderr so warnings cannot slip through. Wired into the make lint fan-out and ci-orchestrator's lint gate; an install group provisions the parsers (best-effort, cross-distro).
Hardens package_mirror so the generated changelogs parse without warnings: every RPM body line gets a '- ' continuation prefix (a column-0 '*' was read as a new %changelog entry -> 'bad date'), and the archived-releases notice is folded into the oldest kept entry instead of trailing after the final entry (free text there made dpkg-parsechangelog warn). Regenerates packaging/debian/changelog and the Fedora spec accordingly.
Adds a test-lint check (test_package_changelog_completeness) asserting both package changelogs list every CHANGELOG.md + docs/changelog release and that their inline window matches CHANGELOG.md, plus unit regressions for the RPM continuation prefix and the Debian footer fold.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
format_debian_entry indented only the first body line per entry, leaving the rest at column 0. dpkg's changelog parser treats un-indented change lines as badly formatted, loses the entry boundary and reports 'cannot parse maintainer email address ""', failing the .deb build in build-ci-images. Every non-empty body line is now indented by two spaces.
trim_and_archive additionally dropped the '# Changelog' H1 preamble on every rewrite (it rebuilt content from the version entries only), which would break the MD041 markdown lint on the next trim. The preamble before the first version header is now preserved.
Regenerated packaging/debian/changelog and packaging/fedora/infinito-nexus.spec from CHANGELOG.md (untrimmed). Also adds the missing YAML document-start to web-app-pihole/tasks/main.yml.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The auto-generated "Older Releases" index in CHANGELOG.md emitted dash
bullets while the rest of the file uses asterisks, failing markdownlint
MD004 (ul-style, consistent) with 49 errors. Switch the archive index
generator to asterisk bullets, convert the existing entries, and align
the unit test assertions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
scripts/meta/wait/mirrors.sh embedded its logic as a `python - <<PY` heredoc
plus a bash skopeo/retry loop, and the heredoc imported iter_role_images from
utils.docker.image_discovery — a module that does not exist (it is
utils.docker.image.discovery). That raised ModuleNotFoundError, so the process
substitution yielded zero refs and the script logged "Nothing to wait for" and
exited 0, silently turning the fork-PR "wait for mirrored images" safeguard
into a no-op.
Move the whole thing into cli/contributing/mirror/wait/ (sibling to sync and
resolver): an argparse entrypoint that enumerates role images via
iter_role_images, computes each GHCR destination via GHCRProvider, and polls
GHCRProvider.tag_exists with --attempts/--sleep-seconds until all are present
or it times out. mirrors.sh becomes a thin env->args wrapper invoking
python -m cli.contributing.mirror.wait, matching scripts/meta/mirror/sync_missing.sh.
Behaviour is identical to the previous bash, minus the broken import.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The fully-checked-Markdown archive helper used to live under
cli/contributing/requirements/archive/ in this repo. It is a generic
spec-folder-management primitive (numbered NNN-topic.md files,
task-list completeness check, README ## Archive section append) and
now ships in pkgmgr.actions.archive so every kpmx-managed repository
can rely on the same convention.
Changes here:
* tests/lint/repository/documentation/test_requirements_completeness.py
imports the archive primitives from pkgmgr.actions.archive instead
of the vendored cli.contributing.requirements.archive, and points
contributors at `pkgmgr archive docs/requirements` in the failure
message.
* cli/contributing/requirements/README.md becomes a stub that
redirects readers to the kpmx archive subcommand.
* The vendored cli/contributing/requirements/archive/ package and its
unit tests under tests/unit/ are deleted.
* The pkgmgr role pin bumps to v1.15.0 to pick up the new archive
subcommand on deployed hosts.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* TC003: move `pathlib.Path` into `TYPE_CHECKING` blocks across the
new `requirements/archive` package and its completeness lint test.
* TC001: same treatment for `VaultHandler` in
`credentials/emit.py` (annotation-only).
* SIM105/S110: replace `try/except/pass` in
`current_play_redirect_domains.py` with
`contextlib.suppress(Exception)`.
* `archive/__main__.py`: mark `README_PATH.read_text(...)` with
`# nocheck: cache-read` — CLI rewrites the README in the same flow,
mirroring the `changelog/archive` pattern.
* Drive-by ruff-format condensations in the archive unit tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* `python -m cli.contributing.requirements.archive` moves the H1 of
every fully-checked `docs/requirements/NNN-*.md` into the
`## Archive` block of the directory README and deletes the file.
Files with any remaining `- [ ]` are skipped. Split SRP-wise into
discovery / inspect / readme / __main__.
* New lint `test_requirements_completeness` warns per unchecked
acceptance criterion (GitHub annotation) and fails when any
requirement is ready for archival, pointing at the CLI. Keeps the
requirements directory small so AI agents process less context.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Relocates `cli/meta/{ports,networks/suggest}` to
`cli/contributing/network/{ports,address/suggest}` so port and subnet
allocators live alongside the rest of the contributor-side tooling
(mirror, update, changelog) instead of `meta`. Mirrors the tests
into `tests/unit/cli/contributing/network/` and renames the contract
docs to `docs/contributing/tools/network/{port,address}.md`. Prog
names update from `cli meta ports suggest` / `cli meta networks
suggest` to `cli contributing network ports suggest` / `cli
contributing network address suggest`; the corresponding lint
references and `utils.meta.scan` docstring follow.
Adds a third subcommand: `cli contributing network diagnose`
(stdlib-only DNS / TCP / TLS / PMTU probe against the standard
infrastructure hosts plus interface MTU, `/etc/resolv.conf`,
`/etc/hosts`, proxy env, CA-bundle summaries). Split across `cli.py`
(orchestration), `config.py` (SPOT for hosts / per-distro install
commands), `format.py` (output + subprocess helpers), `tools.py`
(distro-aware auto-install of `iputils-ping` + `iproute2`),
`report.py` (system-info dumps + IPv6-default-route detection), and
`probes.py` (per-host probes). Exposed as `make debug-network`.
Companion `docs/contributing/actions/debugging/network.md` absorbs
the diagnose-tool walkthrough (incl. how to read MTU-induced TLS
handshake timeouts) so the merged page covers DNS / MTU / IPv6 /
connectivity end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.
utils/
update/
base.py # semver primitives shared by both backends
docker.py # OCI registry lookups + version: line rewriter
repository.py # git ls-remote walker + ref: line rewriter
cli/contributing/update/
docker/ # python -m cli.contributing.update.docker
repository/ # python -m cli.contributing.update.repository
scripts/github/update/
image_versions.sh
repository_refs.sh
Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.
Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Modules imported by tests run inside the same pytest process as the
tests themselves. Raw ``path.read_text(...)`` calls in those modules
bypass the cache the tests just populated. ``roles/`` stays excluded
(deployed scripts have no ``utils/`` on PYTHONPATH); ``utils/cache/``
stays excluded (cache implementation itself).
Five conversions to ``utils.cache.files.read_text``:
- ``utils/handler/yaml.py`` (YAML loader, read-only)
- ``utils/annotations/summarize.py`` (CI log scanner, with
``UnicodeDecodeError`` fallback replacing ``errors="replace"``)
- ``cli/meta/roles/applications/resolution/combined/__main__.py``
(single-shot CLI JSON input)
- ``cli/core/help.py`` (read_folder_title + read_folder_description;
same README is read many times across CLI help iteration)
Eight ``# nocheck: cache-read`` markers, each annotated inline:
- ``utils/claude/settings_sort.py`` (read then rewrite sorted)
- ``utils/docker/version_updater.py`` (read then rewrite config)
- ``cli/administration/deploy/development/network.py`` (tests mock
``Path.read_text`` directly; caching would bypass the mock)
- ``cli/administration/deploy/development/log_utils.py`` (append flow)
- ``cli/contributing/changelog/archive/changelog_md.py``,
``__main__.py``, ``package_mirror.py`` (file mutated in same flow)
Two callers refactored to variable extraction so ruff's 88-char wrap
keeps the ``# nocheck`` marker on the ``read_text(...)`` AST span line
(``cli/.../network.py`` and ``package_mirror.py:105``).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidates image declarations under a single schema: every role's
`<svc>.{image,version}` lives in `meta/services.yml` alongside compose
service entries. The `image` lookup now reads from disk per role,
`iter_role_images` collapses to a single source, and the mirror chain
(resolver + apply_mirror_overrides) drops the dead `images:` branch.
`test-e2e-playwright` resolves `TEST_E2E_PLAYWRIGHT_IMAGE` directly via
`lookup('image', 'playwright', 'ref')` in vars/main.yml; the runtime
set_fact + override-empty-string dance is gone.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
