The duplicate check accepted any open PR whose head branch started with
the dedupe prefix and ended in the fingerprint of the fresh proposal. The
listing also returns PRs from forks, and the fingerprint can be derived
from public data, so a fork PR with such a branch name would have made the
bot skip its own update PR without carrying the change.
The shortcut now only applies to PRs whose head branch lives in this
repository, which only accounts with write access can create. A fork PR
is still compared by the content of its head, as before.
Raised by the CodeRabbit review of PR #781.
Refs #780
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The Docker updater watches image plus version, the repository updater
repository plus ref. A dozen pins fell outside both - an app_version, a
release, a tag in a registry neither speaks - so nothing ever told anyone
they had gone stale.
Such a pin now names its upstream next to itself in an `update:` block:
git_tags, registry_tags, npm, http_regex or a role-local script, with
`match` selecting a tag flavour and `strip` removing a prefix the pin does
not carry. utils/update/source.py resolves them beside the two existing
updaters, `python -m cli.contributing.update.source` bumps them, the
update-version-sources job opens one PR per role, and
tests/external/update/source warns while a bump is pending.
Two lints hold the rule: every semver pin is watched by one of the three
updaters or opted out with a reason, and a semver only ever sits behind
`version`, `<what>_version` or `ref` - which renames moodle's release to
app_version and simpleicons' local_version to local_tag, the latter being
the tag of a locally built image rather than an upstream version. A
declaration naming a key the entity does not carry, or a type the resolver
does not know, fails the lint instead of resolving nothing in silence.
Picked from feature/store-i18n (94d607fd0a) onto current main. Two things
differ from the original: svc-ai-lmstudio no longer carries an app_version
pin, so its declaration is dropped, and web-app-stalwart's webui_version,
added since, declares the git tags of the webui repository.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The update bot recognised an open PR as a duplicate by hashing the blobs
of the files its commit changed, read at the PR head. A follow-up commit
that edits one of those files changes the hash, so the next run opened a
second PR with the identical bump: PR #778 next to PR #775 for
web-svc-collabora, after a fix had edited the bumped services.yml.
The branch name of a bot PR ends in the first seven characters of the
fingerprint of the change the bot proposed. An open PR whose branch ends
in the fingerprint of the fresh proposal now counts as a duplicate before
its head is compared. The head comparison stays as the second criterion.
The test runs pr.sh in a throwaway repository against a stubbed gh. It
fails on the previous script with exit 1, because that one goes on to
push.
Closes#780
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The gate flagged six of my own changes. INFINITO_ANSIBLE_COLLECTIONS_SOURCE
reached default.env without a handler, so the generator dropped it
silently; ansible.sh then carried a shell-side default for the same key,
a second source that drifts. Both are now one declaration, read as
required, so a missing key fails loudly instead of falling back.
File reads route through the session cache except where the same run
rewrites the file, which is the documented opt-out and now says so per
call. The updater reads yaml through utils.cache.yaml rather than calling
safe_load itself.
restore_collection_history.sh was committed without its executable bit,
and the new workflow's name was 31 characters where the dispatch menu
truncates at 25; the explanatory wording lives in run-name instead.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Which of the two sources is faster is not a constant. galaxy.ansible.com
degraded from 207s to 687s over one day and then served 502s outright,
while the pinned git fallback carried the same install in 31s; a week
later the order may be the other way round. The daily job times both and
swaps the default only when the preferred one's median reaches three
times the other's over twelve samples.
A failed install is recorded as None and ranks worse than any duration,
so an outage counts instead of dropping out of the statistic. Below a
full window nothing changes, which keeps a single bad day from flipping
the default.
History lives in an artifact rather than the Actions cache: a schedule
run resolves to the default branch and has read-only cache access there,
so it could not carry the window forward.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pinning the collections froze them, and nothing thaws a pin: dependabot
reads pyproject.toml, the docker updater reads image tags, and neither
sees a version inside requirements/. The new job resolves upstream
releases with git ls-remote --tags and opens a PR the same way the pip,
image and repository-ref updaters do.
Both requirement files are bumped together, because the git fallback has
to install what the Galaxy file declares; moving only one would break
that. A collection whose two files disagree, or whose version is not a
semver, is left alone -- that covers the branch ref and the commit pin,
which state an intent a bumper must not guess.
The job calls pr.sh rather than role_prs.sh: the latter splits per
sub-directory and would open no PR at all for flat paths like
requirements/*.yml.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The version cron updates docker image tags and repository refs, and tests/external/update covers docker, addons and repository. Neither sees a version inside an Ansible task variable, and dependabot reads pyproject.toml rather than roles/. A pin set once therefore froze and nothing thawed it: backup-docker-to-local has not moved since it was written, and the automtu pin one commit back would have gone the same way.
utils/update/pip mirrors the docker module: collect the pins, ask PyPI, rewrite in place. Only exact == pins are collected, because a range or a VCS reference states an intent an updater must not overrule. The external check warns rather than fails, like its docker sibling, so a slow PyPI cannot redden a run and a release the repository has deliberately not taken is not a regression.
The cron job is the docker job with one script swapped, so a changed pin arrives as its own PR under update/pip-pins.
Executed: the collector finds both pins in the repository; a synthetic file rewrites from 1.0.0 to 1.1.1 with the indent, quoting and neighbouring line untouched; automtu==1.0.0 would be reported while a bare name and a git+ reference are passed over.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every scheduled workflow opened with an alarm clock, so the Actions list
read as a column of identical icons that said nothing about the run. The
clock now lives in run-name behind github.event_name == 'schedule', where
it marks the one thing it can mark: a run nobody triggered. The name says
what the run does, in two or three words.
The dispatch menu shows names in one narrow column and truncates them, so
a new lint caps a workflow_dispatch name at 25 characters and points the
longer, explanatory wording at run-name, which carries no such limit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The workflow directory mixed entry points, reusable stages and scheduled jobs into one flat list sorted by topic, so the first question about any file - what starts it - was the one the name did not answer. Every workflow now carries one of three prefixes, derived from its on: block: cron- for a schedule trigger, call- for one only reachable through workflow_call, entry- for everything a repository event or a human starts. Scheduled workflows additionally carry an alarm clock as the first character of their name.
Two lints keep it that way. One derives the expected prefix from the triggers and compares it against the file name; it flagged all 17 offenders before the rename and none after. The other resolves every .github/workflows/<file> string in the repository against the directory, which is what a rename silently breaks in Markdown links, Python constants and shell API URLs alike.
Renaming pulled 69 pre-existing comments into the comment lint's changed-file scope. The vendored CodeQL template got the nocheck marker its scorecard sibling already carried, two workflow headers moved above name: where they count as file headers, and the remaining banners were deleted.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The CI deploy bootstrap hand-listed two packages and let the ubuntu-latest runner image supply the rest, so a new host-side import surfaced as a red deploy job instead of a failed install. Size [project.dependencies] to what runs outside the project image and move the deployment stack into a deploy extra, then have every install site read that instead of naming packages itself: the image build and make install-python take the deploy extra, the CI host and the mirror and instructions workflows take the base.
Declare cryptography, which utils/manager/value_generator.py imports directly and which only reached the environment transitively through ansible. Drop numpy, which nothing in the repository imports. Guard the boundary with a lint test that blocks every deploy package and asserts the runner-host CLI still builds its parser.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Move files that share a delimiter-bounded name prefix within a folder (threshold 2) into a subfolder named after the prefix, dropping the prefix from the filename. Covers plugins/filter (docker, generate, has, merge, get/all), env handler subpackages, utils/roles/entity, utils/tests/swarm/write, role SQL/scripts/templates, keycloak/mailu task-includes, shared tasks/utils/load, and CI scripts. Update every import, query_file/script/include_tasks/src/lookup target, compose bind mount, and python -m path to the new locations.
Extend the prefix-grouped-files lint to threshold 2 with dir-level .nocheck and file-head nocheck opt-outs, and skip groups whose shortest member's stem equals the shared prefix. Keep flat only where a loader forbids nesting: WordPress mu-plugins and the docs/requirements namespace. Ansible filter discovery is recursive, so nested filters stay Jinja-resolvable.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The roles updaters (Docker image versions, repository refs) opened a
single combined PR for all role changes, and the jobs checked out the
trigger commit (actions/checkout default), so updates were computed
against a stale tree and could re-propose already-merged changes.
- Check out 'ref: main' in all update jobs so the diff is always
computed against the latest main tip at run time.
- Add open_role_prs.sh: split the working-tree changes per role and
open one isolated branch/PR per changed role, reusing open_pr.sh for
the dedup/push/PR logic (new UPDATE_DEDUPE_PREFIX hook scopes the
duplicate check per role; default preserves prior single-PR behavior
for the skills job).
- Wire the Docker and repository-refs jobs to open_role_prs.sh; skills
stays a single PR.
Branch: update/<type>-<role>-<date>-<fp>; commit/title gain a (<role>)
suffix and a Scope line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The previous tree-hash dedup compared whole-commit trees, so unrelated
drift on main (e.g. a manual edit to any roles/* file) caused the same
auto-update delta to appear "new" each daily cron and a fresh PR to open.
Now: fingerprint only the files that this run actually committed
(git diff --name-only HEAD~1 HEAD), both locally and on each candidate
open PR with the same UPDATE_BRANCH_PREFIX. If any open PR carries the
identical content for those files, skip push + PR creation entirely.
Branch name now carries the fingerprint's short hash so parallel PRs
of the same class can coexist (the close-superseded loop is dropped).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous gh pr close call left the bot's daily update branch
behind on the remote, so stale update/<prefix>-YYYYMMDD refs
accumulated indefinitely. --delete-branch removes the head ref on
close. Scope-safe because the loop only iterates branches matching
${UPDATE_BRANCH_PREFIX}-.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
actions/checkout records two includeif targets per repo: one under
the host path (/home/runner/work/_temp/...) and one under the
container-internal path (/github/runner_temp/...). Only the host path
exists on the runner; the container path is a placeholder for
docker-based actions.
`truncate` (without -c) aborts on the missing container path with
exit 123. Add `-c` (--no-create) so missing paths are skipped
silently and the existing credentials file still gets blanked.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
actions/checkout v6 doesn't put `http.https://github.com/.extraheader`
inside .git/config; it writes that header to an external file at
/home/runner/work/_temp/git-credentials-*.config and pulls it in via
local `includeif.gitdir:<repo>/.git.path = <credfile>` directives.
That's why every earlier `git config --local --unset-all` /
`--remove-section` round was a silent no-op (verified by the previous
debug commit's dump: the extraheader survives because the source-of-
truth lives outside .git/config).
Read every includeif `.path` value from local config, dedupe, and
`truncate -s 0` the targets so the included header chunk becomes
empty. The App installation token then enters the wire via the
URL-embedded `x-access-token:${GH_TOKEN}` credentials with no
competing Authorization header.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous strip targeted only `--local` config and the resulting
CI log showed local was already empty, yet the push still authed
as github-actions[bot]. The auth has to be coming from --global,
--system, --worktree, or GH_TOKEN itself is GITHUB_TOKEN.
Dump every scope's http/credential/url entries before and after the
strip, and probe /installation/repositories with GH_TOKEN so the
HTTP status alone (no secret in logs) tells us whether it's an App
installation token or the workflow GITHUB_TOKEN.
Will replace this with the real fix once the next CI run surfaces
where the rogue Authorization actually originates.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previous attempts targeted a specific config key path
(`http.https://github.com/.extraheader`). If actions/checkout v6
stores it under a different subsection naming, our unset/remove-section
no-ops silently and the GITHUB_TOKEN-bearing header keeps overriding
the App-token URL credentials.
Walk every `http.*.extraheader` entry the local config holds, pipe
the matched key names through xargs to unset-all each one, and dump
the remaining `http.*` config before pushing so the next CI run
proves what was stripped. The push itself still uses an
x-access-token URL so the App's installation token is the only
Authorization that reaches the wire.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`--replace-all` on http.https://github.com/.extraheader still left the
wire with two Authorization headers ("remote: Duplicate header" → 400).
Drop the existing extraheader **and** the whole `http.https://github.com/`
subsection from local config, then push to a URL that embeds
`x-access-token:${GH_TOKEN}` directly so only one Authorization header
(the App's installation token) reaches the server.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Last attempt (unset + URL-embedded token) still pushed as
github-actions[bot] because actions/checkout's
http.https://github.com/.extraheader Authorization survived the
unset (likely silenced by `|| true`) and outranked URL-embedded
credentials on the wire.
Force-replace the header value with the App installation token's
HTTP Basic equivalent (`x-access-token:<token>` base64'd, same shape
checkout itself uses for GITHUB_TOKEN). Drops the URL rewrite, drops
the unset-then-rewrite dance, and guarantees a single Authorization
header that GitHub honours as the App.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`git config --unset-all "http.https://github.com/.extraheader"` is a
git config key path, not a fetchable URL, but
tests/external/repository/test_urls_reachable.py extracts the URL-like
substring and probes it, which returns 404 and fails the suite.
Tag the line with `# nocheck: url` so the URL test skips it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`actions/checkout` configures git's http.extraheader to authenticate
as GITHUB_TOKEN, which update.yml scopes to `contents: read`. The
subsequent `git push --force` in open_pr.sh therefore tripped 403
("Permission to infinito-nexus/core.git denied to github-actions[bot]")
even though the token-minting step had already produced an App
installation token with `contents: write` in GH_TOKEN.
Drop checkout's extraheader and rewrite origin's URL to embed
GH_TOKEN as x-access-token before pushing, so the App's permissions
actually reach the wire.
Also pin two actionlint `-ignore` patterns in scripts/lint/action.sh
to silence false positives for the `client-id` input on
actions/create-github-app-token@v3 (the action's action.yml carries
the input and marks `app-id` deprecated; actionlint 1.7.12's bundled
schema lags behind the upstream release).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
open_pr.sh's PR-dedup loop relied on `gh pr diff <num>` which the
GitHub API refuses with HTTP 406 once a PR exceeds 300 changed files.
Under `set -euo pipefail` that killed the workflow whenever any
unrelated large PR was open against the same base branch.
Replace the diff-hash check with a tree-SHA comparison: fetch each
PR's head commit shallowly, compare `git log -1 --format=%T` against
HEAD's tree. Same-tree means same end-state regardless of base SHA,
whitespace, file order, or patch context. Unfetchable heads (deleted
branches, lost permissions) fall through as non-duplicates instead of
aborting the run.
image_versions.sh / repository_refs.sh also pick up an explicit
`# shellcheck disable=SC1091` above their `. .env` source since the
dotenv file is generated at runtime and never present during
shellcheck's static parse.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`update.yml` jobs (Docker image versions, repository refs) shelled
straight into `python3 -m cli.contributing.update.{docker,repository}`
without sourcing the project dotenv. After eb4bcd23a removed the
`.get(..., default)` fallback in resolve_max_fetch_workers(),
KeyError: 'INFINITO_WORKER_FETCH' surfaced on every run.
Both wrapper scripts now `make dotenv` (idempotent) and `set -a; . .env;
set +a` before exec'ing the updater, restoring the SPOT env chain on
CI runners that don't go through compose.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two new steps wrapped around the existing push/create flow:
1. Dedupe before push: hash the local diff vs origin/<base-branch> and
compare against gh pr diff of every open PR (any author, any branch)
targeting the same base. On a match, exit 0 without pushing and
without creating a PR. This is what stops the same image-version
bump from being opened repeatedly.
2. Supersede after create: close every other open PR whose branch
starts with UPDATE_BRANCH_PREFIX with a "Superseded by #N." comment.
The prefix filter keeps the bot from ever closing repository-refs,
skills, or human-authored PRs, even though dedupe itself sees them.
CRLF normalization on both sides of the hash compare so a runner that
checks out the patch with native-CRLF translation still produces a
stable hash.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Group the shell helpers under scripts/github/ into five subfolders
that match how the workflow files consume them:
runner/ host-runner hygiene (disk, swap, Docker daemon, diagnostics)
cancel/ run cancellation on PR-close and branch-delete events
resolve/ derivation of workflow inputs and outputs from repo state
update/ scheduled Update: Versions pipeline
release/ release-time gates
Drop the now-redundant subfolder prefix from each filename, e.g.
dump_runner_state.sh becomes dump_state.sh and cancel_pull_request_runs.sh
becomes pull_request_runs.sh. Each subfolder carries a README that
describes its scope per documentation.md rules; the top-level README
points at the subfolders without inventorying files.
Propagate the new paths through ten workflow files, three docs
(workflow.md, pipeline.md, secrets.md), and two tests. Fix a real
runtime bug in enlarge_swap.sh, which still referenced the
pre-rename dump_runner_state.sh path. Extend the inline-block-size
lint hint to describe the new subfolder convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Folds the parallel Docker-image and git-ref version-update plumbing
into a single subsystem so the auto-update workflow, the external
freshness tests, and the unit tests all share one module layout.
utils/
update/
base.py # semver primitives shared by both backends
docker.py # OCI registry lookups + version: line rewriter
repository.py # git ls-remote walker + ref: line rewriter
cli/contributing/update/
docker/ # python -m cli.contributing.update.docker
repository/ # python -m cli.contributing.update.repository
scripts/github/update/
image_versions.sh
repository_refs.sh
Tests mirror the same shape under tests/{unit,external}/update/{docker,repository}/.
Also widens the Docker bump scope to all roles (previously gated on
`web-*`) so non-web roles like sys-svc-container get bumped too;
pins Bookwyrm's python image at 3.11-bookworm via
`# nocheck: docker-version` with a TODO until upstream supports
Python >= 3.12.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
