mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 10:59:59 +00:00
refactor(ci): prefix workflows by trigger
The workflow directory mixed entry points, reusable stages and scheduled jobs into one flat list sorted by topic, so the first question about any file - what starts it - was the one the name did not answer. Every workflow now carries one of three prefixes, derived from its on: block: cron- for a schedule trigger, call- for one only reachable through workflow_call, entry- for everything a repository event or a human starts. Scheduled workflows additionally carry an alarm clock as the first character of their name. Two lints keep it that way. One derives the expected prefix from the triggers and compares it against the file name; it flagged all 17 offenders before the rename and none after. The other resolves every .github/workflows/<file> string in the repository against the directory, which is what a rename silently breaks in Markdown links, Python constants and shell API URLs alike. Renaming pulled 69 pre-existing comments into the comment lint's changed-file scope. The vendored CodeQL template got the nocheck marker its scorecard sibling already carried, two workflow headers moved above name: where they count as file headers, and the remaining banners were deleted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
2e3438cc99
commit
71888d86af
70 files changed
+380
-182
No files matched your search
+16
-16
@@ -10,8 +10,8 @@ inputs and descriptions: [workflows.md](../../docs/contributing/tools/github/act
|
||||
```mermaid
|
||||
flowchart TB
|
||||
push["push: main, feature/**, hotfix/**, fix/**"] --> epl["entry-push-latest.yml"]
|
||||
pr["pull_request: opened, synchronize, reopened, ready_for_review"] --> eprc["entry-pull-request-change.yml"]
|
||||
dispatch["workflow_dispatch"] --> eman["entry-manual.yml"]
|
||||
pr["pull_request: opened, synchronize, reopened, ready_for_review"] --> eprc["entry-pr-change-orchestrate.yml"]
|
||||
dispatch["workflow_dispatch"] --> eman["entry-manual-steer.yml"]
|
||||
|
||||
epl --> orch["ci-orchestrator.yml"]
|
||||
epl -->|"version tag on main"| relv["release-version.yml"]
|
||||
@@ -25,7 +25,7 @@ flowchart TB
|
||||
|
||||
lintwf["lint.yml: make lint + hadolint"]
|
||||
testwf["test.yml: make test"]
|
||||
codeql["security-codeql.yml"]
|
||||
codeql["cron-security-codeql.yml"]
|
||||
buildci["build-ci-images: images-build-ci.yml"] --> dns["test-dns.yml"]
|
||||
mirror["images-mirror-missing.yml"]
|
||||
seq["sequencing: serial or parallel, per line"]
|
||||
@@ -166,7 +166,7 @@ first failed mode:
|
||||
|
||||
`skipped` counts as passed, so a mode absent from `modes` never blocks the
|
||||
chain. The parallel layout ignores the switch — with no order there is nothing
|
||||
to stop. The setting is a checkbox on `entry-manual.yml`; the other entry
|
||||
to stop. The setting is a checkbox on `entry-manual-steer.yml`; the other entry
|
||||
points take the default.
|
||||
|
||||
### Job budget
|
||||
@@ -181,8 +181,8 @@ of its single discover job.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
prclose["pull_request_target: closed, converted_to_draft"] --> eprcancel["entry-pull-request-cancel.yml"]
|
||||
branchdelete["delete: branch"] --> delbranch["delete-branch.yml"]
|
||||
prclose["pull_request_target: closed, converted_to_draft"] --> eprcancel["entry-pr-closed-cancel-workflows.yml"]
|
||||
branchdelete["delete: branch"] --> delbranch["entry-delete-branch.yml"]
|
||||
eprcancel -.->|"cancels concurrency group"| runningci["running entry + child workflow runs"]
|
||||
delbranch -.->|"cancels concurrency group"| runningci
|
||||
```
|
||||
@@ -191,15 +191,15 @@ flowchart TB
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
daily["schedule: daily 00:00 UTC"] --> mirrorall["images-mirror-all.yml"]
|
||||
daily --> stale["cleanup-stale.yml"]
|
||||
daily --> relhighest["release-highest.yml"]
|
||||
weekly["schedule: weekly Sat 00:00 UTC"] --> updatewf["update.yml"]
|
||||
weekly --> cleanupci["images-cleanup-ci.yml"]
|
||||
weeklymon["schedule: weekly Mon 00:00 UTC"] --> scorecard["security-scorecard.yml"]
|
||||
daily["schedule: daily 00:00 UTC"] --> mirrorall["cron-images-mirror-all.yml"]
|
||||
daily --> stale["cron-cleanup-stale.yml"]
|
||||
daily --> relhighest["cron-release-highest.yml"]
|
||||
weekly["schedule: weekly Sat 00:00 UTC"] --> updatewf["cron-update.yml"]
|
||||
weekly --> cleanupci["cron-images-cleanup-ci.yml"]
|
||||
weeklymon["schedule: weekly Mon 00:00 UTC"] --> scorecard["cron-security-scorecard.yml"]
|
||||
branchprot["branch_protection_rule"] --> scorecard
|
||||
pushmain["push: main"] --> updatewf
|
||||
prtarget["pull_request_target: opened, reopened"] --> depclose["dependabot-close.yml"]
|
||||
prtarget["pull_request_target: opened, reopened"] --> depclose["entry-pr-open-dependabot-close.yml"]
|
||||
|
||||
relhighest -.->|"gh workflow run"| relver["release-version.yml"]
|
||||
relver --> imgbuildci["images-build-ci.yml"]
|
||||
@@ -207,7 +207,7 @@ flowchart TB
|
||||
manual --> deploywf["test-instructions.yml: run a role README Production command"]
|
||||
```
|
||||
|
||||
Also manually dispatchable: `images-mirror-all.yml`, `images-cleanup-ci.yml`,
|
||||
`cleanup-stale.yml`, `update.yml`, `release-highest.yml`, `release-version.yml`,
|
||||
`lint.yml`, `test.yml`, `test-deploy-swarm.yml`, `test-dns.yml`,
|
||||
Also manually dispatchable: `cron-images-mirror-all.yml`, `cron-images-cleanup-ci.yml`,
|
||||
`cron-cleanup-stale.yml`, `cron-update.yml`, `cron-release-highest.yml`, `call-release-version.yml`,
|
||||
`call-lint.yml`, `call-test.yml`, `call-test-deploy-swarm.yml`, `call-test-dns.yml`,
|
||||
`test-workspace.yml`, `test-runner-smoke.yml`.
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -117,7 +117,7 @@ jobs:
|
||||
shell: bash
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
WORKFLOW_FILE: entry-pull-request-change.yml
|
||||
WORKFLOW_FILE: entry-pr-change-orchestrate.yml
|
||||
TARGET_EVENT: pull_request_target
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
@@ -142,16 +142,16 @@ jobs:
|
||||
run: ./scripts/github/gate/fork_prereqs.sh
|
||||
|
||||
security-codeql:
|
||||
uses: ./.github/workflows/security-codeql.yml
|
||||
uses: ./.github/workflows/cron-security-codeql.yml
|
||||
|
||||
lint:
|
||||
uses: ./.github/workflows/lint.yml
|
||||
uses: ./.github/workflows/call-lint.yml
|
||||
|
||||
test:
|
||||
uses: ./.github/workflows/test.yml
|
||||
uses: ./.github/workflows/call-test.yml
|
||||
|
||||
build-ci-images:
|
||||
uses: ./.github/workflows/images-build-ci.yml
|
||||
uses: ./.github/workflows/call-images-build-ci.yml
|
||||
with:
|
||||
distros: "arch debian ubuntu fedora centos"
|
||||
concurrency_channel: ci-orchestrator-build-consumer
|
||||
@@ -160,12 +160,12 @@ jobs:
|
||||
test-dns:
|
||||
needs:
|
||||
- build-ci-images
|
||||
uses: ./.github/workflows/test-dns.yml
|
||||
uses: ./.github/workflows/call-test-dns.yml
|
||||
with:
|
||||
distros: "arch debian ubuntu fedora centos"
|
||||
|
||||
images-mirror-missing:
|
||||
uses: ./.github/workflows/images-mirror-missing.yml
|
||||
uses: ./.github/workflows/call-images-mirror-missing.yml
|
||||
with:
|
||||
concurrency_channel: ci-orchestrator-mirror-consumer
|
||||
image_wait_attempts: ${{ inputs.image_wait_attempts }}
|
||||
@@ -240,7 +240,7 @@ jobs:
|
||||
- test
|
||||
- build-ci-images
|
||||
- images-mirror-missing
|
||||
uses: ./.github/workflows/test-deploy-swarm.yml
|
||||
uses: ./.github/workflows/call-test-deploy-swarm.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.priority }}
|
||||
@@ -260,7 +260,7 @@ jobs:
|
||||
- lint
|
||||
- test
|
||||
- sequencing
|
||||
uses: ./.github/workflows/test-deploy-single-node.yml
|
||||
uses: ./.github/workflows/call-test-deploy-single-node.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.priority }}
|
||||
@@ -289,7 +289,7 @@ jobs:
|
||||
- test
|
||||
- sequencing
|
||||
- test-deploy-swarm-priority
|
||||
uses: ./.github/workflows/test-deploy-compose.yml
|
||||
uses: ./.github/workflows/call-test-deploy-compose.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.priority }}
|
||||
@@ -317,7 +317,7 @@ jobs:
|
||||
- test
|
||||
- sequencing
|
||||
- test-deploy-compose-priority
|
||||
uses: ./.github/workflows/test-deploy-host.yml
|
||||
uses: ./.github/workflows/call-test-deploy-host.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.priority }}
|
||||
@@ -348,7 +348,7 @@ jobs:
|
||||
- test-deploy-swarm-priority
|
||||
- lint
|
||||
- test
|
||||
uses: ./.github/workflows/test-deploy-swarm.yml
|
||||
uses: ./.github/workflows/call-test-deploy-swarm.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.whitelist }}
|
||||
@@ -380,7 +380,7 @@ jobs:
|
||||
- test-deploy-swarm-priority
|
||||
- lint
|
||||
- test
|
||||
uses: ./.github/workflows/test-deploy-single-node.yml
|
||||
uses: ./.github/workflows/call-test-deploy-single-node.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.whitelist }}
|
||||
@@ -416,7 +416,7 @@ jobs:
|
||||
- test-deploy-swarm
|
||||
- lint
|
||||
- test
|
||||
uses: ./.github/workflows/test-deploy-compose.yml
|
||||
uses: ./.github/workflows/call-test-deploy-compose.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.whitelist }}
|
||||
@@ -451,7 +451,7 @@ jobs:
|
||||
- test-deploy-compose
|
||||
- lint
|
||||
- test
|
||||
uses: ./.github/workflows/test-deploy-host.yml
|
||||
uses: ./.github/workflows/call-test-deploy-host.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.whitelist }}
|
||||
@@ -464,26 +464,26 @@ jobs:
|
||||
needs:
|
||||
- test-deploy-swarm
|
||||
if: ${{ always() && vars.CI_SELF_HOSTED_RUNNER_COUNT != '0' && vars.CI_SELF_HOSTED_RUNNER_COUNT != '' && (needs.test-deploy-swarm.result == 'success' || needs.test-deploy-swarm.result == 'failure') }}
|
||||
uses: ./.github/workflows/test-runner-smoke.yml
|
||||
uses: ./.github/workflows/call-test-runner-smoke.yml
|
||||
with:
|
||||
ref: ${{ github.sha }}
|
||||
secrets: inherit
|
||||
|
||||
test-install-make:
|
||||
uses: ./.github/workflows/test-install-make.yml
|
||||
uses: ./.github/workflows/call-test-install-make.yml
|
||||
test-install-pkgmgr:
|
||||
uses: ./.github/workflows/test-install-pkgmgr.yml
|
||||
uses: ./.github/workflows/call-test-install-pkgmgr.yml
|
||||
|
||||
test-workspace:
|
||||
if: ${{ inputs.workspace == 'true' || (inputs.workspace == 'auto' && (inputs.whitelist == '' || inputs.whitelist == '__ALL__')) }}
|
||||
uses: ./.github/workflows/test-workspace.yml
|
||||
uses: ./.github/workflows/call-test-workspace.yml
|
||||
needs:
|
||||
- images-mirror-missing
|
||||
secrets: inherit
|
||||
|
||||
test-instructions:
|
||||
if: ${{ inputs.instructions == 'true' || (inputs.instructions == 'auto' && (inputs.whitelist == '' || inputs.whitelist == '__ALL__')) }}
|
||||
uses: ./.github/workflows/test-instructions.yml
|
||||
uses: ./.github/workflows/call-test-instructions.yml
|
||||
needs:
|
||||
- images-mirror-missing
|
||||
- build-ci-images
|
||||
@@ -102,7 +102,7 @@ jobs:
|
||||
name: 🏗️ Build missing CI images
|
||||
needs: check-ci-images
|
||||
if: needs.check-ci-images.outputs.missing == 'true'
|
||||
uses: ./.github/workflows/images-build-ci.yml
|
||||
uses: ./.github/workflows/call-images-build-ci.yml
|
||||
with:
|
||||
distros: ${{ needs.check-ci-images.outputs.distros }}
|
||||
checkout_ref: ${{ needs.check-ci-images.outputs.version_tag }}
|
||||
File renamed without changes.
File renamed without changes.
+2
-2
@@ -46,7 +46,7 @@ jobs:
|
||||
compose:
|
||||
name: 📦 Compose
|
||||
if: ${{ contains(inputs.modes, 'compose') }}
|
||||
uses: ./.github/workflows/test-deploy-compose.yml
|
||||
uses: ./.github/workflows/call-test-deploy-compose.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.whitelist }}
|
||||
@@ -58,7 +58,7 @@ jobs:
|
||||
host:
|
||||
name: 💻 Host
|
||||
if: ${{ contains(inputs.modes, 'host') }}
|
||||
uses: ./.github/workflows/test-deploy-host.yml
|
||||
uses: ./.github/workflows/call-test-deploy-host.yml
|
||||
with:
|
||||
distros: ${{ inputs.distros }}
|
||||
whitelist: ${{ inputs.whitelist }}
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -1,4 +1,4 @@
|
||||
name: "🧹 Cleanup: Stale Repository Data"
|
||||
name: "⏰ Cleanup: Stale Repository Data"
|
||||
|
||||
on:
|
||||
schedule:
|
||||
@@ -20,7 +20,6 @@ jobs:
|
||||
- name: 🏷️ Mark stale issues and pull requests
|
||||
uses: actions/stale@v10
|
||||
with:
|
||||
# Issues
|
||||
days-before-issue-stale: 90
|
||||
days-before-issue-close: 7
|
||||
stale-issue-label: "stale"
|
||||
@@ -31,7 +30,6 @@ jobs:
|
||||
close-issue-message: >
|
||||
This issue has been automatically closed due to inactivity.
|
||||
|
||||
# Pull Requests
|
||||
days-before-pr-stale: 180
|
||||
days-before-pr-close: 30
|
||||
stale-pr-label: "stale"
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
name: "🧹 Images: Cleanup CI (GHCR)"
|
||||
name: "⏰ Images: Cleanup CI (GHCR)"
|
||||
|
||||
on:
|
||||
schedule:
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
name: "🪞 Mirror: Docker Hub → GHCR"
|
||||
name: "⏰ Mirror: Docker Hub → GHCR"
|
||||
|
||||
on:
|
||||
schedule:
|
||||
@@ -1,4 +1,4 @@
|
||||
name: "🚀 Release: Highest Missing Version"
|
||||
name: "⏰ Release: Highest Missing Version"
|
||||
|
||||
on:
|
||||
schedule:
|
||||
@@ -1,4 +1,4 @@
|
||||
name: "🔁 Rerun: Cancelled Branch Runs"
|
||||
name: "⏰ Rerun: Cancelled Branch Runs"
|
||||
|
||||
concurrency:
|
||||
group: rerun-cancelled
|
||||
@@ -1,3 +1,5 @@
|
||||
# nocheck: comments-valid vendored CodeQL starter workflow; upstream comments kept verbatim
|
||||
#
|
||||
# For most projects, this workflow file will not need changing; you simply need
|
||||
# to commit it to your repository.
|
||||
#
|
||||
@@ -9,7 +11,7 @@
|
||||
# the `language` matrix defined below to confirm you have the correct set of
|
||||
# supported CodeQL languages.
|
||||
#
|
||||
name: "🔒 Scan: CodeQL (Advanced)"
|
||||
name: "⏰ Scan: CodeQL (Advanced)"
|
||||
|
||||
concurrency:
|
||||
group: codeql-${{ github.repository }}-${{ github.ref }}
|
||||
+1
-1
@@ -3,7 +3,7 @@
|
||||
# policy, and support documentation.
|
||||
# nocheck: comments-valid vendored OSSF scorecard workflow; upstream comments kept verbatim
|
||||
|
||||
name: 🛡️ Scorecard supply-chain security
|
||||
name: "⏰ Scorecard supply-chain security"
|
||||
on:
|
||||
# For Branch-Protection check. Only the default branch is supported. See
|
||||
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection
|
||||
@@ -1,4 +1,4 @@
|
||||
name: "🔄 Update: Versions"
|
||||
name: "⏰ Update: Versions"
|
||||
|
||||
on:
|
||||
push:
|
||||
@@ -1,14 +1,15 @@
|
||||
# Enters the same concurrency group as entry-push-latest.yml for the deleted
|
||||
# branch, which cancels that run and all its child workflow_call runs.
|
||||
#
|
||||
# On `delete` events `github.ref_name` resolves to the default branch, not the
|
||||
# deleted ref, so the group keys on `github.event.ref`. Keying on `ref_name`
|
||||
# would collide with the push-to-main CI and cancel it.
|
||||
|
||||
name: "🚫 Cancel: Runs on Branch Delete"
|
||||
|
||||
on:
|
||||
delete:
|
||||
|
||||
# Enters the same concurrency group as entry-push-latest.yml for the deleted
|
||||
# branch, which cancels that run and all its child workflow_call runs.
|
||||
# NOTE: on `delete` events, `github.ref_name` resolves to the default branch
|
||||
# (the workflow runs from the default branch context), not the deleted ref.
|
||||
# Use `github.event.ref` to get the actually deleted branch name — otherwise
|
||||
# this group would collide with the push-to-main CI and cancel it.
|
||||
concurrency:
|
||||
group: global-ci-${{ github.repository }}-${{ github.event.ref }}
|
||||
cancel-in-progress: true
|
||||
@@ -18,7 +19,6 @@ permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
# Fallback: cancel any remaining active runs via the GitHub API.
|
||||
cancel-branch-runs:
|
||||
if: ${{ github.event.ref_type == 'branch' }}
|
||||
runs-on: ubuntu-latest
|
||||
File renamed without changes.
@@ -96,7 +96,7 @@ jobs:
|
||||
ci-orchestrator:
|
||||
name: 🎛️ Orchestrate CI (manual)
|
||||
needs: pick-distro
|
||||
uses: ./.github/workflows/ci-orchestrator.yml
|
||||
uses: ./.github/workflows/call-orchestrator.yml
|
||||
with:
|
||||
distros: ${{ needs.pick-distro.outputs.distros }}
|
||||
whitelist: ${{ inputs.whitelist }}
|
||||
+3
-3
@@ -139,7 +139,7 @@ jobs:
|
||||
packages: write
|
||||
security-events: write
|
||||
issues: write
|
||||
uses: ./.github/workflows/ci-orchestrator.yml
|
||||
uses: ./.github/workflows/call-orchestrator.yml
|
||||
with:
|
||||
distros: "arch debian ubuntu fedora centos"
|
||||
whitelist: ${{ needs.detect-affected-roles.outputs.whitelist }}
|
||||
@@ -168,7 +168,7 @@ jobs:
|
||||
|
||||
build-ci-images-fork:
|
||||
needs: prepare-fork-merge-ref
|
||||
uses: ./.github/workflows/images-build-ci.yml
|
||||
uses: ./.github/workflows/call-images-build-ci.yml
|
||||
with:
|
||||
distros: "arch debian ubuntu fedora centos"
|
||||
checkout_ref: ${{ needs.prepare-fork-merge-ref.outputs.trusted == 'true' && github.event.pull_request.head.sha || needs.prepare-fork-merge-ref.outputs.checkout_ref }}
|
||||
@@ -179,7 +179,7 @@ jobs:
|
||||
|
||||
images-mirror-missing-fork:
|
||||
needs: prepare-fork-merge-ref
|
||||
uses: ./.github/workflows/images-mirror-missing.yml
|
||||
uses: ./.github/workflows/call-images-mirror-missing.yml
|
||||
with:
|
||||
repo_root: _pr_source
|
||||
source_repository: ${{ github.repository }}
|
||||
File renamed without changes.
+3
-3
@@ -1,5 +1,3 @@
|
||||
name: "🚫 Dependabot: Close while auto-updates disabled"
|
||||
|
||||
# Closes Dependabot PRs while update PRs are gated off via the
|
||||
# CI_ENABLE_AUTO_UPDATES repository variable. Dependabot itself cannot read
|
||||
# repository variables, so the gate is enforced here.
|
||||
@@ -8,6 +6,8 @@ name: "🚫 Dependabot: Close while auto-updates disabled"
|
||||
# access to repository vars/secrets and a write-scoped GITHUB_TOKEN
|
||||
# even for Dependabot's PRs. This workflow does NOT check out PR code.
|
||||
|
||||
name: "🚫 Dependabot: Close while auto-updates disabled"
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
types: [opened, reopened]
|
||||
@@ -25,4 +25,4 @@ jobs:
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_URL: ${{ github.event.pull_request.html_url }}
|
||||
run: gh pr close "$PR_URL" --comment "Auto-closed by \`dependabot-close.yml\`. The repository variable \`CI_ENABLE_AUTO_UPDATES\` is not set to \`true\`, so update PRs are disabled. Set it to \`true\` under **Settings → Secrets and variables → Actions → Variables** to allow Dependabot PRs."
|
||||
run: gh pr close "$PR_URL" --comment "Auto-closed by \`entry-pr-open-dependabot-close.yml\`. The repository variable \`CI_ENABLE_AUTO_UPDATES\` is not set to \`true\`, so update PRs are disabled. Set it to \`true\` under **Settings → Secrets and variables → Actions → Variables** to allow Dependabot PRs."
|
||||
@@ -129,7 +129,7 @@ jobs:
|
||||
- run-policy
|
||||
- pick-distro
|
||||
- resolve-priority
|
||||
uses: ./.github/workflows/ci-orchestrator.yml
|
||||
uses: ./.github/workflows/call-orchestrator.yml
|
||||
with:
|
||||
distros: ${{ needs.pick-distro.outputs.distros }}
|
||||
priority: ${{ needs.resolve-priority.outputs.priority }}
|
||||
@@ -172,7 +172,7 @@ jobs:
|
||||
- pick-distro
|
||||
- release-guard
|
||||
if: needs.run-policy.outputs.should_run == 'true' && needs.release-guard.outputs.allowed == 'true'
|
||||
uses: ./.github/workflows/release-version.yml
|
||||
uses: ./.github/workflows/call-release-version.yml
|
||||
with:
|
||||
tag: ${{ needs.pick-distro.outputs.version_tag }}
|
||||
secrets: inherit
|
||||
@@ -706,7 +706,7 @@ swarm-zombie: install-act
|
||||
SWARM_NAME=$(or $(name),$(app)); \
|
||||
INFINITO_SWARM_STEP_TIMEOUT_MINUTES=$(or $(step_timeout),690); \
|
||||
INFINITO_DISTROS=$(SWARM_DISTROS)" \
|
||||
ACT_WORKFLOW=.github/workflows/test-deploy-swarm.yml \
|
||||
ACT_WORKFLOW=.github/workflows/call-test-deploy-swarm.yml \
|
||||
ACT_JOB=swarm \
|
||||
ACT_MATRIX='apps:$(app);variant:$(or $(variant),0)' \
|
||||
ACT_INPUTS="whitelist=$(app) distros=$(SWARM_DISTROS)" \
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Trigger the manual CI run (entry-manual.yml) for the current branch.
|
||||
"""Trigger the manual CI run (entry-manual-steer.yml) for the current branch.
|
||||
|
||||
entry-manual.yml reads the "__ALL__" whitelist sentinel as "force a full
|
||||
entry-manual-steer.yml reads the "__ALL__" whitelist sentinel as "force a full
|
||||
deploy across all roles".
|
||||
"""
|
||||
|
||||
@@ -11,7 +11,7 @@ import sys
|
||||
|
||||
from cli.administration.deploy.ci import runs
|
||||
|
||||
_WORKFLOW = "entry-manual.yml"
|
||||
_WORKFLOW = "entry-manual-steer.yml"
|
||||
_ALL = "__ALL__"
|
||||
|
||||
|
||||
|
||||
@@ -59,7 +59,7 @@ _DEPLOY_CALLERS = frozenset(
|
||||
}
|
||||
)
|
||||
|
||||
_ORCHESTRATOR = ".github/workflows/ci-orchestrator.yml"
|
||||
_ORCHESTRATOR = ".github/workflows/call-orchestrator.yml"
|
||||
_DYNAMIC_MATRIX_ESTIMATE = 5
|
||||
_DEFAULT_JOB_LIMIT = 256
|
||||
_MIN_MODE_SLOTS = 3
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Activated by the ``🧩 Subset`` label: the entry-pull-request workflow only
|
||||
runs this module when the label is present. It gives fork contributors the
|
||||
selective-CI capability that ``entry-manual.yml`` (``workflow_dispatch``)
|
||||
selective-CI capability that ``entry-manual-steer.yml`` (``workflow_dispatch``)
|
||||
grants maintainers, which forks cannot trigger.
|
||||
|
||||
It reads the PR body from ``$PR_BODY``, extracts the first fenced code block
|
||||
|
||||
@@ -10,7 +10,7 @@ Every push runs the same gate you can run locally:
|
||||
make test # lint + unit + integration + external
|
||||
```
|
||||
|
||||
On GitHub Actions this is followed by the **deploy-test matrix** — real app deploys across Debian/Ubuntu/Arch/Fedora in throwaway containers, with Playwright/CLI end-to-end checks (`.github/workflows/test-deploy-*.yml`). This is the heavy part and the reason self-hosted runners exist.
|
||||
On GitHub Actions this is followed by the **deploy-test matrix** — real app deploys across Debian/Ubuntu/Arch/Fedora in throwaway containers, with Playwright/CLI end-to-end checks (`.github/workflows/call-test-deploy-*.yml`). This is the heavy part and the reason self-hosted runners exist.
|
||||
|
||||
## Self-hosted runners (opt-in overflow)
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ For the swarm deploy loop, see [Swarm Loop](swarm.md).
|
||||
- If the workflow uses a distro matrix, you MUST iterate on one distro at a time instead of rerunning the whole matrix during the default debug loop.
|
||||
- Debian SHOULD be the preferred distro for that focused workflow iteration unless the failure is clearly distro-specific or the user asked for a different distro.
|
||||
- When you constrain an Act matrix run through `ACT_MATRIX`, you MUST use Act's `key:value` syntax instead of `key=value`. Otherwise Act may ignore the filter and rerun the whole matrix.
|
||||
- For `.github/workflows/test-workspace.yml`, the preferred focused Debian example is `make act-workflow ACT_WORKFLOW=.github/workflows/test-workspace.yml ACT_JOB=test-workspace ACT_MATRIX="dev_runtime_image:$(python3 -c 'from utils.distros import dev_runtime_image; print(dev_runtime_image("debian"))')"`.
|
||||
- For `.github/workflows/call-test-workspace.yml`, the preferred focused Debian example is `make act-workflow ACT_WORKFLOW=.github/workflows/call-test-workspace.yml ACT_JOB=test-workspace ACT_MATRIX="dev_runtime_image:$(python3 -c 'from utils.distros import dev_runtime_image; print(dev_runtime_image("debian"))')"`.
|
||||
- You SHOULD avoid jumping straight to repeated remote CI reruns when `make act-workflow` can validate the workflow locally and the user agreed to use it.
|
||||
|
||||
## Recovery & gotchas
|
||||
|
||||
@@ -66,13 +66,13 @@ Compute scope from `git diff --name-only origin/main...<branch>`.
|
||||
- `distros`: a change counts as **distro-specific** only if the diff is confined to either
|
||||
- files under `packaging/<distro>/` for one or more specific distros, or
|
||||
- role tasks gated by a distro conditional (e.g. `when: ansible_os_family == '...'`, `when: ansible_distribution == '...'`), touching only the branches for one or more specific distros.
|
||||
Pass those distros as a space-separated list via `-f distros="<distros>"`. In every other case, omit `-f distros`; `entry-manual.yml` then applies its own default (currently `debian` only, it does NOT run all distros).
|
||||
Pass those distros as a space-separated list via `-f distros="<distros>"`. In every other case, omit `-f distros`; `entry-manual-steer.yml` then applies its own default (currently `debian` only, it does NOT run all distros).
|
||||
|
||||
Before triggering, capture the currently latest run id for this workflow+branch as `<prev-run-id>` (empty string if none exists). This is required so the post-trigger resolve can distinguish the new run from any stale completed run on the same branch:
|
||||
|
||||
```
|
||||
gh run list \
|
||||
--workflow=entry-manual.yml \
|
||||
--workflow=entry-manual-steer.yml \
|
||||
--branch=<branch> \
|
||||
--event=workflow_dispatch \
|
||||
--limit=1 \
|
||||
@@ -83,7 +83,7 @@ gh run list \
|
||||
Then trigger:
|
||||
|
||||
```
|
||||
gh workflow run entry-manual.yml --ref <branch> \
|
||||
gh workflow run entry-manual-steer.yml --ref <branch> \
|
||||
[-f distros="<distros>"] \
|
||||
[-f whitelist="<role>"]
|
||||
```
|
||||
|
||||
@@ -33,7 +33,7 @@ The per-job logs carry the real failing task and its error, and the artefacts ca
|
||||
|
||||
You SHOULD use targeted manual CI jobs instead of rerunning the full pipeline if you only need one focused check.
|
||||
|
||||
Prefer the manual workflow in [entry-manual.yml](../../../../.github/workflows/entry-manual.yml):
|
||||
Prefer the manual workflow in [entry-manual-steer.yml](../../../../.github/workflows/entry-manual-steer.yml):
|
||||
|
||||
- Select your branch.
|
||||
- Use `debian` unless you have a clear reason to use a different distro.
|
||||
|
||||
@@ -51,8 +51,8 @@ A `v*` tag push fans out into two CI workflows:
|
||||
|
||||
| Workflow | Purpose |
|
||||
|---|---|
|
||||
| [release-version.yml](../../../.github/workflows/release-version.yml) | Backfills CI images for the version tag if missing, retags them to release tags, and publishes the version-specific image set. |
|
||||
| [release-highest.yml](../../../.github/workflows/release-highest.yml) | Decides whether the new version is the highest published version and, if so, advances the `latest` floating tag. |
|
||||
| [call-release-version.yml](../../../.github/workflows/call-release-version.yml) | Backfills CI images for the version tag if missing, retags them to release tags, and publishes the version-specific image set. |
|
||||
| [cron-release-highest.yml](../../../.github/workflows/cron-release-highest.yml) | Decides whether the new version is the highest published version and, if so, advances the `latest` floating tag. |
|
||||
|
||||
Both run from the trusted workflow ref and check out the version tag for the actual build payload, so the release artefacts match the tagged commit byte-for-byte.
|
||||
|
||||
|
||||
@@ -66,7 +66,7 @@ make test-external INFINITO_TEST_PATTERN=test_image_versions.py
|
||||
|
||||
## CI Workflow 🤖
|
||||
|
||||
External tests run inside `.github/workflows/test.yml` via `make test`.
|
||||
External tests run inside `.github/workflows/call-test.yml` via `make test`.
|
||||
It MAY be triggered manually and is intentionally separate from the default
|
||||
`make test` flow. It is also wired into the orchestrated code-testing gate in
|
||||
CI so the live check can run there without changing local default validation.
|
||||
@@ -6,14 +6,14 @@ The branch prefix MUST match the type of change. The [PR workflow](pull-request.
|
||||
|
||||
| Branch prefix | Scope | Matching files | CI behavior |
|
||||
|---|---|---|---|
|
||||
| `agent` | Changes to agent instructions or prompts | `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, `docs/agents/*` | Skips [ci-orchestrator](../../../../.github/workflows/ci-orchestrator.yml), finishes green through the lightweight scope gate. |
|
||||
| `documentation` | Documentation-only changes | `**/*.md`, `**/*.rst` (outside agent paths) | Skips [ci-orchestrator](../../../../.github/workflows/ci-orchestrator.yml), finishes green through the lightweight scope gate. |
|
||||
| `feature` | New features or enhancements | `*` | Runs the full [ci-orchestrator](../../../../.github/workflows/ci-orchestrator.yml) pipeline. |
|
||||
| `fix` | Bug fixes | `*` | Runs the full [ci-orchestrator](../../../../.github/workflows/ci-orchestrator.yml) pipeline. |
|
||||
| `update` | Automated or maintenance updates | `*` | Runs the full [ci-orchestrator](../../../../.github/workflows/ci-orchestrator.yml) pipeline. |
|
||||
| `chore` | Maintenance, dependency, and tooling updates (incl. branches created by [update.yml](../../../../.github/workflows/update.yml)) | `*` | Runs the full [ci-orchestrator](../../../../.github/workflows/ci-orchestrator.yml) pipeline. |
|
||||
| `alert-autofix` | Automated alert-triggered fixes | `*` | Runs the full [ci-orchestrator](../../../../.github/workflows/ci-orchestrator.yml) pipeline. |
|
||||
| `dependabot` | Automated dependency updates | `*` | Runs the full [ci-orchestrator](../../../../.github/workflows/ci-orchestrator.yml) pipeline. |
|
||||
| `agent` | Changes to agent instructions or prompts | `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, `docs/agents/*` | Skips [orchestrator](../../../../.github/workflows/call-orchestrator.yml), finishes green through the lightweight scope gate. |
|
||||
| `documentation` | Documentation-only changes | `**/*.md`, `**/*.rst` (outside agent paths) | Skips [orchestrator](../../../../.github/workflows/call-orchestrator.yml), finishes green through the lightweight scope gate. |
|
||||
| `feature` | New features or enhancements | `*` | Runs the full [orchestrator](../../../../.github/workflows/call-orchestrator.yml) pipeline. |
|
||||
| `fix` | Bug fixes | `*` | Runs the full [orchestrator](../../../../.github/workflows/call-orchestrator.yml) pipeline. |
|
||||
| `update` | Automated or maintenance updates | `*` | Runs the full [orchestrator](../../../../.github/workflows/call-orchestrator.yml) pipeline. |
|
||||
| `chore` | Maintenance, dependency, and tooling updates (incl. branches created by [cron-update.yml](../../../../.github/workflows/cron-update.yml)) | `*` | Runs the full [orchestrator](../../../../.github/workflows/call-orchestrator.yml) pipeline. |
|
||||
| `alert-autofix` | Automated alert-triggered fixes | `*` | Runs the full [orchestrator](../../../../.github/workflows/call-orchestrator.yml) pipeline. |
|
||||
| `dependabot` | Automated dependency updates | `*` | Runs the full [orchestrator](../../../../.github/workflows/call-orchestrator.yml) pipeline. |
|
||||
|
||||
## Enforcement 🔒
|
||||
|
||||
|
||||
@@ -4,15 +4,15 @@ This document describes the CI pipeline structure, entry points, and gates used
|
||||
|
||||
## Overview 🗺️
|
||||
|
||||
CI is triggered automatically on pull-request events and on pushes to the branches listed in the entry-workflow row in [workflows.md](../../tools/github/actions/workflows.md). The pipeline is composed of reusable workflow files under [.github/workflows/](../../../../.github/workflows/). The central coordinator is [ci-orchestrator.yml](../../../../.github/workflows/ci-orchestrator.yml), which is called by the entry workflows.
|
||||
CI is triggered automatically on pull-request events and on pushes to the branches listed in the entry-workflow row in [workflows.md](../../tools/github/actions/workflows.md). The pipeline is composed of reusable workflow files under [.github/workflows/](../../../../.github/workflows/). The central coordinator is [call-orchestrator.yml](../../../../.github/workflows/call-orchestrator.yml), which is called by the entry workflows.
|
||||
|
||||
## Entry Points 🚪
|
||||
|
||||
Every external CI trigger MUST route through one of the entry workflows. Each entry translates its event into a call to the orchestrator; the catalog of triggers and inputs lives in [workflows.md](../../tools/github/actions/workflows.md).
|
||||
|
||||
- [entry-pull-request-change.yml](../../../../.github/workflows/entry-pull-request-change.yml): detects PR scope and conditionally calls the orchestrator.
|
||||
- [entry-pr-change-orchestrate.yml](../../../../.github/workflows/entry-pr-change-orchestrate.yml): detects PR scope and conditionally calls the orchestrator.
|
||||
- [entry-push-latest.yml](../../../../.github/workflows/entry-push-latest.yml): calls the orchestrator for pushes on the supported branch prefixes and additionally invokes the release workflow on version tags. For the contributor-facing release procedure see [release.md](../../actions/release.md).
|
||||
- [entry-manual.yml](../../../../.github/workflows/entry-manual.yml): dispatches the orchestrator manually for a chosen distro set and whitelist.
|
||||
- [entry-manual-steer.yml](../../../../.github/workflows/entry-manual-steer.yml): dispatches the orchestrator manually for a chosen distro set and whitelist.
|
||||
|
||||
The push entry synchronizes the current repository's `main` branch from the configured source repository before CI policy and deploy discovery run.
|
||||
The default source is `infinito-nexus/core`, and `CI_SYNC_MAIN_SOURCE_REPOSITORY` MAY override or disable the sync as documented in [configuration.md](../../tools/github/actions/configuration.md).
|
||||
@@ -23,7 +23,7 @@ This sync **force-overwrites** the local `main`, so on a synced repository you M
|
||||
Before the full pipeline runs, CI detects the scope of the PR based on the changed files and the branch prefix (see [branch.md](branch.md)).
|
||||
|
||||
- Documentation-only and agent-only PRs MUST pass a lightweight gate without running the full orchestrator.
|
||||
- All other scopes MUST run the full [ci-orchestrator.yml](../../../../.github/workflows/ci-orchestrator.yml).
|
||||
- All other scopes MUST run the full [call-orchestrator.yml](../../../../.github/workflows/call-orchestrator.yml).
|
||||
- The branch prefix MUST match the detected scope, otherwise the `validate-pr-branch-prefix` job fails.
|
||||
|
||||
## Pipeline Stages 🏗️
|
||||
@@ -40,15 +40,15 @@ The `security-codeql` job runs the CodeQL scan (catalogued in the `Security and
|
||||
|
||||
### 3. Linting 🧹
|
||||
|
||||
The `lint` job runs [lint.yml](../../../../.github/workflows/lint.yml): `make lint` fans every lint check out in parallel, plus a hadolint SARIF job. It MUST pass before the pipeline continues.
|
||||
The `lint` job runs [call-lint.yml](../../../../.github/workflows/call-lint.yml): `make lint` fans every lint check out in parallel, plus a hadolint SARIF job. It MUST pass before the pipeline continues.
|
||||
|
||||
### 4. CI Image Build 🐳
|
||||
|
||||
The `build-ci-images` stage uses [images-build-ci.yml](../../../../.github/workflows/images-build-ci.yml) to build Docker images for the target distributions (`arch`, `debian`, `ubuntu`, `fedora`, `centos`). These images are consumed by all subsequent test jobs.
|
||||
The `build-ci-images` stage uses [call-images-build-ci.yml](../../../../.github/workflows/call-images-build-ci.yml) to build Docker images for the target distributions (`arch`, `debian`, `ubuntu`, `fedora`, `centos`). These images are consumed by all subsequent test jobs.
|
||||
|
||||
### 5. Code Tests 🧪
|
||||
|
||||
The `test` job runs [test.yml](../../../../.github/workflows/test.yml): `make test` fans the unit, integration, lint and external suites out in parallel on the runner host. It MUST pass `code-quality-gate`.
|
||||
The `test` job runs [call-test.yml](../../../../.github/workflows/call-test.yml): `make test` fans the unit, integration, lint and external suites out in parallel on the runner host. It MUST pass `code-quality-gate`.
|
||||
|
||||
### 6. Code Quality Gate 🚦
|
||||
|
||||
@@ -56,11 +56,11 @@ The `code-quality-gate` requires linting, code tests, and security checks to all
|
||||
|
||||
### 7. DNS Tests 🌐
|
||||
|
||||
[test-dns.yml](../../../../.github/workflows/test-dns.yml) validates DNS configuration across all target distributions.
|
||||
[call-test-dns.yml](../../../../.github/workflows/call-test-dns.yml) validates DNS configuration across all target distributions.
|
||||
|
||||
### 8. Image Mirroring 🪞
|
||||
|
||||
[images-mirror-missing.yml](../../../../.github/workflows/images-mirror-missing.yml) mirrors any missing upstream images in parallel with the DNS tests so later deploy jobs pull from GHCR instead of Docker Hub, MCR, or other external registries. This shields CI from upstream rate limits, geo-blocking, and transient registry outages.
|
||||
[call-images-mirror-missing.yml](../../../../.github/workflows/call-images-mirror-missing.yml) mirrors any missing upstream images in parallel with the DNS tests so later deploy jobs pull from GHCR instead of Docker Hub, MCR, or other external registries. This shields CI from upstream rate limits, geo-blocking, and transient registry outages.
|
||||
|
||||
Fork PRs cannot publish mirror packages directly, so their runs wait for the `pull_request_target` mirror producer before continuing. The producer mirrors with the Docker Hub credentials only for trusted PRs ([🛡️ Trusted](#trusted-fork-prs-)); untrusted forks mirror anonymously (no organization secrets). See [mirror.md](../image/mirror.md) for the mirroring architecture and naming convention.
|
||||
|
||||
@@ -72,10 +72,10 @@ The two deploy-test workflows listed in the `Infrastructure tests` table of [wor
|
||||
|
||||
#### Diff-driven app selection 🎯
|
||||
|
||||
Both [test-deploy-compose.yml](../../../../.github/workflows/test-deploy-compose.yml) and [test-deploy-swarm.yml](../../../../.github/workflows/test-deploy-swarm.yml) narrow their app matrix to the set of roles actually impacted by the branch's diff against `origin/main`. The `discover` job resolves an effective whitelist before [output_apps.sh](../../../../scripts/github/resolve/output_apps.sh) runs, using the following precedence:
|
||||
Both [call-test-deploy-compose.yml](../../../../.github/workflows/call-test-deploy-compose.yml) and [call-test-deploy-swarm.yml](../../../../.github/workflows/call-test-deploy-swarm.yml) narrow their app matrix to the set of roles actually impacted by the branch's diff against `origin/main`. The `discover` job resolves an effective whitelist before [output_apps.sh](../../../../scripts/github/resolve/output_apps.sh) runs, using the following precedence:
|
||||
|
||||
1. **Sentinel `__ALL__` in the `whitelist` input** (case-insensitive). The diff logic MUST be skipped and an empty whitelist MUST be emitted, which deploys everything in the workflow's scope. This is the explicit opt-out from diff narrowing for manual dispatch.
|
||||
2. **Any other non-empty `whitelist`** (forwarded from `entry-manual.yml` and similar). The explicit value MUST win over the diff and is passed through verbatim.
|
||||
2. **Any other non-empty `whitelist`** (forwarded from `entry-manual-steer.yml` and similar). The explicit value MUST win over the diff and is passed through verbatim.
|
||||
3. **Diff vs `origin/main`**, applied only when the `whitelist` input is empty:
|
||||
- **No diff at all** OR **any changed path outside `roles/<role>/...`**: no whitelist is set, so the full deploy across the workflow's scope runs.
|
||||
- **All changed paths under `roles/<role>/...`**: the changed roles become the seed set, and the whitelist is the transitive closure of those seeds expanded upwards over `run_after`, `dependencies`, and shared services. In other words, every role whose prerequisite set as defined by [combined resolver](../../../../cli/meta/roles/applications/resolution/combined/resolver.py) contains one of the seeds is included, together with the seeds themselves.
|
||||
@@ -101,7 +101,7 @@ roles:
|
||||
```
|
||||
````
|
||||
|
||||
With the label set, the `detect-affected-roles` job in [entry-pull-request-change.yml](../../../../.github/workflows/entry-pull-request-change.yml) calls [pr_affected_roles.sh](../../../../scripts/github/resolve/pr_affected_roles.sh), which runs the [cli.meta.ci.subset_roles](../../../../cli/meta/ci/subset_roles.py) module. The module reads the first fenced block whose YAML carries a `roles:` key (the heading text is not matched) and emits that list as the `whitelist` output, which flows into the deploy tests as precedence rung 2 above. Without the label, the diff-driven precedence applies.
|
||||
With the label set, the `detect-affected-roles` job in [entry-pr-change-orchestrate.yml](../../../../.github/workflows/entry-pr-change-orchestrate.yml) calls [pr_affected_roles.sh](../../../../scripts/github/resolve/pr_affected_roles.sh), which runs the [cli.meta.ci.subset_roles](../../../../cli/meta/ci/subset_roles.py) module. The module reads the first fenced block whose YAML carries a `roles:` key (the heading text is not matched) and emits that list as the `whitelist` output, which flows into the deploy tests as precedence rung 2 above. Without the label, the diff-driven precedence applies.
|
||||
|
||||
The subset path fails the run when:
|
||||
|
||||
@@ -128,7 +128,7 @@ The final `done` job aggregates all deploy, install, and development gates. CI i
|
||||
|
||||
- PR pipelines use `cancel-in-progress: true` so only the newest run per PR and event type is active.
|
||||
- The orchestrator and the push entry workflow (`entry-push-latest.yml`) default to `cancel-in-progress: true` and both respect the repository variable `CI_CANCEL_IN_PROGRESS`.
|
||||
- The manual entry workflow (`entry-manual.yml`) sets `cancel-in-progress: true` on its own group and passes `force_cancel_in_progress: true` into the orchestrator, so a manual run cancels on both levels regardless of `CI_CANCEL_IN_PROGRESS`.
|
||||
- The manual entry workflow (`entry-manual-steer.yml`) sets `cancel-in-progress: true` on its own group and passes `force_cancel_in_progress: true` into the orchestrator, so a manual run cancels on both levels regardless of `CI_CANCEL_IN_PROGRESS`.
|
||||
|
||||
See [configuration.md](../../tools/github/actions/configuration.md) for all repository variables that control CI behaviour.
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ Before marking a PR as **ready for review**, CI in your fork MUST be green. The
|
||||
|
||||
| Change type | Isolated? | Required CI |
|
||||
|---|---|---|
|
||||
| Single role | ✅ No dependencies | MAY use [manual CI](../../../../.github/workflows/entry-manual.yml) for that role only |
|
||||
| Single role | ✅ No dependencies | MAY use [manual CI](../../../../.github/workflows/entry-manual-steer.yml) for that role only |
|
||||
| Single role | ❌ Has dependencies | MUST run full pipeline |
|
||||
| Multiple roles or shared code | N/A | MUST run full pipeline |
|
||||
| CI/CD, tooling, systemic | N/A | MUST run full pipeline |
|
||||
|
||||
@@ -68,7 +68,7 @@ The mirror workflow runs as stage 8 of the CI pipeline. See [ci.md](../git/pipel
|
||||
### CI Flow 📋
|
||||
|
||||
1. Image discovery scans role declarations in `meta/services.yml`.
|
||||
2. [images-mirror-missing.yml](../../../../.github/workflows/images-mirror-missing.yml) copies only missing upstream refs into GHCR via `cli.contributing.mirror.sync --only-missing`. Optional Docker Hub credentials reduce source-side rate limits during that sync.
|
||||
2. [call-images-mirror-missing.yml](../../../../.github/workflows/call-images-mirror-missing.yml) copies only missing upstream refs into GHCR via `cli.contributing.mirror.sync --only-missing`. Optional Docker Hub credentials reduce source-side rate limits during that sync.
|
||||
3. Fork PRs cannot publish packages themselves, so their untrusted `pull_request` runs wait in `scripts/meta/wait/mirrors.sh` until a trusted producer run has published the required refs.
|
||||
4. Inventory generation writes the resulting mirror refs from `mirrors.yml` back into host vars.
|
||||
5. Deploy and test jobs resolve images from `ghcr.io/{namespace}/{repository}/mirror/...` instead of pulling directly from Docker Hub, MCR, or other upstream registries.
|
||||
|
||||
@@ -8,14 +8,14 @@ Repository variables are set under **Settings → Secrets and variables → Acti
|
||||
|
||||
| Variable | Workflow | Default (unset) | Set to activate |
|
||||
|---|---|---|---|
|
||||
| `CI_CANCEL_IN_PROGRESS` | [entry-push-latest.yml](../../../../../.github/workflows/entry-push-latest.yml), [ci-orchestrator.yml](../../../../../.github/workflows/ci-orchestrator.yml) | Cancels in-progress runs on new push | `false` to keep in-progress runs alive |
|
||||
| `CI_CANCEL_IN_PROGRESS` | [entry-push-latest.yml](../../../../../.github/workflows/entry-push-latest.yml), [call-orchestrator.yml](../../../../../.github/workflows/call-orchestrator.yml) | Cancels in-progress runs on new push | `false` to keep in-progress runs alive |
|
||||
| `CI_SYNC_MAIN_SOURCE_REPOSITORY` | [entry-push-latest.yml](../../../../../.github/workflows/entry-push-latest.yml) | Syncs `main` from `infinito-nexus/core` before CI scope discovery | `<owner>/<repo>` to use another source, or `false`, empty, or the current repository to skip |
|
||||
| `CI_RUN_ON_MAIN` | [entry-push-latest.yml](../../../../../.github/workflows/entry-push-latest.yml) | Pushes to `main` skip CI | `true` to run CI on `main` pushes too |
|
||||
| `CI_ENABLE_AUTO_UPDATES` | [update.yml](../../../../../.github/workflows/update.yml), [dependabot-close.yml](../../../../../.github/workflows/dependabot-close.yml) | Update jobs skipped; Dependabot PRs auto-closed | `true` to allow update PRs (workflow-driven and Dependabot) |
|
||||
| `INFINITO_PLAYWRIGHT_KEEP` | [test-deploy-compose.yml](../../../../../.github/workflows/test-deploy-compose.yml) | Playwright keeps trace, screenshot and video only when a test fails | `true` to keep them for every test (passing runs included) |
|
||||
| `CI_RERUN_INTERVAL_MINUTES` | [rerun-cancelled.yml](../../../../../.github/workflows/rerun-cancelled.yml) | Sweeps for cancelled runs every full hour | Minutes between two sweeps, rounded up to whole hours |
|
||||
| `CI_RERUN_MAX_ATTEMPTS` | [rerun-cancelled.yml](../../../../../.github/workflows/rerun-cancelled.yml) | Revives a run however often it is cancelled | Attempt count at which a run is left alone; `0` for no ceiling |
|
||||
| `CI_RERUN_MAX_AGE_HOURS` | [rerun-cancelled.yml](../../../../../.github/workflows/rerun-cancelled.yml) | Ignores runs untouched for more than 24 hours | Age in hours beyond which a cancelled run is left alone |
|
||||
| `CI_ENABLE_AUTO_UPDATES` | [cron-update.yml](../../../../../.github/workflows/cron-update.yml), [entry-pr-open-dependabot-close.yml](../../../../../.github/workflows/entry-pr-open-dependabot-close.yml) | Update jobs skipped; Dependabot PRs auto-closed | `true` to allow update PRs (workflow-driven and Dependabot) |
|
||||
| `INFINITO_PLAYWRIGHT_KEEP` | [call-test-deploy-compose.yml](../../../../../.github/workflows/call-test-deploy-compose.yml) | Playwright keeps trace, screenshot and video only when a test fails | `true` to keep them for every test (passing runs included) |
|
||||
| `CI_RERUN_INTERVAL_MINUTES` | [rerun-cancelled.yml](../../../../../.github/workflows/cron-rerun-cancelled.yml) | Sweeps for cancelled runs every full hour | Minutes between two sweeps, rounded up to whole hours |
|
||||
| `CI_RERUN_MAX_ATTEMPTS` | [rerun-cancelled.yml](../../../../../.github/workflows/cron-rerun-cancelled.yml) | Revives a run however often it is cancelled | Attempt count at which a run is left alone; `0` for no ceiling |
|
||||
| `CI_RERUN_MAX_AGE_HOURS` | [rerun-cancelled.yml](../../../../../.github/workflows/cron-rerun-cancelled.yml) | Ignores runs untouched for more than 24 hours | Age in hours beyond which a cancelled run is left alone |
|
||||
|
||||
## `CI_CANCEL_IN_PROGRESS` 🛑
|
||||
|
||||
@@ -116,7 +116,7 @@ The gate is applied inside [push_ci_policy.sh](../../../../../scripts/meta/resol
|
||||
|
||||
## `CI_ENABLE_AUTO_UPDATES` 🔄
|
||||
|
||||
Controls whether automated update PRs are created. Covers both the workflow-driven jobs in [update.yml](../../../../../.github/workflows/update.yml) (Docker image versions, agent skills) and PRs opened by Dependabot (gated via [dependabot-close.yml](../../../../../.github/workflows/dependabot-close.yml), which auto-closes them).
|
||||
Controls whether automated update PRs are created. Covers both the workflow-driven jobs in [cron-update.yml](../../../../../.github/workflows/cron-update.yml) (Docker image versions, agent skills) and PRs opened by Dependabot (gated via [entry-pr-open-dependabot-close.yml](../../../../../.github/workflows/entry-pr-open-dependabot-close.yml), which auto-closes them).
|
||||
|
||||
The workflow-driven jobs additionally require the `BOT_APP_CLIENT_ID` and `BOT_APP_PRIVATE_KEY` repository secrets, see [secrets.md](secrets.md). Without those secrets, runs fail at the token-minting step and no PR is opened.
|
||||
|
||||
@@ -138,13 +138,13 @@ Delete the variable or change its value to anything other than `true`.
|
||||
|
||||
**How it works:**
|
||||
|
||||
In [update.yml](../../../../../.github/workflows/update.yml) each job carries a job-level guard:
|
||||
In [cron-update.yml](../../../../../.github/workflows/cron-update.yml) each job carries a job-level guard:
|
||||
|
||||
```yaml
|
||||
if: vars.CI_ENABLE_AUTO_UPDATES == 'true'
|
||||
```
|
||||
|
||||
Dependabot cannot read repository variables itself, so [dependabot-close.yml](../../../../../.github/workflows/dependabot-close.yml) listens on `pull_request_target` (`opened`, `reopened`) and closes any PR authored by `dependabot[bot]` while `CI_ENABLE_AUTO_UPDATES != 'true'`. The workflow does not check out PR code, which keeps the elevated `pull_request_target` context safe.
|
||||
Dependabot cannot read repository variables itself, so [entry-pr-open-dependabot-close.yml](../../../../../.github/workflows/entry-pr-open-dependabot-close.yml) listens on `pull_request_target` (`opened`, `reopened`) and closes any PR authored by `dependabot[bot]` while `CI_ENABLE_AUTO_UPDATES != 'true'`. The workflow does not check out PR code, which keeps the elevated `pull_request_target` context safe.
|
||||
|
||||
| Variable value | Workflow update jobs | Dependabot PRs |
|
||||
|---|---|---|
|
||||
@@ -154,7 +154,7 @@ Dependabot cannot read repository variables itself, so [dependabot-close.yml](..
|
||||
|
||||
## `INFINITO_PLAYWRIGHT_KEEP` 🎬
|
||||
|
||||
Controls whether Playwright keeps trace, screenshot, and video for every test or only for failing tests in the compose deploy-test workflow ([test-deploy-compose.yml](../../../../../.github/workflows/test-deploy-compose.yml)).
|
||||
Controls whether Playwright keeps trace, screenshot, and video for every test or only for failing tests in the compose deploy-test workflow ([call-test-deploy-compose.yml](../../../../../.github/workflows/call-test-deploy-compose.yml)).
|
||||
For the full propagation chain, the inventory override, and the local equivalents, see [Playwright Tests](../../../actions/testing/playwright.md#artefact-retention-).
|
||||
|
||||
**Default behaviour (variable not set or set to any value other than `true`):**
|
||||
@@ -189,7 +189,7 @@ INFINITO_PLAYWRIGHT_KEEP: ${{ vars.INFINITO_PLAYWRIGHT_KEEP }}
|
||||
|
||||
## `CI_RERUN_*` 🔁
|
||||
|
||||
Three variables tune [rerun-cancelled.yml](../../../../../.github/workflows/rerun-cancelled.yml), the sweep that restarts the failed jobs of a branch whose latest run was cancelled.
|
||||
Three variables tune [rerun-cancelled.yml](../../../../../.github/workflows/cron-rerun-cancelled.yml), the sweep that restarts the failed jobs of a branch whose latest run was cancelled.
|
||||
|
||||
| Variable | Unset | Effect when set |
|
||||
|---|---|---|
|
||||
|
||||
@@ -8,14 +8,14 @@ Repository secrets MUST be set under **Settings → Secrets and variables → Ac
|
||||
|
||||
| Secret | Workflow | Purpose | Required when |
|
||||
|---|---|---|---|
|
||||
| `BOT_APP_CLIENT_ID` | [update.yml](../../../../../.github/workflows/update.yml) | GitHub App Client ID (OAuth-style `Iv…` identifier shown on the App's General page) used to mint a short-lived installation token for the update PR. | `CI_ENABLE_AUTO_UPDATES == 'true'` |
|
||||
| `BOT_APP_PRIVATE_KEY` | [update.yml](../../../../../.github/workflows/update.yml) | PEM-encoded private key of the same GitHub App. Used to sign the JWT that exchanges for the installation token. | `CI_ENABLE_AUTO_UPDATES == 'true'` |
|
||||
| `BOT_APP_CLIENT_ID` | [cron-update.yml](../../../../../.github/workflows/cron-update.yml) | GitHub App Client ID (OAuth-style `Iv…` identifier shown on the App's General page) used to mint a short-lived installation token for the update PR. | `CI_ENABLE_AUTO_UPDATES == 'true'` |
|
||||
| `BOT_APP_PRIVATE_KEY` | [cron-update.yml](../../../../../.github/workflows/cron-update.yml) | PEM-encoded private key of the same GitHub App. Used to sign the JWT that exchanges for the installation token. | `CI_ENABLE_AUTO_UPDATES == 'true'` |
|
||||
|
||||
## `BOT_APP_CLIENT_ID` and `BOT_APP_PRIVATE_KEY` 🤖
|
||||
|
||||
### Why these secrets exist 🎯
|
||||
|
||||
Actions performed with the workflow-provided `GITHUB_TOKEN` do not start further workflow runs. When [update.yml](../../../../../.github/workflows/update.yml) opens an update PR with `GITHUB_TOKEN`, the `pull_request` and `pull_request_target` events that drive [entry-pull-request-change.yml](../../../../../.github/workflows/entry-pull-request-change.yml) do not fire on that PR, so the CI orchestrator never runs against it.
|
||||
Actions performed with the workflow-provided `GITHUB_TOKEN` do not start further workflow runs. When [cron-update.yml](../../../../../.github/workflows/cron-update.yml) opens an update PR with `GITHUB_TOKEN`, the `pull_request` and `pull_request_target` events that drive [entry-pr-change-orchestrate.yml](../../../../../.github/workflows/entry-pr-change-orchestrate.yml) do not fire on that PR, so the CI orchestrator never runs against it.
|
||||
|
||||
Opening the PR through a GitHub App installation token bypasses this restriction. The App counts as a distinct actor, so PR-lifecycle events fire normally and the full CI pipeline runs against every update PR.
|
||||
|
||||
@@ -51,7 +51,7 @@ A classic personal access token MAY be used as a fallback for unrelated automati
|
||||
|
||||
### How the workflow consumes the secrets ⚙️
|
||||
|
||||
Each PR-opening job in [update.yml](../../../../../.github/workflows/update.yml) runs an `actions/create-github-app-token@v2` step gated on `steps.diff.outputs.changed == 'true'`, then forwards the minted token as `GH_TOKEN` and the resolved `APP_SLUG` to the subsequent PR step. The job's `permissions:` block keeps the workflow-provided `GITHUB_TOKEN` read-only; write access is provided exclusively by the App token, scoped to the App's repository permissions.
|
||||
Each PR-opening job in [cron-update.yml](../../../../../.github/workflows/cron-update.yml) runs an `actions/create-github-app-token@v2` step gated on `steps.diff.outputs.changed == 'true'`, then forwards the minted token as `GH_TOKEN` and the resolved `APP_SLUG` to the subsequent PR step. The job's `permissions:` block keeps the workflow-provided `GITHUB_TOKEN` read-only; write access is provided exclusively by the App token, scoped to the App's repository permissions.
|
||||
|
||||
### Commit identity propagation 🪪
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ GitHub Actions is GitHub's built-in CI/CD system. Each YAML file under `.github/
|
||||
- `workflow_call`: the workflow is a *reusable workflow* and is only started by another workflow that calls it via `uses:`.
|
||||
- `delete`: fires when a branch or tag is deleted.
|
||||
|
||||
Workflows communicate through *inputs* (on `workflow_call` / `workflow_dispatch`), *outputs*, *concurrency groups* (which cancel or serialize runs with the same group key), and the `GITHUB_TOKEN` / repository secrets and variables. The entry points of this repository (`entry-*.yml`) translate external triggers into a call to the central [ci-orchestrator.yml](../../../../../.github/workflows/ci-orchestrator.yml). Most other workflows are reusable building blocks invoked from the orchestrator.
|
||||
Workflows communicate through *inputs* (on `workflow_call` / `workflow_dispatch`), *outputs*, *concurrency groups* (which cancel or serialize runs with the same group key), and the `GITHUB_TOKEN` / repository secrets and variables. The entry points of this repository (`entry-*.yml`) translate external triggers into a call to the central [call-orchestrator.yml](../../../../../.github/workflows/call-orchestrator.yml). Most other workflows are reusable building blocks invoked from the orchestrator.
|
||||
|
||||
## Workflow catalog 📚
|
||||
|
||||
@@ -25,70 +25,70 @@ Trigger column legend: **auto** = fires automatically (push, pull_request, sched
|
||||
|
||||
| Workflow | Description | Trigger | Inputs |
|
||||
|---|---|---|---|
|
||||
| [entry-pull-request-change.yml](../../../../../.github/workflows/entry-pull-request-change.yml): `🔀 Pull Request` | Detects PR scope from changed files + branch prefix, then conditionally calls the orchestrator. A maintainer `🛡️ Trusted` label builds the privileged images from the PR head — see [pipeline.md](../../../artefact/git/pipeline.md#trusted-fork-prs-). A `🧩 Subset` label pins the deploy matrix to the roles listed in the PR body — see [pipeline.md](../../../artefact/git/pipeline.md#subset-label-). | auto (`pull_request` & `pull_request_target`: opened, synchronize, reopened, ready_for_review, labeled) | none |
|
||||
| [entry-push-latest.yml](../../../../../.github/workflows/entry-push-latest.yml): `📤 Push` | Syncs `main` from the configured source repository, then runs the orchestrator on pushes to `main`, `feature/**`, `hotfix/**`, `fix/**`, `chore/**`, `alert-autofix-*`; on tagged `main` it also calls `release-version.yml`. Respects the `CI_CANCEL_IN_PROGRESS`, `CI_SYNC_MAIN_SOURCE_REPOSITORY`, and `CI_RUN_ON_MAIN` repository variables, see [configuration.md](configuration.md). | auto (`push`) | none |
|
||||
| [entry-manual.yml](../../../../../.github/workflows/entry-manual.yml): `🕹️ Manual` | Manual dispatch of the full orchestrator for a chosen distro set and app whitelist. The `whitelist` input accepts the `__ALL__` sentinel for forced full deploy; see the "Diff-driven app selection" subsection in [pipeline.md](../../../artefact/git/pipeline.md). | manual | `distros` (default `debian`), `whitelist` (optional; `__ALL__` forces full deploy, empty triggers diff derivation, any other value is verbatim), `lifecycles`, `modes` (default `swarm compose`), `priority` (optional; priority line first, regular line runs the rest after it succeeds) |
|
||||
| [entry-pull-request-cancel.yml](../../../../../.github/workflows/entry-pull-request-cancel.yml): `🚫 Cancel: PR Runs on Close` | Cancels active workflow runs for a PR when it is closed or converted to draft. | auto (`pull_request_target`: closed, converted_to_draft) | none |
|
||||
| [delete-branch.yml](../../../../../.github/workflows/delete-branch.yml): `🚫 Cancel: Runs on Branch Delete` | Enters the push-entry concurrency group for the deleted branch so that any in-flight run on it is cancelled; a fallback step cancels remaining runs via the API. | auto (`delete`) | none |
|
||||
| [entry-pr-change-orchestrate.yml](../../../../../.github/workflows/entry-pr-change-orchestrate.yml): `🔀 Pull Request` | Detects PR scope from changed files + branch prefix, then conditionally calls the orchestrator. A maintainer `🛡️ Trusted` label builds the privileged images from the PR head — see [pipeline.md](../../../artefact/git/pipeline.md#trusted-fork-prs-). A `🧩 Subset` label pins the deploy matrix to the roles listed in the PR body — see [pipeline.md](../../../artefact/git/pipeline.md#subset-label-). | auto (`pull_request` & `pull_request_target`: opened, synchronize, reopened, ready_for_review, labeled) | none |
|
||||
| [entry-push-latest.yml](../../../../../.github/workflows/entry-push-latest.yml): `📤 Push` | Syncs `main` from the configured source repository, then runs the orchestrator on pushes to `main`, `feature/**`, `hotfix/**`, `fix/**`, `chore/**`, `alert-autofix-*`; on tagged `main` it also calls `call-release-version.yml`. Respects the `CI_CANCEL_IN_PROGRESS`, `CI_SYNC_MAIN_SOURCE_REPOSITORY`, and `CI_RUN_ON_MAIN` repository variables, see [configuration.md](configuration.md). | auto (`push`) | none |
|
||||
| [entry-manual-steer.yml](../../../../../.github/workflows/entry-manual-steer.yml): `🕹️ Manual` | Manual dispatch of the full orchestrator for a chosen distro set and app whitelist. The `whitelist` input accepts the `__ALL__` sentinel for forced full deploy; see the "Diff-driven app selection" subsection in [pipeline.md](../../../artefact/git/pipeline.md). | manual | `distros` (default `debian`), `whitelist` (optional; `__ALL__` forces full deploy, empty triggers diff derivation, any other value is verbatim), `lifecycles`, `modes` (default `swarm compose`), `priority` (optional; priority line first, regular line runs the rest after it succeeds) |
|
||||
| [entry-pr-closed-cancel-workflows.yml](../../../../../.github/workflows/entry-pr-closed-cancel-workflows.yml): `🚫 Cancel: PR Runs on Close` | Cancels active workflow runs for a PR when it is closed or converted to draft. | auto (`pull_request_target`: closed, converted_to_draft) | none |
|
||||
| [entry-delete-branch.yml](../../../../../.github/workflows/entry-delete-branch.yml): `🚫 Cancel: Runs on Branch Delete` | Enters the push-entry concurrency group for the deleted branch so that any in-flight run on it is cancelled; a fallback step cancels remaining runs via the API. | auto (`delete`) | none |
|
||||
|
||||
### Orchestration 🎵
|
||||
|
||||
| Workflow | Description | Trigger | Inputs |
|
||||
|---|---|---|---|
|
||||
| [ci-orchestrator.yml](../../../../../.github/workflows/ci-orchestrator.yml): `🎵 CI: Orchestrator` | Central coordinator. Runs fork-prereq wait, security, linting, CI image build, code tests, code-quality gate, DNS tests, mirror, deploy tests (optionally split into a priority line and a regular line via `priority`; each mode's discover job renders its own plan table via `cli.meta.ci.plan` into the run summary), install tests, `test-workspace`, `test-instructions`, final `done` gate. | reusable | `distros` (required), `whitelist` (optional), `priority` (optional; app ids deployed first per mode, then excluded from the regular line, which only starts after every priority job succeeded), `image_wait_attempts` (default `1980`), `image_wait_sleep_seconds` (default `10`), `workspace` (default `auto`), `instructions` (default `auto`) |
|
||||
| [call-orchestrator.yml](../../../../../.github/workflows/call-orchestrator.yml): `🎵 CI: Orchestrator` | Central coordinator. Runs fork-prereq wait, security, linting, CI image build, code tests, code-quality gate, DNS tests, mirror, deploy tests (optionally split into a priority line and a regular line via `priority`; each mode's discover job renders its own plan table via `cli.meta.ci.plan` into the run summary), install tests, `test-workspace`, `test-instructions`, final `done` gate. | reusable | `distros` (required), `whitelist` (optional), `priority` (optional; app ids deployed first per mode, then excluded from the regular line, which only starts after every priority job succeeded), `image_wait_attempts` (default `1980`), `image_wait_sleep_seconds` (default `10`), `workspace` (default `auto`), `instructions` (default `auto`) |
|
||||
|
||||
### Security and linting 🛡️🔍
|
||||
|
||||
| Workflow | Description | Trigger | Inputs |
|
||||
|---|---|---|---|
|
||||
| [security-codeql.yml](../../../../../.github/workflows/security-codeql.yml): `🔒 Scan: CodeQL (Advanced)` | CodeQL static analysis. Invoked by [ci-orchestrator.yml](../../../../../.github/workflows/ci-orchestrator.yml) so every CI run produces a single scan; additionally runs on a weekly cron so coverage does not drop when no pushes land on `main`. See [pipeline.md](../../../artefact/git/pipeline.md) for the gating behaviour. | reusable, auto (`schedule`: weekly Mon 00:00 UTC) | none |
|
||||
| [lint.yml](../../../../../.github/workflows/lint.yml): `🧹 Lint` | Runs `make lint` (every lint check in parallel: action, ansible, javascript, makefile, markdown, mermaid, packages, playwright, python, shellcheck) plus a hadolint SARIF job on `Dockerfile`. | reusable, manual | none |
|
||||
| [cron-security-codeql.yml](../../../../../.github/workflows/cron-security-codeql.yml): `⏰ Scan: CodeQL (Advanced)` | CodeQL static analysis. Invoked by [call-orchestrator.yml](../../../../../.github/workflows/call-orchestrator.yml) so every CI run produces a single scan; additionally runs on a weekly cron so coverage does not drop when no pushes land on `main`. See [pipeline.md](../../../artefact/git/pipeline.md) for the gating behaviour. | reusable, auto (`schedule`: weekly Mon 00:00 UTC) | none |
|
||||
| [call-lint.yml](../../../../../.github/workflows/call-lint.yml): `🧹 Lint` | Runs `make lint` (every lint check in parallel: action, ansible, javascript, makefile, markdown, mermaid, packages, playwright, python, shellcheck) plus a hadolint SARIF job on `Dockerfile`. | reusable, manual | none |
|
||||
|
||||
### CI images 🐳
|
||||
|
||||
| Workflow | Description | Trigger | Inputs |
|
||||
|---|---|---|---|
|
||||
| [images-build-ci.yml](../../../../../.github/workflows/images-build-ci.yml): `🖼️ Build: CI Images (all distros)` | Builds the per-distro CI base images consumed by all test jobs. | reusable | `distros` (required), `checkout_ref` (optional), `image_tag` (default `ci-${github.sha}`), `concurrency_channel` (default `default`) |
|
||||
| [images-cleanup-ci.yml](../../../../../.github/workflows/images-cleanup-ci.yml): `🧹 Images: Cleanup CI (GHCR)` | Deletes CI images from GHCR older than N days. | auto (`schedule`: weekly Mon 00:00 UTC), manual | `days` (default `7`) |
|
||||
| [call-images-build-ci.yml](../../../../../.github/workflows/call-images-build-ci.yml): `🖼️ Build: CI Images (all distros)` | Builds the per-distro CI base images consumed by all test jobs. | reusable | `distros` (required), `checkout_ref` (optional), `image_tag` (default `ci-${github.sha}`), `concurrency_channel` (default `default`) |
|
||||
| [cron-images-cleanup-ci.yml](../../../../../.github/workflows/cron-images-cleanup-ci.yml): `⏰ Images: Cleanup CI (GHCR)` | Deletes CI images from GHCR older than N days. | auto (`schedule`: weekly Mon 00:00 UTC), manual | `days` (default `7`) |
|
||||
|
||||
### Image mirroring 🪞
|
||||
|
||||
| Workflow | Description | Trigger | Inputs |
|
||||
|---|---|---|---|
|
||||
| [images-mirror-missing.yml](../../../../../.github/workflows/images-mirror-missing.yml): `🪞 Mirror: Docker Hub → GHCR (only missing)` | Mirrors only the upstream images that are not yet in GHCR. Called from the orchestrator before deploy tests. | reusable | `ghcr_namespace`, `ghcr_prefix` (default `mirror`), `repo_root` (default `.`), `source_repository`, `source_ref`, plus throttling knobs |
|
||||
| [images-mirror-all.yml](../../../../../.github/workflows/images-mirror-all.yml): `🪞 Mirror: Docker Hub → GHCR` | Full nightly mirror of every referenced upstream image into GHCR. | auto (`schedule`: daily 00:00 UTC), manual | `ghcr_namespace`, `ghcr_prefix` (default `mirror`), `repo_root` (default `.`), `images_per_hour` (throttle) |
|
||||
| [images-mirror-cleanup.yml](../../../../../.github/workflows/images-mirror-cleanup.yml): `🧹 Images: Cleanup GHCR (Private)` | Deletes GHCR mirror packages by prefix and visibility. Supports `dry_run`. | manual | `ghcr_namespace`, `ghcr_prefix` (default `mirror`), `visibility` (`private`/`public`/`internal`, default `private`), `dry_run` |
|
||||
| [call-images-mirror-missing.yml](../../../../../.github/workflows/call-images-mirror-missing.yml): `🪞 Mirror: Docker Hub → GHCR (only missing)` | Mirrors only the upstream images that are not yet in GHCR. Called from the orchestrator before deploy tests. | reusable | `ghcr_namespace`, `ghcr_prefix` (default `mirror`), `repo_root` (default `.`), `source_repository`, `source_ref`, plus throttling knobs |
|
||||
| [cron-images-mirror-all.yml](../../../../../.github/workflows/cron-images-mirror-all.yml): `⏰ Mirror: Docker Hub → GHCR` | Full nightly mirror of every referenced upstream image into GHCR. | auto (`schedule`: daily 00:00 UTC), manual | `ghcr_namespace`, `ghcr_prefix` (default `mirror`), `repo_root` (default `.`), `images_per_hour` (throttle) |
|
||||
| [entry-manual-mirror-cleanup.yml](../../../../../.github/workflows/entry-manual-mirror-cleanup.yml): `🧹 Images: Cleanup GHCR (Private)` | Deletes GHCR mirror packages by prefix and visibility. Supports `dry_run`. | manual | `ghcr_namespace`, `ghcr_prefix` (default `mirror`), `visibility` (`private`/`public`/`internal`, default `private`), `dry_run` |
|
||||
|
||||
### Code tests 🧪
|
||||
|
||||
| Workflow | Description | Trigger | Inputs |
|
||||
|---|---|---|---|
|
||||
| [test.yml](../../../../../.github/workflows/test.yml): `🧪 Test` | Runs `make test` (unit, integration, lint and external tests in parallel) on the runner host. | reusable, manual | none |
|
||||
| [call-test.yml](../../../../../.github/workflows/call-test.yml): `🧪 Test` | Runs `make test` (unit, integration, lint and external tests in parallel) on the runner host. | reusable, manual | none |
|
||||
|
||||
### Infrastructure tests 🌐📦💻📥
|
||||
|
||||
| Workflow | Description | Trigger | Inputs |
|
||||
|---|---|---|---|
|
||||
| [test-dns.yml](../../../../../.github/workflows/test-dns.yml): `💬 Test: DNS` | Validates DNS resolution across target distributions. | reusable, manual | `distros` (required on call; default `debian` on manual) |
|
||||
| [test-deploy-single-node.yml](../../../../../.github/workflows/test-deploy-single-node.yml): `🏝️ Test: Deploy (single-node)` | Single point of truth for the non-swarm deploy tests: fans out to `test-deploy-compose.yml` and `test-deploy-host.yml`, gated per mode via the `modes` input. Called from the orchestrator. | reusable | `distros` (required), `whitelist`, `lifecycles`, `modes` (default `compose host`), `blacklist` (optional; app ids dropped from discovery) |
|
||||
| [test-deploy-compose.yml](../../../../../.github/workflows/test-deploy-compose.yml): `📦 Test: Deploy (compose)` | Deploy test (compose mode) across all deployment types (`INFINITO_DISCOVER_ALL`; lifecycle-filtered to the `INFINITO_LIFECYCLES` envelope, so `server`/`universal`/`workstation` scopes but not just `web-*`). When `whitelist` is empty, the discover job derives one from the branch's diff vs `origin/main`; see the "Diff-driven app selection" subsection in [pipeline.md](../../../artefact/git/pipeline.md). | reusable | `distros` (required), `whitelist` (optional; explicit value wins over diff), `lifecycles`, `blacklist` (optional) |
|
||||
| [test-deploy-host.yml](../../../../../.github/workflows/test-deploy-host.yml): `🖥️ Test: Deploy (host)` | Deploy test (host mode) for invokable non-stack roles that configure the host instead of shipping a container stack. Same diff-driven whitelist resolution as `test-deploy-compose.yml`; no Playwright stage (host roles have no web UI). | reusable | `distros` (required), `whitelist` (optional; explicit value wins over diff), `lifecycles`, `blacklist` (optional) |
|
||||
| [test-deploy-swarm.yml](../../../../../.github/workflows/test-deploy-swarm.yml): `🐝 Test: Deploy (swarm)` | Deploy test (swarm mode) across the full application matrix; each app gets a simulated 3-node DinD cluster with an NFS server, then a drain/reschedule state-survival assertion. Same diff-driven whitelist resolution as `test-deploy-compose.yml`. | reusable, manual | `distros` (required), `whitelist` (optional; explicit value wins over diff), `lifecycles`, `blacklist` (optional; reusable trigger only) |
|
||||
| [test-instructions.yml](../../../../../.github/workflows/test-instructions.yml): `📖 Test: Instructions` | Extracts the `### Production` command from a role README (a random role when `role` is empty) and executes it against the runner: `<your-server>` becomes `localhost` (ansible_connection local) and the Docker socket replaces the TTY flag. Verifies the documented one-command deploy actually works, seeding the run´s freshly built CI image (needs `images-build-ci.yml`) instead of the published default. Gates the final `done` job. | reusable, manual | `distros` (required), `role` (optional; empty picks one at random), `priority` (optional), `whitelist` (optional) |
|
||||
| [test-install-make.yml](../../../../../.github/workflows/test-install-make.yml): `📥 Test: Install Make` | Validates the `make install` entry points. | reusable | none |
|
||||
| [test-install-pkgmgr.yml](../../../../../.github/workflows/test-install-pkgmgr.yml): `📥 Test: Install Package Manager` | Validates installation via the upstream package manager. | reusable | none |
|
||||
| [test-workspace.yml](../../../../../.github/workflows/test-workspace.yml): `💻 Test: Workspace` | Builds and exercises the dev-runtime image matrix. | reusable, manual | none |
|
||||
| [call-test-dns.yml](../../../../../.github/workflows/call-test-dns.yml): `💬 Test: DNS` | Validates DNS resolution across target distributions. | reusable, manual | `distros` (required on call; default `debian` on manual) |
|
||||
| [call-test-deploy-single-node.yml](../../../../../.github/workflows/call-test-deploy-single-node.yml): `🏝️ Test: Deploy (single-node)` | Single point of truth for the non-swarm deploy tests: fans out to `call-test-deploy-compose.yml` and `call-test-deploy-host.yml`, gated per mode via the `modes` input. Called from the orchestrator. | reusable | `distros` (required), `whitelist`, `lifecycles`, `modes` (default `compose host`), `blacklist` (optional; app ids dropped from discovery) |
|
||||
| [call-test-deploy-compose.yml](../../../../../.github/workflows/call-test-deploy-compose.yml): `📦 Test: Deploy (compose)` | Deploy test (compose mode) across all deployment types (`INFINITO_DISCOVER_ALL`; lifecycle-filtered to the `INFINITO_LIFECYCLES` envelope, so `server`/`universal`/`workstation` scopes but not just `web-*`). When `whitelist` is empty, the discover job derives one from the branch's diff vs `origin/main`; see the "Diff-driven app selection" subsection in [pipeline.md](../../../artefact/git/pipeline.md). | reusable | `distros` (required), `whitelist` (optional; explicit value wins over diff), `lifecycles`, `blacklist` (optional) |
|
||||
| [call-test-deploy-host.yml](../../../../../.github/workflows/call-test-deploy-host.yml): `🖥️ Test: Deploy (host)` | Deploy test (host mode) for invokable non-stack roles that configure the host instead of shipping a container stack. Same diff-driven whitelist resolution as `call-test-deploy-compose.yml`; no Playwright stage (host roles have no web UI). | reusable | `distros` (required), `whitelist` (optional; explicit value wins over diff), `lifecycles`, `blacklist` (optional) |
|
||||
| [call-test-deploy-swarm.yml](../../../../../.github/workflows/call-test-deploy-swarm.yml): `🐝 Test: Deploy (swarm)` | Deploy test (swarm mode) across the full application matrix; each app gets a simulated 3-node DinD cluster with an NFS server, then a drain/reschedule state-survival assertion. Same diff-driven whitelist resolution as `call-test-deploy-compose.yml`. | reusable, manual | `distros` (required), `whitelist` (optional; explicit value wins over diff), `lifecycles`, `blacklist` (optional; reusable trigger only) |
|
||||
| [call-test-instructions.yml](../../../../../.github/workflows/call-test-instructions.yml): `📖 Test: Instructions` | Extracts the `### Production` command from a role README (a random role when `role` is empty) and executes it against the runner: `<your-server>` becomes `localhost` (ansible_connection local) and the Docker socket replaces the TTY flag. Verifies the documented one-command deploy actually works, seeding the run´s freshly built CI image (needs `call-images-build-ci.yml`) instead of the published default. Gates the final `done` job. | reusable, manual | `distros` (required), `role` (optional; empty picks one at random), `priority` (optional), `whitelist` (optional) |
|
||||
| [call-test-install-make.yml](../../../../../.github/workflows/call-test-install-make.yml): `📥 Test: Install Make` | Validates the `make install` entry points. | reusable | none |
|
||||
| [call-test-install-pkgmgr.yml](../../../../../.github/workflows/call-test-install-pkgmgr.yml): `📥 Test: Install Package Manager` | Validates installation via the upstream package manager. | reusable | none |
|
||||
| [call-test-workspace.yml](../../../../../.github/workflows/call-test-workspace.yml): `💻 Test: Workspace` | Builds and exercises the dev-runtime image matrix. | reusable, manual | none |
|
||||
|
||||
### Release and maintenance 🚀🔄
|
||||
|
||||
| Workflow | Description | Trigger | Inputs |
|
||||
|---|---|---|---|
|
||||
| [release-version.yml](../../../../../.github/workflows/release-version.yml): `🚀 Release: Version Logic` | Releases a specific version tag (build + publish). Called from `entry-push-latest.yml` on tagged pushes. | reusable, manual | `tag` (required, e.g. `v1.2.3`) |
|
||||
| [release-highest.yml](../../../../../.github/workflows/release-highest.yml): `🚀 Release: Highest Missing Version` | Scheduled backfill: finds the highest tag without a release and triggers `release-version.yml` for it. | auto (`schedule`: daily 00:00 UTC), manual | none |
|
||||
| [cleanup-stale.yml](../../../../../.github/workflows/cleanup-stale.yml): `🧹 Cleanup: Stale Repository Data` | Marks and closes stale issues and PRs, deletes inactive branches, and prunes old GHCR CI image versions. | auto (`schedule`: daily 00:00 UTC), manual | none |
|
||||
| [update.yml](../../../../../.github/workflows/update.yml): `🔄 Update: Versions` | Updates Docker image versions and other pinned dependencies; opens an update PR via a GitHub App installation token so PR-lifecycle events fire on the resulting PR. Both jobs are gated behind the `CI_ENABLE_AUTO_UPDATES` repository variable, see [configuration.md](configuration.md). Requires the `BOT_APP_CLIENT_ID` and `BOT_APP_PRIVATE_KEY` repository secrets, see [secrets.md](secrets.md). | auto (`push` to `main`, `schedule`: daily 00:30 UTC), manual | none |
|
||||
| [dependabot-close.yml](../../../../../.github/workflows/dependabot-close.yml): `🚫 Dependabot: Close while auto-updates disabled` | Auto-closes Dependabot PRs while `CI_ENABLE_AUTO_UPDATES` is not set to `true`, so Dependabot honours the same gate as the workflow-driven update jobs. See [configuration.md](configuration.md). | auto (`pull_request_target`: opened, reopened) | none |
|
||||
| [rerun-cancelled.yml](../../../../../.github/workflows/rerun-cancelled.yml): `🔁 Rerun: Cancelled Branch Runs` | Sweeps every branch: when its latest run ended `cancelled`, the failed jobs are rerun via `gh run rerun --failed` while the successful ones are kept, so a concurrency cancellation costs a resume instead of a full pipeline. The attempt ceiling, the age cutoff, and the sweep interval come from `CI_RERUN_MAX_ATTEMPTS`, `CI_RERUN_MAX_AGE_HOURS` and `CI_RERUN_INTERVAL_MINUTES`, see [configuration.md](configuration.md). Sweeps hourly by default, with no attempt ceiling. A new trigger cancels a sweep that is still running. Manual dispatch always sweeps, ignoring the interval. | auto (`schedule`: hourly, gated to the configured interval), manual | none |
|
||||
| [call-release-version.yml](../../../../../.github/workflows/call-release-version.yml): `🚀 Release: Version Logic` | Releases a specific version tag (build + publish). Called from `entry-push-latest.yml` on tagged pushes. | reusable, manual | `tag` (required, e.g. `v1.2.3`) |
|
||||
| [cron-release-highest.yml](../../../../../.github/workflows/cron-release-highest.yml): `⏰ Release: Highest Missing Version` | Scheduled backfill: finds the highest tag without a release and triggers `call-release-version.yml` for it. | auto (`schedule`: daily 00:00 UTC), manual | none |
|
||||
| [cron-cleanup-stale.yml](../../../../../.github/workflows/cron-cleanup-stale.yml): `⏰ Cleanup: Stale Repository Data` | Marks and closes stale issues and PRs, deletes inactive branches, and prunes old GHCR CI image versions. | auto (`schedule`: daily 00:00 UTC), manual | none |
|
||||
| [cron-update.yml](../../../../../.github/workflows/cron-update.yml): `⏰ Update: Versions` | Updates Docker image versions and other pinned dependencies; opens an update PR via a GitHub App installation token so PR-lifecycle events fire on the resulting PR. Both jobs are gated behind the `CI_ENABLE_AUTO_UPDATES` repository variable, see [configuration.md](configuration.md). Requires the `BOT_APP_CLIENT_ID` and `BOT_APP_PRIVATE_KEY` repository secrets, see [secrets.md](secrets.md). | auto (`push` to `main`, `schedule`: daily 00:30 UTC), manual | none |
|
||||
| [entry-pr-open-dependabot-close.yml](../../../../../.github/workflows/entry-pr-open-dependabot-close.yml): `🚫 Dependabot: Close while auto-updates disabled` | Auto-closes Dependabot PRs while `CI_ENABLE_AUTO_UPDATES` is not set to `true`, so Dependabot honours the same gate as the workflow-driven update jobs. See [configuration.md](configuration.md). | auto (`pull_request_target`: opened, reopened) | none |
|
||||
| [cron-rerun-cancelled.yml](../../../../../.github/workflows/cron-rerun-cancelled.yml): `⏰ Rerun: Cancelled Branch Runs` | Sweeps every branch: when its latest run ended `cancelled`, the failed jobs are rerun via `gh run rerun --failed` while the successful ones are kept, so a concurrency cancellation costs a resume instead of a full pipeline. The attempt ceiling, the age cutoff, and the sweep interval come from `CI_RERUN_MAX_ATTEMPTS`, `CI_RERUN_MAX_AGE_HOURS` and `CI_RERUN_INTERVAL_MINUTES`, see [configuration.md](configuration.md). Sweeps hourly by default, with no attempt ceiling. A new trigger cancels a sweep that is still running. Manual dispatch always sweeps, ignoring the interval. | auto (`schedule`: hourly, gated to the configured interval), manual | none |
|
||||
|
||||
## Changing workflows ✍️
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
<!-- nocheck: workflow-references names a workflow path that was planned, never created -->
|
||||
|
||||
# 023 - Docker Swarm Deployment with NFS-backed Shared Volumes
|
||||
|
||||
## User Story
|
||||
|
||||
@@ -40,7 +40,7 @@ dispatch_http=$(${CURL} -sf -o /dev/null -w "%{http_code}" \
|
||||
-X POST \
|
||||
-H "Authorization: Bearer ${RUNNER_API_TOKEN}" \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
"https://api.github.com/repos/${RUNNER_GITHUB_OWNER}/${RUNNER_GITHUB_REPO}/actions/workflows/test-runner-smoke.yml/dispatches" \
|
||||
"https://api.github.com/repos/${RUNNER_GITHUB_OWNER}/${RUNNER_GITHUB_REPO}/actions/workflows/call-test-runner-smoke.yml/dispatches" \
|
||||
-d "{\"ref\":\"${RUNNER_GIT_REF}\"}" \
|
||||
|| echo "000")
|
||||
|
||||
@@ -57,7 +57,7 @@ else
|
||||
if ! runs_json=$(${CURL} -sf \
|
||||
-H "Authorization: Bearer ${RUNNER_API_TOKEN}" \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
"https://api.github.com/repos/${RUNNER_GITHUB_OWNER}/${RUNNER_GITHUB_REPO}/actions/workflows/test-runner-smoke.yml/runs?per_page=10"); then
|
||||
"https://api.github.com/repos/${RUNNER_GITHUB_OWNER}/${RUNNER_GITHUB_REPO}/actions/workflows/call-test-runner-smoke.yml/runs?per_page=10"); then
|
||||
echo "FAIL: GitHub API unreachable while polling for smoke run"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -126,7 +126,7 @@ Volume layout under `DEPLOYMENT_MODE: swarm` with `storage.backend: nfs`:
|
||||
directory is intentionally out of scope for v1 (locking / `fsync`
|
||||
semantics). See 023's Future Extensions.
|
||||
|
||||
CI gate: [.github/workflows/test-deploy-swarm.yml](../../.github/workflows/test-deploy-swarm.yml)
|
||||
CI gate: [.github/workflows/call-test-deploy-swarm.yml](../../.github/workflows/call-test-deploy-swarm.yml)
|
||||
provisions a 3-node DinD swarm, deploys this role as a stack, drains
|
||||
the worker running the application service, and asserts that wiki
|
||||
content survives the reschedule.
|
||||
|
||||
@@ -4,7 +4,7 @@ Shell helpers that drive the scheduled `Update: Versions` workflow.
|
||||
|
||||
## Scope 📋
|
||||
|
||||
This directory contains the helpers invoked by [update.yml](../../../.github/workflows/update.yml) to bump pinned upstream versions and open a pull request once per day. The helpers are thin wrappers: the image-version and repository-ref bumpers delegate to Python CLIs under [cli/contributing/](../../../cli/contributing/) that scan `roles/*/meta/services.yml` and rewrite version pins in place; the PR-open helper commits, force-pushes the update branch, and creates or refreshes the PR via `gh`. Helpers MUST NOT be called from workflows other than `Update: Versions`; the `CI_ENABLE_AUTO_UPDATES` gate is enforced once, at the workflow's job level, and bypassing it from another caller would create PRs even when auto-updates are intentionally disabled.
|
||||
This directory contains the helpers invoked by [cron-update.yml](../../../.github/workflows/cron-update.yml) to bump pinned upstream versions and open a pull request once per day. The helpers are thin wrappers: the image-version and repository-ref bumpers delegate to Python CLIs under [cli/contributing/](../../../cli/contributing/) that scan `roles/*/meta/services.yml` and rewrite version pins in place; the PR-open helper commits, force-pushes the update branch, and creates or refreshes the PR via `gh`. Helpers MUST NOT be called from workflows other than `⏰ Update: Versions`; the `CI_ENABLE_AUTO_UPDATES` gate is enforced once, at the workflow's job level, and bypassing it from another caller would create PRs even when auto-updates are intentionally disabled.
|
||||
|
||||
Each shim MUST resolve the repository root relative to its own location so it can run from any working directory, MUST forward arguments to the underlying CLI so `--dry-run` and similar flags keep working from the shell, and MUST exit non-zero on CLI failure so the workflow surfaces the error.
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
: "${ACT_WORKFLOW:?ACT_WORKFLOW is not set (e.g. .github/workflows/test-workspace.yml)}"
|
||||
: "${ACT_WORKFLOW:?ACT_WORKFLOW is not set (e.g. .github/workflows/call-test-workspace.yml)}"
|
||||
: "${ACT_EVENT:=workflow_dispatch}"
|
||||
: "${ACT_JOB:=}"
|
||||
: "${ACT_MATRIX:=}"
|
||||
|
||||
@@ -173,7 +173,7 @@ the cluster declaration (image, containers, network, DNS play) in
|
||||
The matrix orchestrator
|
||||
(`utils/tests/swarm/matrix.py`) drives the deploy stage
|
||||
per variant round; `routine/00_one.sh` drives the surrounding stages for one
|
||||
distro, and the workflow `.github/workflows/test-deploy-swarm.yml` drives the
|
||||
distro, and the workflow `.github/workflows/call-test-deploy-swarm.yml` drives the
|
||||
distro loop. Run one app locally with
|
||||
`make swarm-zombie app=<id>` (keeps the cluster for inspection) or the
|
||||
whole matrix via `make roundtrip`.
|
||||
@@ -11,7 +11,7 @@ Counterpart to ``tests/external/update/docker/test_image_versions.py``
|
||||
from-source builds, plugin and build-helper repos).
|
||||
The shared discovery / version logic lives in
|
||||
:mod:`utils.update.repository`; the CI auto-update job at
|
||||
``.github/workflows/update.yml`` (``update-repository-refs``) consumes
|
||||
``.github/workflows/cron-update.yml`` (``update-repository-refs``) consumes
|
||||
the same module to open PRs against ``main``.
|
||||
|
||||
External test: depends on live ``git ls-remote`` calls against the
|
||||
@@ -78,7 +78,6 @@ class TestRepositoryVersions(unittest.TestCase):
|
||||
line=u.entry.line,
|
||||
)
|
||||
|
||||
# Always pass — outdated refs are warnings, not hard failures.
|
||||
self.assertIsNotNone(entries)
|
||||
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ from itertools import pairwise
|
||||
from tests.utils import PROJECT_ROOT
|
||||
from utils.cache.yaml import load_yaml
|
||||
|
||||
ORCHESTRATOR = PROJECT_ROOT / ".github" / "workflows" / "ci-orchestrator.yml"
|
||||
ORCHESTRATOR = PROJECT_ROOT / ".github" / "workflows" / "call-orchestrator.yml"
|
||||
|
||||
LINES = {"priority": "-priority", "regular": ""}
|
||||
CHAIN = ("test-deploy-swarm", "test-deploy-compose", "test-deploy-host")
|
||||
|
||||
@@ -8,15 +8,15 @@ from utils.cache.yaml import load_yaml_any
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
SECURITY_WORKFLOW_PATH = PROJECT_ROOT / ".github" / "workflows" / "security-codeql.yml"
|
||||
SECURITY_WORKFLOW_PATH = (
|
||||
PROJECT_ROOT / ".github" / "workflows" / "cron-security-codeql.yml"
|
||||
)
|
||||
|
||||
# Hidden directories that are the only ones worth scanning (the cached walker
|
||||
# already prunes .git/.venv/__pycache__/node_modules etc.).
|
||||
SCANNED_HIDDEN_DIRS = {
|
||||
".github",
|
||||
}
|
||||
"""Hidden directories worth scanning; the cached walker prunes the rest."""
|
||||
|
||||
# Mapping: CodeQL language name -> filename patterns
|
||||
CODEQL_FILENAME_INDICATORS: dict[str, list[str]] = {
|
||||
"c-cpp": ["*.c", "*.cc", "*.cpp", "*.cxx", "*.h", "*.hh", "*.hpp", "*.hxx"],
|
||||
"csharp": ["*.cs"],
|
||||
@@ -28,8 +28,8 @@ CODEQL_FILENAME_INDICATORS: dict[str, list[str]] = {
|
||||
"rust": ["*.rs"],
|
||||
"swift": ["*.swift"],
|
||||
}
|
||||
"""CodeQL language name -> filename patterns."""
|
||||
|
||||
# Mapping: CodeQL language name -> repository-relative path patterns
|
||||
CODEQL_PATH_INDICATORS: dict[str, list[str]] = {
|
||||
"actions": [
|
||||
".github/workflows/*.yml",
|
||||
@@ -74,8 +74,6 @@ class TestSecurityWorkflowCoverage(unittest.TestCase):
|
||||
|
||||
for path in iter_project_files():
|
||||
rel_file = os.path.relpath(path, PROJECT_ROOT)
|
||||
# Skip top-level hidden directories that are not SCANNED_HIDDEN_DIRS
|
||||
# (the cached walker already prunes the vendor/cache junk).
|
||||
top = rel_file.split(os.sep, 1)[0]
|
||||
if top.startswith(".") and top not in SCANNED_HIDDEN_DIRS:
|
||||
continue
|
||||
@@ -89,7 +87,8 @@ class TestSecurityWorkflowCoverage(unittest.TestCase):
|
||||
problems: list[str] = []
|
||||
if missing_languages:
|
||||
problems.append(
|
||||
"Detected languages missing from .github/workflows/security-codeql.yml:"
|
||||
"Detected languages missing from "
|
||||
".github/workflows/cron-security-codeql.yml:"
|
||||
)
|
||||
problems.extend(f" - {language}" for language in missing_languages)
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ class TestSubsetRoles(unittest.TestCase):
|
||||
|
||||
The "no subset label" case (existing diff behaviour stays unchanged) is
|
||||
covered by test_diff_affected_roles.py: this module only ever runs when
|
||||
the label gates it in entry-pull-request-change.yml.
|
||||
the label gates it in entry-pr-change-orchestrate.yml.
|
||||
"""
|
||||
|
||||
def _run(self, body: str):
|
||||
|
||||
@@ -6,8 +6,8 @@ Why
|
||||
---
|
||||
|
||||
Semver pinning is the precondition for the daily auto-update job at
|
||||
[update-image-versions](../../../../.github/workflows/update.yml) and
|
||||
[update-repository-refs](../../../../.github/workflows/update.yml):
|
||||
[update-image-versions](../../../../.github/workflows/cron-update.yml) and
|
||||
[update-repository-refs](../../../../.github/workflows/cron-update.yml):
|
||||
those jobs only bump entries whose current value matches the semver
|
||||
shape (``v?\\d+(\\.\\d+){0,3}(-flavor)?``). A pin like ``latest``,
|
||||
``stable``, ``master``, ``main``, or a commit SHA silently opts the
|
||||
|
||||
@@ -24,7 +24,9 @@ from . import PROJECT_ROOT
|
||||
HANDLERS = PROJECT_ROOT / "roles" / "sys-svc-compose" / "handlers"
|
||||
SWARM = HANDLERS / "swarm.yml"
|
||||
COMPOSE = HANDLERS / "compose.yml"
|
||||
COMPOSE_WORKFLOW = PROJECT_ROOT / ".github" / "workflows" / "test-deploy-compose.yml"
|
||||
COMPOSE_WORKFLOW = (
|
||||
PROJECT_ROOT / ".github" / "workflows" / "call-test-deploy-compose.yml"
|
||||
)
|
||||
|
||||
BUILDS = {
|
||||
"swarm: pre-deploy build of local images": (SWARM, "_swarm_pre_deploy_build"),
|
||||
|
||||
@@ -37,7 +37,7 @@ from utils.env.parser import parse_static_env
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
WORKFLOWS_DIR = PROJECT_ROOT / ".github" / "workflows"
|
||||
WORKSPACE_WORKFLOW = WORKFLOWS_DIR / "test-workspace.yml"
|
||||
WORKSPACE_WORKFLOW = WORKFLOWS_DIR / "call-test-workspace.yml"
|
||||
_MATRIX_REF_RE = re.compile(
|
||||
r"^\$\{\{\s*fromJson\(\s*needs\.([\w-]+)\.outputs\.([\w-]+)\s*\)\s*\}\}$",
|
||||
re.IGNORECASE,
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
"""Lint: every workflow file named anywhere in the repository exists.
|
||||
|
||||
Scope
|
||||
=====
|
||||
Any occurrence of ``.github/workflows/<file>.yml`` or ``actions/workflows/
|
||||
<file>.yml`` in a tracked text file: `uses:` lines, Python constants, Makefile
|
||||
variables, shell API URLs and Markdown links alike.
|
||||
|
||||
Why
|
||||
===
|
||||
Renaming a workflow silently breaks every one of those. A `uses:` line fails
|
||||
the run, a Python constant raises at test time, but a Markdown link or an API
|
||||
URL in a shell script fails only in production, months later. Matching the
|
||||
string against the directory listing catches all of them at once.
|
||||
|
||||
A file opts out with a ``nocheck: workflow-references`` marker in its first 30
|
||||
lines, for documents that name a workflow that does not exist yet.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from utils.annotations.suppress import is_suppressed_in_head
|
||||
from utils.cache.files import iter_project_files, read_text
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_NOCHECK_RULE = "workflow-references"
|
||||
_WORKFLOW_DIR = PROJECT_ROOT / ".github" / "workflows"
|
||||
_SUFFIXES = {".md", ".yml", ".yaml", ".py", ".sh", ".j2", ".rst", ".txt"}
|
||||
_REFERENCE_RE = re.compile(r"(?:\.github|actions)/workflows/([A-Za-z0-9._-]+\.ya?ml)")
|
||||
|
||||
|
||||
def _scanned(path: Path) -> bool:
|
||||
return path.suffix in _SUFFIXES or path.name == "Makefile"
|
||||
|
||||
|
||||
def missing_references() -> list[str]:
|
||||
"""Every `<file>:<line>: <workflow>` whose workflow is not on disk."""
|
||||
known = {entry.name for entry in _WORKFLOW_DIR.iterdir() if entry.is_file()}
|
||||
offenders: list[str] = []
|
||||
for path_str in iter_project_files():
|
||||
path = Path(path_str)
|
||||
if not _scanned(path):
|
||||
continue
|
||||
try:
|
||||
lines = read_text(path_str).splitlines()
|
||||
except (OSError, UnicodeDecodeError):
|
||||
continue
|
||||
if is_suppressed_in_head(lines, _NOCHECK_RULE):
|
||||
continue
|
||||
rel = path.relative_to(PROJECT_ROOT).as_posix()
|
||||
for number, line in enumerate(lines, 1):
|
||||
for name in _REFERENCE_RE.findall(line):
|
||||
if name not in known:
|
||||
offenders.append(f"{rel}:{number}: {name}")
|
||||
return offenders
|
||||
|
||||
|
||||
class TestWorkflowReferences(unittest.TestCase):
|
||||
def test_referenced_workflows_exist(self) -> None:
|
||||
offenders = missing_references()
|
||||
if offenders:
|
||||
self.fail(
|
||||
f"{len(offenders)} reference(s) point at a workflow file that does "
|
||||
"not exist. Rename the reference along with the file, or mark the "
|
||||
f"document with 'nocheck: {_NOCHECK_RULE}' when it names a workflow "
|
||||
"that is still to be created:\n" + "\n".join(sorted(offenders))
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,119 @@
|
||||
"""Lint: a workflow file name declares what starts the workflow.
|
||||
|
||||
Scope
|
||||
=====
|
||||
Every `.github/workflows/*.yml`.
|
||||
|
||||
Rule
|
||||
====
|
||||
The file name carries exactly one of three prefixes, derived from the `on:`
|
||||
block and checked in this order:
|
||||
|
||||
* ``cron-`` the workflow has a `schedule` trigger. Its `name:` must also start
|
||||
with ⏰, so a run nobody triggered is recognisable in the Actions list.
|
||||
* ``call-`` no schedule, but a `workflow_call` trigger: a pipeline stage that
|
||||
only another workflow starts. An extra `workflow_dispatch` for debugging does
|
||||
not change that.
|
||||
* ``entry-`` neither: a repository event or a human starts it.
|
||||
|
||||
Why
|
||||
===
|
||||
The first question about any workflow is what starts it, and the directory
|
||||
otherwise mixes entry points, reusable stages and scheduled jobs into one flat
|
||||
list sorted by topic.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from ruamel.yaml import YAML
|
||||
|
||||
from utils.cache.files import iter_project_files
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_WORKFLOW_DIR = ".github/workflows/"
|
||||
_CLOCK = "⏰"
|
||||
_SCHEDULE = "cron-"
|
||||
_REUSABLE = "call-"
|
||||
_ENTRY = "entry-"
|
||||
|
||||
_yaml = YAML(typ="safe")
|
||||
_yaml.allow_duplicate_keys = True
|
||||
|
||||
|
||||
def _load(path: str):
|
||||
try:
|
||||
with Path(path).open(encoding="utf-8") as handle:
|
||||
return _yaml.load(handle)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _trigger_names(data) -> set[str]:
|
||||
"""The keys of the `on:` block, which YAML 1.1 parsers hand back under `True`."""
|
||||
if not isinstance(data, dict):
|
||||
return set()
|
||||
for key in ("on", True):
|
||||
if key in data:
|
||||
triggers = data[key]
|
||||
if isinstance(triggers, dict):
|
||||
return {str(name) for name in triggers}
|
||||
if isinstance(triggers, list):
|
||||
return {str(name) for name in triggers}
|
||||
if triggers is not None:
|
||||
return {str(triggers)}
|
||||
return set()
|
||||
|
||||
|
||||
def expected_prefix(data) -> str:
|
||||
triggers = _trigger_names(data)
|
||||
if "schedule" in triggers:
|
||||
return _SCHEDULE
|
||||
if "workflow_call" in triggers:
|
||||
return _REUSABLE
|
||||
return _ENTRY
|
||||
|
||||
|
||||
def issues(rel: str, data) -> list[str]:
|
||||
found: list[str] = []
|
||||
wanted = expected_prefix(data)
|
||||
name = Path(rel).name
|
||||
if not name.startswith(wanted):
|
||||
carried = next(
|
||||
(p for p in (_SCHEDULE, _REUSABLE, _ENTRY) if name.startswith(p)),
|
||||
"no prefix",
|
||||
)
|
||||
found.append(f"carries {carried!r}, must carry {wanted!r}")
|
||||
if wanted == _SCHEDULE:
|
||||
title = data.get("name") if isinstance(data, dict) else None
|
||||
if not isinstance(title, str) or not title.startswith(_CLOCK):
|
||||
found.append(f"name: must start with {_CLOCK}")
|
||||
return found
|
||||
|
||||
|
||||
class TestWorkflowTriggerPrefix(unittest.TestCase):
|
||||
def test_workflow_names_declare_their_trigger(self) -> None:
|
||||
offenders: list[str] = []
|
||||
for path_str in iter_project_files(extensions=(".yml", ".yaml")):
|
||||
rel = Path(path_str).relative_to(PROJECT_ROOT).as_posix()
|
||||
if not rel.startswith(_WORKFLOW_DIR):
|
||||
continue
|
||||
found = issues(rel, _load(path_str))
|
||||
if found:
|
||||
offenders.append(f"{rel}: {'; '.join(found)}")
|
||||
|
||||
if offenders:
|
||||
self.fail(
|
||||
f"{len(offenders)} workflow(s) break the prefix convention. A "
|
||||
f"workflow file MUST start with {_SCHEDULE!r} when it has a "
|
||||
f"`schedule` trigger (and then carry a {_CLOCK} name), with "
|
||||
f"{_REUSABLE!r} when it is only reachable through `workflow_call`, "
|
||||
f"and with {_ENTRY!r} otherwise:\n" + "\n".join(sorted(offenders))
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -224,8 +224,8 @@ class TestConfigFromTitle(unittest.TestCase):
|
||||
"""A retrigger has to reproduce the source run's configuration.
|
||||
|
||||
The REST API answers ``inputs: null`` for a finished workflow_dispatch, so
|
||||
the run title entry-manual.yml builds is the only record of it. Guessing
|
||||
wrong is silent: entry-manual.yml defaults to debian alone, so a failure
|
||||
the run title entry-manual-steer.yml builds is the only record of it. Guessing
|
||||
wrong is silent: entry-manual-steer.yml defaults to debian alone, so a failure
|
||||
that only reproduces on centos comes back green.
|
||||
"""
|
||||
|
||||
|
||||
@@ -63,7 +63,7 @@ class TestTriggerMain(unittest.TestCase):
|
||||
rc, calls = self._run([])
|
||||
self.assertEqual(rc, 0)
|
||||
self.assertEqual(
|
||||
calls, [("entry-manual.yml", "feature/x", "__ALL__", "", {}, "o/r")]
|
||||
calls, [("entry-manual-steer.yml", "feature/x", "__ALL__", "", {}, "o/r")]
|
||||
)
|
||||
|
||||
def test_apps_explicit_list(self) -> None:
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
# nocheck: workflow-references synthetic fixture workflows, not repository files
|
||||
from __future__ import annotations
|
||||
|
||||
import textwrap
|
||||
@@ -83,7 +84,7 @@ def _tree() -> TemporaryDirectory:
|
||||
"""
|
||||
jobs:
|
||||
orchestrate:
|
||||
uses: ./.github/workflows/ci-orchestrator.yml
|
||||
uses: ./.github/workflows/call-orchestrator.yml
|
||||
""",
|
||||
)
|
||||
_write(
|
||||
@@ -95,7 +96,7 @@ def _tree() -> TemporaryDirectory:
|
||||
release:
|
||||
uses: ./.github/workflows/installs.yml
|
||||
orchestrate:
|
||||
uses: ./.github/workflows/ci-orchestrator.yml
|
||||
uses: ./.github/workflows/call-orchestrator.yml
|
||||
""",
|
||||
)
|
||||
return tmp
|
||||
|
||||
@@ -17,9 +17,9 @@ from utils.roles.display import display_names
|
||||
WORKFLOWS = PROJECT_ROOT / ".github" / "workflows"
|
||||
|
||||
_DEPLOY = {
|
||||
"docker": ("test-deploy-compose.yml", "compose"),
|
||||
"swarm": ("test-deploy-swarm.yml", "swarm"),
|
||||
"host": ("test-deploy-host.yml", "host"),
|
||||
"docker": ("call-test-deploy-compose.yml", "compose"),
|
||||
"swarm": ("call-test-deploy-swarm.yml", "swarm"),
|
||||
"host": ("call-test-deploy-host.yml", "host"),
|
||||
}
|
||||
ORCHESTRATOR_PREFIX = {
|
||||
"docker": "🎶 Orchestrate CI / test-deploy-compose / ",
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Single point of truth for test fixtures, for the same reason
|
||||
``utils.github.run_name`` parses rather than hard-codes the format: a
|
||||
hand-typed title drifts the moment ``entry-manual.yml`` changes its
|
||||
hand-typed title drifts the moment ``entry-manual-steer.yml`` changes its
|
||||
``run-name``, and the retrigger that reads configuration back out of it
|
||||
would silently start recovering nothing.
|
||||
"""
|
||||
@@ -15,7 +15,7 @@ _GLYPH_OF = {value: glyph for glyph, value in run_name.VALUE_GLYPHS.items()}
|
||||
|
||||
|
||||
def render(config: dict[str, str]) -> str:
|
||||
"""The run name ``entry-manual.yml`` builds for *config*.
|
||||
"""The run name ``entry-manual-steer.yml`` builds for *config*.
|
||||
|
||||
Args:
|
||||
config: dispatch inputs that deviate from their workflow default,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Recover a manual CI run's inputs from its run name.
|
||||
|
||||
``entry-manual.yml`` declares a ``run-name:`` that interpolates the dispatch
|
||||
``entry-manual-steer.yml`` declares a ``run-name:`` that interpolates the dispatch
|
||||
inputs. Once a run has started, that title is the only record of them: the
|
||||
REST API answers ``inputs: null`` for a workflow_dispatch run. Anything that
|
||||
wants to know what a run was dispatched with therefore parses the title — and
|
||||
@@ -22,7 +22,7 @@ import re
|
||||
from utils.cache.files import PROJECT_ROOT
|
||||
from utils.cache.yaml import load_yaml
|
||||
|
||||
WORKFLOW = PROJECT_ROOT / ".github" / "workflows" / "entry-manual.yml"
|
||||
WORKFLOW = PROJECT_ROOT / ".github" / "workflows" / "entry-manual-steer.yml"
|
||||
|
||||
VALUE_GLYPHS = {"是": "true", "否": "false", "序": "serial", "并": "parallel"}
|
||||
|
||||
|
||||
Reference in new issue
Block a user