Commit Graph
25 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 5.5 9281a17195 ci(push): let a release push on main supersede the running run and deploy every chunk
A push to main whose head commit starts with "Release version " (the subject pkgmgr writes for every release) now cancels the main run in progress, so the release starts at once instead of queueing behind it. Every other push to main still leaves the running pipeline alone.

When the pushed main commit carries a vX.Y.Z tag, the orchestrator runs with chunk_gate false: every chunk deploys and reports, and a failed chunk still turns the run red, so release-guard and release-version stay skipped.

entry-cancel-superseded.yml mirrors the new rule behind its push guard, parenthesised so && does not bind the release clause; the mirror test composes the expected rule the same way.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 19:57:30 +02:00
Alejandro Roman 4b9a834546 Merge remote-tracking branch 'upstream/main' into feature/role-stalwart-mail-swarm 2026-09-21 03:25:20 +04:00
kevinveenbirkenbachandClaude Opus 5 d8c6bd242d test(ci): require every secret and variable a workflow reads to be documented
An unset secret or variable resolves to the empty string, not to an error, so
an undocumented one is invisible twice: the administrator cannot know to set it,
and the run that needed it degrades quietly instead of failing. The guard reads
every ${{ secrets.X }} and ${{ vars.X }} out of .github/workflows and demands
the name in the page that owns it, secrets.md or configuration.md.

It compares the reference against the page rather than the page against itself,
so it also catches a doc naming something the workflows no longer read.

Three gaps were already there. DOCKERHUB_USERNAME and DOCKERHUB_TOKEN
authenticate the image mirror against Docker Hub and lift the anonymous
pull-rate limit; both are declared required: false and the login step is gated
on them being non-empty, so their absence does not fail a run, it walks the
mirror into the rate limit. Neither was documented anywhere.

PLAYWRIGHT_KEEP_ALL is the sharper one: configuration.md listed
INFINITO_PLAYWRIGHT_KEEP, which is the name the workflow forwards it under, not
the name an administrator sets. Following the documentation produced a variable
nothing reads.

Commented-out lines are skipped. cron-security-scorecard.yml carries
`# repo_token: ${{ secrets.SCORECARD_TOKEN }}` as an option nobody has taken,
and demanding a doc row for it would describe an input no run asks for.

GITHUB_TOKEN is exempt in _PROVIDED_SECRETS because GitHub mints it per run; it
is not administered and cannot be set. A second case asserts the scan finds
workflow files at all, so the guard cannot pass vacuously.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 19:59:10 +02:00
kevinveenbirkenbachandClaude Opus 5 786899700f feat(litellm): serve OpenAI, Anthropic and OpenRouter from declared keys
The gateway could reach exactly one remote provider, through a bare
OPENROUTER_API_KEY group_var with an empty-string default and an `{% if %}`
around a single hardcoded model entry. OpenAI and Anthropic had no path at all.

All three are now declared in roles/svc-ai-litellm/meta/secrets.yml, which
mapping.py already calls the application config schema, so a key lands at
applications.svc-ai-litellm.secrets.credentials.<provider>_api_key like every
other credential. Each carries the `type` and `regex` its value must satisfy and
defaults to the central API.<provider>.api_key block, which keeps one
operator-facing place to fill a provider in while the role owns the shape.

type and regex had to become real to be worth declaring. 192 `validation:`
blocks sit in the role schemas today and not one line of Python enforces them,
so a `regex` would have joined a pile of decoration. validate_supplied_value
now runs on every path a value arrives from outside — the --set override, a
value already in the inventory, and the value standing next to a `default:`.
A generated value stays exempt because the algorithm that produced it is named
by the same schema and generate_value takes no schema argument. min_length is
deliberately left unenforced: the generators ignore it, so holding them to it
would fail where nothing is wrong.

The pattern is written for the paste, not for the vendor. OpenAI moved issuance
from sk- to sk-proj- without notice and Anthropic issues sk-ant-api03-; a schema
pinning a prefix would reject a valid key and abort inventory creation. `\S{20,}`
accepts every form those three issue today and one nobody has announced, and
rejects a trailing newline, an embedded space, a truncated copy and a
<placeholder>. A key pasted into the wrong provider's slot is caught at deploy
instead, where the route it publishes answers nothing.

The gateway iterates one LITELLM_REMOTE_BACKENDS list instead of three `{% if %}`
blocks; config.yaml.j2 and env.j2 read the same list and selectattr('key')
decides. LITELLM_CHAT_MODEL falls back to the first configured remote alias
rather than to a hardcoded openrouter/auto.

For CI the keys travel as INFINITO_API_{OPENAI,ANTHROPIC,OPENROUTER}_KEY through
default.env, the passthrough allowlist, the infinito service in compose.yml, the
workflow env block and the development inventory. The compose leg is not
optional: the env lint rejected the change without it, because a key the
inventory reads but compose does not forward resolves to empty inside the
container with no error. The workflow reads `secrets.X || vars.X` so a variable
works as asked while a secret wins, which matters because a variable is not
masked and the development inventory sets MASK_CREDENTIALS_IN_LOGS: false.

An unset key is not an error anywhere: the provider's routes stay unpublished
and the CLI test asserts their absence rather than skipping.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 19:57:21 +02:00
kevinveenbirkenbachandClaude Opus 5 e4353d9742 refactor(ci): derive the manual cancel override from the event
force_cancel_in_progress existed so a manual sweep could supersede a running orchestrator on any ref. It was a flag every caller had to remember to pass, and only one ever did.

A called workflow inherits github.event_name from its caller, so the orchestrator can see for itself that a human dispatched the run. The input is gone; the group reads the event instead. Automatic triggers keep following the ref alone -- everything but main -- and a manual dispatch supersedes everywhere, main included, because someone typed it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 16:29:11 +02:00
kevinveenbirkenbachandClaude Opus 5 653dc632e4 refactor(ci): key cancellation on the ref, not a variable
CI_CANCEL_IN_PROGRESS gated whether a new run cancels the one before it, and it had to be remembered per repository: unset upstream meant cancel, set to false meant do not, and a fork inherited neither. Cancellation now follows the ref. Every branch but main supersedes its own runs; main never does, because a half-finished pipeline on the default branch is worse than a queued one.

call-orchestrator.yml compares github.ref rather than github.ref_name: a called workflow inherits the caller's ref, which is refs/pull/<n>/merge for a pull_request caller. The manual entry workflow keeps overriding the rule through force_cancel_in_progress.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 16:29:10 +02:00
kevinveenbirkenbachandClaude Opus 5 450350de6b docs(ci): describe sweeps, chunks and priority coverage
Replaces the deploy-line sequencing section, which documented the
three-mode split, the serial/parallel decision and mode_fail_fast --
the only place any of that was written down.

Records what a run now does instead: why chunks exist at all (the queue
clock), how the chunk size and count are derived, what priority means
now that a named role is deployed in every combination it can take
rather than sampled, and how the mode and tor rotations are decoupled
for the regular rows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 06:09:05 +02:00
kevinveenbirkenbachandClaude Opus 5 7f81d26340 fix(lint): close the blank line the rerun doc left behind
3a1ef70cb removed the rerun workflow's rows from the actions configuration
page and left two blank lines at the end of the file, which MD012 rejects.
That is what failed the Lint job on 777f7f0fa.

The two Python files carry the reflow ruff 0.16.2 produces; 6320da703
brought that version in with the main merge. Committing it keeps the tree
matching the formatter, since scripts/lint/python.sh runs ruff format in
write mode and would otherwise rewrite them on every run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-12 11:08:49 +02:00
kevinveenbirkenbachandClaude Opus 5 3a1ef70cbb chore(ci): drop the cancelled-run rerun automation
The hourly sweep fought the rest of the CI control plane instead of
helping it: it preempted itself (17 of the last 29 runs concluded
cancelled 1-2s after their successor was created, for 8s of work behind
a 74-80min queue wait) and shipped MAX_ATTEMPTS=0, which the script
reads as no ceiling while its own README promised the opposite. The 24h
age cutoff could not bound it either, because every rerun refreshes the
run's updatedAt and so keeps it inside the window.

Removes the workflow, the two scripts it was the sole consumer of, its
unit test, and the CI_RERUN_* documentation that would otherwise point
at a deleted file. The workflow is disabled in both origin and fork so
no sweep fires before this lands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 20:20:20 +02:00
kevinveenbirkenbachandClaude Opus 5 c74b10177e docs(ci): name the rerun sweep by its current file
The link targets were updated with the rename, the link texts were not, so four places advertised a workflow file that no longer exists. No lint catches this: the path resolves, only the word beside it is wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 17:19:46 +02:00
kevinveenbirkenbachandClaude Opus 5 71888d86af refactor(ci): prefix workflows by trigger
The workflow directory mixed entry points, reusable stages and scheduled jobs into one flat list sorted by topic, so the first question about any file - what starts it - was the one the name did not answer. Every workflow now carries one of three prefixes, derived from its on: block: cron- for a schedule trigger, call- for one only reachable through workflow_call, entry- for everything a repository event or a human starts. Scheduled workflows additionally carry an alarm clock as the first character of their name.

Two lints keep it that way. One derives the expected prefix from the triggers and compares it against the file name; it flagged all 17 offenders before the rename and none after. The other resolves every .github/workflows/<file> string in the repository against the directory, which is what a rename silently breaks in Markdown links, Python constants and shell API URLs alike.

Renaming pulled 69 pre-existing comments into the comment lint's changed-file scope. The vendored CodeQL template got the nocheck marker its scorecard sibling already carried, two workflow headers moved above name: where they count as file headers, and the remaining banners were deleted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 17:05:51 +02:00
kevinveenbirkenbachandClaude Opus 5 2e3438cc99 feat(ci): rerun the failed jobs of cancelled runs
A run cancelled by a concurrency group loses the work it had already finished. An hourly sweep now walks every branch, and when its latest run ended cancelled it restarts only the failed jobs through gh run rerun --failed, keeping the successful ones.

CI_RERUN_INTERVAL_MINUTES, CI_RERUN_MAX_ATTEMPTS and CI_RERUN_MAX_AGE_HOURS tune the sweep. A cron expression cannot read a repository variable, so the workflow is triggered every full hour and a gate lets only the hours through that open the configured window; the gate reads the hour, not the minute, so a late start still counts for its hour. An unset attempt ceiling means no ceiling, bounded only by the 50 reruns GitHub allows per run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 17:05:51 +02:00
kevinveenbirkenbachandClaude Opus 5 c9de2e1a01 fix(ci): let manual runs cancel the orchestrator
The orchestrator is a reusable workflow whose single caller job carries
its own concurrency group, not the caller's. That group was pinned to
cancel-in-progress: false, so a newer run could never displace a running
one on the inner level and parked as pending with zero jobs instead.

Run 30390890384 hit exactly that: its run-level status had degraded to
queued while the matrix waited for runners, so the outer global-ci cancel
did not fire against it, and run 30391785197 blocked behind the inner
group indefinitely.

The orchestrator now takes a force_cancel_in_progress input and otherwise
follows CI_CANCEL_IN_PROGRESS like the push entry workflow does. The
manual entry workflow passes true, so it cancels on both levels and no
longer depends on the outer cancel firing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 23:53:42 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f08d71534a feat(tooling): operator alias, shortcut, skill, cheat and onboard system
Add the i8* terminal aliases (make install-alias) and ai8* agent
conversation shortcuts (docs/.../agents/alias.md), move skill
distribution to the external INFINITO_SKILLS_REPOSITORY (make
install-skills), and add make cheat, make alias and make onboard. Introduce
custom.env operator overrides (win over defaults and handlers) and pin
INFINITO_ALIAS_MD / INFINITO_ALIAS_REPOSITORY / INFINITO_SKILLS_REPOSITORY
in default.env. Drop the in-repo skills-lock.json and its update-skills CI
job. AGENTS.md instructs agents to suggest the matching cli/prompt alias
when an operator writes something an alias covers. Lints enforce alias
sorting, i8/ai8 prefixes, no consonant clusters, full make-target
coverage, and terminal/agent shortcut non-overlap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-10 18:43:35 +02:00
kevinveenbirkenbach 0798a21b2a Merge branch 'main' into feature/swarm-nfs 2026-06-27 21:52:19 +02:00
kevinveenbirkenbachandClaude Opus 4.8 f3fcaa8004 feat(ci): sync main from source repository before CI scope discovery
entry-push-latest.yml gains a sync-main job that force-pushes the current repository's main from a source repo (default infinito-nexus/core) before CI policy and deploy discovery derive scope from origin/main. The source is configurable and can be disabled via CI_SYNC_MAIN_SOURCE_REPOSITORY; the same-repository case and disabled values skip the sync.

The sync-main checkout uses fetch-depth: 0 and the push is --force to avoid shallow-update and non-fast-forward rejections. Docs warn that main is force-overwritten and must not be used as a working branch on synced repositories.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 18:32:29 +02:00
kevinveenbirkenbachandClaude Opus 4.8 8547a800ed refactor(ci): unify deploy tests to compose+swarm, retire INFINITO_DEPLOY_TYPE
Delete the per-deploy-type workflows (test-deploy-local/universal/
workstation) and the redundant act-all/act-app runner scripts (replaced
by make act-workflow). Only compose and swarm deploy tests remain.

Both now discover the full application matrix (every deployment type,
lifecycle-filtered to alpha/beta/rc/stable) instead of just the server/
web-* scope, via apps.sh resolving applications.type without --type.

Retire INFINITO_DEPLOY_TYPE entirely: discovery is now unconditional and
the deploy step never read it. Removed from apps.sh, ci/all.sh, the local
deploy scripts, the two workflows, default.env, the env passthrough list,
the dotenv generator test, the Makefile compose-deploy target, and docs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 18:31:35 +02:00
kevinveenbirkenbachandClaude Opus 4.7 16a4298dbb refactor(ci): rename test-deploy-server to test-deploy-compose
Use whale (🐳) for the per-app matrix job to mirror the swarm
workflow's bee (🐝) pattern, and magnifying glass (🔍) for the
discover step. Keep the workflow header on the existing 📦 prefix to
stay consistent with the other test-deploy-* siblings.

Rename the workflow file and align all references:
  - ci-orchestrator.yml job key + uses: path
  - artifact names: inventory-server-* -> inventory-compose-*,
                    playwright-server-* -> playwright-compose-*
  - scripts/github/inventory/zip_compose_for_upload.sh output path
  - 3 docs files

INFINITO_DEPLOY_TYPE: server stays - that key is wired to the
invokable-rule registry in utils/roles/validation/invokable.py and is
a separate refactor concern.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-05 18:45:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 29a01ff538 chore(workflows): migrate create-github-app-token from app-id to client-id
actions/create-github-app-token@v3 emits "Input 'app-id' has been
deprecated with message: Use 'client-id' instead." on every run.
Swap the input + rename the corresponding repository secret from
BOT_APP_ID to BOT_APP_CLIENT_ID so the value reflects what the
action actually expects: the OAuth-style Client ID (`Iv…` format
shown on the App's General page), not the numeric App ID.

Requires the operator to create a fresh `BOT_APP_CLIENT_ID`
repository secret on infinito-nexus/core before merging; the old
`BOT_APP_ID` secret becomes orphaned and SHOULD be deleted in the
same settings pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-28 12:47:59 +02:00
kevinveenbirkenbachandClaude Opus 4.7 a77a570409 refactor(env): consolidate INFINITO_* namespace and centralize path/image SPOTs
Closes a stack of env-pipeline regressions left over from the recent
INFINITO_* namespace refactor by:

* DOMAIN_PRIMARY now reads INFINITO_DOMAIN strictly (was: legacy DOMAIN
  env var → defaulted to infinito.localhost, which never resolved
  because CoreDNS serves the INFINITO_DOMAIN value, infinito.example).
* group_vars/all/08_networks.yml IPs (ip4/ip6/dns) now read
  INFINITO_IP4 / INFINITO_IP6 / INFINITO_DNS_IP.
* PLAYWRIGHT_KEEP_ALL renamed to INFINITO_PLAYWRIGHT_KEEP end-to-end
  (env-builder, role default, run_one.yml docker exec, playwright.config.js,
  lint allowlist, docs anchor, CI workflow vars).
* PKGMGR_IMAGE renamed to INFINITO_PARENT_IMAGE; Dockerfile ARG drops
  the inline default so env/default.env is the only SPOT.
* INFINITO_SRC_DIR centralized: Dockerfile ARG (no default) + ENV;
  compose forwards it as build-arg, volume mount path, and container env;
  every Python CLI + shell helper + CI workflow now reads it strictly
  (os.environ[...] / "${INFINITO_SRC_DIR:?...}") instead of the literal
  /opt/src/infinito.
* New entries in env/default.env: INFINITO_IP6=::1, INFINITO_PARENT_IMAGE,
  INFINITO_PLAYWRIGHT_KEEP=false. All three propagated through
  utils/env/handlers/passthrough.STATIC_KEYS and (where consumed in the
  container) through compose.yml's environment block.

Validated end-to-end with `make test` (2360 tests across lint, lint-*,
test-lint, test-external, test-unit, test-integration).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 02:30:34 +02:00
kevinveenbirkenbachandClaude Opus 4.7 0b5672d7d5 ci(update): mint GitHub App token so update PRs trigger full CI
Workflow-driven update PRs opened with GITHUB_TOKEN do not start
further workflow runs, so the CI orchestrator never executed against
them. Each PR-opening job in update.yml now mints a short-lived
GitHub App installation token via actions/create-github-app-token@v2
and forwards APP_SLUG to open_update_pr.sh, which derives the
verified-commit identity from the App user.

The job-level permissions block drops to read-only since write access
is provided exclusively by the App token. Behaviour with the secrets
unset is fail-loud: the token step fails, the PR step is skipped,
and no PR with an unintended identity is ever created.

Requires the BOT_APP_ID and BOT_APP_PRIVATE_KEY repository secrets
documented in docs/contributing/tools/github/actions/secrets.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 00:50:25 +02:00
kevinveenbirkenbachandClaude Opus 4.7 14700e0658 feat(test-e2e-playwright): add PLAYWRIGHT_KEEP_ALL artefact-retention toggle
Forwards `PLAYWRIGHT_KEEP_ALL` from host env / GitHub Actions repo var
through `TEST_E2E_PLAYWRIGHT_KEEP_ALL` (inventory-overridable) into the
container and flips trace / screenshot / video to `on` for every test
when truthy.

SPOT for the propagation chain lives in
`docs/contributing/actions/testing/playwright.md#artefact-retention-`;
CI Configuration and the agent iteration loops link to it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-10 22:06:31 +02:00
kevinveenbirkenbachandClaude Opus 4.7 7d7301614d docs(changelog): per-release archive CLI + release docs + SPOT cleanup
Add cli/contributing/changelog/archive: a CLI that trims CHANGELOG.md
to the last N entries (default 7) and writes every older release to
its own file under docs/changelog/ named <padded-semver>-<date>.md
(7.0.0 → 007.000.000-2026-05-08.md). The padded form makes a directory
listing already chronological, and the index in CHANGELOG.md is
rebuilt from the archive directory listing on every run, so it never
drifts away from what is on disk. The tool is idempotent and never
overwrites an existing archive file.

Run the migration once to split the 44-entry CHANGELOG.md into 7
inline entries plus 37 archived per-release files, with the index at
the bottom of CHANGELOG.md linking each archive in descending order.

Add docs/contributing/actions/release.md as the contributor-facing
release procedure (versioning, active window vs. archive, cutting a
release end-to-end, CI hand-off on tag push) and add a docs README
under docs/changelog/ that explains the archive directory layout and
filename schema. Cross-link release.md from pipeline.md and add
"releases" to the actions/ scope description.

Documentation cleanup pass on every page touched in the previous
mapping refactor:

- Remove "is the SPOT for X" / "Single Point of Truth" self-labels
  across docs/contributing/, docs/agents/, and docs/changelog/. The
  rule in documentation.md forbids self-declared SPOT status; status
  is conferred by readers cross-linking, not by the page asserting
  it. Rephrase to descriptive phrasing ("This page covers X",
  "describes X", "lists X") throughout.
- Drop the orphaned docs/contributing/tools/actions.md duplicate
  (101 lines paralleling tools/github/actions/workflows.md, with no
  inbound links).
- Fix em-dashes used as thought breaks in dockerfile.md, origin.md,
  agents/style.css.md, and suppression.md per documentation.md.
- Unwrap soft-wrapped sentences in iam/common.md, iam/ldap.md,
  iam/oidc.md, image/origin.md, requirements.md, and dockerfile.md
  so each logical line carries one full sentence.

Add a unit test for the archive CLI under
tests/unit/cli/contributing/changelog/archive/test_main.py covering
_pad_version, _split_into_entries, idempotency, dry-run, no-overwrite
of existing archives, and the multi-run index rebuild.

Quote `# noqa:` in CHANGELOG.md prose with backticks so the
noqa-only-ruff-codes lint does not parse "for" as a non-ruff code.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 16:19:42 +02:00
kevinveenbirkenbachandClaude Opus 4.7 3e30694ee3 ci(updates): gate auto-update PRs via repo var
Add CI_ENABLE_AUTO_UPDATES (default unset/false). Workflow update
jobs and Dependabot PRs are skipped/auto-closed unless the variable
is set to true, so update noise is opt-in.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-26 16:16:58 +02:00
kevinveenbirkenbachandClaude Opus 4.7 4f2dd95fe9 feat(ci): add CI_RUN_ON_MAIN gate, reorg actions docs
Add repository variable CI_RUN_ON_MAIN that gates whether pushes to
main trigger CI. When unset (default), pushes to main skip CI;
set to 'true' to enable. Pushes to other branches are unaffected.

Reorganize GitHub Actions docs into a dedicated tools/github/actions/
directory and consolidate every CI repository variable in a single
SPOT (configuration.md). Move actions.md to actions/workflows.md and
branch/configuration.md to actions/configuration.md, then update
every cross-reference. Trim README files to scope-only per
docs/contributing/artefact/files/docs/readme_md.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 16:31:52 +02:00