Files
computer-playbook/cli/administration/inventory/provision/credentials_generator.py
T
kevinveenbirkenbachandClaude Opus 5 7162b7bf4c Merge fork/feature/svc-net-tor into feature/ai-agent-employees
834 commits from the Tor onion-service branch against 299 of ours, from merge
base 7b646bdece: 127 conflicted files, 3506 files changed.

The two sides are largely orthogonal. Theirs adds svc-net-tor and a `tor` service
across the role tree, plus two sweeping refactors: shipped scripts sorted by
language (files/ruby/, files/python/, files/shell/, scripts/lint/*.sh) and the
unit suite moved under tests/unit/python/. Ours is requirement 036, the native AI
gateway, plus the MCP role work. Most conflicts were both sides appending to the
same service map, resolved as unions.

Resolutions that were not unions:

Credentials moved namespace. Theirs renamed meta/schema.yml to meta/secrets.yml
and re-addressed credentials as secrets.credentials.<name>. Git carried our
litellm_api_key blocks through the rename, but every consumer we added still read
'credentials.<name>', which no longer resolves. All 45 call sites are migrated.

plugins/filter/container_volumes.py is deleted by theirs and relocated to
plugins/lookup/container_volumes.py, where it carries the same tmpfs defect our
f85996b43 had just fixed: a hard-coded {"size": ...} that silently drops a per
mount `mode`. The fix is re-applied in the relocated file, as that commit's merge
note anticipated.

compose.yml keeps only INFINITO_PLAYWRIGHT_REPORTS_BASE_DIR. Theirs also forwards
INFINITO_PLAYWRIGHT_STAGE_BASE_DIR; that line is deliberately dropped, per the
analysis recorded in 45804e9ad -- STAGE has a drift-proof role-vars fallback,
while forwarding it promotes a setdefault-pinnable .env value above that SPOT.

tasks/stages/01_constructor.yml takes their include, ./tasks/utils/setup/group.yml,
because their refactor removed tasks/groups/*-roles.yml, leaving our include
target nonexistent. Our task name is kept, since svc-virt is in the loop.

Makefile swarm-zombie is a union: their validate plus teardown, then our
lab_subnet cleanup, which their side dropped while leaving lab_subnet.sh as the
only orphaned reference in the file.

01_bootstrap.sh keeps our deterministic --ip pinning, which distinguishes an
already-attached node from a real failure, over their `|| true`.

Their language-sorted layout orphaned references our side still held: zammad's
two scripts/apply/*.rb, jenkins' java-with-ca-trust.sh, and the notify topics
their handlers renamed to hyphenated form. All corrected.

roles/sys-svc-compose/files/shell/swarm/stack_ready.sh gains our latched-update
block: paused, rollback_paused, rollback_started and rollback_completed exit 2
rather than polling a state that cannot resolve without another deploy.

config_from_run is unioned to (title, logged, jobs); the production caller in
trigger/__main__.py now passes jobs, without which our suite suppression would
have been inert.

READMEs are regenerated rather than hand-merged: 15 LiteLLM rows and 91 Tor rows
now derive from the merged metadata, which is itself evidence the meta merge is
coherent.

Verified: 5161 unit tests pass. The single failure, test_query.py TestSortSpec,
also fails on fork/feature/svc-net-tor at b51d2f9919, measured in a baseline
worktree -- so this merge introduces no new unit-test failure. Two modules are
excluded from that run because `baudolo`, a dependency their branch introduces,
is not installed on this host.
Not verified: no deploy has run against the merged tree, and make quality has not
been run end to end here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 16:32:30 +02:00

159 lines
4.5 KiB
Python

from __future__ import annotations
import concurrent.futures
import subprocess
import sys
from typing import TYPE_CHECKING
from ruamel.yaml import YAML
from ruamel.yaml.comments import CommentedMap
from cli.administration.inventory.credentials.emit import (
credentials_map,
declared_credentials,
)
from .role_resolver import resolve_role_path
from .ruamel_io import dump_document, ensure_map, load_document
if TYPE_CHECKING:
from pathlib import Path
def _fatal(msg: str) -> None:
raise SystemExit(f"[FATAL] {msg}")
def _generate_credentials_snippet_for_app(
app_id: str,
roles_dir: Path,
host_vars_file: Path,
vault_password_file: Path,
project_root: Path,
env: dict[str, str] | None,
variant: int | None = None,
) -> CommentedMap | None:
role_path = resolve_role_path(app_id, roles_dir, project_root, env=env)
if role_path is None:
return None
cmd = [
sys.executable,
"-m",
"cli.administration.inventory.credentials",
"--role-path",
str(role_path),
"--inventory-file",
str(host_vars_file),
"--vault-password-file",
str(vault_password_file),
"--snippet",
"--allow-empty-plain",
]
if variant is not None:
cmd += ["--variant", str(variant)]
result = subprocess.run(cmd, text=True, capture_output=True, env=env, check=False)
if result.returncode != 0:
_fatal(
f"Command failed ({result.returncode}): {' '.join(cmd)}\n"
f"STDOUT:\n{result.stdout or ''}\nSTDERR:\n{result.stderr or ''}"
)
snippet_text = (result.stdout or "").strip()
if not snippet_text:
return None
yaml_rt = YAML(typ="rt")
try:
data = yaml_rt.load(snippet_text)
except Exception as exc:
raise SystemExit(
f"Failed to parse credentials snippet for {app_id}: {exc}"
) from exc
if data is None:
return None
if not isinstance(data, CommentedMap):
cm = CommentedMap()
for k, v in dict(data).items():
cm[k] = v
return cm
return data
def generate_credentials_for_roles(
application_ids: list[str],
roles_dir: Path,
host_vars_file: Path,
vault_password_file: Path,
project_root: Path,
env: dict[str, str] | None,
workers: int = 4,
app_variants: dict[str, int] | None = None,
) -> None:
if not application_ids:
return
max_workers = max(1, workers)
snippets: list[CommentedMap] = []
variants = app_variants or {}
with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
future_to_app: dict[concurrent.futures.Future, str] = {}
for app_id in application_ids:
future = executor.submit(
_generate_credentials_snippet_for_app,
app_id,
roles_dir,
host_vars_file,
vault_password_file,
project_root,
env,
variants.get(app_id),
)
future_to_app[future] = app_id
for future in concurrent.futures.as_completed(future_to_app):
app_id = future_to_app[future]
try:
snippet = future.result()
except Exception as exc:
_fatal(f"Worker for {app_id} failed with exception: {exc}")
if snippet is not None:
snippets.append(snippet)
if not snippets:
return
doc = load_document(host_vars_file)
for snippet in snippets:
apps_snip = snippet.get("applications", {}) or {}
if isinstance(apps_snip, dict):
apps_doc = ensure_map(doc, "applications")
for app_id, app_block_snip in apps_snip.items():
if not isinstance(app_block_snip, dict):
continue
app_doc = ensure_map(apps_doc, app_id)
creds_doc = credentials_map(app_doc)
creds_snip = declared_credentials(app_block_snip) or {}
if not isinstance(creds_snip, dict):
continue
for key, val in creds_snip.items():
if key not in creds_doc or (
creds_doc.get(key) == "" and val not in ("", None)
):
creds_doc[key] = val
if (
"ansible_become_password" in snippet
and "ansible_become_password" not in doc
):
doc["ansible_become_password"] = snippet["ansible_become_password"]
dump_document(host_vars_file, doc)