refactor(repo): clear all project-root-import lint violations + helper inlining

Eliminates the last 102 violations of test_project_root_import by
making PROJECT_ROOT in each package's __init__.py the single source
and inlining every wrapper helper.

Migrations
==========
- ~70 files: ``Path(__file__).resolve().parents[N]`` → ``from <pkg>
  import PROJECT_ROOT`` (single-expression shape, scripted).
- ~12 files: two-statement helpers (``T = Path(__file__).resolve();
  R = T.parents[N]``) and multi-line ``os.path.abspath(os.path.join(...
  os.pardir...))`` chains rewritten by a follow-up scripted pass.
- ~20 files migrated by hand for shapes the scripts couldn't safely
  rewrite (helper functions like ``_repo_root_from_test_file``,
  embedded ``os.pardir`` chains, etc.).

Helpers removed (inlined to PROJECT_ROOT)
=========================================
- All ``def repo_root_from_here() / _project_root_from_here()`` and
  same-shape variants in cli/ (``cli/deploy/development/{down,common,
  build,deploy}.py``, ``cli/meta/categories/invokable/command.py``,
  ``cli/meta/applications/resolution/{combined/repo_paths,
  dependencies/__main__,run_after/__main__}.py``).
- All zero-arg ``def <name>() -> Path: return PROJECT_ROOT`` helpers
  (utils/invokable.py, utils/roles/resource_validation.py, cli/meta/
  runtime/__init__.py, cli/meta/callorder/__main__.py, plus a dozen
  test files).
- 4 ``PROJECT_ROOT = PROJECT_ROOT`` self-assignments removed from
  test files where the class attribute became dead after migration.

Test-mock updates
=================
``patch.object(repo_paths, 'repo_root_from_here', return_value=ROOT)``
→ ``patch.object(repo_paths, 'PROJECT_ROOT', ROOT)`` (27 patches across
8 unit-test files); same shape for ``patch('utils.invokable._repo_root',
return_value=…)``.

Suppressions documented
=======================
5 files marked ``# nocheck: project-root-import`` for the catalog's
documented exemptions:
- 2 lints walking from a discovered glob match (<role>/templates/...)
  up to its role dir, NOT the repo root.
- ``cli/validate/inventory/__main__.py`` invoked as standalone
  subprocess (relative imports unavailable).
- ``plugins/filter/value_generator.py`` loaded by Ansible outside the
  project package (relative imports broken).
- ``roles/test-e2e-playwright/filter_plugins/discover_playwright_roles.py``.

Bugs surfaced and fixed in the same commit
==========================================
The migration uncovered two latent role-content violations that were
silently masked by buggy ``_repo_root()`` helpers in two integration
tests (the helpers walked upward looking for marker dirs and returned
``tests/integration/`` instead of the repo root, so the tests scanned
nothing under ``roles/`` for years):
- roles/web-app-minio: extracted 6-task OIDC block into 02_oidc.yml +
  ``include_tasks`` (the documented "Good" pattern from
  test_block_size; parsing cost is paid only when OIDC is enabled).
- roles/web-app-bluesky/tasks/01_core.yml: collapsed 3 sequential
  ``when: BLUESKY_WEB_ENABLED | bool`` tasks into one ``block:`` with
  the guard at block level (4 → 2 occurrences, under threshold).

Two semantic-bug fixes carried alongside
========================================
- cli/core/app.py: ``cli_dir`` was previously ``parents[1]`` (= cli/),
  not the repo root; rewritten to ``PROJECT_ROOT / "cli"`` to match
  the variable's intent.
- cli/make/__main__.py: pre-existing bug — ``repo_root`` was actually
  the ``cli/`` directory (so ``cli/Makefile`` was the search target,
  which never existed). Now correctly uses PROJECT_ROOT.
- scripts/lint/sort_claude_settings.sh: invokes utils/claude/
  settings_sort.py as a module (``python -m``) so its
  ``from . import PROJECT_ROOT`` resolves.

Verification
============
- ``ruff check .`` → clean
- ``tests.lint.repository.test_project_root_import`` → OK (102 → 0)
- 1621 unit tests → all pass
- Integration suite: 44 failures (baseline, pre-existing, none from
  this commit), 1 skipped

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
kevinveenbirkenbachandClaude Opus 4.7 committed 2026-05-07 16:38:41 +02:00
1 parent dbc1e925f7
commit 9f62f9bb0e
174 files changed
+636 -667

No files matched your search

+2
View File
@@ -1 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[2]
+3 -1
View File
@@ -16,6 +16,8 @@ from cli.core.help import (
)
from cli.core.run import RunConfig, open_log_file, run_command_once
from . import PROJECT_ROOT
@dataclass
class Flags:
@@ -109,7 +111,7 @@ def main() -> None:
flags = parse_flags(argv)
args = argv[1:]
cli_dir = Path(__file__).resolve().parents[1]
cli_dir = PROJECT_ROOT / "cli"
# sanity: cli_dir should contain __init__.py and __main__.py of dispatcher
# but we do not hard-fail here
+5 -1
View File
@@ -1,6 +1,10 @@
from __future__ import annotations
from .command import (
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
from .command import ( # noqa: E402
main,
remove_container,
run_in_container,
+2 -2
View File
@@ -7,8 +7,8 @@ import subprocess
import sys
import time
import uuid
from pathlib import Path
from typing import List, Tuple
from . import PROJECT_ROOT
INFINITO_SRC_DIR = "/opt/src/infinito"
@@ -153,7 +153,7 @@ def start_ci_container(
name = f"infinito-ci-{uuid.uuid4().hex[:8]}"
print(f">>> Starting CI container '{name}' (host Docker socket mode)...")
project_root = str(Path(__file__).resolve().parents[3])
project_root = str(PROJECT_ROOT)
subprocess.run(
[
"docker",
+5 -1
View File
@@ -1,5 +1,9 @@
from __future__ import annotations
from .cli import main
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
from .cli import main # noqa: E402
__all__ = ["main"]
+3 -2
View File
@@ -4,7 +4,8 @@ import argparse
import os
import subprocess
from .common import repo_root_from_here, resolve_distro
from .common import resolve_distro
from . import PROJECT_ROOT
def add_parser(sub: argparse._SubParsersAction) -> None:
@@ -35,7 +36,7 @@ def add_parser(sub: argparse._SubParsersAction) -> None:
def handler(args: argparse.Namespace) -> int:
repo_root = repo_root_from_here()
repo_root = PROJECT_ROOT
script = repo_root / "scripts" / "image" / "build.sh"
repository_name_script = (
repo_root / "scripts" / "meta" / "resolve" / "repository" / "name.sh"
+2 -6
View File
@@ -1,10 +1,10 @@
from __future__ import annotations
import os
from pathlib import Path
from typing import TYPE_CHECKING
from .deps import apps_with_deps, resolve_run_after
from . import PROJECT_ROOT
if TYPE_CHECKING:
from .compose import Compose
@@ -20,10 +20,6 @@ DEV_INVENTORY_VARS_FILE: str = os.environ.get(
VALID_DISTROS: tuple[str, ...] = ("arch", "debian", "ubuntu", "fedora", "centos")
def repo_root_from_here() -> Path:
return Path(__file__).resolve().parents[3]
def compose_file_args() -> list[str]:
"""Compose `-f` flags shared by up and down flows."""
from .profile import Profile
@@ -91,7 +87,7 @@ def make_compose() -> Compose:
distro = resolve_distro()
# Surface env-script gap here rather than as a cryptic compose error later.
resolve_container()
return Compose(repo_root=repo_root_from_here(), distro=distro)
return Compose(repo_root=PROJECT_ROOT, distro=distro)
def resolve_deploy_ids_for_app(compose: Compose, app_id: str) -> list[str]:
+2 -2
View File
@@ -13,10 +13,10 @@ from cli.create.inventory.services_disabler import (
from .common import (
make_compose,
repo_root_from_here,
resolve_container,
)
from .inventory import filter_plan_to_variant, plan_dev_inventory_matrix
from . import PROJECT_ROOT
def _env_variant() -> int | None:
@@ -115,7 +115,7 @@ def _purge_app_entities(*, container: str, app_ids: list[str]) -> None:
"""
if not app_ids:
return
repo_root = repo_root_from_here()
repo_root = PROJECT_ROOT
purge_script = (
repo_root / "scripts" / "tests" / "deploy" / "local" / "purge" / "entity.sh"
)
+2 -5
View File
@@ -8,10 +8,7 @@ from pathlib import Path
from .common import cache_env_overrides, compose_file_args, resolve_distro
from .profile import Profile
def _repo_root_from_here() -> Path:
return Path(__file__).resolve().parents[3]
from . import PROJECT_ROOT
CI_DOCKER_ROOT = Path("/mnt/docker")
@@ -82,5 +79,5 @@ def add_parser(sub: argparse._SubParsersAction) -> None:
def handler(args: argparse.Namespace) -> int:
down_stack(repo_root=_repo_root_from_here(), distro=resolve_distro())
down_stack(repo_root=PROJECT_ROOT, distro=resolve_distro())
return 0
@@ -1 +1,5 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
"""CLI entry point package for Docker image version updates."""
+2 -1
View File
@@ -5,6 +5,7 @@ import argparse
from pathlib import Path
from utils.docker.version_updater import apply_updates, find_outdated_updates
from . import PROJECT_ROOT
def main() -> int:
@@ -14,7 +15,7 @@ def main() -> int:
parser.add_argument(
"--repo-root",
type=Path,
default=Path(__file__).resolve().parents[3],
default=PROJECT_ROOT,
help="Repository root to scan and update",
)
parser.add_argument(
+7 -1
View File
@@ -1,6 +1,12 @@
from __future__ import annotations
from . import __main__ as _main
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
# `__main__` does `from . import PROJECT_ROOT` at import time, so the
# constant has to be defined first.
from . import __main__ as _main # noqa: E402
# Expose configuration that tests monkeypatch
ROLES_DIR = _main.ROLES_DIR
+3 -2
View File
@@ -11,9 +11,10 @@ import yaml
from utils.cache.yaml import dump_yaml, load_yaml_any
from pathlib import Path
from . import PROJECT_ROOT
# Directory containing roles; can be overridden by tests
MODULE_DIR = Path(__file__).resolve().parent
ROLES_DIR = (MODULE_DIR.parents[3] / "roles").resolve()
ROLES_DIR = (PROJECT_ROOT / "roles").resolve()
def process_role(role_dir: Path, prefix: str, preview: bool, overwrite: bool):
+7 -4
View File
@@ -8,10 +8,13 @@ We re-export the public API so existing imports keep working.
from __future__ import annotations
from . import __main__ as _main
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[2]
# `__main__` does `from . import PROJECT_ROOT` at import time, so the
# constant has to be defined first.
from . import __main__ as _main # noqa: E402
__all__ = getattr(_main, "__all__", [n for n in dir(_main) if not n.startswith("_")])
globals().update({name: getattr(_main, name) for name in __all__})
# Prefer explicit __all__ if the original module defined it.
__all__ = getattr(_main, "__all__", [n for n in dir(_main) if not n.startswith("_")])
+3 -3
View File
@@ -25,9 +25,9 @@ def main():
# Default to 'build' if no target is specified
make_args = args.targets or ["build"]
# Determine repository root (one level up from cli/)
script_dir = os.path.dirname(os.path.realpath(__file__))
repo_root = os.path.abspath(os.path.join(script_dir, os.pardir))
from . import PROJECT_ROOT
repo_root = str(PROJECT_ROOT)
# Check for Makefile
makefile_path = os.path.join(repo_root, "Makefile")
@@ -1 +1,5 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[5]
# Intentionally empty; executable via: python -m cli.meta.applications.combined_resolution
@@ -1,15 +1,11 @@
# cli/meta/applications/resolution/combined/repo_paths.py
from __future__ import annotations
from pathlib import Path
def repo_root_from_here() -> Path:
return Path(__file__).resolve().parents[5]
from . import PROJECT_ROOT
def roles_dir() -> Path:
return repo_root_from_here() / "roles"
return PROJECT_ROOT / "roles"
def role_dir(role_name: str) -> Path:
@@ -1 +1,5 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[5]
# Intentionally empty; module is executable via -m.
@@ -18,18 +18,15 @@ from pathlib import Path
from typing import Dict, List, Set
from utils.cache.yaml import load_yaml
from . import PROJECT_ROOT
class DependenciesResolutionError(RuntimeError):
"""Raised when dependencies resolution fails (e.g., cycle detected)."""
def repo_root_from_here() -> Path:
return Path(__file__).resolve().parents[5]
def roles_dir() -> Path:
return repo_root_from_here() / "roles"
return PROJECT_ROOT / "roles"
def role_meta_path(role_name: str) -> Path:
@@ -1 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[5]
@@ -25,17 +25,15 @@ from utils.roles.meta_lookup import (
get_role_run_after,
)
from . import PROJECT_ROOT
class RunAfterResolutionError(RuntimeError):
"""Raised when run_after resolution fails (e.g., cycle detected)."""
def repo_root_from_here() -> Path:
return Path(__file__).resolve().parents[5]
def roles_dir() -> Path:
return repo_root_from_here() / "roles"
return PROJECT_ROOT / "roles"
def load_run_after(role_name: str) -> List[str]:
@@ -1 +1,5 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[5]
# Intentionally empty; importable package.
@@ -12,7 +12,9 @@ from utils.service_registry import (
resolve_service_dependency_roles_from_config,
)
_ROLES_ROOT = Path(__file__).parents[5] / "roles"
from . import PROJECT_ROOT
_ROLES_ROOT = PROJECT_ROOT / "roles"
def _stable_dedup(items: Iterable[str]) -> List[str]:
+7 -4
View File
@@ -8,10 +8,13 @@ We re-export the public API so existing imports keep working.
from __future__ import annotations
from . import __main__ as _main
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[4]
# `__main__` does `from . import PROJECT_ROOT` at import time, so the
# constant has to be defined first.
from . import __main__ as _main # noqa: E402
__all__ = getattr(_main, "__all__", [n for n in dir(_main) if not n.startswith("_")])
globals().update({name: getattr(_main, name) for name in __all__})
# Prefer explicit __all__ if the original module defined it.
__all__ = getattr(_main, "__all__", [n for n in dir(_main) if not n.startswith("_")])
+3 -7
View File
@@ -14,6 +14,8 @@ import argparse
from utils.cache.yaml import load_yaml
from . import PROJECT_ROOT
def get_role(application_id, roles_path):
"""
@@ -55,13 +57,7 @@ def main():
parser.add_argument(
"-r",
"--roles-path",
default=os.path.join(
os.path.dirname(os.path.realpath(__file__)),
os.pardir,
os.pardir,
os.pardir,
"roles",
),
default=str(PROJECT_ROOT / "roles"),
help="Path to the roles directory (default: roles/ at project root)",
)
+2
View File
@@ -1 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
+2 -14
View File
@@ -35,25 +35,13 @@ from pathlib import Path
from typing import Any, Dict, List, Optional, Tuple
from utils.cache.yaml import load_yaml_any as _load_yaml_cached
from . import PROJECT_ROOT
def eprint(msg: str) -> None:
print(msg, file=sys.stderr)
def repo_root_from_this_script() -> Path:
"""
Compute repo root *only* from this script's absolute location.
Expected layout:
<repo_root>/cli/meta/callorder/__main__.py
Therefore:
repo_root = Path(__file__).resolve().parents[3]
"""
return Path(__file__).resolve().parents[3]
def list_group_files(groups_dir: Path) -> List[Path]:
"""
Return group YAML files under tasks/groups (skip .gitignore and non-yml).
@@ -169,7 +157,7 @@ def main(argv: Optional[List[str]] = None) -> int:
args = ap.parse_args(argv)
repo_root = repo_root_from_this_script()
repo_root = PROJECT_ROOT
groups = build_groups(repo_root)
global_order = flatten_callorder(groups)
+5 -1
View File
@@ -1,5 +1,9 @@
from __future__ import annotations
from .command import main
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[4]
from .command import main # noqa: E402
__all__ = ["main"]
+2 -9
View File
@@ -12,7 +12,6 @@ from __future__ import annotations
import argparse
import sys
from pathlib import Path
import yaml
@@ -24,13 +23,7 @@ from plugins.filter.invokable_paths import (
get_non_invokable_paths,
)
def _project_root_from_here() -> Path:
"""
Determine repo root by fixed parent depth.
command.py -> invokable -> categories -> meta -> cli -> repo_root
"""
return Path(__file__).resolve().parents[4]
from . import PROJECT_ROOT
def main() -> None:
@@ -69,7 +62,7 @@ def main() -> None:
args = parser.parse_args()
# Ensure repo root is importable (CLI use outside repo root)
repo_root = _project_root_from_here()
repo_root = PROJECT_ROOT
sys.path.insert(0, str(repo_root))
try:
+2 -14
View File
@@ -2,19 +2,7 @@ from __future__ import annotations
import os
from pathlib import Path
def get_project_root() -> Path:
"""
Return the project root based on the location of this file.
Assumes this file lives in:
<project-root>/cli/meta/runtime/__init__.py
Therefore:
project-root = parents[3]
"""
return Path(__file__).resolve().parents[3]
from . import PROJECT_ROOT
def detect_runtime(project_root: Path | None = None) -> str:
@@ -51,7 +39,7 @@ def detect_runtime(project_root: Path | None = None) -> str:
return "github"
# 4) local dev compose stack (project-root relative marker)
root = project_root or get_project_root()
root = project_root or PROJECT_ROOT
if (root / "env.development").exists():
return "dev"
+2 -2
View File
@@ -3,7 +3,7 @@ from __future__ import annotations
import argparse
import json
from . import detect_runtime, get_project_root
from . import PROJECT_ROOT, detect_runtime
def _build_parser() -> argparse.ArgumentParser:
@@ -31,7 +31,7 @@ def main() -> int:
parser = _build_parser()
args = parser.parse_args()
root = get_project_root()
root = PROJECT_ROOT
runtime = detect_runtime(project_root=root)
if args.json:
+7 -4
View File
@@ -8,10 +8,13 @@ We re-export the public API so existing imports keep working.
from __future__ import annotations
from . import __main__ as _main
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
# `__main__` does `from . import PROJECT_ROOT` at import time, so the
# constant has to be defined first.
from . import __main__ as _main # noqa: E402
__all__ = getattr(_main, "__all__", [n for n in dir(_main) if not n.startswith("_")])
globals().update({name: getattr(_main, name) for name in __all__})
# Prefer explicit __all__ if the original module defined it.
__all__ = getattr(_main, "__all__", [n for n in dir(_main) if not n.startswith("_")])
+10 -5
View File
@@ -4,10 +4,15 @@ import re
import sys
from pathlib import Path
# Ensure imports work when invoked as a script (subprocess without PYTHONPATH).
repo_root = Path(__file__).resolve().parents[3]
if str(repo_root) not in sys.path:
sys.path.insert(0, str(repo_root))
# This script is also invoked as a standalone subprocess
# (`python __main__.py …`), in which case ``from . import PROJECT_ROOT``
# would fail with ``attempted relative import with no known parent
# package``. Compute the repo root locally and prepend it to sys.path
# before any project-package import resolves.
# nocheck: project-root-import sys.path bootstrap before package imports resolve
PROJECT_ROOT = Path(__file__).resolve().parents[3]
if str(PROJECT_ROOT) not in sys.path:
sys.path.insert(0, str(PROJECT_ROOT))
from utils.cache.applications import get_application_defaults # noqa: E402
from utils.cache.files import read_text # noqa: E402
@@ -121,7 +126,7 @@ def main():
p.add_argument("inventory_dir")
p.add_argument(
"--roles-dir",
default=str(repo_root / "roles"),
default=str(PROJECT_ROOT / "roles"),
help="Path to the repository roles directory.",
)
args = p.parse_args()
+2
View File
@@ -1 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[2]
+2 -1
View File
@@ -3,8 +3,8 @@
from __future__ import annotations
from functools import lru_cache
from pathlib import Path
import sys
from pathlib import Path
from typing import Type
@@ -17,6 +17,7 @@ def _value_generator_cls() -> Type[object]:
- utils/manager/__init__.py exists (package import works)
- plugins/filter/ is located at repo root
"""
# nocheck: project-root-import ansible loads filter plugins outside the project package, so relative imports don't resolve
repo_root = Path(__file__).resolve().parents[2]
utils_path = repo_root / "utils"
+3 -2
View File
@@ -376,8 +376,9 @@ def _discover_profiles_from_files(compose_files: List[Path]) -> List[str]:
die(f"Compose file does not exist: {f}")
try:
doc = (
yaml.safe_load(f.read_text(encoding="utf-8")) or {}
) # nocheck: direct-yaml
yaml.safe_load(f.read_text(encoding="utf-8")) # nocheck: direct-yaml
or {}
)
except Exception as e:
die(f"Failed to parse compose file '{f}': {e}")
@@ -39,6 +39,7 @@ def discover_playwright_roles(
# Current marker for Playwright-enabled app roles:
# .../roles/<role>/templates/playwright.env.j2
for env_file in base.rglob("templates/playwright.env.j2"):
# nocheck: project-root-import walking from a discovered glob match (<role>/templates/...) up to its role dir, not the repo root
role_name = env_file.parents[1].name
found.append(role_name)
+11 -11
View File
@@ -6,19 +6,19 @@
# handlers, otherwise `compose build` fires against a missing build context.
# Pre-create the services directory and clone the repo here, before any
# compose-touching include.
- name: "Ensure social-app services directory exists"
ansible.builtin.file:
path: "{{ BLUESKY_SOCIAL_APP_DIR | dirname }}"
state: directory
mode: "0755"
when: BLUESKY_WEB_ENABLED | bool
- name: "Pre-compose web-enabled staging (services dir + social-app + login-broker)"
block:
- name: "Ensure social-app services directory exists"
ansible.builtin.file:
path: "{{ BLUESKY_SOCIAL_APP_DIR | dirname }}"
state: directory
mode: "0755"
- name: "Clone social-app build context (pre-compose)"
ansible.builtin.include_tasks: "03_social_app.yml"
when: BLUESKY_WEB_ENABLED | bool
- name: "Clone social-app build context (pre-compose)"
ansible.builtin.include_tasks: "03_social_app.yml"
- name: "Stage login-broker build context (pre-compose)"
ansible.builtin.include_tasks: "06_login_broker.yml"
- name: "Stage login-broker build context (pre-compose)"
ansible.builtin.include_tasks: "06_login_broker.yml"
when: BLUESKY_WEB_ENABLED | bool
# Bind-mount safety: if a previous role-iteration ran without
+170
View File
@@ -0,0 +1,170 @@
---
# mc URL-embedded credentials (`MC_HOST_<alias>=https://USER:PASS@host`)
# are NOT URL-decoded by mc when read from the environment, while the
# MinIO server stores the URL-decoded password. With administrator
# credentials containing $ / € / ! / ' the two sides disagree and every
# admin call fails with "request signature we calculated does not
# match". Authenticate via `mc alias set` with explicit args instead;
# mc applies the credentials verbatim, matching the server.
# MinIO finishes booting its IAM subsystem (where policies and the
# OIDC IdP live) AFTER `mc admin info` already reports the server
# as up. Probing IAM directly via `mc admin policy ls` is the only
# readiness signal that matches what the policy admin API needs;
# `mc admin info` returns rc=0 in a window where the very next
# `mc admin policy info` still fails with `Server not initialized
# yet, please try again.`.
- name: "Wait for MinIO IAM subsystem to finish initializing"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin policy ls minio'
args: { executable: /bin/bash }
register: mc_admin_policy_ls
retries: 60
delay: 5
until: mc_admin_policy_ls.rc == 0
changed_when: false
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
# Register the OIDC provider under the explicit name `keycloak`
# via `mc idp openid add`. The Console's login page only renders
# the SSO button for NAMED providers (its
# `redirectRules` field is built from named entries only); the
# legacy `MINIO_IDENTITY_OPENID_*` env-var form registers the
# provider as nameless `(default)` and the SSO button never
# appears, even when the server-side IdP is fully functional.
- name: "Register Keycloak as named OIDC IdP on MinIO"
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
# All `mc idp openid add` parameters are passed to the inner sh
# via env vars rather than inline-quoted args. The display name
# contains parentheses (e.g. `SSO Login (INFINITO.EXAMPLE)`) and
# any inline shell-quoting at this nesting depth (Ansible shell
# → container run -c → inner sh) lets bash interpret `(` as a
# command-grouping start. Env-passthrough side-steps that
# entirely.
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
MINIO_OIDC_CONFIG_URL: "{{ OIDC.CLIENT.DISCOVERY_DOCUMENT }}"
MINIO_OIDC_CLIENT_ID: "{{ OIDC.CLIENT.ID }}"
MINIO_OIDC_CLIENT_SECRET: "{{ OIDC.CLIENT.SECRET }}"
MINIO_OIDC_DISPLAY_NAME: "{{ OIDC.BUTTON_TEXT }}"
MINIO_OIDC_CLAIM_NAME: "{{ RBAC.GROUP.CLAIM }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
-e MINIO_OIDC_CONFIG_URL \
-e MINIO_OIDC_CLIENT_ID \
-e MINIO_OIDC_CLIENT_SECRET \
-e MINIO_OIDC_DISPLAY_NAME \
-e MINIO_OIDC_CLAIM_NAME \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc idp openid add minio keycloak \
"config_url=$MINIO_OIDC_CONFIG_URL" \
"client_id=$MINIO_OIDC_CLIENT_ID" \
"client_secret=$MINIO_OIDC_CLIENT_SECRET" \
"display_name=$MINIO_OIDC_DISPLAY_NAME" \
"claim_name=$MINIO_OIDC_CLAIM_NAME" \
scopes=openid,profile,email,groups \
redirect_uri_dynamic=on'
args: { executable: /bin/bash }
register: mc_idp_add
changed_when: '"Successfully" in (mc_idp_add.stdout | default("")) or "Successfully" in (mc_idp_add.stderr | default(""))'
failed_when:
- mc_idp_add.rc != 0
- '"already exists" not in (mc_idp_add.stderr | default(""))'
- name: "Restart MinIO so the named OIDC IdP is picked up by the Console"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin service restart minio --json'
args: { executable: /bin/bash }
register: mc_admin_service_restart
changed_when: mc_admin_service_restart.rc == 0
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
when: mc_idp_add.changed | bool
- name: "Wait for MinIO IAM subsystem to come back after restart"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin policy ls minio'
args: { executable: /bin/bash }
register: mc_admin_policy_ls_after_restart
retries: 60
delay: 5
until: mc_admin_policy_ls_after_restart.rc == 0
changed_when: false
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
when: mc_idp_add.changed | bool
- name: "Check policy (RAW with slash) exists"
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin policy info minio {{ MINIO_OIDC_POLICY_NAME | quote }}'
args: { executable: /bin/bash }
register: mc_policy_info_raw
failed_when: false
changed_when: false
- name: "Create policy (RAW with slash) if missing"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
printf '%s' '{{ (MINIO_OIDC_POLICY_CONTENT | from_yaml | to_json) | b64encode }}' \
| base64 -d \
| container run --rm -i \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin policy create minio {{ MINIO_OIDC_POLICY_NAME | quote }} /dev/stdin'
args: { executable: /bin/bash }
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
async: "{{ ASYNC_TIME if ASYNC_ENABLED | bool else omit }}"
poll: "{{ ASYNC_POLL if ASYNC_ENABLED | bool else omit }}"
when:
- mc_policy_info_raw.rc != 0
+2 -171
View File
@@ -17,177 +17,8 @@
loop_var: minio_proxy
label: "{{ minio_proxy.domain }} -> {{ minio_proxy.http_port }}"
- block:
# mc URL-embedded credentials (`MC_HOST_<alias>=https://USER:PASS@host`)
# are NOT URL-decoded by mc when read from the environment, while the
# MinIO server stores the URL-decoded password. With administrator
# credentials containing $ / € / ! / ' the two sides disagree and every
# admin call fails with "request signature we calculated does not
# match". Authenticate via `mc alias set` with explicit args instead;
# mc applies the credentials verbatim, matching the server.
# MinIO finishes booting its IAM subsystem (where policies and the
# OIDC IdP live) AFTER `mc admin info` already reports the server
# as up. Probing IAM directly via `mc admin policy ls` is the only
# readiness signal that matches what the policy admin API needs;
# `mc admin info` returns rc=0 in a window where the very next
# `mc admin policy info` still fails with `Server not initialized
# yet, please try again.`.
- name: "Wait for MinIO IAM subsystem to finish initializing"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin policy ls minio'
args: { executable: /bin/bash }
register: mc_admin_policy_ls
retries: 60
delay: 5
until: mc_admin_policy_ls.rc == 0
changed_when: false
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
# Register the OIDC provider under the explicit name `keycloak`
# via `mc idp openid add`. The Console's login page only renders
# the SSO button for NAMED providers (its
# `redirectRules` field is built from named entries only); the
# legacy `MINIO_IDENTITY_OPENID_*` env-var form registers the
# provider as nameless `(default)` and the SSO button never
# appears, even when the server-side IdP is fully functional.
- name: "Register Keycloak as named OIDC IdP on MinIO"
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
# All `mc idp openid add` parameters are passed to the inner sh
# via env vars rather than inline-quoted args. The display name
# contains parentheses (e.g. `SSO Login (INFINITO.EXAMPLE)`) and
# any inline shell-quoting at this nesting depth (Ansible shell
# → container run -c → inner sh) lets bash interpret `(` as a
# command-grouping start. Env-passthrough side-steps that
# entirely.
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
MINIO_OIDC_CONFIG_URL: "{{ OIDC.CLIENT.DISCOVERY_DOCUMENT }}"
MINIO_OIDC_CLIENT_ID: "{{ OIDC.CLIENT.ID }}"
MINIO_OIDC_CLIENT_SECRET: "{{ OIDC.CLIENT.SECRET }}"
MINIO_OIDC_DISPLAY_NAME: "{{ OIDC.BUTTON_TEXT }}"
MINIO_OIDC_CLAIM_NAME: "{{ RBAC.GROUP.CLAIM }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
-e MINIO_OIDC_CONFIG_URL \
-e MINIO_OIDC_CLIENT_ID \
-e MINIO_OIDC_CLIENT_SECRET \
-e MINIO_OIDC_DISPLAY_NAME \
-e MINIO_OIDC_CLAIM_NAME \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc idp openid add minio keycloak \
"config_url=$MINIO_OIDC_CONFIG_URL" \
"client_id=$MINIO_OIDC_CLIENT_ID" \
"client_secret=$MINIO_OIDC_CLIENT_SECRET" \
"display_name=$MINIO_OIDC_DISPLAY_NAME" \
"claim_name=$MINIO_OIDC_CLAIM_NAME" \
scopes=openid,profile,email,groups \
redirect_uri_dynamic=on'
args: { executable: /bin/bash }
register: mc_idp_add
changed_when: '"Successfully" in (mc_idp_add.stdout | default("")) or "Successfully" in (mc_idp_add.stderr | default(""))'
failed_when:
- mc_idp_add.rc != 0
- '"already exists" not in (mc_idp_add.stderr | default(""))'
- name: "Restart MinIO so the named OIDC IdP is picked up by the Console"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin service restart minio --json'
args: { executable: /bin/bash }
register: mc_admin_service_restart
changed_when: mc_admin_service_restart.rc == 0
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
when: mc_idp_add.changed | bool
- name: "Wait for MinIO IAM subsystem to come back after restart"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin policy ls minio'
args: { executable: /bin/bash }
register: mc_admin_policy_ls_after_restart
retries: 60
delay: 5
until: mc_admin_policy_ls_after_restart.rc == 0
changed_when: false
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
when: mc_idp_add.changed | bool
- name: "Check policy (RAW with slash) exists"
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
container run --rm \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin policy info minio {{ MINIO_OIDC_POLICY_NAME | quote }}'
args: { executable: /bin/bash }
register: mc_policy_info_raw
failed_when: false
changed_when: false
- name: "Create policy (RAW with slash) if missing"
environment:
MINIO_ROOT_USER: "{{ lookup('users', 'administrator').username }}"
MINIO_ROOT_PASSWORD: "{{ lookup('users', 'administrator').password }}"
shell: |
set -euo pipefail
printf '%s' '{{ (MINIO_OIDC_POLICY_CONTENT | from_yaml | to_json) | b64encode }}' \
| base64 -d \
| container run --rm -i \
-e MINIO_ROOT_USER \
-e MINIO_ROOT_PASSWORD \
--entrypoint /bin/sh \
{{ MINIO_MC_IMAGE }} \
-c 'mc alias set minio {{ MINIO_API_URL | quote }} "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null \
&& mc admin policy create minio {{ MINIO_OIDC_POLICY_NAME | quote }} /dev/stdin'
args: { executable: /bin/bash }
no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}"
async: "{{ ASYNC_TIME if ASYNC_ENABLED | bool else omit }}"
poll: "{{ ASYNC_POLL if ASYNC_ENABLED | bool else omit }}"
when:
- mc_policy_info_raw.rc != 0
- name: "Configure MinIO OIDC (Keycloak as named IdP + policy)"
include_tasks: 02_oidc.yml
when: MINIO_OIDC_ENABLED | bool
# When LDAP is the active auth backend (variant 1), MinIO STS rejects
+5 -1
View File
@@ -8,4 +8,8 @@ set -euo pipefail
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd)"
exec python3 "${REPO_ROOT}/utils/claude/settings_sort.py" "$@"
# Invoke as a module (`python -m`) so the script's `from . import
# PROJECT_ROOT` resolves against the real package, not as a standalone
# script with no parent.
cd -- "${REPO_ROOT}"
exec python3 -m utils.claude.settings_sort "$@"
+2
View File
@@ -1 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
+3 -2
View File
@@ -28,7 +28,6 @@ but any non-comment line resets the search):
from __future__ import annotations
import unittest
from pathlib import Path
from utils.annotations.message import warning
from utils.docker.version_updater import (
@@ -46,7 +45,9 @@ from utils.docker.version_updater import (
from utils.docker.image.discovery import iter_role_images
_REPO_ROOT = Path(__file__).resolve().parents[3]
from . import PROJECT_ROOT
_REPO_ROOT = PROJECT_ROOT
_ROLES_ROOT = _REPO_ROOT / "roles"
+2
View File
@@ -1 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
+2 -1
View File
@@ -33,8 +33,9 @@ import requests
from utils.annotations.message import error, warning
from utils.annotations.suppress import is_suppressed_at
from . import PROJECT_ROOT
_REPO_ROOT = Path(__file__).resolve().parents[3]
_REPO_ROOT = PROJECT_ROOT
_URL_RE = re.compile(r"https?://[^\s<>'\"`\]]+")
_TEMPLATE_MARKERS = ("${", "{{", "}}", "{%", "%}")
_PUBLIC_HOST_RE = re.compile(r"^[A-Za-z0-9.-]+$")
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
@@ -6,6 +6,7 @@ import threading
import unittest
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from . import PROJECT_ROOT
class TestGetUrlRetryActionPluginIntegration(unittest.TestCase):
@@ -43,7 +44,7 @@ class TestGetUrlRetryActionPluginIntegration(unittest.TestCase):
return server, thread, counters, base_url
def _run_get_url_retry_playbook_with_default_and_override_blocks(self):
repo_root = Path(__file__).resolve().parents[3]
repo_root = PROJECT_ROOT
server = None
thread = None
@@ -6,6 +6,7 @@ import threading
import unittest
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from . import PROJECT_ROOT
class TestUriRetryActionPluginIntegration(unittest.TestCase):
@@ -43,7 +44,7 @@ class TestUriRetryActionPluginIntegration(unittest.TestCase):
return server, thread, counters, base_url
def _run_uri_retry_playbook_with_default_and_override_blocks(self):
repo_root = Path(__file__).resolve().parents[3]
repo_root = PROJECT_ROOT
server = None
thread = None
@@ -2,6 +2,7 @@ import re
import unittest
from pathlib import Path
from . import PROJECT_ROOT
BUILD_LOOKUP_RE = re.compile(
r"""\{\{\s*lookup\(\s*['"]template['"]\s*,\s*['"]roles/sys-svc-container/templates/build\.yml\.j2['"]\s*\)\s*\|\s*indent\(\s*4\s*\)\s*\}\}"""
@@ -29,10 +30,6 @@ class TestComposeBuildTemplateRequiresImageTag(unittest.TestCase):
- a YAML merge key `<<:` on the same indentation level (since it may inject `image:`).
"""
@staticmethod
def _repo_root() -> Path:
return Path(__file__).resolve().parents[3]
@staticmethod
def _iter_compose_templates(repo_root: Path):
roles_dir = repo_root / "roles"
@@ -81,7 +78,7 @@ class TestComposeBuildTemplateRequiresImageTag(unittest.TestCase):
return None
def test_build_lookup_requires_image_or_merge_on_same_level(self):
repo_root = self._repo_root()
repo_root = PROJECT_ROOT
compose_files = self._iter_compose_templates(repo_root)
self.assertTrue(
@@ -20,6 +20,8 @@ from dataclasses import dataclass
from pathlib import Path
from typing import List, Tuple
from . import PROJECT_ROOT
BUILD_RE = re.compile(r"^(?P<indent>[ \t]*)build:\s*(#.*)?$")
IMAGE_RE = re.compile(r"^(?P<indent>[ \t]*)image:\s*(?P<rest>.+)?$")
@@ -35,16 +37,6 @@ class Finding:
note: str
def _repo_root_from_test_file(test_file: str) -> Path:
"""
Compute repo root from this test file location.
tests/integration/<cluster>/<this_file> -> repo root is parents[3]
"""
p = Path(test_file).resolve()
# .../tests/integration/<cluster>/test_*.py
return p.parents[3]
def _is_ignored_line(line: str) -> bool:
return bool(BLANK_RE.match(line) or COMMENTED_RE.match(line))
@@ -156,8 +148,7 @@ def _scan_file_for_missing_image(path: Path) -> List[Finding]:
class TestComposeBuildRequiresImage(unittest.TestCase):
def test_all_compose_templates_and_files_have_image_for_build(self) -> None:
repo_root = _repo_root_from_test_file(__file__)
roles_dir = repo_root / "roles"
roles_dir = PROJECT_ROOT / "roles"
patterns = [
"*/templates/compose.yml.j2",
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
@@ -1,7 +1,7 @@
import unittest
import yaml
from pathlib import Path
from urllib.parse import urlparse
from . import PROJECT_ROOT
class TestCspConfigurationConsistency(unittest.TestCase):
@@ -59,7 +59,7 @@ class TestCspConfigurationConsistency(unittest.TestCase):
return parsed.scheme in ("http", "https", "ws", "wss") and bool(parsed.netloc)
def test_csp_configuration_structure(self):
roles_dir = Path(__file__).resolve().parents[3] / "roles"
roles_dir = PROJECT_ROOT / "roles"
errors = []
for role_path in sorted(roles_dir.iterdir()):
+3 -7
View File
@@ -3,6 +3,8 @@ import glob
import yaml
import unittest
from . import PROJECT_ROOT
def find_none_values(data, prefix=None):
"""
@@ -38,13 +40,7 @@ class TestConfigurationNoNone(unittest.TestCase):
# volumes.yml). The legacy roles/*/config/main.yml file no longer
# exists. Recurse into every meta/*.yml file and assert no key
# resolves to a YAML null.
roles_root = os.path.join(
os.path.dirname(__file__),
os.pardir,
os.pardir,
os.pardir,
"roles",
)
roles_root = str(PROJECT_ROOT / "roles")
pattern = os.path.join(roles_root, "*", "meta", "*.yml")
files = glob.glob(pattern)
self.assertTrue(
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
@@ -1,11 +1,11 @@
import re
import unittest
from pathlib import Path
from . import PROJECT_ROOT
class TestDockerComposeTemplates(unittest.TestCase):
# Search for all roles/*/templates/compose.yml.j2
PROJECT_ROOT = Path(__file__).resolve().parents[3]
TEMPLATE_PATTERN = "roles/*/templates/compose.yml.j2"
# Allowed lines before BASE_INCLUDE
@@ -28,7 +28,7 @@ class TestDockerComposeTemplates(unittest.TestCase):
3. BASE_INCLUDE appears before NET_INCLUDE when both are required
4. Only allowed lines appear before BASE_INCLUDE (invalid lines issue warnings)
"""
template_paths = sorted(self.PROJECT_ROOT.glob(self.TEMPLATE_PATTERN))
template_paths = sorted(PROJECT_ROOT.glob(self.TEMPLATE_PATTERN))
self.assertTrue(
template_paths, f"No templates found for pattern {self.TEMPLATE_PATTERN}"
)
@@ -8,6 +8,7 @@ from typing import Any, Dict, Iterable, List, Tuple
import yaml
from utils.docker.image.ref import is_valid_image_name
from . import PROJECT_ROOT
# Docker tag (version)
TAG_RE = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9_.-]{0,127}$")
@@ -65,7 +66,7 @@ class TestDockerServicesImageVersionValid(unittest.TestCase):
- `version` is OPTIONAL → only validated if present
- No coupling required between image/version
"""
repo_root = Path(__file__).resolve().parents[3]
repo_root = PROJECT_ROOT
failures: List[str] = []
config_files = _iter_role_config_files(repo_root)
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
+2 -2
View File
@@ -1,8 +1,8 @@
import unittest
from pathlib import Path
from collections import defaultdict
from utils.cache.applications import get_application_defaults
from . import PROJECT_ROOT
class TestDomainUniqueness(unittest.TestCase):
@@ -12,7 +12,7 @@ class TestDomainUniqueness(unittest.TestCase):
collect all entries under domains.canonical and domains.aliases across all applications,
and assert that no domain appears more than once.
"""
repo_root = Path(__file__).resolve().parents[3]
repo_root = PROJECT_ROOT
apps = get_application_defaults(roles_dir=repo_root / "roles")
domain_to_apps = defaultdict(set)
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
@@ -4,12 +4,13 @@ import subprocess
import tempfile
import unittest
from pathlib import Path
from . import PROJECT_ROOT
class TestStrongPasswordFilterPathIntegration(unittest.TestCase):
@unittest.skipUnless(shutil.which("ansible"), "ansible not found")
def test_strong_password_filter_uses_repo_relative_utils_path(self):
repo_root = Path(__file__).resolve().parents[3]
repo_root = PROJECT_ROOT
source_plugin = repo_root / "plugins" / "filter" / "value_generator.py"
with tempfile.TemporaryDirectory() as tmpdir:
+2
View File
@@ -1 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
@@ -1,11 +1,11 @@
import re
import unittest
import yaml
from pathlib import Path
from collections import defaultdict
from . import PROJECT_ROOT
# Directory containing group_vars/all/*.yml
GROUPVARS_DIR = Path(__file__).resolve().parents[3] / "group_vars" / "all"
GROUPVARS_DIR = PROJECT_ROOT / "group_vars" / "all"
JINJA_RE = re.compile(r"{{\s*([^}]+)\s*}}")
# Matches variables like foo.bar, foo["bar"], foo['bar']
VAR_PATTERN = re.compile(
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
+6 -6
View File
@@ -1,6 +1,7 @@
import re
import unittest
from pathlib import Path
from . import PROJECT_ROOT
class TestJinjaIncludePaths(unittest.TestCase):
@@ -11,12 +12,11 @@ class TestJinjaIncludePaths(unittest.TestCase):
- Includes are resolved relative to project root, template directory, or file's parent.
"""
PROJECT_ROOT = Path(__file__).resolve().parents[3]
INCLUDE_STMT_RE = re.compile(r"{%\s*include\s+(.+?)\s*%}")
LITERAL_PATH_RE = re.compile(r"^[\'\"]([^\'\"]+)[\'\"]$")
def test_all_jinja_includes_exist(self):
template_paths = list(self.PROJECT_ROOT.glob("**/*.j2"))
template_paths = list(PROJECT_ROOT.glob("**/*.j2"))
self.assertTrue(
template_paths, "No .j2 templates found anywhere in the project"
)
@@ -32,7 +32,7 @@ class TestJinjaIncludePaths(unittest.TestCase):
include_path = m.group(1)
# check absolute project-relative path
abs_target = self.PROJECT_ROOT / include_path
abs_target = PROJECT_ROOT / include_path
# check path relative to the template file
rel_target = tpl.parent / include_path
@@ -51,10 +51,10 @@ class TestJinjaIncludePaths(unittest.TestCase):
or rel_target.exists()
or (role_target and role_target.exists())
):
rel_tpl = tpl.relative_to(self.PROJECT_ROOT)
rel_tpl = tpl.relative_to(PROJECT_ROOT)
missing.append(
f"{rel_tpl}: included file '{include_path}' not found "
f"(checked in project root, {tpl.parent.relative_to(self.PROJECT_ROOT)} or templates folder)"
f"(checked in project root, {tpl.parent.relative_to(PROJECT_ROOT)} or templates folder)"
)
if missing:
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[4]
+2 -1
View File
@@ -31,6 +31,7 @@ from utils.cache.applications import get_application_defaults
from utils.cache.files import iter_project_files_with_content
from utils.cache.users import get_user_defaults
from utils.cache.yaml import load_yaml_any
from . import PROJECT_ROOT
# Rule key consumed by every `tests/integration/lookups/config/test_*.py`.
@@ -261,7 +262,7 @@ _SCANNED_EXTENSIONS: Tuple[str, ...] = (".yml", ".yaml", ".j2")
@functools.lru_cache(maxsize=1)
def get_context() -> ScanContext:
"""Return the cached :class:`ScanContext` for the current repo."""
root = Path(__file__).resolve().parents[4]
root = PROJECT_ROOT
roles_root = root / "roles"
application_defaults = get_application_defaults(roles_dir=roles_root)
user_defaults = get_user_defaults(roles_dir=roles_root)
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
@@ -25,9 +25,8 @@ from __future__ import annotations
import re
import unittest
from pathlib import Path
from typing import List
from . import PROJECT_ROOT
PASSWORD_ASSIGN_RE = re.compile(
r"""^
@@ -50,11 +49,6 @@ JINJA_DOUBLE_WRAPPED_RE = re.compile(r'^"\{\{\s*(?P<expr>.+?)\s*\}\}"$')
DOTENV_FILTER = "dotenv_quote"
def _repo_root_from_test_file() -> Path:
# tests/integration/<cluster>/test_*.py -> repo root is three levels up
return Path(__file__).resolve().parents[3]
def _is_full_line_comment(line: str) -> bool:
return line.lstrip().startswith("#")
@@ -66,7 +60,7 @@ def _is_jinja_control_line(line: str) -> bool:
class TestEnvPasswordsQuotedAndFiltered(unittest.TestCase):
def test_password_env_vars_are_dotenv_quoted_without_double_quoting(self):
root = _repo_root_from_test_file()
root = PROJECT_ROOT
env_templates = sorted(root.rglob("env.j2"))
failures: List[str] = []
@@ -6,6 +6,7 @@ import unittest
from dataclasses import dataclass
from pathlib import Path
from typing import Iterable, List
from . import PROJECT_ROOT
QUOTE_FILTER_RE = re.compile(r"\|\s*quote\b", re.IGNORECASE)
@@ -63,9 +64,7 @@ def _scan_file(path: Path) -> List[Finding]:
class TestPasswordDefinitionsNoQuote(unittest.TestCase):
def test_password_definitions_must_not_use_quote_filter(self) -> None:
repo_root = (
Path(__file__).resolve().parents[3]
) # tests/integration/<cluster>/ -> repo root
repo_root = PROJECT_ROOT # tests/integration/<cluster>/ -> repo root
all_findings: List[Finding] = []
for yml in _iter_roles_yml_files(repo_root):
@@ -6,6 +6,7 @@ import unittest
from dataclasses import dataclass
from pathlib import Path
from typing import Iterable, List
from . import PROJECT_ROOT
JINJA_EXPR_RE = re.compile(r"{{(.*?)}}", re.DOTALL)
@@ -141,9 +142,7 @@ def _scan_shell_block(file_path: Path, start_line: int, block: str) -> List[Find
class TestPasswordQuoteInShellTasks(unittest.TestCase):
def test_passwords_are_quoted_in_shell_tasks(self) -> None:
repo_root = (
Path(__file__).resolve().parents[3]
) # tests/integration/<cluster>/ -> repo root
repo_root = PROJECT_ROOT # tests/integration/<cluster>/ -> repo root
all_findings: List[Finding] = []
for yml in _iter_roles_yml_files(repo_root):
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
@@ -1,11 +1,11 @@
import unittest
from pathlib import Path
import re
import sys
from plugins.filter.invokable_paths import get_invokable_paths
from . import PROJECT_ROOT
# Ensure project root is on PYTHONPATH so plugins.filter.* can be imported
ROOT = Path(__file__).parents[3]
ROOT = PROJECT_ROOT
sys.path.insert(0, str(ROOT))
STAGES_DIR = ROOT / "tasks" / "stages"
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[4]
@@ -1 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[5]
@@ -1,9 +1,9 @@
import yaml
import unittest
from pathlib import Path
from plugins.filter.invokable_paths import get_invokable_paths
from . import PROJECT_ROOT
ROLES_DIR = Path(__file__).resolve().parents[5] / "roles"
ROLES_DIR = PROJECT_ROOT / "roles"
class TestApplicationIdConsistency(unittest.TestCase):
@@ -1,12 +1,12 @@
#!/usr/bin/env python3
import unittest
import yaml
from pathlib import Path
from . import PROJECT_ROOT
class TestRunAfterRoles(unittest.TestCase):
def setUp(self):
self.roles_dir = Path(__file__).resolve().parents[5] / "roles"
self.roles_dir = PROJECT_ROOT / "roles"
self.valid_role_names = {p.name for p in self.roles_dir.iterdir() if p.is_dir()}
def test_run_after_roles_are_valid(self):
@@ -1,12 +1,10 @@
import os
import sys
import re
import unittest
from . import PROJECT_ROOT
# ensure project root is on PYTHONPATH so we can import your CLI code
ROOT = os.path.abspath(
os.path.join(os.path.dirname(__file__), os.pardir, os.pardir, os.pardir, os.pardir)
)
ROOT = str(PROJECT_ROOT)
sys.path.insert(0, ROOT)
from cli.meta.applications.all import find_application_ids # noqa: E402
@@ -1,12 +1,10 @@
import os
import re
import sys
import unittest
from . import PROJECT_ROOT
# ensure project root is on PYTHONPATH so we can import the CLI code
ROOT = os.path.abspath(
os.path.join(os.path.dirname(__file__), os.pardir, os.pardir, os.pardir, os.pardir)
)
ROOT = str(PROJECT_ROOT)
sys.path.insert(0, ROOT)
from cli.meta.applications.all import find_application_ids # noqa: E402
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[4]
@@ -5,6 +5,7 @@ import yaml
import re
from utils.cache.files import read_text
from . import PROJECT_ROOT
class TestIncludeImportExistence(unittest.TestCase):
@@ -16,10 +17,7 @@ class TestIncludeImportExistence(unittest.TestCase):
"""
def setUp(self):
tests_dir = os.path.dirname(__file__)
self.project_root = os.path.abspath(
os.path.join(tests_dir, os.pardir, os.pardir, os.pardir, os.pardir)
)
self.project_root = str(PROJECT_ROOT)
self.roles_dir = os.path.join(self.project_root, "roles")
self.files_to_scan = []
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[5]
@@ -3,18 +3,12 @@ import os
import glob
from utils.cache.yaml import load_yaml_any
from . import PROJECT_ROOT
class TestRoleDependencies(unittest.TestCase):
def test_dependencies_exist(self):
# Determine the path to the roles directory relative to this test file
tests_dir = os.path.dirname(__file__)
project_root = os.path.abspath(
os.path.join(
tests_dir, os.pardir, os.pardir, os.pardir, os.pardir, os.pardir
)
)
roles_dir = os.path.join(project_root, "roles")
roles_dir = os.path.join(str(PROJECT_ROOT), "roles")
# Find all meta/main.yml files under roles/*/meta/main.yml
pattern = os.path.join(roles_dir, "*", "meta", "main.yml")
+2 -2
View File
@@ -1,6 +1,6 @@
import unittest
import re
from pathlib import Path
from . import PROJECT_ROOT
# Regex:
# - one or more lowercase letters, digits or hyphens
@@ -10,7 +10,7 @@ ROLE_NAME_PATTERN = re.compile(r"^[a-z0-9-]+(?:_[a-z0-9-]+)?$")
class TestRoleNames(unittest.TestCase):
def test_role_names_follow_naming_convention(self):
roles_dir = Path(__file__).resolve().parents[3] / "roles"
roles_dir = PROJECT_ROOT / "roles"
self.assertTrue(
roles_dir.is_dir(), f"'roles/' directory not found at {roles_dir}"
)
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[4]
@@ -3,6 +3,8 @@ from pathlib import Path
import yaml
from typing import Any, Dict, List, Tuple, Union
from . import PROJECT_ROOT
# -------- YAML loader that's tolerant of Ansible-specific tags (e.g. !vault) -----
class AnsibleTolerantLoader(yaml.SafeLoader):
@@ -25,22 +27,6 @@ yaml.add_multi_constructor("!", _ansible_tag_passthrough, Loader=AnsibleTolerant
# -------------------------------------------------------------------------------
def _repo_root() -> Path:
"""Find a plausible project root by walking upward from this file."""
here = Path(__file__).resolve()
cur = here.parent
markers = {".git", "ansible.cfg", "roles", "playbook.yml"}
for _ in range(7):
if any((cur / m).exists() for m in markers):
return cur
parent = cur.parent
if parent == cur:
break
cur = parent
# Fallback: project/tests/integration/roles/when/ -> pick parent of 'tests'
return here.parents[4] if len(here.parents) >= 5 else here.parent
Yaml = Union[Dict[str, Any], List[Any], Any]
@@ -103,7 +89,7 @@ class BlockWhenSizeTest(unittest.TestCase):
MAX_TASKS = 3 # performance threshold
def test_blocks_with_when_and_sections_have_max_three_tasks(self):
root = _repo_root()
root = PROJECT_ROOT
violations: List[str] = []
for yml in _iter_yaml_files(root):
@@ -1,6 +1,7 @@
import unittest
from pathlib import Path
from typing import Any, Dict, Iterable, List, Tuple
from . import PROJECT_ROOT
try:
import yaml # PyYAML
@@ -13,24 +14,6 @@ except ImportError as e:
THRESHOLD = 3 # fail if the same when-condition occurs on more than this many tasks
def _find_repo_root_containing(marker_names: Iterable[str], max_up: int = 8) -> Path:
"""
Walk upwards from this file to find the repo root. We assume the project root
contains at least one of `marker_names` (e.g., 'roles', '.git', 'playbooks').
"""
here = Path(__file__).resolve().parent
cur = here
for _ in range(max_up):
for marker in marker_names:
if (cur / marker).exists():
return cur
if cur.parent == cur:
break
cur = cur.parent
# Fallback: repo root assumed 4 levels up from tests/integration/roles/when
return Path(__file__).resolve().parents[4]
def _normalize_when(value: Any) -> str:
"""
Normalize Ansible 'when' to a comparable string:
@@ -150,9 +133,7 @@ class WhenConditionDuplicationTest(unittest.TestCase):
"""
def test_excessive_repeated_when_in_tasks_files(self):
repo_root = _find_repo_root_containing(
marker_names=(".git", "roles", "playbooks")
)
repo_root = PROJECT_ROOT
tasks_globs = [
"**/tasks/**/*.yml",
"**/tasks/**/*.yaml",
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
+2 -2
View File
@@ -1,15 +1,15 @@
import unittest
from collections import defaultdict
from pathlib import Path
from utils.service_registry import build_service_registry_from_roles_dir
from . import PROJECT_ROOT
class TestServicesCanonical(unittest.TestCase):
"""Discovered canonical aliases from role-local configs must be internally consistent."""
def setUp(self):
self.repo_root = Path(__file__).resolve().parents[3]
self.repo_root = PROJECT_ROOT
self.service_registry = build_service_registry_from_roles_dir(
self.repo_root / "roles"
)
@@ -7,10 +7,12 @@ from utils.service_registry import (
ordered_primary_service_entries,
)
from . import PROJECT_ROOT
class TestFrontendServiceSpot(unittest.TestCase):
def setUp(self):
self.repo_root = Path(__file__).resolve().parents[3]
self.repo_root = PROJECT_ROOT
self.roles_dir = self.repo_root / "roles"
self.applications = load_applications_from_roles_dir(self.roles_dir)
self.service_registry = build_service_registry_from_roles_dir(self.roles_dir)
@@ -4,6 +4,8 @@ from pathlib import Path
import yaml
from . import PROJECT_ROOT
_MAILU_ROLE = "web-app-mailu"
_EMAIL_SERVICE_KEY = "email"
@@ -40,10 +42,8 @@ class TestMailuServiceDependency(unittest.TestCase):
must declare services.email with enabled: true and shared: true
in its config/main.yml."""
PROJECT_ROOT = Path(__file__).resolve().parents[3]
def setUp(self):
self.roles_root = self.PROJECT_ROOT / "roles"
self.roles_root = PROJECT_ROOT / "roles"
self.assertTrue(
self.roles_root.is_dir(),
f"Roles directory not found: {self.roles_root}",
@@ -90,7 +90,7 @@ class TestMailuServiceDependency(unittest.TestCase):
reason_str = " and ".join(reasons)
email_svc = self._email_service_conf(config_path)
rel = config_path.relative_to(self.PROJECT_ROOT)
rel = config_path.relative_to(PROJECT_ROOT)
if not email_svc.get("enabled"):
errors.append(
@@ -1,5 +1,4 @@
import unittest
from pathlib import Path
from plugins.lookup.service import LookupModule
from utils.service_registry import (
@@ -7,12 +6,14 @@ from utils.service_registry import (
load_applications_from_roles_dir,
)
from . import PROJECT_ROOT
class TestServicesResolvable(unittest.TestCase):
"""Discovered service keys and provider roles must resolve via the service lookup."""
def setUp(self):
self.repo_root = Path(__file__).resolve().parents[3]
self.repo_root = PROJECT_ROOT
self.roles_dir = self.repo_root / "roles"
self.applications = load_applications_from_roles_dir(self.roles_dir)
self.service_registry = build_service_registry_from_roles_dir(self.roles_dir)
@@ -1,14 +1,14 @@
import unittest
from pathlib import Path
import yaml
from plugins.lookup.service import LookupModule
from . import PROJECT_ROOT
class TestServiceTransitiveDependencies(unittest.TestCase):
def setUp(self):
self.repo_root = Path(__file__).resolve().parents[3]
self.repo_root = PROJECT_ROOT
self.applications = {
"web-app-dashboard": self._load_yaml(
"roles", "web-app-dashboard", "meta", "services.yml"
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[3]
@@ -5,6 +5,7 @@ from typing import Any, Dict, Iterable, List, Optional, Set, Tuple
import yaml
from utils.cache.files import read_text
from . import PROJECT_ROOT
def _safe_yaml_load_all(path: Path) -> List[Any]:
@@ -188,7 +189,7 @@ class TestFactsAreNotOverriddenByVars(unittest.TestCase):
"""
def test_no_fact_is_overridden_via_vars_on_includes(self) -> None:
repo_root = Path(__file__).resolve().parents[3]
repo_root = PROJECT_ROOT
task_files = _find_task_files(repo_root)
facts = _collect_defined_facts(task_files) # fact -> {paths}
@@ -110,6 +110,7 @@ class TestPlaywrightEnvKeysUsed(unittest.TestCase):
unreferenced_keys: List[str] = []
for env_path in sorted(roles_dir.glob("*/templates/playwright.env.j2")):
# nocheck: project-root-import walking from a discovered glob match (<role>/templates/...) up to its role dir, not the repo root
role_dir = env_path.parents[1]
role_name = role_dir.name
spec_path = role_dir / "files" / "playwright.spec.js"
+2 -6
View File
@@ -5,6 +5,7 @@ import unittest
from dataclasses import dataclass
from pathlib import Path
from typing import Iterable, Set
from . import PROJECT_ROOT
@dataclass(frozen=True)
@@ -18,11 +19,6 @@ class Finding:
return f"{rel}:{self.line}: {self.snippet}"
def _repo_root() -> Path:
# tests/integration/<cluster>/<file>.py -> repo root
return Path(__file__).resolve().parents[3]
def _iter_target_files(repo_root: Path) -> Iterable[Path]:
target = repo_root / "cli" / "deploy"
if not target.is_dir():
@@ -127,7 +123,7 @@ class TestNoShLcPipefail(unittest.TestCase):
`pipefail` requires bash. Using it via `sh -lc` is not portable and breaks
on Debian/Ubuntu (dash) with: `set: Illegal option -o pipefail`.
"""
repo_root = _repo_root()
repo_root = PROJECT_ROOT
findings: list[Finding] = []
for py_file in _iter_target_files(repo_root):
findings.extend(_scan_file(py_file))
+3
View File
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[4]
@@ -19,8 +19,9 @@ from __future__ import annotations
import re
import unittest
from pathlib import Path
from . import PROJECT_ROOT
_REPO_ROOT = Path(__file__).resolve().parents[4]
_REPO_ROOT = PROJECT_ROOT
_ROLES_ROOT = _REPO_ROOT / "roles"
# Matches a FROM line that references at least one ${VAR}
@@ -23,8 +23,9 @@ import unittest
from pathlib import Path
from utils.annotations.message import warning
from . import PROJECT_ROOT
_REPO_ROOT = Path(__file__).resolve().parents[4]
_REPO_ROOT = PROJECT_ROOT
_ROLES_ROOT = _REPO_ROOT / "roles"
# Matches any Jinja2 control-flow or structural tag: {% ... %}
@@ -39,6 +40,7 @@ def _has_j2_logic(dockerfile_j2: Path) -> bool:
def _collect_templated_dockerfiles() -> list[Path]:
templated: list[Path] = []
for dockerfile_j2 in sorted(_ROLES_ROOT.glob("*/templates/Dockerfile.j2")):
# nocheck: project-root-import walking from a discovered glob match (<role>/templates/...) up to its role dir, not the repo root
plain_dockerfile = dockerfile_j2.parents[1] / "files" / "Dockerfile"
if plain_dockerfile.is_file():
continue
Loaded 100 of 174 files, more files were not shown because too many files have changed in this diff. Show more