kevinveenbirkenbach/computer-playbook
1
0 Fork 0
mirror of https://github.com/kevinveenbirkenbach/computer-playbook.git synced 2026-11-11 13:17:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
6,268 Commits 20 Branches 79 Tags
05f2d9d6cc8e4d87db43d3141506fdebf58b1663
Commit Graph
4 Commits
Author SHA1 Message Date
kevinveenbirkenbachandClaude Opus 4.7 4b37e123d3 test(lint): cap production Python files at 500 lines 
Adds a SPOT-aware lint that walks the repo via the cached
project-tree iterator (no `git` subprocess) and fails on any tracked
non-test Python file longer than 500 lines. Files under `tests/` are
intentionally excluded — coverage breadth there reflects feature
matrix, not design weight. A per-file `# nocheck` (or `#nocheck`)
marker in the first 30 lines opts the file out; the marker is
intentionally visible so the cost of carrying long modules is not
silently buried.

Gitignore-aware filtering moves to its own SPOT
-----------------------------------------------

The fnmatch-based `.gitignore` parser/matcher previously lived inline
in `tests/lint/repository/test_test_files_location.py` (and in
near-duplicates across other tests). Pull it into
`utils.cache.gitignore` as a single source of truth:

* `load_gitignore_patterns(root)` — cached pattern loader.
* `is_path_gitignored(rel_path, patterns)` — pure matcher.
* `iter_non_ignored_files(...)` — convenience iterator in
  `utils.cache.files` that composes the cached project walk with the
  matcher.

`utils.cache._reset_cache_for_tests` now resets the gitignore module
too, so test fixtures that mutate the tree see consistent state.

Refactor / opt-out for existing offenders
-----------------------------------------

Three production files exceeded the cap; addressed each:

* `roles/sys-svc-compose-ca/files/compose_ca.py` — opted out via
  `# nocheck: file-size`. The script is delivered to target hosts
  via `ansible.builtin.copy` and runs with only the stdlib; splitting
  would require also delivering a package, which the role does not
  do.
* `tasks/utils/migrate_meta_layout.py` — converted to a package
  (`tasks/utils/migrate_meta_layout/`) with focused submodules:
    - `__main__` — entrypoint + run-orchestration.
    - `yaml_io` — load/dump/empty-dir/deep-merge helpers.
    - `credentials` — schema-vs-config collision detection +
      runtime-to-schema conversion.
    - `role_builder` — per-role meta file emission.
    - `indices` — networks + ports + relays index builders.
  Run via `python3 -m tasks.utils.migrate_meta_layout`. Largest
  submodule is `role_builder.py` at 250 lines.
* `cli/fix/move_unnecessary_dependencies/__main__.py` — split
  alongside two new sibling modules in the same package:
    - `analysis` — provider/consumer static analysis.
    - `apply` — meta rewrite + guarded include-block prepending.
    - `yaml_io` — ruamel round-trip helpers + role-tree discovery.
  Largest submodule is `analysis.py` at 174 lines.

`make test` green (162 tests, 1 skipped); the new lint passes.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
 2026-04-29 17:15:13 +02:00
kevinveenbirkenbachandClaude Opus 4.7 b3de2a4d7c refactor(meta): land req-011 meta/info.yml + utils/cache cleanup 
req-011 — descriptive role-level metadata
  * Move logo / homepage / video / display out of
    meta/main.yml.galaxy_info into a new project-owned
    roles/<role>/meta/info.yml (file-root convention, no `info:` wrapper).
  * 78 role meta/main.yml files migrated via ruamel round-trip
    (key-order / quoting / inline-comments preserved). Roles with none
    of the four fields are left untouched (no empty info.yml created).
  * web-app-dashboard/lookup_plugins/docker_cards.py reads logo +
    display from meta/info.yml; description / galaxy_tags continue to
    come from galaxy_info (Galaxy-spec fields).
  * docker_cards.py now routes ALL its file reads through the
    process-wide caches: load_yaml_any for vars/main.yml,
    meta/main.yml, meta/info.yml; read_text from utils.cache.files for
    README.md. The orphan `import yaml` is gone.

New utils/cache/files (renamed from tests/utils/fs)
  * The process-wide filesystem cache (read_text, iter_project_files,
    iter_project_files_with_content, _all_project_files) is not
    test-specific; it now lives under utils/cache/files.py next to the
    other process-wide caches. 34 consumer imports rewritten across
    tests/ and docs/. The empty tests/utils/ package is removed.
  * New unit suite tests/unit/utils/cache/test_files.py — 12 cases
    covering read_text, iter_project_files (skip-dirs, extensions,
    exclude_tests, exclude_dirs, walk-cache invariants), the combined
    iterator, non-UTF-8 silent skip, and _reset.

utils/cache redundancy cleanup
  * applications.py + users.py + cli/create/inventory/services_disabler.py
    drop their direct yaml.safe_load calls and their private
    _load_yaml_mapping/_load_yaml_variant_list copies; everything routes
    through utils.cache.yaml.load_yaml / load_yaml_any. The dead
    utils.cache.base._load_yaml_mapping is removed.
  * users.py:_load_store_users (tokens.yml read) likewise migrated.
  * _load_yaml_variant_list moves out of base.py to applications.py
    (its only consumer) and is renamed _load_variants_overrides — also
    routes through the cached loader now.
  * utils.cache.yaml._reset_cache_for_tests is renamed _reset (kept as
    alias) for parity with the per-domain reset helpers.
  * utils.cache.files grows a _reset() that clears its two LRU caches.
  * utils.cache.__init__._reset_cache_for_tests now orchestrates ALL
    six caches (applications, users, domains, base, yaml, files) so
    tests get clean state across cache layers — needed because
    applications/users now share the yaml cache and would otherwise
    serve stale parses across reset boundaries.
  * utils.cache.files imports PROJECT_ROOT from base instead of
    redefining it.
  * tests/unit/utils/cache/test_data.py + test_base.py: obsolete
    TestLoadYamlMapping / TestLoadYamlVariantList classes deleted
    (covered by test_yaml.py + test_variants.py).
  * README.md updated for the new module list (files, yaml) and the
    six-module orchestrator.

Galaxy schema lint
  * tests/lint/repository/test_role_meta_main_galaxy_schema.py: split
    galaxy_info field requirements into hard-required {author,
    description, license} and soft-recommended {min_ansible_version,
    platforms}. Soft-recommended -> emit a GitHub Actions ::warning::
    annotation via utils.annotations.message.warning (titled
    "meta/main.yml galaxy_info" with file= pointing at the offender),
    no test failure. Rationale: min_ansible_version + platforms are
    only enforced by Galaxy when *publishing*, which Infinito.Nexus
    does not do.
  * Galaxy lint goes green (was 226 offenders before req-011 +
    license_url/repository/documentation strip; was 62 before this
    relaxation; now 0 hard offenders, 62 soft warnings).
  * roles/svc-opt-keyboard-color/meta/main.yml: backfill `license`
    (only role missing the hard-required field).

New lint
  * tests/lint/repository/test_role_meta_info.py — enforces meta/info.yml
    file-root convention, the four-field allowlist (logo, homepage,
    video, display), and shape (logo.class string, homepage/video
    non-empty strings, display bool). Also fails if any
    meta/main.yml.galaxy_info still carries one of the four migrated
    fields.

Docs
  * docs/contributing/design/services/layout.md: new section
    "Descriptive Role-Level Metadata — meta/info.yml" with allowed-
    fields table, optionality rule, materialised path, and consumer
    note. File-Layout table grows the meta/info.yml row;
    meta/main.yml row gets the explicit "no logo/homepage/video/display"
    note. req-011 added to header and Related Pages.
  * utils/cache/README.md: files.py + yaml.py rows added; six-module
    reset orchestrator documented.

Validation
  * Unit:        1573 passed.
  * Lint:        55 passed.
  * Integration: 161 passed (1 skipped) via pre-commit hook.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-27 12:27:17 +02:00
kevinveenbirkenbachandClaude Opus 4.7 9c35909752 refactor(utils/cache): split data.py into base/applications/users/domains 
The 934-line `utils/cache/data.py` monolith mixed cross-cutting helpers
(constants, yaml-loaders, deep-merge, signatures, render machinery)
with three independent domain caches (applications, users, domains).
Per the KISS/SPOT/DRY guidelines, split it along domain boundaries so
each file has a single, well-scoped responsibility:

  utils/cache/__init__.py     — package; owns _reset_cache_for_tests()
                                orchestrator
  utils/cache/base.py         — cross-cutting helpers: constants,
                                yaml-loaders, _deep_merge, cache-key +
                                content-fingerprint signatures, the
                                cross-domain re-entry guard, and the
                                templar-render machinery (the only
                                ansible-coupled symbol, lazy-imports
                                its dependency)
  utils/cache/applications.py — variants + defaults +
                                get_merged_applications. Strictly
                                ansible-free at import time so the
                                GitHub Actions runner-host CLI path
                                (cli.deploy.development.init →
                                plan_dev_inventory_matrix →
                                get_variants) keeps working without
                                ansible installed
  utils/cache/users.py        — user defs, token store hydration,
                                alias materialization,
                                get_user_defaults, get_merged_users
  utils/cache/domains.py      — get_merged_domains, derived from the
                                merged applications view
  utils/cache/data.py         — DELETED. Façade is gone; the package
                                exports nothing the domain modules
                                don't already

Backward-compat: the facade was deleted (per operator request) and all
36+ consumers — 18 plugins, 2 CLI entry points, 9 test files, 4 doc
files, plus a role-internal lookup_plugin — were rewired to import
directly from the owning domain module. `_reset_cache_for_tests` is
now exposed at `from utils.cache import _reset_cache_for_tests`.

Tests added (per-module, focused):
  tests/unit/utils/cache/test_base.py            (31 cases)
  tests/unit/utils/cache/test_applications.py    ( 9 cases)
  tests/unit/utils/cache/test_users_module.py    (21 cases; suffix
                                                  disambiguates from
                                                  the existing
                                                  tests/unit/plugins/
                                                  lookup/test_users.py)
  tests/unit/utils/cache/test_domains.py         ( 6 cases)

The ansible-block invariants from CI runs 24934007615 and 24935979190
are now pinned via subprocess isolation: each ansible-less import test
spawns a fresh `python3 -c '…'` with a meta_path filter denying every
`ansible*` import and asserts the import + invocation succeed in that
clean process. The earlier in-process sys.modules-manipulation shims
fragmented module identity and broke under Python 3.13 inside the
infinito container; subprocess isolation has neither problem.

Doc-style audit (per docs/contributing/documentation.md):
  - Removed em-dash from utils/cache/README.md (split into two
    sentences)
  - Updated stale `utils.runtime_data` reference in
    docs/contributing/design/matrix-deploy.md
  - Removed two SPOT self-declarations
    (docs/agents/files/group_vars/applications.md and
    docs/contributing/artefact/files/plugins/lookup/users.md) per
    the documentation rule that SPOT status is conferred by
    cross-links, not by the page asserting it about itself

Empirical proof:
  - All four domain modules import without ansible (subprocess
    isolation tests)
  - get_variants(roles_dir=ROLES_DIR) returns 99 apps in the
    ansible-blocked subprocess
  - 142 cache tests + 1085 wider-suite tests pass; the only
    remaining failures (test_host_vars free-function fixture bug,
    test_runtime_performance timing flake) are pre-existing and
    unrelated to this refactor (verified via `git stash`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-26 00:27:57 +02:00
kevinveenbirkenbach 12dba01156 refactor(utils): consolidate cache modules into utils/cache/ package 
Both in-process cache layers — the cached YAML loader and the cached
runtime-data getters — now live next to each other under
`utils/cache/`. The package gains a README that documents the
"orthogonal layers, no third cache" rule so future contributors hit
the right primitive on the first look.

Moves:
- `utils/yaml_cache.py` -> `utils/cache/yaml.py`
- `utils/runtime_data.py` -> `utils/cache/data.py`
- `tests/unit/utils/test_yaml_cache.py` -> `tests/unit/utils/cache/test_yaml.py`
- `tests/unit/utils/test_runtime_data.py` -> `tests/unit/utils/cache/test_data.py`
- `tests/unit/utils/test_variants.py` -> `tests/unit/utils/cache/test_variants.py`

New:
- `utils/cache/__init__.py` (empty per project convention)
- `utils/cache/README.md` describing the two layers, when to reach for
  which, lifetime, test-only helpers, and design guidelines (RFC 2119
  keywords, emoji-after-headline, no self-SPOT declaration, no
  slash-in-link-text).
- `tests/unit/utils/cache/__init__.py`

Mechanical updates:
- All imports across cli/, plugins/lookup/, tests/, and roles/ rewritten
  from `utils.yaml_cache` -> `utils.cache.yaml` and
  `utils.runtime_data` -> `utils.cache.data` (40+ files).
- `REPO_ROOT` parents-index in the moved tests bumped from `parents[3]`
  to `parents[4]` to reflect the deeper nesting.
- Six broken markdown links in docs/ updated to the new module paths
  (test_markdown_links lint test stays green).
- Two test files that used the `from utils import runtime_data` form
  now use `from utils.cache import data as runtime_data` so the local
  alias remains.

Tests: 848 passed across unit + lint suites (1 pre-existing collection
error in test_host_vars.py unrelated to this change).
 2026-04-25 17:17:14 +02:00