Files
computer-playbook/tests/lint/filesystem/python/test_no_module_utils_shadowing.py
T
kevinveenbirkenbachandClaude Opus 5 3c2666f262 fix(resource): read host CPU and RAM from one lookup
Ansible's Linux hardware collector reads /sys/block/<dev>/size without a None
guard, and that read runs before the CPU facts in the same collector. One
unreadable block device therefore drops every hardware fact at once and without
a warning. Seven expressions across four files dereferenced processor_vcpus or
memtotal_mb directly and died on the first render. Measured on the interpreter
the deploy actually uses, gather_subset=all returns none of those facts.

plugins/lookup/resource.py now measures and computes every host-resource
constant. The two measured inputs resolve as an inventory override, then the
host's own hardware facts, then a probe of the machine the plugin runs on. The
probe applies only to a target reached over the local connection, because a
lookup executes on the Ansible controller and would otherwise report the
controller's hardware for a remote host.

No value is ever substituted. What that abort may interrupt depends on context.
A play carries inventory_hostname and has to stop, while the inventory
generator and the lint suite render the same applications payload with no host
and must not fail over a value they never read. The first case raises
UnresolvableValueError, the one class the best-effort render layers re-raise
instead of swallowing; the second raises an ordinary AnsibleError that they
absorb, exactly as the missing fact did before.

The meta files of web-app-nextcloud and svc-opt-swapfile call the lookup
directly rather than through a group_var. _render_with_templar replaces the
templar's variables with a flat dict, which drops the lazy resolution a play
uses to expand a variable that itself holds a lookup, so the indirection
renders 0M there while the same expression yields 2133M in a play.

The container count is computed only for the keys that divide by it. Reading
applications pulls the payload back through the templar while it is still being
produced, and those meta files sit in that payload.

utils/cache/errors.py lives in the cache package rather than at the top of
utils/: ansible.cfg exposes ./utils as module_utils, so a top-level errors.py
becomes ansible.module_utils.errors and shadows the real one inside every
module payload zip, which broke uri and get_url with an ImportError pointing
into a temporary zip. A lint now guards that class of collision.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 13:44:05 +02:00

73 lines
2.6 KiB
Python

"""No top-level file under the configured `module_utils` path shadows Ansible's own.
`ansible.cfg` points `module_utils` at `./utils`, so every top-level `.py` there
becomes importable as `ansible.module_utils.<stem>` and is bundled into the
payload zip Ansible ships for each task. A stem that Ansible already uses wins
over the real module inside that zip, and every module importing from it dies at
run time, far from the file that caused it.
A `utils/errors.py` did exactly that: `ansible.module_utils.errors` is where
Ansible keeps `AliasError`, so `uri` and `get_url` failed with an ImportError
pointing into a temporary zip rather than at the repository.
Fix a hit by moving the file into a subpackage (`utils/<topic>/<name>.py`).
Nested modules land under `ansible.module_utils.<topic>.<name>` and cannot
collide.
"""
from __future__ import annotations
import configparser
import importlib.util
import unittest
from typing import TYPE_CHECKING
from . import PROJECT_ROOT
if TYPE_CHECKING:
from pathlib import Path
_ANSIBLE_CFG = PROJECT_ROOT / "ansible.cfg"
def _module_utils_dir() -> Path:
"""The directory `ansible.cfg` exposes as `ansible.module_utils`."""
parser = configparser.ConfigParser()
parser.read(_ANSIBLE_CFG)
configured = parser.get("defaults", "module_utils", fallback="").strip()
if not configured:
raise unittest.SkipTest(
f"{_ANSIBLE_CFG} sets no 'module_utils' path, so nothing can shadow."
)
return (PROJECT_ROOT / configured).resolve()
def _shadows_ansible(stem: str) -> bool:
try:
return importlib.util.find_spec(f"ansible.module_utils.{stem}") is not None
except (ImportError, AttributeError, ValueError):
return False
class TestNoModuleUtilsShadowing(unittest.TestCase):
def test_no_top_level_helper_shadows_an_ansible_module_util(self) -> None:
module_utils_dir = _module_utils_dir()
offenders = sorted(
path.name
for path in module_utils_dir.glob("*.py")
if path.stem != "__init__" and _shadows_ansible(path.stem)
)
self.assertEqual(
offenders,
[],
f"These files in {module_utils_dir.relative_to(PROJECT_ROOT)} are "
f"exposed as 'ansible.module_utils.<stem>' and shadow a module "
f"Ansible ships under that exact name. Every task whose module "
f"imports the real one then fails inside the payload zip. Move each "
f"into a subpackage instead. Found {offenders}.",
)
if __name__ == "__main__": # pragma: no cover
unittest.main()