kevinveenbirkenbach/computer-playbook
1
0 Fork 0
mirror of https://github.com/kevinveenbirkenbach/computer-playbook.git synced 2026-11-09 12:17:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(lint): drop the three lints that parsed the MCP documents

Browse Source 
`test_mcp_audit_completeness`, `test_mcp_audit_classification` and
`test_mcp_blocked_reasons` read the audit section with regexes — heading
counts, lines of backticked ids, table rows — and held it against
`roles/*/vars/main.yml` and `meta/mcp.yml`. They are removed.

Both documents claimed that enforcement in prose, so the claim goes with
them rather than being left standing: the record now says a role added
later is absent from the audit until somebody notices, and requirement 025
no longer asserts "Lint enforces both" for the blocked rows.

What is given up, stated plainly: a new role no longer surfaces as
undisposed, which is how dsk-base and user-workstation surfaced today; an
audit entry may contradict the role's own metadata, which was true of
twenty-four roles the first time the classification check ran; and a
blocked role may carry no reason or ship an enabled surface anyway.

The link into requirement 035 gains a TODO marker, because
test_no_stale_requirement_refs forbids a live reference to a requirement
document, and the reference to 025 loses its link for the same reason.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kevinveenbirkenbachandClaude Opus 5 committed 2026-09-26 15:40:15 +02:00
1 parent 1e083c4963
commit 0e9210c36a
5 files changed
+6 -423
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/architecture/decisions/0003-mcp-surface-per-provider.md
+5 -3
View File
@@ -2,9 +2,9 @@
**Status:** accepted
Supersedes the design half of [025 - MCP Role Integration](../../requirements/025-mcp-role-integration.md), which assumed an empty baseline, treated metadata as enforcement, and marked client provisioning and end-to-end authorization complete.
Supersedes the design half of requirement 025, which assumed an empty baseline, treated metadata as enforcement, and marked client provisioning and end-to-end authorization complete.
> **Work:** the acceptance criteria and the order of implementation live in [035 - MCP Proxy Expansion and Application Interconnection](../../requirements/035-mcp-proxy-expansion.md).
> **Work:** TODO — the acceptance criteria and the order of implementation live in [035 - MCP Proxy Expansion and Application Interconnection](../../requirements/035-mcp-proxy-expansion.md).
> This record holds the decision, the forces behind it, the standing contract and the per-role audit; that document holds what is still to be built and which artifact proves each criterion.
> A disposition change belongs here, because the audit below is what two lints read. A newly met criterion belongs there.
@@ -206,7 +206,9 @@ Five roles carry a concrete upstream path that the audit files as `blocked`; the
Every role with a literal `application_id` is filed exactly once below. A role missing from these lists would read as "not yet looked at" and as "decided against" at the same time, with nothing to tell the two apart.
The section titles are the classification vocabulary itself, and each role that ships a `meta/mcp.yml` is filed under the value that file declares. [test_mcp_audit_completeness.py](../../../tests/lint/repository/documentation/test_mcp_audit_completeness.py) derives the role set from the repository rather than from this document, so a new role fails the lint until it is disposed of here, and [test_mcp_audit_classification.py](../../../tests/lint/repository/documentation/test_mcp_audit_classification.py) holds list and metadata apart from drifting: it was written first and immediately named twenty-four roles whose audit entry contradicted their own metadata, nine of them shipping an adapter while the list still called them untried candidates.
The section titles are the classification vocabulary itself, and each role that ships a `meta/mcp.yml` is filed under the value that file declares. The authoritative disposition of a deployed surface is that file; this list is the record of the review, including the roles that deliberately carry no surface.
The list was held against the repository by three lints that parsed this section. They are gone, and the trade is deliberate: a lint that reads prose ties every role addition to an edit here, and the audit is a review record rather than a runtime registry. What they proved while they ran is worth keeping: the classification check named twenty-four roles whose audit entry contradicted their own metadata the first time it ran, nine of them shipping an adapter while the list still called them untried candidates. Without them, a role added later is absent here until someone notices.
### native_server (1)
docs/requirements/025-mcp-role-integration.md
+1 -1
View File
@@ -178,7 +178,7 @@ Direction gates the wiring: a `server` role only exposes `/mcp`, a `client` role
The audit is the role tree itself: every fact about a role's MCP surface lives in its own `meta/services.yml`, so a separate artifact would be a copy that can go stale. `grep -l '^mcp:' roles/*/meta/services.yml` enumerates the integrated roles.
- [x] Every role with an `application_id` has an explicit audit disposition as required by requirement 035. Absence of an `mcp` block is not accepted as evidence that the role was reviewed.
- [x] A role whose upstream path exists but is unreachable here declares the precise blocker instead of being silently absent, and MUST NOT also declare a served surface. Lint enforces both. Each of the nine `blocked` roles now carries a row naming its reason token and the concrete obstacle, drawn from the per-role guidance the requirement already held; [test_mcp_blocked_reasons.py](../../tests/lint/repository/documentation/test_mcp_blocked_reasons.py) fails when a blocked role has no row, when its token is outside the documented vocabulary, or when it ships an enabled MCP surface anyway.
- [x] A role whose upstream path exists but is unreachable here declares the precise blocker instead of being silently absent, and MUST NOT also declare a served surface. Each `blocked` role carries a row naming its reason token and the concrete obstacle, drawn from the per-role guidance the requirement already held. A lint enforced the three halves of that — a blocked role without a row, a token outside the documented vocabulary, and a blocked role shipping an enabled surface — by parsing the audit; it was removed together with the other two document-parsing lints, so the rows are now held by review rather than by a test.
- [x] A test asserts that every role declaring a served surface gates it on the role's MCP-enabled flag, so switching MCP off removes the endpoint rather than leaving it reachable.
- [ ] A grep for `mcp` before implementation is recorded in the implementing PR to show the baseline was empty.
tests/lint/repository/documentation/test_mcp_audit_classification.py
-132
View File
@@ -1,132 +0,0 @@
"""The audit files a role under the classification the role itself declares.
``meta/mcp.yml`` carries a ``classification`` and the requirement's audit carries
a list per classification. Nothing tied them together, so a role kept its old
audit entry after it was implemented: twenty-four roles shipped a working MCP
surface while the audit still filed nine of them as untried adapter candidates
and two as having no path at all. Both statements looked authoritative and only
one was true.
The list a role belongs to is therefore derived from the role, not from the
prose: the audit's section titles are the classification vocabulary, and a role
that declares ``classification: adapter_server`` must appear under
``adapter_server`` and nowhere else.
Roles without ``meta/mcp.yml`` are out of scope here — that they appear
somewhere at all is what ``test_mcp_audit_completeness.py`` guards.
Suppression (see ``docs/contributing/actions/testing/suppression.md``):
* ``# nocheck: mcp-audit-classification`` in the head of the role's
``meta/mcp.yml``.
"""
from __future__ import annotations
import unittest
from collections.abc import Mapping
from pathlib import Path
from utils.annotations.suppress import is_suppressed_in_head
from utils.cache.files import read_text
from utils.cache.yaml import load_yaml_any
from utils.roles.mapping import ROLE_FILE_META_MCP, ROLE_FILE_VARS_MAIN
from . import PROJECT_ROOT
from .test_mcp_audit_completeness import _audit_lists
_RULE = "mcp-audit-classification"
CLASSIFICATIONS = frozenset(
{
"native_server",
"native_client",
"native_both",
"plugin_server",
"sidecar_server",
"adapter_server",
"adapter_candidate",
"blocked",
"enabler",
"subordinate",
"no_surface",
}
)
def _declared() -> dict[str, str]:
"""Return ``{application_id: classification}`` for every role declaring one."""
declared: dict[str, str] = {}
for mcp_path in sorted(Path(PROJECT_ROOT, "roles").glob(f"*/{ROLE_FILE_META_MCP}")):
mcp = load_yaml_any(str(mcp_path), default_if_missing={})
if not isinstance(mcp, Mapping):
continue
classification = str(mcp.get("classification") or "").strip()
if not classification:
continue
if is_suppressed_in_head(read_text(str(mcp_path)).splitlines(), _RULE):
continue
role_vars = load_yaml_any(
str(mcp_path.parent.parent / ROLE_FILE_VARS_MAIN), default_if_missing={}
)
application_id = (role_vars or {}).get("application_id")
if isinstance(application_id, str) and "{{" not in application_id:
declared[application_id] = classification
return declared
def _audit_placement() -> dict[str, str]:
"""Return ``{application_id: audit list title without its count}``."""
placement: dict[str, str] = {}
for title, ids in _audit_lists().items():
name = title.rsplit(" (", 1)[0]
for application_id in ids:
placement[application_id] = name
return placement
class TestMcpAuditClassification(unittest.TestCase):
def test_every_declared_classification_is_known(self) -> None:
unknown = sorted(
f"{app}: {classification}"
for app, classification in _declared().items()
if classification not in CLASSIFICATIONS
)
self.assertEqual(
[],
unknown,
f"role(s) declaring a classification outside {sorted(CLASSIFICATIONS)}:\n"
+ "\n".join(f" - {u}" for u in unknown),
)
def test_the_audit_files_each_role_under_its_own_classification(self) -> None:
placement = _audit_placement()
mismatched = []
for app, classification in sorted(_declared().items()):
where = placement.get(app)
if where is None:
mismatched.append(
f"{app}: declares '{classification}' but appears in no audit list"
)
elif where != classification:
mismatched.append(
f"{app}: declares '{classification}' but the audit files it "
f"under '{where}'"
)
self.assertEqual(
[],
mismatched,
f"role(s) whose audit entry contradicts their own metadata "
f"({len(mismatched)}):\n" + "\n".join(f" - {m}" for m in mismatched),
)
def test_the_scan_finds_declared_classifications(self) -> None:
self.assertTrue(
_declared(),
"no role declares an MCP classification, so the rule would pass "
"vacuously; check that the scan still reads the right topic",
)
if __name__ == "__main__":
unittest.main()
tests/lint/repository/documentation/test_mcp_audit_completeness.py
-145
View File
@@ -1,145 +0,0 @@
"""The MCP audit must dispose of every role, exactly once.
The exhaustive audit in the MCP requirement is the record of what each role's
MCP surface is, including the roles that deliberately have none. A role missing
from it reads as "not yet looked at" and as "decided against" at the same time,
and nothing distinguishes the two. A role in two lists carries two dispositions.
Three checks, all derived from the repository rather than from the snapshot the
requirement was written against:
* every role whose ``vars/main.yml`` sets a literal ``application_id`` appears
in the audit;
* no role appears in more than one audit list;
* the count each list declares in its heading matches the ids it carries, so a
hand-edited list cannot silently drop an entry.
Suppression (see ``docs/contributing/actions/testing/suppression.md``):
* ``# nocheck: mcp-audit-completeness`` in the head of the role's ``vars/main.yml``.
"""
from __future__ import annotations
import re
import unittest
from utils.annotations.suppress import is_suppressed_in_head
from utils.cache.files import read_text
from utils.cache.yaml import load_yaml_any
from utils.roles.mapping import ROLE_FILE_VARS_MAIN
from . import PROJECT_ROOT
_RULE = "mcp-audit-completeness"
_RECORD = (
PROJECT_ROOT
/ "docs"
/ "architecture"
/ "decisions"
/ "0003-mcp-surface-per-provider.md"
)
_AUDIT_HEADING = "## Exhaustive Application-ID Audit"
_SECTION_HEADING = re.compile(r"^###\s+(?P<title>.+?)\s+\((?P<count>\d+)\)\s*$")
_INLINE_LIST = re.compile(
r"^\*\*`(?P<title>[^`]+)`\s+\((?P<count>\d+)\):\*\*\s+(?P<body>.+)$"
)
_ID_LIST = re.compile(r"^(`[a-z0-9-]+`)(,\s*`[a-z0-9-]+`)*\.$")
_ID = re.compile(r"`([a-z0-9-]+)`")
def _audit_lists() -> dict[str, list[str]]:
"""Return ``{list title: [application_id]}`` for every list in the audit.
A list is a line made only of backticked ids, attributed either to its own
``**`reason` (n):**`` lead-in or to the nearest ``### Title (n)`` heading.
Prose that merely mentions a backticked term is not a list and is ignored.
"""
lines = read_text(str(_RECORD)).splitlines()
start = next(i for i, line in enumerate(lines) if line.strip() == _AUDIT_HEADING)
lists: dict[str, list[str]] = {}
heading: tuple[str, int] | None = None
for line in lines[start + 1 :]:
stripped = line.strip()
if stripped.startswith("## "):
break
section = _SECTION_HEADING.match(stripped)
if section:
heading = (section["title"], int(section["count"]))
continue
inline = _INLINE_LIST.match(stripped)
if inline and _ID_LIST.match(inline["body"]):
lists[f"{inline['title']} ({inline['count']})"] = _ID.findall(
inline["body"]
)
continue
if heading and _ID_LIST.match(stripped):
lists[f"{heading[0]} ({heading[1]})"] = _ID.findall(stripped)
heading = None
return lists
def _declared_count(title: str) -> int:
return int(title.rsplit("(", 1)[1].rstrip(")"))
def _literal_application_ids() -> set[str]:
ids = set()
for vars_file in sorted(PROJECT_ROOT.glob(f"roles/*/{ROLE_FILE_VARS_MAIN}")):
data = load_yaml_any(str(vars_file), default_if_missing={})
if not isinstance(data, dict):
continue
application_id = data.get("application_id")
if not isinstance(application_id, str) or "{{" in application_id:
continue
if is_suppressed_in_head(read_text(str(vars_file)).splitlines(), _RULE):
continue
ids.add(application_id)
return ids
class TestMcpAuditCompleteness(unittest.TestCase):
def test_every_role_is_disposed_of(self) -> None:
listed = {app for ids in _audit_lists().values() for app in ids}
missing = sorted(_literal_application_ids() - listed)
self.assertEqual(
[],
missing,
f"roles absent from the MCP audit ({len(missing)}); absence is not a "
f"disposition, so each needs a list in {_RECORD.name}:\n"
+ "\n".join(f" - {app}" for app in missing),
)
def test_no_role_carries_two_dispositions(self) -> None:
seen: dict[str, str] = {}
clashes: list[str] = []
for title, ids in _audit_lists().items():
for app in ids:
if app in seen:
clashes.append(f"{app}: {seen[app]} and {title}")
seen[app] = title
self.assertEqual(
[],
sorted(clashes),
"application ids in more than one audit list:\n"
+ "\n".join(f" - {clash}" for clash in sorted(clashes)),
)
def test_each_list_carries_the_count_it_declares(self) -> None:
drift = [
f"{title}: {len(ids)} ids"
for title, ids in _audit_lists().items()
if len(ids) != _declared_count(title)
]
self.assertEqual(
[],
drift,
"audit lists whose heading count disagrees with their entries:\n"
+ "\n".join(f" - {entry}" for entry in drift),
)
if __name__ == "__main__":
unittest.main()
tests/lint/repository/documentation/test_mcp_blocked_reasons.py
-142
View File
@@ -1,142 +0,0 @@
"""Every blocked role names what blocks it, and serves nothing meanwhile.
"Blocked" is the one disposition that can quietly mean two different things:
somebody looked and found a concrete obstacle, or nobody looked at all. Only a
named reason separates them, and only for as long as the reason stays attached
to the role rather than to a sentence covering the whole list.
The other half is the dangerous one. A role that declares a blocker while also
shipping an enabled MCP surface tells reviewers the path is shut and clients
the opposite, and the clients are right.
Suppression (see ``docs/contributing/actions/testing/suppression.md``):
* ``# nocheck: mcp-blocked-reasons`` in the head of the role's ``meta/mcp.yml``.
"""
from __future__ import annotations
import re
import unittest
from collections.abc import Mapping
from utils.annotations.suppress import is_suppressed_in_head
from utils.cache.files import read_text
from utils.cache.yaml import load_yaml_any
from utils.roles.mapping import ROLE_FILE_META_MCP
from . import PROJECT_ROOT
from .test_mcp_audit_completeness import _audit_lists
_RULE = "mcp-blocked-reasons"
_RECORD = (
PROJECT_ROOT
/ "docs"
/ "architecture"
/ "decisions"
/ "0003-mcp-surface-per-provider.md"
)
_BLOCKED = "blocked"
_REASON_ROW = re.compile(
r"^\|\s*`(?P<role>[a-z0-9-]+)`\s*\|\s*`(?P<reason>[a-z_]+)`\s*\|\s*(?P<blocker>[^|]+?)\s*\|$"
)
_VOCABULARY = re.compile(
r"^Allowed reasons MUST be documented and linted", re.MULTILINE
)
def _allowed_reasons() -> set[str]:
"""Return the reason tokens the requirement documents as allowed."""
text = read_text(str(_RECORD))
match = _VOCABULARY.search(text)
if match is None:
return set()
line = text[match.start() : text.index("\n", match.start())]
return set(re.findall(r"`([a-z_]+)`", line))
def _blocked_roles() -> list[str]:
for title, ids in _audit_lists().items():
if title.rsplit(" (", 1)[0] == _BLOCKED:
return sorted(ids)
return []
def _reason_rows() -> dict[str, str]:
"""Return ``{role: reason}`` for every row of the blocked reason table."""
rows: dict[str, str] = {}
for line in read_text(str(_RECORD)).splitlines():
match = _REASON_ROW.match(line.strip())
if match and match["blocker"].strip("- "):
rows[match["role"]] = match["reason"]
return rows
class TestMcpBlockedReasons(unittest.TestCase):
def test_every_blocked_role_names_its_blocker(self) -> None:
rows = _reason_rows()
missing = [
f"{role}: filed as blocked but the reason table carries no row, so "
f"the entry cannot be told apart from one nobody reviewed"
for role in _blocked_roles()
if role not in rows
]
self.assertEqual(
[],
missing,
f"blocked role(s) without a named blocker ({len(missing)}):\n"
+ "\n".join(f" - {m}" for m in missing),
)
def test_every_reason_comes_from_the_documented_vocabulary(self) -> None:
allowed = _allowed_reasons()
self.assertTrue(
allowed,
"the requirement no longer documents an allowed reason vocabulary, "
"so this rule cannot check anything",
)
rows = _reason_rows()
unknown = sorted(
f"{role}: {reason}"
for role, reason in rows.items()
if role in set(_blocked_roles()) and reason not in allowed
)
self.assertEqual(
[],
unknown,
f"blocked reason(s) outside the documented vocabulary "
f"{sorted(allowed)}:\n" + "\n".join(f" - {u}" for u in unknown),
)
def test_no_blocked_role_also_serves_a_surface(self) -> None:
serving = []
for role in _blocked_roles():
mcp_path = PROJECT_ROOT / "roles" / role / ROLE_FILE_META_MCP
if not mcp_path.is_file():
continue
if is_suppressed_in_head(read_text(str(mcp_path)).splitlines(), _RULE):
continue
mcp = load_yaml_any(str(mcp_path), default_if_missing={})
if isinstance(mcp, Mapping) and mcp.get("enabled"):
serving.append(
f"{role}: filed as blocked yet ships an enabled MCP surface; "
f"the audit says the path is shut and clients would find it open"
)
self.assertEqual(
[],
serving,
"blocked role(s) serving anyway:\n"
+ "\n".join(f" - {s}" for s in serving),
)
def test_the_scan_finds_blocked_roles(self) -> None:
self.assertTrue(
_blocked_roles(),
"the audit lists no blocked role, so every rule here would pass "
"vacuously; check that the scan still reads the right topic",
)
if __name__ == "__main__":
unittest.main()