Files
computer-playbook/plugins/lookup/container_dns.py
kevinveenbirkenbachandClaude Opus 5 52b4504035 fix(container-dns): let the bridge be declared instead of guessed
Whether a resolver answers on the docker bridge was inferred from the host's
own resolv.conf: all entries loopback was read as "a listener of ours is there,
and containers must reach it on the bridge". A systemd-resolved stub binds
127.0.0.53, NetworkManager's dnsmasq binds 127.0.0.1, and a node dnsmasq binds
both those and the bridge. All three look identical from resolv.conf and only
the third is reachable from a container.

So ordinary hosts were handed an address with nothing behind it. The
Instructions job died there: daemon.json carried "dns": ["10.208.0.1"] alone,
and the openresty build could not resolve dl-cdn.alpinelinux.org while the
daemon, resolving through the host, pulled the same base image fine. Two netns,
two outcomes.

e308e18923 had already fixed this once, with a reachability gate whose message
names the same URL. b09f2cbd82 dropped that gate and ce43263745 put the
resolv.conf inference in its place, so the guard was gone and only the missing
docker0 fact still hid it. 285ba1e76c removed that accident and the failure
surfaced.

svc-net-tor and NETWORK_CONTAINER_BRIDGE_RESOLVER now say it instead, the
latter false by default and true in the swarm lab, which provisions the dnsmasq
its nodes resolve through. Extra-vars are global, so the lab's NFS and backup
hosts carry it although no listener is installed there; the clearnet resolver
behind the bridge keeps them working.

The docker0 fact is gone from the address too. It is not merely late, it is
wrong while it is late: facts are read before the daemon config is applied, so
docker0 still holds docker's default and writing default-address-pools moves
the bridge out from under it. A swarm node shipped "dns": ["172.17.0.1", ...]
against a live 10.208.0.1, which is why its containers asked a dead address and
fell through to the public resolver, where infinito.example does not exist. The
pool answers before the daemon exists and answers the same afterwards, and it
is read per slice rather than per supernet, which differ once a pool does not
start on a slice boundary.

A list that would carry only the bridge now gets NETWORK_PUBLIC_DNS_RESOLVERS
behind it, so a declaration that turns out to be wrong costs a wasted first
query rather than all name resolution. An empty declaration raises.

Rendered from the repository's own variable files: the Instructions host yields
[] and the template omits the key, which is what de95a95c65 emitted while that
job was green; lab and tor nodes yield the live bridge ahead of the clearnet
resolver even with a stale docker0 fact present; an ordinary host yields only
its declared resolver.

The suite asserted the two behaviours this replaces, including one case added
with 285ba1e76c demanding the stale fact win. They are inverted, and a
cross-product over nameserver shapes, tor membership, clearnet and the docker0
fact now asserts that no shape ends up holding the bridge alone - the old suite
checked each shape by itself, which is how this passed through it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 00:32:13 +02:00

199 lines
8.7 KiB
Python

from __future__ import annotations
from ipaddress import ip_network
from typing import Any
from ansible.errors import AnsibleError
from ansible.plugins.lookup import LookupBase
from utils.networks.address import subnet_gateway
from utils.templating.ansible import _trust_as_template
_TOR_ROLE = "svc-net-tor"
def _serves_on_the_bridge(variables: dict[str, Any]) -> bool:
"""Return whether a resolver on the docker bridge is there to be used.
Args:
variables: the variables in scope for the lookup.
This used to be inferred from the host's own ``resolv.conf``: every entry a
loopback address was read as "a listener of ours answers, and containers
must reach it on the bridge instead". The inference is wrong on any host
running systemd-resolved or NetworkManager's dnsmasq, whose stubs bind
``127.0.0.53`` and ``127.0.0.1`` and nothing else. Those hosts were handed a
bridge address with no server behind it, which cost them all container DNS -
an image build failed on ``apk`` resolving nothing while the daemon itself,
resolving through the host, pulled the same image fine.
Two things do bind the bridge and both say so: ``svc-net-tor``, whose
dnsmasq listens there so ``.onion`` resolves, and any deployment that sets
``NETWORK_CONTAINER_BRIDGE_RESOLVER``.
"""
if _TOR_ROLE in (variables.get("group_names") or []):
return True
return bool(variables.get("NETWORK_CONTAINER_BRIDGE_RESOLVER"))
def render(value: Any, templar: Any) -> str:
"""Return an inventory value with its Jinja resolved.
Args:
value: whatever the inventory holds for the field.
templar: the lookup's templar, or None outside a play.
``networks.internet.dns`` arrives through ``-e @inventories/...`` and is
read straight out of ``variables``, which hands back whatever the file
contains. While that was a literal address nobody noticed; an expression
there reached the daemon as its own source text and docker refused the
config with ``ParseAddr("{{ ... }}")``.
``str()`` drops the ``TrustedAsTemplate`` tag the loader attached, and the
templar returns an untagged string unrendered. Re-tagging before templating
is what every sibling lookup does; without it this function would trade the
daemon's parse error for its own.
"""
text = str(value or "")
if "{{" not in text:
return text
rendered = (
str(templar.template(_trust_as_template(text))) if templar is not None else text
)
if "{{" in rendered:
raise AnsibleError(
f"networks.internet.dns is {text!r} and did not resolve to an "
f"address. Emitting it would put that source text in daemon.json, "
f"where docker refuses the whole config."
)
return rendered
def bridge_address(variables: dict[str, Any]) -> str:
"""Return the docker bridge gateway containers can reach the host on.
Args:
variables: the variables in scope for the lookup.
``ansible_facts`` only carries ``docker0`` once the daemon has run, and the
daemon config is written before that on a fresh node - facts are gathered at
play start, and nothing re-gathers them after the install. Reading the fact
alone therefore yields nothing on the very run that has to emit the address,
which leaves the inner containers pointed at the clearnet resolver and the
DinD DNS check failing on a domain only the node dnsmasq knows.
``default-address-pools`` is where docker takes that address from, so the
declaration answers before the daemon exists and answers identically
afterwards. That also keeps daemon.json byte-identical across passes, which
the fact-derived value could not: it appeared only from the second run on
and restarted the container runtime under a live stack.
The fact is not merely late, it is wrong while it is late. Facts are read
before the daemon config is applied, so ``docker0`` still carries docker's
own default; writing the pools then moves the bridge underneath it. A swarm
node measured ``172.17.0.1`` in daemon.json against a live ``10.208.0.1``,
which is why the declaration answers here and the fact never does.
The pool's ``size`` bounds the slice docker hands the bridge, so the gateway
is derived from that slice rather than from the enclosing supernet - they
agree only while the pool starts on a slice boundary.
"""
pools = variables.get("NETWORK_DOCKER_ADDRESS_POOLS") or []
first = (pools[0] or {}) if pools else {}
base = str(first.get("base") or "")
if not base:
return ""
size = first.get("size")
try:
network = ip_network(base, strict=True)
slice_ = (
network if size is None else next(network.subnets(new_prefix=int(size)))
)
return subnet_gateway(str(slice_))
except ValueError as exc:
raise AnsibleError(
f"networks: the first docker address pool {base!r} holds no gateway "
f"address, so containers have no way back to the host resolver."
) from exc
def public_resolver(variables: dict[str, Any]) -> str:
"""Return the resolver to fall back on when the bridge may serve nothing.
Args:
variables: the variables in scope for the lookup.
Read strictly: a missing declaration must fail the render rather than emit
a hardcoded address, which the repository forbids and lints for. Only the
first entry is taken because a resolver list is capped at three and the
bridge already holds one of those slots.
"""
resolvers = variables.get("NETWORK_PUBLIC_DNS_RESOLVERS")
if not resolvers:
raise AnsibleError(
"networks: NETWORK_PUBLIC_DNS_RESOLVERS is empty, so a host whose "
"docker bridge serves no DNS would be left without any resolver."
)
return str(resolvers[0])
def resolve_container_dns(variables: dict[str, Any], templar: Any = None) -> list[str]:
"""Return the resolver list for the container runtime, most specific first.
On a Tor node the node dnsmasq resolves ``.onion`` through Tor's DNSPort
and everything else through the clearnet upstream, which is what makes
per-app proxy settings unnecessary. It is only reachable on the docker
bridge address because docker discards loopback resolvers, and it is
absent before docker itself is installed, hence the guard.
The same reachability problem exists without Tor wherever the host resolves
through a listener of its own on loopback, which is how the simulated
cluster nodes are wired. Handing containers the declared clearnet address
there was what a literal bridge address in the inventory used to paper
over, and that literal went stale the moment the daemon's address pool
moved the bridge off Docker's default.
Both deploy modes get the bridge: a Tor node runs the same dnsmasq
listener in compose and in swarm, so both resolve onions alike.
The clearnet resolver stays as a second entry so that losing dnsmasq costs
onion names rather than all name resolution. Duplicates are dropped: where
the bridge *is* the clearnet resolver, emitting it twice makes daemon.json
differ between a run before and a run after ``docker0`` exists, and that
difference restarts the container runtime underneath a live stack.
Whether a resolver answers on the bridge is not something the host's own
``resolv.conf`` can say - a systemd-resolved stub and a node dnsmasq both
read as loopback-only, and only the second one also binds the bridge. Until
that is declared rather than inferred, the bridge may be an address nothing
serves, so it never goes out alone: a list that would carry it by itself
gets the project's public resolver behind it. Without that the daemon has a
single dead nameserver and every container and image build loses DNS.
"""
bridge = bridge_address(variables)
clearnet = render(
((variables.get("networks") or {}).get("internet") or {}).get("dns"), templar
)
resolvers = (bridge if _serves_on_the_bridge(variables) else "", clearnet)
emitted = list(dict.fromkeys(str(r) for r in resolvers if r))
if emitted == [bridge]:
emitted.append(public_resolver(variables))
return emitted
class LookupModule(LookupBase):
"""
Usage:
{{ lookup('container_dns') }}
Resolver list for the docker daemon's ``dns`` key. Takes no terms.
"""
def run(self, terms, variables: dict[str, Any] | None = None, **kwargs):
if terms:
raise ValueError("lookup('container_dns') takes no positional terms.")
variables = variables or getattr(self._templar, "available_variables", {}) or {}
return [resolve_container_dns(variables, self._templar)]