fix(stalwart): fix all comments, fix tor approach, SPOT fixes and chagne some values to follow standars + remove comemnts

This commit is contained in:
Alejandro Roman committed 2026-09-02 22:05:45 +04:00
1 parent e809765e23
commit 942fdb5fad
25 files changed
+802 -2

No files matched your search

+112
View File
@@ -0,0 +1,112 @@
# Tor SMTP Gateway (svc-net-tor-smtp)
## Description
A small SMTP relay that carries `.onion` recipients over Tor. The active mail
provider (Stalwart) cannot open a SOCKS connection of its own, and a `.onion`
address cannot be reached any other way, so the provider hands every `.onion`
recipient to this gateway as an ordinary SMTP relay target and the gateway dials
`<recipient-domain>:25` through Tor's SOCKS5 listener.
## Overview
```
Stalwart ──(relay route: rcpt ends_with .onion)──▶ tor-smtp:1525
tor-smtp ──(SOCKS5, remote DNS)──▶ Tor ──▶ <recipient>.onion:25
```
Clearnet mail is untouched — it stays on the provider's normal MX delivery. The
gateway only ever accepts `.onion` recipients; it is not an open relay.
## Cosmos
The diagram places Tor SMTP Gateway in the Infinito.Nexus cosmos: the container
it deploys (capabilities), the Tor daemon it consumes (dependencies), and its
outward reach to `.onion` mail hosts (cosmos).
```mermaid
flowchart LR
subgraph deps [Dependencies]
dep_svc_net_tor["svc-net-tor 🐳🐝"]
end
subgraph role [svc-net-tor-smtp 🐳🐝]
svc_tor_smtp["tor-smtp"]
end
subgraph dependents [Dependents]
dpt_web_app_stalwart["web-app-stalwart 🐳🐝"]
end
subgraph cosmos [Cosmos]
onion["*.onion MX ✉️"]
end
dpt_web_app_stalwart -- "relay .onion" --> svc_tor_smtp
svc_tor_smtp -- "SOCKS5" --> dep_svc_net_tor
dep_svc_net_tor -. "Tor circuit" .-> onion
```
Solid edges are fixed relationships; dashed edges are conditional. Node markers
show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm).
## Features
- **Onion-only relay:** accepts a recipient only when its domain ends with
`.onion`; every clearnet recipient is refused with `550`.
- **Remote resolution:** dials through Tor's SOCKS5 with `proxy_rdns`, so the
`.onion` name is resolved by Tor and never looked up locally.
- **One hop per domain:** recipients sharing a `.onion` domain are delivered in a
single SMTP transaction.
- **Clearnet untouched:** normal SMTP reputation, DKIM and SPF delivery never
pass through Tor.
## Quick Setup
### Development
Clone, set up the workstation, and deploy Tor SMTP Gateway (svc-net-tor-smtp) onto the local stack:
```bash
git clone https://github.com/infinito-nexus/core.git
cd core
make onboard
make compose-deploy mode=reinstall apps=svc-net-tor-smtp full_cycle=false
```
### Production
Run the published image to provision the inventory and deploy Tor SMTP Gateway (svc-net-tor-smtp) to a managed server (the mounted volume persists the inventory):
```bash
APP=svc-net-tor-smtp
HOST=<your-server>
TLS_MODE=self_signed
SSH_PUBLIC_KEY="<your-ssh-public-key>"
docker run --rm -it \
-v "$PWD/inventories:/etc/infinito.nexus/inventories" \
-e APP="$APP" -e HOST="$HOST" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \
ghcr.io/infinito-nexus/core/debian bash -c '
INVENTORY=/etc/infinito.nexus/inventories/production
infinito administration inventory provision "$INVENTORY" \
--inventory-file "$INVENTORY/devices.yml" \
--host "$HOST" \
--include "$APP" \
--vars "{\"TLS_MODE\": \"$TLS_MODE\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" &&
infinito administration deploy dedicated "$INVENTORY/devices.yml" \
--password-file "$INVENTORY/.password" \
--diff -vv'
```
## Design Decisions
- **A dedicated role, not Stalwart config.** Stalwart has no outbound SOCKS
support ([stalwartlabs/stalwart#644](https://github.com/stalwartlabs/stalwart/issues/644)),
so the Tor hop lives in a separate service that only the `.onion` route uses.
- **`proxy_rdns` is on.** A `.onion` has no DNS entry, so Tor must resolve it —
the name is never resolved locally.
- **Co-located with `svc-net-tor`.** It dials that role's SOCKS listener, so it
is manager-pinned like the Tor daemon.
## Credits
Implemented by **[Alejandro Roman Ibanez](https://github.com/AlejandroRomanIbanez)**.
Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world).
Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license).
+14
View File
@@ -0,0 +1,14 @@
# Tor SMTP gateway: receives SMTP from the active mail provider and relays each
# .onion recipient through Tor's SOCKS5 listener. aiosmtpd is the SMTP server,
# PySocks the SOCKS client — both pinned so the image is reproducible.
ARG PY_BASE
FROM ${PY_BASE}
RUN pip install --no-cache-dir "aiosmtpd==1.4.6" "PySocks==1.7.1"
COPY tor_smtp_gateway.py /usr/local/bin/tor_smtp_gateway.py
RUN adduser --system --no-create-home --group gateway
USER gateway
ENTRYPOINT ["python", "/usr/local/bin/tor_smtp_gateway.py"]
@@ -0,0 +1,130 @@
#!/usr/bin/env python3
"""SMTP-to-Tor gateway: accept a message, deliver it to the recipient's ``.onion``.
The active mail provider (Stalwart) cannot open a SOCKS connection of its own
(stalwartlabs/stalwart#644), and a ``.onion`` recipient can be reached no other
way — it has no DNS entry, so MX resolution and a direct TCP connect both fail.
So the provider's outbound routing hands every ``.onion`` recipient to this
gateway as a plain SMTP relay target, and the gateway dials
``<recipient-domain>:25`` through Tor's SOCKS5 listener. ``proxy_rdns`` is on:
the hostname is resolved by Tor, never locally, which is the whole point.
Only ``.onion`` recipients are accepted — the gateway is not an open relay and
never touches clearnet delivery, which stays on the provider's normal MX path.
Environment (compose supplies the first four from the role config; the last two
carry operational defaults):
TOR_SMTP_LISTEN_PORT bind port (required)
TOR_SOCKS_HOST Tor SOCKS host (required)
TOR_SOCKS_PORT Tor SOCKS port (required)
TOR_SMTP_HELO EHLO name announced to the peer (required — the node's
mail identity; never the bind address, which strict
receivers reject)
TOR_SMTP_LISTEN_HOST bind address (default 0.0.0.0)
TOR_SMTP_TIMEOUT per-hop seconds (default 120)
"""
from __future__ import annotations
import asyncio
import logging
import os
import signal
import smtplib
import socks
from aiosmtpd.controller import Controller
log = logging.getLogger("tor-smtp-gateway")
ONION_SMTP_PORT = 25
LISTEN_HOST = os.environ.get("TOR_SMTP_LISTEN_HOST", "0.0.0.0") # noqa: S104 - relay bind; reachability is gated by the compose network, not this bind
LISTEN_PORT = int(os.environ["TOR_SMTP_LISTEN_PORT"])
SOCKS_HOST = os.environ["TOR_SOCKS_HOST"]
SOCKS_PORT = int(os.environ["TOR_SOCKS_PORT"])
HELO_NAME = os.environ["TOR_SMTP_HELO"]
TIMEOUT = int(os.environ.get("TOR_SMTP_TIMEOUT", "120"))
class _SocksSMTP(smtplib.SMTP):
"""An ``smtplib.SMTP`` whose socket is a Tor SOCKS5 tunnel to a ``.onion``."""
def __init__(self, onion: str, port: int, timeout: int) -> None:
self._onion = onion
self._onion_port = port
super().__init__(local_hostname=HELO_NAME, timeout=timeout)
def _get_socket(self, host, port, timeout): # noqa: ARG002 - host/port fixed to the onion
return socks.create_connection(
(self._onion, self._onion_port),
timeout=timeout,
proxy_type=socks.SOCKS5,
proxy_addr=SOCKS_HOST,
proxy_port=SOCKS_PORT,
proxy_rdns=True,
)
def _deliver(onion: str, mail_from: str, rcpts: list[str], content: bytes) -> None:
smtp = _SocksSMTP(onion, ONION_SMTP_PORT, TIMEOUT)
try:
smtp.connect(onion, ONION_SMTP_PORT)
smtp.ehlo_or_helo_if_needed()
smtp.sendmail(mail_from, rcpts, content)
finally:
try:
smtp.quit()
except Exception: # noqa: BLE001 - a failed QUIT must not mask a successful send
smtp.close()
def _group_by_domain(rcpts: list[str]) -> dict[str, list[str]]:
grouped: dict[str, list[str]] = {}
for rcpt in rcpts:
domain = rcpt.rsplit("@", 1)[-1].lower()
grouped.setdefault(domain, []).append(rcpt)
return grouped
class OnionRelay:
"""aiosmtpd handler that relays each recipient's message over Tor."""
async def handle_RCPT(self, server, session, envelope, address, rcpt_options): # noqa: N802, ARG002 - aiosmtpd hook name
if not address.rsplit("@", 1)[-1].lower().endswith(".onion"):
return "550 5.7.1 tor-smtp-gateway relays .onion recipients only"
envelope.rcpt_tos.append(address)
return "250 OK"
async def handle_DATA(self, server, session, envelope): # noqa: N802, ARG002 - aiosmtpd hook name
loop = asyncio.get_running_loop()
for domain, rcpts in _group_by_domain(envelope.rcpt_tos).items():
try:
await loop.run_in_executor(
None, _deliver, domain, envelope.mail_from, rcpts, envelope.content
)
except Exception as exc: # noqa: BLE001 - any failure is a transient relay error
log.warning("onion relay to %s failed: %s", domain, exc)
return f"451 4.4.1 onion relay to {domain} failed"
return "250 2.0.0 accepted for onion delivery"
def main() -> None:
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
controller = Controller(OnionRelay(), hostname=LISTEN_HOST, port=LISTEN_PORT)
controller.start()
log.info("tor-smtp-gateway listening on %s:%s → SOCKS %s:%s", LISTEN_HOST, LISTEN_PORT, SOCKS_HOST, SOCKS_PORT)
stop = asyncio.Event()
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
for sig in (signal.SIGTERM, signal.SIGINT):
loop.add_signal_handler(sig, stop.set)
try:
loop.run_until_complete(stop.wait())
finally:
controller.stop()
loop.close()
if __name__ == "__main__":
main()
+32
View File
@@ -0,0 +1,32 @@
---
# nocheck: info-media no official demo video
galaxy_info:
author: Alejandro Roman Ibanez
description: SMTP gateway that relays .onion recipients through Tor's SOCKS listener
license: Infinito.Nexus Community License (Non-Commercial)
galaxy_tags:
- tor
- onion
- smtp
- mail
- networking
- self-hosted
issue_tracker_url: https://s.infinito.nexus/issues
min_ansible_version: "2.9"
platforms:
- name: ArchLinux
versions:
- all
- name: Debian
versions:
- all
- name: EL
versions:
- all
- name: Fedora
versions:
- all
- name: Ubuntu
versions:
- all
dependencies: []
+10
View File
@@ -0,0 +1,10 @@
---
local:
subnet: 192.168.109.0/24
overlay:
modes: [compose, swarm]
topology: shared_net
aliases: [tor-smtp]
consumer:
kind: services_flags
key: tor-smtp
+32
View File
@@ -0,0 +1,32 @@
---
# nocheck: dockerfile-custom
# nocheck: playwright-service-flag — SMTP egress daemon, no browser surface
tor-smtp: # nocheck: category-modes
primary: true
bond: 1
enabled: true
shared: true
custom: true
modes:
compose:
enabled: true
swarm:
enabled: true
# nocheck: default-placement-manager dials svc-net-tor's host-loopback SOCKS listener, which is manager-pinned; the gateway must share that node
placement: manager
image: python
version: 3.13-slim
name: tor-smtp
ports:
internal:
smtp: 1525
# Exception: the slim image ships no curl/nc — a stdlib socket connect is the
# probe, and it reads the listen port from the same env the daemon binds (SPOT).
healthcheck:
test: ["CMD", "python", "-c", "import os,socket; socket.create_connection(('127.0.0.1', int(os.environ['TOR_SMTP_LISTEN_PORT'])), 3).close()"]
min_storage: 100MB
cpus: "0.25"
mem_reservation: 64m
mem_limit: 128m
pids_limit: 64
lifecycle: alpha
+25
View File
@@ -0,0 +1,25 @@
---
- include_tasks: utils/once/flag.yml
- name: "📂 Include sys-svc-compose (build deferred to stage the daemon first)"
include_role:
name: sys-svc-compose
vars:
compose_handlers_flush: false
compose_repository_pull: false
application_id: "svc-net-tor-smtp"
- name: "📁 Create container network"
include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/network/create.yml') }}"
vars:
network_role_id: "{{ application_id }}"
- name: "🎬 Stage the gateway daemon into the build context"
stack_host_copy:
src: "python/tor_smtp_gateway.py"
dest: "{{ [ lookup('container', application_id, 'directories.instance'), 'tor_smtp_gateway.py' ] | path_join }}"
mode: "0644"
when: IS_STACK_HOST | bool
- name: "🚿 Flush compose handlers (build + up with the daemon present)"
meta: flush_handlers
+9
View File
@@ -0,0 +1,9 @@
---
- include_tasks: 00_core.yml
when: run_once_svc_net_tor_smtp is not defined
vars:
application_id: svc-net-tor-smtp
- include_tasks: utils/load/handlers.yml
vars:
handler_role_name: "sys-svc-compose"
@@ -0,0 +1,20 @@
{% include 'roles/sys-svc-compose/templates/base.yml.j2' %}
{% set service_name = 'tor-smtp' %}
{{ service_name }}:
{{ lookup('template', 'roles/sys-svc-container/templates/build.yml.j2') | indent(4) }}
args:
PY_BASE: "{{ TOR_SMTP_IMAGE }}:{{ TOR_SMTP_VERSION }}"
{{ lookup('container_image', application_id, service_name) }}
{{ lookup('compose_only', 'container_name', TOR_SMTP_CONTAINER) }}
{% include 'roles/sys-svc-container/templates/base.yml.j2' %}
environment:
TOR_SMTP_LISTEN_PORT: "{{ TOR_SMTP_LISTEN_PORT }}"
TOR_SOCKS_HOST: "{{ TOR_SMTP_SOCKS_HOST }}"
TOR_SOCKS_PORT: "{{ TOR_SMTP_SOCKS_PORT }}"
TOR_SMTP_HELO: "{{ TOR_SMTP_HELO }}"
{{ lookup('container_extra_hosts') | indent(4) }}
{{ lookup('container_healthcheck', service_name) | indent(4) }}
{{ lookup('container_networks') | indent(4) }}
{{ lookup('compose_volumes') }}
{{ lookup('compose_networks') }}
+13
View File
@@ -0,0 +1,13 @@
---
application_id: "svc-net-tor-smtp"
TOR_SMTP_IMAGE: "{{ lookup('config', application_id, 'services.tor-smtp.image') }}"
TOR_SMTP_VERSION: "{{ lookup('config', application_id, 'services.tor-smtp.version') }}"
TOR_SMTP_CONTAINER: "{{ lookup('config', application_id, 'services.tor-smtp.name') }}"
TOR_SMTP_LISTEN_PORT: "{{ lookup('config', application_id, 'services.tor-smtp.ports.internal.smtp') }}"
TOR_SMTP_SOCKS: "{{ lookup('tor_socks_proxy') | regex_replace('^socks5h://', '') }}"
TOR_SMTP_SOCKS_HOST: "{{ TOR_SMTP_SOCKS.split(':')[0] }}"
TOR_SMTP_SOCKS_PORT: "{{ TOR_SMTP_SOCKS.split(':')[1] }}"
TOR_SMTP_HELO: "{{ DOMAIN_PRIMARY }}"
@@ -11,7 +11,7 @@ set -eu
# servers, so a misconfigured host surfaces in CI instead of in an outage.
#
# Usage:
# test.sh [<docker_compose_instance_directory>]
# verify.sh [<docker_compose_instance_directory>]
#
# Environment:
# CERT_TEST_REMOTE_ENDPOINTS Space-separated "host[:port]" list (port
@@ -22,7 +22,7 @@
- name: "✅ Verify deployed certificate material"
ansible.builtin.script:
cmd: "{{ [ role_path, 'files/shell/test.sh' ] | path_join }} {{ lookup('container', application_id, 'directories.instance') }}"
cmd: "{{ [ role_path, 'files/shell/verify.sh' ] | path_join }} {{ lookup('container', application_id, 'directories.instance') }}"
environment:
CERT_TEST_REMOTE_ENDPOINTS: >-
{{ cert_deploy_remote_endpoints
+8
View File
@@ -144,3 +144,11 @@ tor:
bond: 0.5
enabled: "{{ 'svc-net-tor' in group_names }}"
shared: true # nocheck: dynamic-flag
# nocheck: playwright-service-flag — SMTP egress relay for .onion recipients, no UI surface
# nocheck: variants-coverage — the onion is never pinned on for this role; see the sso-variant-tor note in meta/variants.yml
# nocheck: bond-baseline — same onion-gated rationale as the tor flag above
# nocheck: dynamic-flag — gated on the onion (svc-net-tor), not on svc-net-tor-smtp itself: the gateway is a derived companion, never a standalone inventory group
tor-smtp:
bond: 0.5
enabled: "{{ 'svc-net-tor' in group_names }}"
shared: true # nocheck: dynamic-flag
@@ -66,3 +66,7 @@
- name: "🔎 Verify the production mail server's public configuration"
include_tasks: 13_production_facts.yml
when: STALWART_PRODUCTION_HOST | length > 0
- name: "🧅 Route .onion recipients through the Tor SMTP gateway"
include_tasks: 14_onion_route.yml
when: STALWART_ONION_ACTIVE | bool
@@ -0,0 +1,77 @@
---
# Route .onion recipients through the Tor SMTP gateway (svc-net-tor-smtp).
# Stalwart has no outbound SOCKS of its own, so a relay route hands every
# .onion recipient to the gateway, which dials the target over Tor.
# JMAP request bodies live in templates/jmap/route|strategy/*.json.j2.
# Trip-wire: the MtaOutboundStrategy singleton is NOT enumerated by
# get(ids:null) — strategy/get.json.j2 MUST query ids:["singleton"], or the
# existence check is always false and the update path is skipped.
- name: "📖 List existing Stalwart outbound routes"
include_tasks: jmap_call.yml
vars:
stalwart_jmap_body: "{{ lookup('template', 'jmap/route/get.json.j2') }}"
- name: "🔍 Resolve whether the onion relay route already exists"
ansible.builtin.set_fact:
stalwart_onion_route_exists: >-
{{ (stalwart_jmap.methodResponses[0][1].list | default([]))
| selectattr('name', 'equalto', STALWART_ONION_ROUTE_NAME)
| list | length > 0 }}
- name: "📁 Create the onion relay route to the Tor SMTP gateway"
include_tasks: jmap_call.yml
vars:
stalwart_jmap_body: "{{ lookup('template', 'jmap/route/create.json.j2') }}"
when: not (stalwart_onion_route_exists | bool)
- name: "⚙️ Build the outbound route strategy (local, then onion, else mx)"
ansible.builtin.set_fact:
stalwart_route_expr:
match:
"0":
if: "is_local_domain(rcpt_domain)"
then: "'local'"
"1":
if: "ends_with(rcpt_domain, '.onion')"
then: "'{{ STALWART_ONION_ROUTE_NAME }}'"
else: "'mx'"
- name: "📖 Read the outbound strategy singleton"
include_tasks: jmap_call.yml
vars:
stalwart_jmap_body: "{{ lookup('template', 'jmap/strategy/get.json.j2') }}"
- name: "🔧 Record whether the outbound strategy singleton exists"
ansible.builtin.set_fact:
stalwart_strategy_exists: "{{ (stalwart_jmap.methodResponses[0][1].list | length) > 0 }}"
- name: "📁 Create the outbound strategy with the onion route"
include_tasks: jmap_call.yml
vars:
stalwart_jmap_body: "{{ lookup('template', 'jmap/strategy/create.json.j2') }}"
when: not (stalwart_strategy_exists | bool)
- name: "🔀 Point the outbound strategy at the onion route for .onion recipients"
include_tasks: jmap_call.yml
vars:
stalwart_jmap_body: "{{ lookup('template', 'jmap/strategy/update.json.j2') }}"
when: stalwart_strategy_exists | bool
- name: "📖 Read back the live outbound strategy"
include_tasks: jmap_call.yml
vars:
stalwart_jmap_body: "{{ lookup('template', 'jmap/strategy/get.json.j2') }}"
- name: "✅ Assert .onion recipients are routed to the gateway relay on the live server"
ansible.builtin.assert:
that:
- stalwart_live_route | regex_search("ends_with\\(rcpt_domain, '\\.onion'\\)")
- stalwart_live_route | regex_search("'" ~ STALWART_ONION_ROUTE_NAME ~ "'")
- stalwart_live_route | regex_search("is_local_domain\\(rcpt_domain\\)")
fail_msg: >-
Stalwart's live outbound strategy does not route .onion recipients to the
'{{ STALWART_ONION_ROUTE_NAME }}' gateway route (or dropped local delivery):
{{ stalwart_live_route }}
success_msg: ".onion recipients route to the '{{ STALWART_ONION_ROUTE_NAME }}' gateway relay, local delivery preserved"
vars:
stalwart_live_route: "{{ (stalwart_jmap.methodResponses[0][1].list | first).route | to_json }}"
@@ -0,0 +1,13 @@
{
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
"methodCalls": [["x:MtaRoute/set", {"create": {"onion": {
"@type": "Relay",
"name": {{ STALWART_ONION_ROUTE_NAME | to_json }},
"address": {{ STALWART_ONION_RELAY_HOST | to_json }},
"port": {{ STALWART_ONION_RELAY_PORT | int }},
"protocol": "smtp",
"implicitTls": false,
"allowInvalidCerts": true,
"authSecret": {"@type": "None"}
}}}, "c0"]]
}
@@ -0,0 +1,4 @@
{
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
"methodCalls": [["x:MtaRoute/get", {"ids": null}, "c0"]]
}
@@ -0,0 +1,4 @@
{
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
"methodCalls": [["x:MtaOutboundStrategy/set", {"create": {"s": {"route": {{ stalwart_route_expr | to_json }} }}}, "c0"]]
}
@@ -0,0 +1,4 @@
{
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
"methodCalls": [["x:MtaOutboundStrategy/get", {"ids": ["singleton"]}, "c0"]]
}
@@ -0,0 +1,4 @@
{
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
"methodCalls": [["x:MtaOutboundStrategy/set", {"update": {"singleton": {"route": {{ stalwart_route_expr | to_json }} }}}, "c0"]]
}
+6
View File
@@ -64,3 +64,9 @@ STALWART_IMPORT_MAILU: "{{ lookup('config', application_id, 'services.stalwart.m
STALWART_PRODUCTION_HOST: "{{ MAIL_PRODUCTION_HOST | trim }}"
STALWART_PRODUCTION_RESOLVER: "{{ MAIL_PRODUCTION_RESOLVER | trim }}"
STALWART_ONION_ACTIVE: "{{ 'svc-net-tor' in group_names }}"
STALWART_ONION_GATEWAY_ID: "svc-net-tor-smtp"
STALWART_ONION_ROUTE_NAME: "onion"
STALWART_ONION_RELAY_HOST: "{{ lookup('config', STALWART_ONION_GATEWAY_ID, 'services.tor-smtp.name') }}"
STALWART_ONION_RELAY_PORT: "{{ lookup('config', STALWART_ONION_GATEWAY_ID, 'services.tor-smtp.ports.internal.smtp') | int }}"
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[5]
@@ -0,0 +1,3 @@
from pathlib import Path
PROJECT_ROOT: Path = Path(__file__).resolve().parents[6]
@@ -0,0 +1,136 @@
"""What the Tor SMTP gateway accepts, and where it routes each recipient.
The gateway is not an open relay: it accepts a recipient ONLY when the domain
ends with ``.onion``, and it delivers each ``.onion`` domain's recipients in one
hop. The heavy dependencies (aiosmtpd, PySocks) are stubbed here so the routing
logic is testable without them, and ``_deliver`` is replaced so nothing dials
the network.
"""
from __future__ import annotations
import asyncio
import importlib.util
import os
import sys
import types
import unittest
from unittest import mock
from . import PROJECT_ROOT
MODULE = PROJECT_ROOT / "roles" / "svc-net-tor-smtp" / "files" / "python" / "tor_smtp_gateway.py"
REQUIRED_ENV = {
"TOR_SMTP_LISTEN_PORT": "1525",
"TOR_SOCKS_HOST": "tor",
"TOR_SOCKS_PORT": "9050",
"TOR_SMTP_HELO": "node.example.onion",
}
def _load(env=None):
"""Import the daemon with the required env stubbed. A None override drops
the key, so a test can assert a required var is missing."""
socks_stub = types.ModuleType("socks")
socks_stub.SOCKS5 = 2
socks_stub.create_connection = lambda *a, **k: None
controller_stub = types.ModuleType("aiosmtpd.controller")
controller_stub.Controller = object
aiosmtpd_pkg = types.ModuleType("aiosmtpd")
aiosmtpd_pkg.controller = controller_stub
merged = {k: v for k, v in {**REQUIRED_ENV, **(env or {})}.items() if v is not None}
with mock.patch.dict(sys.modules, {
"socks": socks_stub,
"aiosmtpd": aiosmtpd_pkg,
"aiosmtpd.controller": controller_stub,
}), mock.patch.dict(os.environ, merged, clear=True):
spec = importlib.util.spec_from_file_location("tor_smtp_gateway", MODULE)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
class _Envelope:
def __init__(self, mail_from="bot@x.onion", rcpt_tos=None, content=b"msg"):
self.mail_from = mail_from
self.rcpt_tos = list(rcpt_tos or [])
self.content = content
class TestDomainGrouping(unittest.TestCase):
def setUp(self):
self.mod = _load()
def test_recipients_group_by_domain(self):
grouped = self.mod._group_by_domain(
["a@one.onion", "b@one.onion", "c@two.onion"]
)
self.assertEqual(grouped, {"one.onion": ["a@one.onion", "b@one.onion"], "two.onion": ["c@two.onion"]})
def test_domain_match_is_case_insensitive(self):
grouped = self.mod._group_by_domain(["A@One.Onion"])
self.assertEqual(list(grouped), ["one.onion"])
class TestRcptGate(unittest.IsolatedAsyncioTestCase):
def setUp(self):
self.mod = _load()
self.relay = self.mod.OnionRelay()
async def _rcpt(self, address):
env = _Envelope(rcpt_tos=[])
reply = await self.relay.handle_RCPT(None, None, env, address, [])
return reply, env
async def test_an_onion_recipient_is_accepted(self):
reply, env = await self._rcpt("biber@abc.onion")
self.assertTrue(reply.startswith("250"))
self.assertEqual(env.rcpt_tos, ["biber@abc.onion"])
async def test_a_clearnet_recipient_is_refused(self):
reply, env = await self._rcpt("biber@infinito.test")
self.assertTrue(reply.startswith("550"))
self.assertEqual(env.rcpt_tos, [])
class TestDataDelivery(unittest.IsolatedAsyncioTestCase):
def setUp(self):
self.mod = _load()
self.relay = self.mod.OnionRelay()
async def test_each_onion_domain_gets_one_delivery(self):
calls = []
with mock.patch.object(self.mod, "_deliver", side_effect=lambda *a: calls.append(a)):
env = _Envelope(rcpt_tos=["a@one.onion", "b@one.onion", "c@two.onion"])
reply = await self.relay.handle_DATA(None, None, env)
self.assertTrue(reply.startswith("250"))
self.assertEqual({c[0] for c in calls}, {"one.onion", "two.onion"})
async def test_a_failed_hop_is_a_transient_error(self):
def boom(*_a):
raise OSError("tor down")
with mock.patch.object(self.mod, "_deliver", side_effect=boom):
env = _Envelope(rcpt_tos=["a@one.onion"])
reply = await self.relay.handle_DATA(None, None, env)
self.assertTrue(reply.startswith("451"))
class TestConfiguration(unittest.TestCase):
def test_the_helo_name_comes_from_the_environment(self):
mod = _load({"TOR_SMTP_HELO": "biber.node.onion"})
self.assertEqual(mod.HELO_NAME, "biber.node.onion")
def test_the_helo_never_falls_back_to_the_bind_address(self):
"""EHLO 0.0.0.0 is malformed; a missing HELO must fail loudly, not
silently announce the bind address to the peer."""
with self.assertRaises(KeyError):
_load({"TOR_SMTP_HELO": None})
def test_the_onion_hop_targets_the_well_known_smtp_port(self):
self.assertEqual(_load().ONION_SMTP_PORT, 25)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,137 @@
"""How Stalwart routes .onion recipients to the Tor SMTP gateway.
Stalwart has no outbound SOCKS of its own, so ``14_onion_route.yml`` provisions
a relay ``MtaRoute`` to the gateway and points the ``MtaOutboundStrategy`` route
expression at it for ``.onion`` recipients. The one property that must never
regress: the strategy still sends local recipients to ``local`` and everything
else to ``mx`` — adding the onion branch must not break normal delivery.
"""
from __future__ import annotations
import json
import unittest
from jinja2 import Environment, StrictUndefined
from utils.cache.files import read_text
from utils.cache.yaml import load_yaml_any
from . import PROJECT_ROOT
ROLE = PROJECT_ROOT / "roles" / "web-app-stalwart"
TASK = ROLE / "tasks" / "14_onion_route.yml"
ROUTE_CREATE = ROLE / "templates" / "jmap" / "route" / "create.json.j2"
STRATEGY_GET = ROLE / "templates" / "jmap" / "strategy" / "get.json.j2"
STRATEGY_CREATE = ROLE / "templates" / "jmap" / "strategy" / "create.json.j2"
STRATEGY_UPDATE = ROLE / "templates" / "jmap" / "strategy" / "update.json.j2"
ROUTE_NAME = "onion"
def _render(template_path, **variables) -> str:
env = Environment(
trim_blocks=True,
lstrip_blocks=True,
undefined=StrictUndefined,
autoescape=False, # noqa: S701 - JMAP request bodies, never HTML
)
env.filters["to_json"] = json.dumps
env.filters["int"] = int
return env.from_string(read_text(template_path)).render(**variables)
def _route_expr_from_task() -> dict:
tasks = load_yaml_any(str(TASK), default_if_missing=[])
for task in tasks:
facts = task.get("set_fact") or task.get("ansible.builtin.set_fact") or {}
if "stalwart_route_expr" in facts:
return facts["stalwart_route_expr"]
raise AssertionError("stalwart_route_expr set_fact not found in 14_onion_route.yml")
class TestOutboundStrategyExpression(unittest.TestCase):
def setUp(self):
self.expr = _route_expr_from_task()
def test_local_recipients_still_route_to_local(self):
first = self.expr["match"]["0"]
self.assertEqual(first["if"], "is_local_domain(rcpt_domain)")
self.assertEqual(first["then"], "'local'")
def test_local_is_evaluated_before_onion(self):
"""A local .onion address must be delivered locally, not relayed out."""
keys = list(self.expr["match"].keys())
self.assertEqual(keys, ["0", "1"])
self.assertIn("is_local_domain", self.expr["match"]["0"]["if"])
self.assertIn("ends_with", self.expr["match"]["1"]["if"])
def test_onion_recipients_route_to_the_onion_route(self):
"""then is a Stalwart expression string literal naming the route, from
the same SPOT var the relay route registers itself under."""
onion = self.expr["match"]["1"]
self.assertEqual(onion["if"], "ends_with(rcpt_domain, '.onion')")
self.assertEqual(onion["then"], "'{{ STALWART_ONION_ROUTE_NAME }}'")
def test_everything_else_falls_back_to_mx(self):
self.assertEqual(self.expr["else"], "'mx'")
class TestStrategyTemplateRendersValidJson(unittest.TestCase):
def test_the_singleton_route_is_set_from_the_expression(self):
expr = _route_expr_from_task()
rendered = _render(STRATEGY_UPDATE, stalwart_route_expr=expr)
body = json.loads(rendered)
method, args, _tag = body["methodCalls"][0]
self.assertEqual(method, "x:MtaOutboundStrategy/set")
self.assertEqual(args["update"]["singleton"]["route"], expr)
def test_create_seeds_a_new_singleton_with_the_route(self):
expr = _route_expr_from_task()
rendered = _render(STRATEGY_CREATE, stalwart_route_expr=expr)
body = json.loads(rendered)
method, args, _tag = body["methodCalls"][0]
self.assertEqual(method, "x:MtaOutboundStrategy/set")
(created,) = args["create"].values()
self.assertEqual(created["route"], expr)
def test_get_queries_the_singleton_by_id(self):
"""The singleton is NOT enumerated by get(ids:null); it must be queried
by its fixed "singleton" id or existence detection is always wrong."""
body = json.loads(_render(STRATEGY_GET))
method, args, _tag = body["methodCalls"][0]
self.assertEqual(method, "x:MtaOutboundStrategy/get")
self.assertEqual(args["ids"], ["singleton"])
class TestRelayRouteRendersValidJson(unittest.TestCase):
def _create(self):
rendered = _render(
ROUTE_CREATE,
STALWART_ONION_ROUTE_NAME=ROUTE_NAME,
STALWART_ONION_RELAY_HOST="tor-smtp",
STALWART_ONION_RELAY_PORT=1525,
)
body = json.loads(rendered)
return body["methodCalls"][0]
def test_it_creates_a_relay_route_to_the_gateway(self):
method, args, _tag = self._create()
self.assertEqual(method, "x:MtaRoute/set")
route = args["create"][ROUTE_NAME]
self.assertEqual(route["@type"], "Relay")
self.assertEqual(route["name"], ROUTE_NAME)
self.assertEqual(route["address"], "tor-smtp")
self.assertEqual(route["port"], 1525)
def test_the_relay_hop_is_plaintext_smtp(self):
"""The gateway speaks plaintext SMTP on an internal network; there is no
cert to validate on a .onion relay hop."""
_method, args, _tag = self._create()
route = args["create"][ROUTE_NAME]
self.assertEqual(route["protocol"], "smtp")
self.assertFalse(route["implicitTls"])
if __name__ == "__main__":
unittest.main()