mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 02:54:26 +00:00
fix(stalwart): fix all comments, fix tor approach, SPOT fixes and chagne some values to follow standars + remove comemnts
This commit is contained in:
25 files changed
+802
-2
No files matched your search
@@ -0,0 +1,112 @@
|
||||
# Tor SMTP Gateway (svc-net-tor-smtp)
|
||||
|
||||
## Description
|
||||
|
||||
A small SMTP relay that carries `.onion` recipients over Tor. The active mail
|
||||
provider (Stalwart) cannot open a SOCKS connection of its own, and a `.onion`
|
||||
address cannot be reached any other way, so the provider hands every `.onion`
|
||||
recipient to this gateway as an ordinary SMTP relay target and the gateway dials
|
||||
`<recipient-domain>:25` through Tor's SOCKS5 listener.
|
||||
|
||||
## Overview
|
||||
|
||||
```
|
||||
Stalwart ──(relay route: rcpt ends_with .onion)──▶ tor-smtp:1525
|
||||
tor-smtp ──(SOCKS5, remote DNS)──▶ Tor ──▶ <recipient>.onion:25
|
||||
```
|
||||
|
||||
Clearnet mail is untouched — it stays on the provider's normal MX delivery. The
|
||||
gateway only ever accepts `.onion` recipients; it is not an open relay.
|
||||
|
||||
## Cosmos
|
||||
|
||||
The diagram places Tor SMTP Gateway in the Infinito.Nexus cosmos: the container
|
||||
it deploys (capabilities), the Tor daemon it consumes (dependencies), and its
|
||||
outward reach to `.onion` mail hosts (cosmos).
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph deps [Dependencies]
|
||||
dep_svc_net_tor["svc-net-tor 🐳🐝"]
|
||||
end
|
||||
subgraph role [svc-net-tor-smtp 🐳🐝]
|
||||
svc_tor_smtp["tor-smtp"]
|
||||
end
|
||||
subgraph dependents [Dependents]
|
||||
dpt_web_app_stalwart["web-app-stalwart 🐳🐝"]
|
||||
end
|
||||
subgraph cosmos [Cosmos]
|
||||
onion["*.onion MX ✉️"]
|
||||
end
|
||||
dpt_web_app_stalwart -- "relay .onion" --> svc_tor_smtp
|
||||
svc_tor_smtp -- "SOCKS5" --> dep_svc_net_tor
|
||||
dep_svc_net_tor -. "Tor circuit" .-> onion
|
||||
```
|
||||
|
||||
Solid edges are fixed relationships; dashed edges are conditional. Node markers
|
||||
show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm).
|
||||
|
||||
## Features
|
||||
|
||||
- **Onion-only relay:** accepts a recipient only when its domain ends with
|
||||
`.onion`; every clearnet recipient is refused with `550`.
|
||||
- **Remote resolution:** dials through Tor's SOCKS5 with `proxy_rdns`, so the
|
||||
`.onion` name is resolved by Tor and never looked up locally.
|
||||
- **One hop per domain:** recipients sharing a `.onion` domain are delivered in a
|
||||
single SMTP transaction.
|
||||
- **Clearnet untouched:** normal SMTP reputation, DKIM and SPF delivery never
|
||||
pass through Tor.
|
||||
|
||||
## Quick Setup
|
||||
|
||||
### Development
|
||||
|
||||
Clone, set up the workstation, and deploy Tor SMTP Gateway (svc-net-tor-smtp) onto the local stack:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/infinito-nexus/core.git
|
||||
cd core
|
||||
make onboard
|
||||
make compose-deploy mode=reinstall apps=svc-net-tor-smtp full_cycle=false
|
||||
```
|
||||
|
||||
### Production
|
||||
|
||||
Run the published image to provision the inventory and deploy Tor SMTP Gateway (svc-net-tor-smtp) to a managed server (the mounted volume persists the inventory):
|
||||
|
||||
```bash
|
||||
APP=svc-net-tor-smtp
|
||||
HOST=<your-server>
|
||||
TLS_MODE=self_signed
|
||||
SSH_PUBLIC_KEY="<your-ssh-public-key>"
|
||||
|
||||
docker run --rm -it \
|
||||
-v "$PWD/inventories:/etc/infinito.nexus/inventories" \
|
||||
-e APP="$APP" -e HOST="$HOST" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \
|
||||
ghcr.io/infinito-nexus/core/debian bash -c '
|
||||
INVENTORY=/etc/infinito.nexus/inventories/production
|
||||
infinito administration inventory provision "$INVENTORY" \
|
||||
--inventory-file "$INVENTORY/devices.yml" \
|
||||
--host "$HOST" \
|
||||
--include "$APP" \
|
||||
--vars "{\"TLS_MODE\": \"$TLS_MODE\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" &&
|
||||
infinito administration deploy dedicated "$INVENTORY/devices.yml" \
|
||||
--password-file "$INVENTORY/.password" \
|
||||
--diff -vv'
|
||||
```
|
||||
|
||||
## Design Decisions
|
||||
|
||||
- **A dedicated role, not Stalwart config.** Stalwart has no outbound SOCKS
|
||||
support ([stalwartlabs/stalwart#644](https://github.com/stalwartlabs/stalwart/issues/644)),
|
||||
so the Tor hop lives in a separate service that only the `.onion` route uses.
|
||||
- **`proxy_rdns` is on.** A `.onion` has no DNS entry, so Tor must resolve it —
|
||||
the name is never resolved locally.
|
||||
- **Co-located with `svc-net-tor`.** It dials that role's SOCKS listener, so it
|
||||
is manager-pinned like the Tor daemon.
|
||||
|
||||
## Credits
|
||||
|
||||
Implemented by **[Alejandro Roman Ibanez](https://github.com/AlejandroRomanIbanez)**.
|
||||
Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world).
|
||||
Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license).
|
||||
@@ -0,0 +1,14 @@
|
||||
# Tor SMTP gateway: receives SMTP from the active mail provider and relays each
|
||||
# .onion recipient through Tor's SOCKS5 listener. aiosmtpd is the SMTP server,
|
||||
# PySocks the SOCKS client — both pinned so the image is reproducible.
|
||||
ARG PY_BASE
|
||||
FROM ${PY_BASE}
|
||||
|
||||
RUN pip install --no-cache-dir "aiosmtpd==1.4.6" "PySocks==1.7.1"
|
||||
|
||||
COPY tor_smtp_gateway.py /usr/local/bin/tor_smtp_gateway.py
|
||||
|
||||
RUN adduser --system --no-create-home --group gateway
|
||||
USER gateway
|
||||
|
||||
ENTRYPOINT ["python", "/usr/local/bin/tor_smtp_gateway.py"]
|
||||
@@ -0,0 +1,130 @@
|
||||
#!/usr/bin/env python3
|
||||
"""SMTP-to-Tor gateway: accept a message, deliver it to the recipient's ``.onion``.
|
||||
|
||||
The active mail provider (Stalwart) cannot open a SOCKS connection of its own
|
||||
(stalwartlabs/stalwart#644), and a ``.onion`` recipient can be reached no other
|
||||
way — it has no DNS entry, so MX resolution and a direct TCP connect both fail.
|
||||
So the provider's outbound routing hands every ``.onion`` recipient to this
|
||||
gateway as a plain SMTP relay target, and the gateway dials
|
||||
``<recipient-domain>:25`` through Tor's SOCKS5 listener. ``proxy_rdns`` is on:
|
||||
the hostname is resolved by Tor, never locally, which is the whole point.
|
||||
|
||||
Only ``.onion`` recipients are accepted — the gateway is not an open relay and
|
||||
never touches clearnet delivery, which stays on the provider's normal MX path.
|
||||
|
||||
Environment (compose supplies the first four from the role config; the last two
|
||||
carry operational defaults):
|
||||
TOR_SMTP_LISTEN_PORT bind port (required)
|
||||
TOR_SOCKS_HOST Tor SOCKS host (required)
|
||||
TOR_SOCKS_PORT Tor SOCKS port (required)
|
||||
TOR_SMTP_HELO EHLO name announced to the peer (required — the node's
|
||||
mail identity; never the bind address, which strict
|
||||
receivers reject)
|
||||
TOR_SMTP_LISTEN_HOST bind address (default 0.0.0.0)
|
||||
TOR_SMTP_TIMEOUT per-hop seconds (default 120)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import smtplib
|
||||
|
||||
import socks
|
||||
from aiosmtpd.controller import Controller
|
||||
|
||||
log = logging.getLogger("tor-smtp-gateway")
|
||||
|
||||
ONION_SMTP_PORT = 25
|
||||
|
||||
LISTEN_HOST = os.environ.get("TOR_SMTP_LISTEN_HOST", "0.0.0.0") # noqa: S104 - relay bind; reachability is gated by the compose network, not this bind
|
||||
LISTEN_PORT = int(os.environ["TOR_SMTP_LISTEN_PORT"])
|
||||
SOCKS_HOST = os.environ["TOR_SOCKS_HOST"]
|
||||
SOCKS_PORT = int(os.environ["TOR_SOCKS_PORT"])
|
||||
HELO_NAME = os.environ["TOR_SMTP_HELO"]
|
||||
TIMEOUT = int(os.environ.get("TOR_SMTP_TIMEOUT", "120"))
|
||||
|
||||
|
||||
class _SocksSMTP(smtplib.SMTP):
|
||||
"""An ``smtplib.SMTP`` whose socket is a Tor SOCKS5 tunnel to a ``.onion``."""
|
||||
|
||||
def __init__(self, onion: str, port: int, timeout: int) -> None:
|
||||
self._onion = onion
|
||||
self._onion_port = port
|
||||
super().__init__(local_hostname=HELO_NAME, timeout=timeout)
|
||||
|
||||
def _get_socket(self, host, port, timeout): # noqa: ARG002 - host/port fixed to the onion
|
||||
return socks.create_connection(
|
||||
(self._onion, self._onion_port),
|
||||
timeout=timeout,
|
||||
proxy_type=socks.SOCKS5,
|
||||
proxy_addr=SOCKS_HOST,
|
||||
proxy_port=SOCKS_PORT,
|
||||
proxy_rdns=True,
|
||||
)
|
||||
|
||||
|
||||
def _deliver(onion: str, mail_from: str, rcpts: list[str], content: bytes) -> None:
|
||||
smtp = _SocksSMTP(onion, ONION_SMTP_PORT, TIMEOUT)
|
||||
try:
|
||||
smtp.connect(onion, ONION_SMTP_PORT)
|
||||
smtp.ehlo_or_helo_if_needed()
|
||||
smtp.sendmail(mail_from, rcpts, content)
|
||||
finally:
|
||||
try:
|
||||
smtp.quit()
|
||||
except Exception: # noqa: BLE001 - a failed QUIT must not mask a successful send
|
||||
smtp.close()
|
||||
|
||||
|
||||
def _group_by_domain(rcpts: list[str]) -> dict[str, list[str]]:
|
||||
grouped: dict[str, list[str]] = {}
|
||||
for rcpt in rcpts:
|
||||
domain = rcpt.rsplit("@", 1)[-1].lower()
|
||||
grouped.setdefault(domain, []).append(rcpt)
|
||||
return grouped
|
||||
|
||||
|
||||
class OnionRelay:
|
||||
"""aiosmtpd handler that relays each recipient's message over Tor."""
|
||||
|
||||
async def handle_RCPT(self, server, session, envelope, address, rcpt_options): # noqa: N802, ARG002 - aiosmtpd hook name
|
||||
if not address.rsplit("@", 1)[-1].lower().endswith(".onion"):
|
||||
return "550 5.7.1 tor-smtp-gateway relays .onion recipients only"
|
||||
envelope.rcpt_tos.append(address)
|
||||
return "250 OK"
|
||||
|
||||
async def handle_DATA(self, server, session, envelope): # noqa: N802, ARG002 - aiosmtpd hook name
|
||||
loop = asyncio.get_running_loop()
|
||||
for domain, rcpts in _group_by_domain(envelope.rcpt_tos).items():
|
||||
try:
|
||||
await loop.run_in_executor(
|
||||
None, _deliver, domain, envelope.mail_from, rcpts, envelope.content
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001 - any failure is a transient relay error
|
||||
log.warning("onion relay to %s failed: %s", domain, exc)
|
||||
return f"451 4.4.1 onion relay to {domain} failed"
|
||||
return "250 2.0.0 accepted for onion delivery"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
|
||||
controller = Controller(OnionRelay(), hostname=LISTEN_HOST, port=LISTEN_PORT)
|
||||
controller.start()
|
||||
log.info("tor-smtp-gateway listening on %s:%s → SOCKS %s:%s", LISTEN_HOST, LISTEN_PORT, SOCKS_HOST, SOCKS_PORT)
|
||||
stop = asyncio.Event()
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
for sig in (signal.SIGTERM, signal.SIGINT):
|
||||
loop.add_signal_handler(sig, stop.set)
|
||||
try:
|
||||
loop.run_until_complete(stop.wait())
|
||||
finally:
|
||||
controller.stop()
|
||||
loop.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
# nocheck: info-media no official demo video
|
||||
galaxy_info:
|
||||
author: Alejandro Roman Ibanez
|
||||
description: SMTP gateway that relays .onion recipients through Tor's SOCKS listener
|
||||
license: Infinito.Nexus Community License (Non-Commercial)
|
||||
galaxy_tags:
|
||||
- tor
|
||||
- onion
|
||||
- smtp
|
||||
- mail
|
||||
- networking
|
||||
- self-hosted
|
||||
issue_tracker_url: https://s.infinito.nexus/issues
|
||||
min_ansible_version: "2.9"
|
||||
platforms:
|
||||
- name: ArchLinux
|
||||
versions:
|
||||
- all
|
||||
- name: Debian
|
||||
versions:
|
||||
- all
|
||||
- name: EL
|
||||
versions:
|
||||
- all
|
||||
- name: Fedora
|
||||
versions:
|
||||
- all
|
||||
- name: Ubuntu
|
||||
versions:
|
||||
- all
|
||||
dependencies: []
|
||||
@@ -0,0 +1,10 @@
|
||||
---
|
||||
local:
|
||||
subnet: 192.168.109.0/24
|
||||
overlay:
|
||||
modes: [compose, swarm]
|
||||
topology: shared_net
|
||||
aliases: [tor-smtp]
|
||||
consumer:
|
||||
kind: services_flags
|
||||
key: tor-smtp
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
# nocheck: dockerfile-custom
|
||||
# nocheck: playwright-service-flag — SMTP egress daemon, no browser surface
|
||||
tor-smtp: # nocheck: category-modes
|
||||
primary: true
|
||||
bond: 1
|
||||
enabled: true
|
||||
shared: true
|
||||
custom: true
|
||||
modes:
|
||||
compose:
|
||||
enabled: true
|
||||
swarm:
|
||||
enabled: true
|
||||
# nocheck: default-placement-manager dials svc-net-tor's host-loopback SOCKS listener, which is manager-pinned; the gateway must share that node
|
||||
placement: manager
|
||||
image: python
|
||||
version: 3.13-slim
|
||||
name: tor-smtp
|
||||
ports:
|
||||
internal:
|
||||
smtp: 1525
|
||||
# Exception: the slim image ships no curl/nc — a stdlib socket connect is the
|
||||
# probe, and it reads the listen port from the same env the daemon binds (SPOT).
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import os,socket; socket.create_connection(('127.0.0.1', int(os.environ['TOR_SMTP_LISTEN_PORT'])), 3).close()"]
|
||||
min_storage: 100MB
|
||||
cpus: "0.25"
|
||||
mem_reservation: 64m
|
||||
mem_limit: 128m
|
||||
pids_limit: 64
|
||||
lifecycle: alpha
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
- include_tasks: utils/once/flag.yml
|
||||
|
||||
- name: "📂 Include sys-svc-compose (build deferred to stage the daemon first)"
|
||||
include_role:
|
||||
name: sys-svc-compose
|
||||
vars:
|
||||
compose_handlers_flush: false
|
||||
compose_repository_pull: false
|
||||
application_id: "svc-net-tor-smtp"
|
||||
|
||||
- name: "📁 Create container network"
|
||||
include_tasks: "{{ lookup('path_absolute', 'roles/sys-svc-compose/tasks/utils/network/create.yml') }}"
|
||||
vars:
|
||||
network_role_id: "{{ application_id }}"
|
||||
|
||||
- name: "🎬 Stage the gateway daemon into the build context"
|
||||
stack_host_copy:
|
||||
src: "python/tor_smtp_gateway.py"
|
||||
dest: "{{ [ lookup('container', application_id, 'directories.instance'), 'tor_smtp_gateway.py' ] | path_join }}"
|
||||
mode: "0644"
|
||||
when: IS_STACK_HOST | bool
|
||||
|
||||
- name: "🚿 Flush compose handlers (build + up with the daemon present)"
|
||||
meta: flush_handlers
|
||||
@@ -0,0 +1,9 @@
|
||||
---
|
||||
- include_tasks: 00_core.yml
|
||||
when: run_once_svc_net_tor_smtp is not defined
|
||||
vars:
|
||||
application_id: svc-net-tor-smtp
|
||||
|
||||
- include_tasks: utils/load/handlers.yml
|
||||
vars:
|
||||
handler_role_name: "sys-svc-compose"
|
||||
@@ -0,0 +1,20 @@
|
||||
{% include 'roles/sys-svc-compose/templates/base.yml.j2' %}
|
||||
{% set service_name = 'tor-smtp' %}
|
||||
{{ service_name }}:
|
||||
{{ lookup('template', 'roles/sys-svc-container/templates/build.yml.j2') | indent(4) }}
|
||||
args:
|
||||
PY_BASE: "{{ TOR_SMTP_IMAGE }}:{{ TOR_SMTP_VERSION }}"
|
||||
{{ lookup('container_image', application_id, service_name) }}
|
||||
{{ lookup('compose_only', 'container_name', TOR_SMTP_CONTAINER) }}
|
||||
{% include 'roles/sys-svc-container/templates/base.yml.j2' %}
|
||||
environment:
|
||||
TOR_SMTP_LISTEN_PORT: "{{ TOR_SMTP_LISTEN_PORT }}"
|
||||
TOR_SOCKS_HOST: "{{ TOR_SMTP_SOCKS_HOST }}"
|
||||
TOR_SOCKS_PORT: "{{ TOR_SMTP_SOCKS_PORT }}"
|
||||
TOR_SMTP_HELO: "{{ TOR_SMTP_HELO }}"
|
||||
{{ lookup('container_extra_hosts') | indent(4) }}
|
||||
{{ lookup('container_healthcheck', service_name) | indent(4) }}
|
||||
{{ lookup('container_networks') | indent(4) }}
|
||||
|
||||
{{ lookup('compose_volumes') }}
|
||||
{{ lookup('compose_networks') }}
|
||||
@@ -0,0 +1,13 @@
|
||||
---
|
||||
application_id: "svc-net-tor-smtp"
|
||||
|
||||
TOR_SMTP_IMAGE: "{{ lookup('config', application_id, 'services.tor-smtp.image') }}"
|
||||
TOR_SMTP_VERSION: "{{ lookup('config', application_id, 'services.tor-smtp.version') }}"
|
||||
TOR_SMTP_CONTAINER: "{{ lookup('config', application_id, 'services.tor-smtp.name') }}"
|
||||
TOR_SMTP_LISTEN_PORT: "{{ lookup('config', application_id, 'services.tor-smtp.ports.internal.smtp') }}"
|
||||
|
||||
TOR_SMTP_SOCKS: "{{ lookup('tor_socks_proxy') | regex_replace('^socks5h://', '') }}"
|
||||
TOR_SMTP_SOCKS_HOST: "{{ TOR_SMTP_SOCKS.split(':')[0] }}"
|
||||
TOR_SMTP_SOCKS_PORT: "{{ TOR_SMTP_SOCKS.split(':')[1] }}"
|
||||
|
||||
TOR_SMTP_HELO: "{{ DOMAIN_PRIMARY }}"
|
||||
+1
-1
@@ -11,7 +11,7 @@ set -eu
|
||||
# servers, so a misconfigured host surfaces in CI instead of in an outage.
|
||||
#
|
||||
# Usage:
|
||||
# test.sh [<docker_compose_instance_directory>]
|
||||
# verify.sh [<docker_compose_instance_directory>]
|
||||
#
|
||||
# Environment:
|
||||
# CERT_TEST_REMOTE_ENDPOINTS Space-separated "host[:port]" list (port
|
||||
@@ -22,7 +22,7 @@
|
||||
|
||||
- name: "✅ Verify deployed certificate material"
|
||||
ansible.builtin.script:
|
||||
cmd: "{{ [ role_path, 'files/shell/test.sh' ] | path_join }} {{ lookup('container', application_id, 'directories.instance') }}"
|
||||
cmd: "{{ [ role_path, 'files/shell/verify.sh' ] | path_join }} {{ lookup('container', application_id, 'directories.instance') }}"
|
||||
environment:
|
||||
CERT_TEST_REMOTE_ENDPOINTS: >-
|
||||
{{ cert_deploy_remote_endpoints
|
||||
|
||||
@@ -144,3 +144,11 @@ tor:
|
||||
bond: 0.5
|
||||
enabled: "{{ 'svc-net-tor' in group_names }}"
|
||||
shared: true # nocheck: dynamic-flag
|
||||
# nocheck: playwright-service-flag — SMTP egress relay for .onion recipients, no UI surface
|
||||
# nocheck: variants-coverage — the onion is never pinned on for this role; see the sso-variant-tor note in meta/variants.yml
|
||||
# nocheck: bond-baseline — same onion-gated rationale as the tor flag above
|
||||
# nocheck: dynamic-flag — gated on the onion (svc-net-tor), not on svc-net-tor-smtp itself: the gateway is a derived companion, never a standalone inventory group
|
||||
tor-smtp:
|
||||
bond: 0.5
|
||||
enabled: "{{ 'svc-net-tor' in group_names }}"
|
||||
shared: true # nocheck: dynamic-flag
|
||||
@@ -66,3 +66,7 @@
|
||||
- name: "🔎 Verify the production mail server's public configuration"
|
||||
include_tasks: 13_production_facts.yml
|
||||
when: STALWART_PRODUCTION_HOST | length > 0
|
||||
|
||||
- name: "🧅 Route .onion recipients through the Tor SMTP gateway"
|
||||
include_tasks: 14_onion_route.yml
|
||||
when: STALWART_ONION_ACTIVE | bool
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
# Route .onion recipients through the Tor SMTP gateway (svc-net-tor-smtp).
|
||||
# Stalwart has no outbound SOCKS of its own, so a relay route hands every
|
||||
# .onion recipient to the gateway, which dials the target over Tor.
|
||||
# JMAP request bodies live in templates/jmap/route|strategy/*.json.j2.
|
||||
# Trip-wire: the MtaOutboundStrategy singleton is NOT enumerated by
|
||||
# get(ids:null) — strategy/get.json.j2 MUST query ids:["singleton"], or the
|
||||
# existence check is always false and the update path is skipped.
|
||||
- name: "📖 List existing Stalwart outbound routes"
|
||||
include_tasks: jmap_call.yml
|
||||
vars:
|
||||
stalwart_jmap_body: "{{ lookup('template', 'jmap/route/get.json.j2') }}"
|
||||
|
||||
- name: "🔍 Resolve whether the onion relay route already exists"
|
||||
ansible.builtin.set_fact:
|
||||
stalwart_onion_route_exists: >-
|
||||
{{ (stalwart_jmap.methodResponses[0][1].list | default([]))
|
||||
| selectattr('name', 'equalto', STALWART_ONION_ROUTE_NAME)
|
||||
| list | length > 0 }}
|
||||
|
||||
- name: "📁 Create the onion relay route to the Tor SMTP gateway"
|
||||
include_tasks: jmap_call.yml
|
||||
vars:
|
||||
stalwart_jmap_body: "{{ lookup('template', 'jmap/route/create.json.j2') }}"
|
||||
when: not (stalwart_onion_route_exists | bool)
|
||||
|
||||
- name: "⚙️ Build the outbound route strategy (local, then onion, else mx)"
|
||||
ansible.builtin.set_fact:
|
||||
stalwart_route_expr:
|
||||
match:
|
||||
"0":
|
||||
if: "is_local_domain(rcpt_domain)"
|
||||
then: "'local'"
|
||||
"1":
|
||||
if: "ends_with(rcpt_domain, '.onion')"
|
||||
then: "'{{ STALWART_ONION_ROUTE_NAME }}'"
|
||||
else: "'mx'"
|
||||
|
||||
- name: "📖 Read the outbound strategy singleton"
|
||||
include_tasks: jmap_call.yml
|
||||
vars:
|
||||
stalwart_jmap_body: "{{ lookup('template', 'jmap/strategy/get.json.j2') }}"
|
||||
|
||||
- name: "🔧 Record whether the outbound strategy singleton exists"
|
||||
ansible.builtin.set_fact:
|
||||
stalwart_strategy_exists: "{{ (stalwart_jmap.methodResponses[0][1].list | length) > 0 }}"
|
||||
|
||||
- name: "📁 Create the outbound strategy with the onion route"
|
||||
include_tasks: jmap_call.yml
|
||||
vars:
|
||||
stalwart_jmap_body: "{{ lookup('template', 'jmap/strategy/create.json.j2') }}"
|
||||
when: not (stalwart_strategy_exists | bool)
|
||||
|
||||
- name: "🔀 Point the outbound strategy at the onion route for .onion recipients"
|
||||
include_tasks: jmap_call.yml
|
||||
vars:
|
||||
stalwart_jmap_body: "{{ lookup('template', 'jmap/strategy/update.json.j2') }}"
|
||||
when: stalwart_strategy_exists | bool
|
||||
|
||||
- name: "📖 Read back the live outbound strategy"
|
||||
include_tasks: jmap_call.yml
|
||||
vars:
|
||||
stalwart_jmap_body: "{{ lookup('template', 'jmap/strategy/get.json.j2') }}"
|
||||
|
||||
- name: "✅ Assert .onion recipients are routed to the gateway relay on the live server"
|
||||
ansible.builtin.assert:
|
||||
that:
|
||||
- stalwart_live_route | regex_search("ends_with\\(rcpt_domain, '\\.onion'\\)")
|
||||
- stalwart_live_route | regex_search("'" ~ STALWART_ONION_ROUTE_NAME ~ "'")
|
||||
- stalwart_live_route | regex_search("is_local_domain\\(rcpt_domain\\)")
|
||||
fail_msg: >-
|
||||
Stalwart's live outbound strategy does not route .onion recipients to the
|
||||
'{{ STALWART_ONION_ROUTE_NAME }}' gateway route (or dropped local delivery):
|
||||
{{ stalwart_live_route }}
|
||||
success_msg: ".onion recipients route to the '{{ STALWART_ONION_ROUTE_NAME }}' gateway relay, local delivery preserved"
|
||||
vars:
|
||||
stalwart_live_route: "{{ (stalwart_jmap.methodResponses[0][1].list | first).route | to_json }}"
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
|
||||
"methodCalls": [["x:MtaRoute/set", {"create": {"onion": {
|
||||
"@type": "Relay",
|
||||
"name": {{ STALWART_ONION_ROUTE_NAME | to_json }},
|
||||
"address": {{ STALWART_ONION_RELAY_HOST | to_json }},
|
||||
"port": {{ STALWART_ONION_RELAY_PORT | int }},
|
||||
"protocol": "smtp",
|
||||
"implicitTls": false,
|
||||
"allowInvalidCerts": true,
|
||||
"authSecret": {"@type": "None"}
|
||||
}}}, "c0"]]
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
|
||||
"methodCalls": [["x:MtaRoute/get", {"ids": null}, "c0"]]
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
|
||||
"methodCalls": [["x:MtaOutboundStrategy/set", {"create": {"s": {"route": {{ stalwart_route_expr | to_json }} }}}, "c0"]]
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
|
||||
"methodCalls": [["x:MtaOutboundStrategy/get", {"ids": ["singleton"]}, "c0"]]
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"using": ["urn:ietf:params:jmap:core", "urn:stalwart:jmap"],
|
||||
"methodCalls": [["x:MtaOutboundStrategy/set", {"update": {"singleton": {"route": {{ stalwart_route_expr | to_json }} }}}, "c0"]]
|
||||
}
|
||||
@@ -64,3 +64,9 @@ STALWART_IMPORT_MAILU: "{{ lookup('config', application_id, 'services.stalwart.m
|
||||
|
||||
STALWART_PRODUCTION_HOST: "{{ MAIL_PRODUCTION_HOST | trim }}"
|
||||
STALWART_PRODUCTION_RESOLVER: "{{ MAIL_PRODUCTION_RESOLVER | trim }}"
|
||||
|
||||
STALWART_ONION_ACTIVE: "{{ 'svc-net-tor' in group_names }}"
|
||||
STALWART_ONION_GATEWAY_ID: "svc-net-tor-smtp"
|
||||
STALWART_ONION_ROUTE_NAME: "onion"
|
||||
STALWART_ONION_RELAY_HOST: "{{ lookup('config', STALWART_ONION_GATEWAY_ID, 'services.tor-smtp.name') }}"
|
||||
STALWART_ONION_RELAY_PORT: "{{ lookup('config', STALWART_ONION_GATEWAY_ID, 'services.tor-smtp.ports.internal.smtp') | int }}"
|
||||
@@ -0,0 +1,3 @@
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT: Path = Path(__file__).resolve().parents[5]
|
||||
@@ -0,0 +1,3 @@
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT: Path = Path(__file__).resolve().parents[6]
|
||||
@@ -0,0 +1,136 @@
|
||||
"""What the Tor SMTP gateway accepts, and where it routes each recipient.
|
||||
|
||||
The gateway is not an open relay: it accepts a recipient ONLY when the domain
|
||||
ends with ``.onion``, and it delivers each ``.onion`` domain's recipients in one
|
||||
hop. The heavy dependencies (aiosmtpd, PySocks) are stubbed here so the routing
|
||||
logic is testable without them, and ``_deliver`` is replaced so nothing dials
|
||||
the network.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import importlib.util
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
MODULE = PROJECT_ROOT / "roles" / "svc-net-tor-smtp" / "files" / "python" / "tor_smtp_gateway.py"
|
||||
|
||||
REQUIRED_ENV = {
|
||||
"TOR_SMTP_LISTEN_PORT": "1525",
|
||||
"TOR_SOCKS_HOST": "tor",
|
||||
"TOR_SOCKS_PORT": "9050",
|
||||
"TOR_SMTP_HELO": "node.example.onion",
|
||||
}
|
||||
|
||||
|
||||
def _load(env=None):
|
||||
"""Import the daemon with the required env stubbed. A None override drops
|
||||
the key, so a test can assert a required var is missing."""
|
||||
socks_stub = types.ModuleType("socks")
|
||||
socks_stub.SOCKS5 = 2
|
||||
socks_stub.create_connection = lambda *a, **k: None
|
||||
controller_stub = types.ModuleType("aiosmtpd.controller")
|
||||
controller_stub.Controller = object
|
||||
aiosmtpd_pkg = types.ModuleType("aiosmtpd")
|
||||
aiosmtpd_pkg.controller = controller_stub
|
||||
merged = {k: v for k, v in {**REQUIRED_ENV, **(env or {})}.items() if v is not None}
|
||||
with mock.patch.dict(sys.modules, {
|
||||
"socks": socks_stub,
|
||||
"aiosmtpd": aiosmtpd_pkg,
|
||||
"aiosmtpd.controller": controller_stub,
|
||||
}), mock.patch.dict(os.environ, merged, clear=True):
|
||||
spec = importlib.util.spec_from_file_location("tor_smtp_gateway", MODULE)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
class _Envelope:
|
||||
def __init__(self, mail_from="bot@x.onion", rcpt_tos=None, content=b"msg"):
|
||||
self.mail_from = mail_from
|
||||
self.rcpt_tos = list(rcpt_tos or [])
|
||||
self.content = content
|
||||
|
||||
|
||||
class TestDomainGrouping(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.mod = _load()
|
||||
|
||||
def test_recipients_group_by_domain(self):
|
||||
grouped = self.mod._group_by_domain(
|
||||
["a@one.onion", "b@one.onion", "c@two.onion"]
|
||||
)
|
||||
self.assertEqual(grouped, {"one.onion": ["a@one.onion", "b@one.onion"], "two.onion": ["c@two.onion"]})
|
||||
|
||||
def test_domain_match_is_case_insensitive(self):
|
||||
grouped = self.mod._group_by_domain(["A@One.Onion"])
|
||||
self.assertEqual(list(grouped), ["one.onion"])
|
||||
|
||||
|
||||
class TestRcptGate(unittest.IsolatedAsyncioTestCase):
|
||||
def setUp(self):
|
||||
self.mod = _load()
|
||||
self.relay = self.mod.OnionRelay()
|
||||
|
||||
async def _rcpt(self, address):
|
||||
env = _Envelope(rcpt_tos=[])
|
||||
reply = await self.relay.handle_RCPT(None, None, env, address, [])
|
||||
return reply, env
|
||||
|
||||
async def test_an_onion_recipient_is_accepted(self):
|
||||
reply, env = await self._rcpt("biber@abc.onion")
|
||||
self.assertTrue(reply.startswith("250"))
|
||||
self.assertEqual(env.rcpt_tos, ["biber@abc.onion"])
|
||||
|
||||
async def test_a_clearnet_recipient_is_refused(self):
|
||||
reply, env = await self._rcpt("biber@infinito.test")
|
||||
self.assertTrue(reply.startswith("550"))
|
||||
self.assertEqual(env.rcpt_tos, [])
|
||||
|
||||
|
||||
class TestDataDelivery(unittest.IsolatedAsyncioTestCase):
|
||||
def setUp(self):
|
||||
self.mod = _load()
|
||||
self.relay = self.mod.OnionRelay()
|
||||
|
||||
async def test_each_onion_domain_gets_one_delivery(self):
|
||||
calls = []
|
||||
with mock.patch.object(self.mod, "_deliver", side_effect=lambda *a: calls.append(a)):
|
||||
env = _Envelope(rcpt_tos=["a@one.onion", "b@one.onion", "c@two.onion"])
|
||||
reply = await self.relay.handle_DATA(None, None, env)
|
||||
self.assertTrue(reply.startswith("250"))
|
||||
self.assertEqual({c[0] for c in calls}, {"one.onion", "two.onion"})
|
||||
|
||||
async def test_a_failed_hop_is_a_transient_error(self):
|
||||
def boom(*_a):
|
||||
raise OSError("tor down")
|
||||
|
||||
with mock.patch.object(self.mod, "_deliver", side_effect=boom):
|
||||
env = _Envelope(rcpt_tos=["a@one.onion"])
|
||||
reply = await self.relay.handle_DATA(None, None, env)
|
||||
self.assertTrue(reply.startswith("451"))
|
||||
|
||||
|
||||
class TestConfiguration(unittest.TestCase):
|
||||
def test_the_helo_name_comes_from_the_environment(self):
|
||||
mod = _load({"TOR_SMTP_HELO": "biber.node.onion"})
|
||||
self.assertEqual(mod.HELO_NAME, "biber.node.onion")
|
||||
|
||||
def test_the_helo_never_falls_back_to_the_bind_address(self):
|
||||
"""EHLO 0.0.0.0 is malformed; a missing HELO must fail loudly, not
|
||||
silently announce the bind address to the peer."""
|
||||
with self.assertRaises(KeyError):
|
||||
_load({"TOR_SMTP_HELO": None})
|
||||
|
||||
def test_the_onion_hop_targets_the_well_known_smtp_port(self):
|
||||
self.assertEqual(_load().ONION_SMTP_PORT, 25)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,137 @@
|
||||
"""How Stalwart routes .onion recipients to the Tor SMTP gateway.
|
||||
|
||||
Stalwart has no outbound SOCKS of its own, so ``14_onion_route.yml`` provisions
|
||||
a relay ``MtaRoute`` to the gateway and points the ``MtaOutboundStrategy`` route
|
||||
expression at it for ``.onion`` recipients. The one property that must never
|
||||
regress: the strategy still sends local recipients to ``local`` and everything
|
||||
else to ``mx`` — adding the onion branch must not break normal delivery.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import unittest
|
||||
|
||||
from jinja2 import Environment, StrictUndefined
|
||||
|
||||
from utils.cache.files import read_text
|
||||
from utils.cache.yaml import load_yaml_any
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
ROLE = PROJECT_ROOT / "roles" / "web-app-stalwart"
|
||||
TASK = ROLE / "tasks" / "14_onion_route.yml"
|
||||
ROUTE_CREATE = ROLE / "templates" / "jmap" / "route" / "create.json.j2"
|
||||
STRATEGY_GET = ROLE / "templates" / "jmap" / "strategy" / "get.json.j2"
|
||||
STRATEGY_CREATE = ROLE / "templates" / "jmap" / "strategy" / "create.json.j2"
|
||||
STRATEGY_UPDATE = ROLE / "templates" / "jmap" / "strategy" / "update.json.j2"
|
||||
|
||||
ROUTE_NAME = "onion"
|
||||
|
||||
|
||||
def _render(template_path, **variables) -> str:
|
||||
env = Environment(
|
||||
trim_blocks=True,
|
||||
lstrip_blocks=True,
|
||||
undefined=StrictUndefined,
|
||||
autoescape=False, # noqa: S701 - JMAP request bodies, never HTML
|
||||
)
|
||||
env.filters["to_json"] = json.dumps
|
||||
env.filters["int"] = int
|
||||
return env.from_string(read_text(template_path)).render(**variables)
|
||||
|
||||
|
||||
def _route_expr_from_task() -> dict:
|
||||
tasks = load_yaml_any(str(TASK), default_if_missing=[])
|
||||
for task in tasks:
|
||||
facts = task.get("set_fact") or task.get("ansible.builtin.set_fact") or {}
|
||||
if "stalwart_route_expr" in facts:
|
||||
return facts["stalwart_route_expr"]
|
||||
raise AssertionError("stalwart_route_expr set_fact not found in 14_onion_route.yml")
|
||||
|
||||
|
||||
class TestOutboundStrategyExpression(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.expr = _route_expr_from_task()
|
||||
|
||||
def test_local_recipients_still_route_to_local(self):
|
||||
first = self.expr["match"]["0"]
|
||||
self.assertEqual(first["if"], "is_local_domain(rcpt_domain)")
|
||||
self.assertEqual(first["then"], "'local'")
|
||||
|
||||
def test_local_is_evaluated_before_onion(self):
|
||||
"""A local .onion address must be delivered locally, not relayed out."""
|
||||
keys = list(self.expr["match"].keys())
|
||||
self.assertEqual(keys, ["0", "1"])
|
||||
self.assertIn("is_local_domain", self.expr["match"]["0"]["if"])
|
||||
self.assertIn("ends_with", self.expr["match"]["1"]["if"])
|
||||
|
||||
def test_onion_recipients_route_to_the_onion_route(self):
|
||||
"""then is a Stalwart expression string literal naming the route, from
|
||||
the same SPOT var the relay route registers itself under."""
|
||||
onion = self.expr["match"]["1"]
|
||||
self.assertEqual(onion["if"], "ends_with(rcpt_domain, '.onion')")
|
||||
self.assertEqual(onion["then"], "'{{ STALWART_ONION_ROUTE_NAME }}'")
|
||||
|
||||
def test_everything_else_falls_back_to_mx(self):
|
||||
self.assertEqual(self.expr["else"], "'mx'")
|
||||
|
||||
|
||||
class TestStrategyTemplateRendersValidJson(unittest.TestCase):
|
||||
def test_the_singleton_route_is_set_from_the_expression(self):
|
||||
expr = _route_expr_from_task()
|
||||
rendered = _render(STRATEGY_UPDATE, stalwart_route_expr=expr)
|
||||
body = json.loads(rendered)
|
||||
method, args, _tag = body["methodCalls"][0]
|
||||
self.assertEqual(method, "x:MtaOutboundStrategy/set")
|
||||
self.assertEqual(args["update"]["singleton"]["route"], expr)
|
||||
|
||||
def test_create_seeds_a_new_singleton_with_the_route(self):
|
||||
expr = _route_expr_from_task()
|
||||
rendered = _render(STRATEGY_CREATE, stalwart_route_expr=expr)
|
||||
body = json.loads(rendered)
|
||||
method, args, _tag = body["methodCalls"][0]
|
||||
self.assertEqual(method, "x:MtaOutboundStrategy/set")
|
||||
(created,) = args["create"].values()
|
||||
self.assertEqual(created["route"], expr)
|
||||
|
||||
def test_get_queries_the_singleton_by_id(self):
|
||||
"""The singleton is NOT enumerated by get(ids:null); it must be queried
|
||||
by its fixed "singleton" id or existence detection is always wrong."""
|
||||
body = json.loads(_render(STRATEGY_GET))
|
||||
method, args, _tag = body["methodCalls"][0]
|
||||
self.assertEqual(method, "x:MtaOutboundStrategy/get")
|
||||
self.assertEqual(args["ids"], ["singleton"])
|
||||
|
||||
|
||||
class TestRelayRouteRendersValidJson(unittest.TestCase):
|
||||
def _create(self):
|
||||
rendered = _render(
|
||||
ROUTE_CREATE,
|
||||
STALWART_ONION_ROUTE_NAME=ROUTE_NAME,
|
||||
STALWART_ONION_RELAY_HOST="tor-smtp",
|
||||
STALWART_ONION_RELAY_PORT=1525,
|
||||
)
|
||||
body = json.loads(rendered)
|
||||
return body["methodCalls"][0]
|
||||
|
||||
def test_it_creates_a_relay_route_to_the_gateway(self):
|
||||
method, args, _tag = self._create()
|
||||
self.assertEqual(method, "x:MtaRoute/set")
|
||||
route = args["create"][ROUTE_NAME]
|
||||
self.assertEqual(route["@type"], "Relay")
|
||||
self.assertEqual(route["name"], ROUTE_NAME)
|
||||
self.assertEqual(route["address"], "tor-smtp")
|
||||
self.assertEqual(route["port"], 1525)
|
||||
|
||||
def test_the_relay_hop_is_plaintext_smtp(self):
|
||||
"""The gateway speaks plaintext SMTP on an internal network; there is no
|
||||
cert to validate on a .onion relay hop."""
|
||||
_method, args, _tag = self._create()
|
||||
route = args["create"][ROUTE_NAME]
|
||||
self.assertEqual(route["protocol"], "smtp")
|
||||
self.assertFalse(route["implicitTls"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user