mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-06 02:54:26 +00:00
kevinveenbirkenbachandClaude Opus 5.5
9b7f25f2fe
Every swarm job on ubuntu-26.04 runners (kernel 7.0) died at the NFS probe write with `[Errno 121] Remote I/O error`. ganesha's NFSv4 debug log in the rescue bundles shows the cause on every client, the same second as the failure: Request 2: opcode 46 is OP_GET_DIR_DELEGATION Status of OP_GET_DIR_DELEGATION in position 2 = NFS4ERR_OP_ILLEGAL Linux 6.19 added directory delegations: after ACCESS, CREATE, REMOVE or RENAME the client adds GET_DIR_DELEGATION to the directory's next GETATTR (`should_request_dir_deleg`, gated per request by the nfsv4 parameter `directory_delegations`). nfs-ganesha before 9.11 routes op 46 to `nfs4_op_illegal`, whose reply carries opcode 10044; the client decoder expects 46 and fails the call with EREMOTEIO. Upstream fixed it in nfs-ganesha#1385 (98eb77c4, first in V9.11), which answers NOTSUPP; the client then drops the capability. The distro packages the role installs predate it: Debian trixie 6.5-5, Ubuntu 26.04 6.5-8. Every client and the controller now read the server's installed nfs-ganesha release and, below 9.11, set `directory_delegations=N` right after the mount and before the first access (the mount itself never sends op 46), plus `options nfsv4 directory_delegations=N` in /etc/modprobe.d for later module loads. No ganesha release grants directory delegations, so the end state equals a fixed server. The parameter is kernel-wide and is only ever set to N, so clients sharing one kernel cannot flip it back on each other. Servers on 9.11 or later and the kernel nfsd flavor keep the client default. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
104 lines
4.3 KiB
Markdown
104 lines
4.3 KiB
Markdown
# NFS Client
|
|
|
|
## Description
|
|
|
|
[NFS](https://en.wikipedia.org/wiki/Network_File_System) (Network File
|
|
System) lets a host mount a directory exported by a remote server as if
|
|
it were a local filesystem. The client packages provide the kernel
|
|
helpers and userspace tools to perform that mount.
|
|
|
|
## Overview
|
|
|
|
This role installs the distro-appropriate NFS client packages on every
|
|
host in the Ansible group `svc-swarm-node` and probe-mounts the
|
|
configured `storage.nfs.server` and the export base from the
|
|
svc-storage-nfs-server services.yml SPOT to confirm
|
|
reachability and writability at deploy time. The actual docker volume
|
|
mounts happen later, driven by the Docker engine at container start.
|
|
|
|
## Cosmos
|
|
|
|
The diagram places NFS Client in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks).
|
|
|
|
```mermaid
|
|
flowchart LR
|
|
subgraph deps [Dependencies]
|
|
dep_svc_storage_nfs_server["svc-storage-nfs-server 💻"]
|
|
end
|
|
subgraph role [svc-storage-nfs-client 💻]
|
|
svc_nfs_server["nfs-server"]
|
|
svc_nfs_client["nfs-client"]
|
|
end
|
|
dep_svc_storage_nfs_server -. "0..1" .-> svc_nfs_server
|
|
```
|
|
|
|
Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`.
|
|
|
|
## Features
|
|
|
|
- **Distro-aware packages:** Installs `nfs-common` on Debian/Ubuntu,
|
|
`nfs-utils` on Arch / RHEL / Fedora / Alpine.
|
|
- **Ephemeral probe mount:** Mounts the export, writes a marker, then
|
|
unmounts; failures surface immediately at deploy time, never at first
|
|
container boot.
|
|
- **Strict assertion:** Missing `storage.nfs.server` or
|
|
fails the deploy with a precise error.
|
|
- **Node-local backups:** After the state mount, binds the node's own
|
|
`/var/lib/infinito-backup` over `/var/lib/infinito/backup`, so every
|
|
backup repository on a swarm node is written to local disk with its
|
|
extended attributes, and the backup host pulls it from there.
|
|
Generations written to the share before the bind stay under the
|
|
export's `backup/` directory, hidden from the nodes; remove them there
|
|
by hand.
|
|
- **Directory delegations off against old ganesha:** When the server
|
|
runs nfs-ganesha older than 9.11, every client and the controller set
|
|
the kernel's `nfsv4` parameter `directory_delegations` to `N`, at
|
|
runtime and in `/etc/modprobe.d`. The parameter is kernel-wide, so on
|
|
a lab whose nodes share one kernel it covers every NFS mount of that
|
|
host until the module reloads.
|
|
|
|
## Quick Setup
|
|
|
|
### Development
|
|
|
|
Clone, set up the workstation, and deploy NFS Client onto the local stack:
|
|
|
|
```bash
|
|
git clone https://github.com/infinito-nexus/core.git
|
|
cd core
|
|
make onboard
|
|
make compose-deploy mode=reinstall apps=svc-storage-nfs-client full_cycle=false
|
|
```
|
|
|
|
### Production
|
|
|
|
Install NFS Client directly onto the target machine: clone the repository, install the OS prerequisites and the repository toolchain, then deploy against localhost over a local connection (no SSH, no container):
|
|
|
|
```bash
|
|
git clone https://github.com/infinito-nexus/core.git
|
|
cd core
|
|
bash scripts/install/package.sh
|
|
make install
|
|
source scripts/meta/env/load.sh
|
|
|
|
APP=svc-storage-nfs-client
|
|
DOMAIN=<your-domain>
|
|
TLS_MODE=self_signed
|
|
SSH_PUBLIC_KEY="<your-ssh-public-key>"
|
|
INVENTORY=inventories/production
|
|
infinito administration inventory provision "$INVENTORY" \
|
|
--inventory-file "$INVENTORY/devices.yml" \
|
|
--host localhost \
|
|
--include "$APP" \
|
|
--vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}"
|
|
infinito administration deploy dedicated "$INVENTORY/devices.yml" \
|
|
--password-file "$INVENTORY/.password" \
|
|
--diff -vv
|
|
```
|
|
|
|
## Credits
|
|
|
|
Implemented by **[Kevin Veen-Birkenbach](https://www.veen.world)**.
|
|
Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world).
|
|
Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license).
|