Kevin Veen-Birkenbach
7791bd8c04
Integration tests added/updated:
- tests/integration/test_filters_usage.py: AST-based detection of filter definitions (FilterModule.filters), robust Jinja detection ({{ ... }}, {% ... %}, {% filter ... %}), plus Python call tracking; fails if a filter is used only under tests/.
- tests/integration/test_filters_are_defined.py: inverse check — every filter used in .yml/.yaml/.j2/.jinja2/.tmpl must be defined locally. Scans only inside Jinja blocks and ignores pipes inside strings (e.g., lookup('pipe', "... | grep ... | awk ...")) to avoid false positives like trusted_hosts, woff/woff2, etc.
Bug fixes & robustness:
- Build regexes without %-string formatting to avoid ValueError from literal '%' in Jinja tags.
- Strip quoted strings in usage analysis so sed/grep/awk pipes are not miscounted as filters.
- Prevent self-matches in the defining file.
Cleanup / removal of dead code:
- Removed unused filter plugins and related unit tests:
* filter_plugins/alias_domains_map.py
* filter_plugins/get_application_id.py
* filter_plugins/load_configuration.py
* filter_plugins/safe.py
* filter_plugins/safe_join.py
* roles/svc-db-openldap/filter_plugins/build_ldap_nested_group_entries.py
* roles/sys-ctl-bkp-docker-2-loc/filter_plugins/dict_to_cli_args.py
* corresponding tests under tests/unit/*
- roles/svc-db-postgres/filter_plugins/split_postgres_connections.py: dropped no-longer-needed list_postgres_roles API; adjusted tests.
Misc:
- sys-stk-front-proxy/defaults/main.yml: clarified valid vhost_flavour values (comma-separated).
Ref: https://chatgpt.com/share/68b56bac-c4f8-800f-aeef-6708dbb44199
Backup Docker to Local
Description
This Ansible role automates the process of backing up Docker volumes to a local folder. It pulls the backup-docker-to-local, installs required software, configures systemd services for both standard and "everything" backup modes, and seeds backup database entries as needed.
Overview
Optimized for Archlinux, this role ensures that Docker volume backups are performed reliably with minimal manual intervention. It integrates with several dependent roles to verify backup success and manage related tasks, including:
- sys-svc-directory-validator – Validates backup directories.
- sys-ctl-cln-faild-bkps – Cleans up unsuccessful backup attempts.
- sys-timer – Schedules recurring backup tasks.
- sys-bkp-provider – Manages backup sources.
- sys-lock – Ensures coordinated maintenance operations.
Purpose
Backup Docker Volumes to Local is a comprehensive solution that leverages rsync to create incremental backups of Docker volumes, providing seamless recovery for both file and database data. Ideal for ensuring the integrity and security of your container data, this role sets up the necessary environment to safeguard your Docker volumes.
Features
- Required Software Installation: Installs necessary packages (e.g., lsof, python-pandas) via pacman.
- Git Repository Pull: Automatically pulls the latest version of the backup-docker-to-local.
- Systemd Service Configuration: Deploys and reloads two systemd service templates to manage backup tasks.
- Database Seeding: Includes tasks to seed and manage a backup database (
databases.csv) for tracking backup details. - Dependency Integration: Works in conjunction with the dependent roles listed above to verify and manage backups.