This commit updates multiple roles to ensure compatibility with Ansible 2.20.
Several include paths and task-loading mechanisms required adjustments,
as Ansible 2.20 applies stricter evaluation rules for complex Jinja expressions
and no longer resolves certain relative include paths the way Ansible 2.18 did.
Key changes:
- Replaced legacy once_finalize.yml and once_flag.yml with the new structure
under tasks/utils/once/finalize.yml and tasks/utils/once/flag.yml.
- Updated all include_tasks statements to use 'path_join' with playbook_dir,
ensuring deterministic and absolute file resolution across roles.
- Fixed all network helper includes by converting direct relative paths such as
'roles/docker-compose/tasks/utils/network.yml' to proper Jinja-evaluated paths.
- Normalized MATOMO_* variable names for consistency with the updated variable
scope behavior in Ansible 2.20.
- Removed deprecated patterns that were implicitly supported in Ansible 2.18
but break under the more strict variable and path resolution model in 2.20.
These changes are part of the full migration step required to ensure the
infinito-nexus roles remain stable, deterministic, and forward-compatible with
Ansible 2.20.
Details of the discussion and reasoning can be found in this conversation:
https://chatgpt.com/share/69300a8d-24d4-800f-bec0-e895a695618a
This change introduces a WHITELISTED_HANDLERS mechanism, allowing specific
handlers to be intentionally skipped due to conditional 'False' evaluations
without causing test failures. Improves flexibility while keeping the
architectural policy enforced for all other handlers.
Reference: https://chatgpt.com/share/692f6841-c19c-800f-8d6c-aa1ef48dcf7e
- Replace legacy utils/run_once.yml with the new helpers utils/once_flag.yml and utils/once_finalize.yml
- Introduce utils/compose_up.yml to ensure docker-compose stacks are up and to flush handlers safely without coupling to run-once flags
- Migrate all affected roles (desk-*, dev-*, sys-ctl-*, sys-svc-*, web-app-*, web-svc-*, util-*) to the new run-once helpers
- Rework sys-svc-msmtp to auto-load Mailu once per deploy, check reachability, and reuse the running stack instead of requiring multiple playbook passes
- Adjust web-app-mailu to integrate cert deployment, handler flushing, and run-once handling so Mailu is fully initialized in a single deploy
- Improve Matomo, CDN, logout and CSP/health-check related roles to cooperate with the new compose_up / once_* pattern
- Simplify alarm/backup/timer/service orchestration (sys-ctl-alm-*, sys-bkp-provider, sys-timer-cln-bkps, etc.) by moving run-once logic into dedicated 01_core.yml files
- Update integration tests so utils/once_flag.yml and utils/once_finalize.yml are recognised as valid run-once providers, keeping the global run_once_* guarantees consistent
- Align frontend injection and service dependencies so Mastodon- and Mailu-related services can be brought up coherently within a single deployment cycle rather than several iterations
- Dynamically calculate PEERTUBE_MAX_OLD_SPACE_SIZE (~35% of container RAM, clamped between 768–3072 MB)
- Dynamically calculate PEERTUBE_TRANSCODING_CONCURRENCY (~½ vCPUs, min 1, max 8)
- Added default resource limits for Redis and Peertube containers
- Updated test suite to include human_to_bytes filter in built-in filter list
https://chatgpt.com/share/690914d2-6100-800f-a850-94e6d226e7c9
This update extends the regex to detect block-style variable definitions such as:
{% set var %} ... {% endset %}
Previously, only inline 'set var =' syntax was recognized, causing false positives
like '_snippet' being flagged as undefined in Jinja templates.
Reference: https://chatgpt.com/share/68f6799a-eb80-800f-ab5c-7c196d4c4661
- Added regex masking to skip {{ var }} usages inside {% raw %}…{% endraw %} blocks.
- Simplified code by removing redundant comments.
- Cleaned up task file for XWiki role by removing outdated note.
Ref: https://chatgpt.com/share/68cd2558-e92c-800f-a80a-a79d3c81476e
Updated tests/integration/test_vars_usage_in_yaml.py:
- Variables immediately followed by '(' are now treated as function calls,
not as set variables. This prevents false errors.
- Fixed detection of redirect_domain_mappings so it is no longer flagged
as unused.
See: https://chatgpt.com/share/68c3542d-f44c-800f-a483-b3e43739f315
Removed obsolete handlers from roles (VirtualBox, backup-to-USB, OpenLDAP)
and introduced an integration test under tests/integration/test_handlers_invoked.py
that ensures all handlers defined in roles/*/handlers are actually notified
somewhere in the code base. This keeps the repository clean by preventing
unused or forgotten handlers from accumulating.
Ref: https://chatgpt.com/share/68b6b28e-4388-800f-87d2-34dfb34b8d36
Integration tests added/updated:
- tests/integration/test_filters_usage.py: AST-based detection of filter definitions (FilterModule.filters), robust Jinja detection ({{ ... }}, {% ... %}, {% filter ... %}), plus Python call tracking; fails if a filter is used only under tests/.
- tests/integration/test_filters_are_defined.py: inverse check — every filter used in .yml/.yaml/.j2/.jinja2/.tmpl must be defined locally. Scans only inside Jinja blocks and ignores pipes inside strings (e.g., lookup('pipe', "... | grep ... | awk ...")) to avoid false positives like trusted_hosts, woff/woff2, etc.
Bug fixes & robustness:
- Build regexes without %-string formatting to avoid ValueError from literal '%' in Jinja tags.
- Strip quoted strings in usage analysis so sed/grep/awk pipes are not miscounted as filters.
- Prevent self-matches in the defining file.
Cleanup / removal of dead code:
- Removed unused filter plugins and related unit tests:
* filter_plugins/alias_domains_map.py
* filter_plugins/get_application_id.py
* filter_plugins/load_configuration.py
* filter_plugins/safe.py
* filter_plugins/safe_join.py
* roles/svc-db-openldap/filter_plugins/build_ldap_nested_group_entries.py
* roles/sys-ctl-bkp-docker-2-loc/filter_plugins/dict_to_cli_args.py
* corresponding tests under tests/unit/*
- roles/svc-db-postgres/filter_plugins/split_postgres_connections.py: dropped no-longer-needed list_postgres_roles API; adjusted tests.
Misc:
- sys-stk-front-proxy/defaults/main.yml: clarified valid vhost_flavour values (comma-separated).
Ref: https://chatgpt.com/share/68b56bac-c4f8-800f-aeef-6708dbb44199
- Converted group_vars/all/13_ldap.yml from lower-case to ALL-CAPS nested keys.
- Updated all roles, tasks, templates, and filter_plugins to reference LDAP.* instead of ldap.*.
- Fixed Keycloak JSON templates to properly quote Jinja variables.
- Adjusted svc-db-openldap filter plugins and unit tests to handle new LDAP structure.
- Updated integration test to only check uniqueness of TOP-LEVEL ALL-CAPS constants, ignoring nested keys.
See: https://chatgpt.com/share/68b01017-efe0-800f-a508-7d7e2f1c8c8d
This test scans all roles for tasks including:
- include_role:
name: sys-service
If present, the role must define a non-empty 'system_service_id' in vars/main.yml.
Helps enforce consistency and prevent misconfiguration.
Ref: https://chatgpt.com/share/68a536e5-c384-800f-937a-f9d91249950c
- Unified service templates into generic systemctl templates
- Introduced reusable filter plugins for script path handling
- Updated path variables and service/timer definitions
- Migrated roles (backup, cleanup, repair, etc.) to use systemctl role
- Added sys-daemon role for core systemd cleanup
- Simplified timer handling via sys-timer role
Note: This is a large refactor and some errors may still exist. Further testing and adjustments will be needed.
This commit restructures systemctl service definitions and category mappings.
Motivation: Alarm-related bugs revealed inconsistencies in service and role handling.
Preparation step: lays the groundwork for fixing the alarm issues by aligning categories, roles, and service templates.
- Support fully qualified ansible.builtin.set_fact
- Parse inline set_fact mappings (e.g. set_fact: { a: 1, b: 2 })
- Continue scanning inside vars/set_fact blocks for Jinja {% set %}, {% for %}, and {% macro %}
- Ensures variables defined by set_fact are correctly recognized as defined
This test scans roles/*/handlers/main.yml and fails if a handler's 'name' contains a Jinja variable ({{ ... }}).
Reason:
- Handler names must be static to ensure reliable 'notify' resolution.
- Dynamic names can break handler matching, cause undefined-variable errors, and produce unstable logs.
Recommendation:
- Keep handler names static and, if dynamic behavior is needed, use a static 'listen:' key.
https://chatgpt.com/share/689b37dc-e1e4-800f-bd56-00b43c7701f6
This test scans all roles/*/meta/main.yml for meta dependencies that are
likely unnecessary and could be replaced with guarded include_role/import_role
calls to improve performance.
A dependency is flagged as unnecessary when:
- The consumer role does not use provider variables in defaults/vars/handlers
(no early variable requirement), and
- Any usage of provider variables or handler notifications in tasks occurs
only after an explicit include/import of the provider in the same file,
or there is no usage at all.
Purpose:
Helps reduce redundant parsing/execution of roles and improves Ansible
performance by converting heavy global dependencies into conditional,
guarded includes where possible.
https://chatgpt.com/share/689a59ee-52f4-800f-8349-4f477dc97c7c
This test scans all .yml/.yaml files under any tasks/ directory and flags cases where the same
'when' condition appears on more than 3 tasks in the same file. Excessive duplication of identical
conditions can harm Ansible performance because the condition is re-evaluated for every task.
The test suggests replacing repeated conditions with an include_tasks call or a block guarded
by the condition to evaluate it only once.
https://chatgpt.com/share/6899c605-6f40-800f-a954-ccb62f8bbcf1
Updated test_mode_reset.py to also validate roles that contain a reset
task file (*_reset.yml or reset.yml) even when no mode_reset keyword is
found. The test now:
- Detects roles with reset files but no include, and fails accordingly.
- Ignores commented include_tasks and when lines.
- Ensures exactly one non-commented include of the reset file exists.
- Requires that the include is guarded in the same task block by a
when containing mode_reset | bool (with optional extra conditions).
This prevents silent omissions of reset task integration.
https://chatgpt.com/share/6899b745-7150-800f-98f3-ca714486f5ba
