diff --git a/docs/contributing/artefact/files/plugins/lookup/applications.md b/docs/contributing/artefact/files/plugins/lookup/applications.md index 04ca54e35e..ac13d7e6e1 100644 --- a/docs/contributing/artefact/files/plugins/lookup/applications.md +++ b/docs/contributing/artefact/files/plugins/lookup/applications.md @@ -36,7 +36,7 @@ Ansible forks a fresh worker process for every task. The in-process cache inside The constructor stage renders once through the [applications_carrier](../../../../../../plugins/action/applications_carrier.py) action and parks the result together with its cache key in the host fact `_INFINITO_APPLICATIONS_RENDERED` (the action calls `lookup('applications', carrier=True)`). Every later worker inherits that fact at fork time; `get_merged_applications` serves it whenever the key still matches and renders only when it does not (a changed `applications`, `users`, `DOMAIN_PRIMARY`, `SYSTEM_EMAIL_DOMAIN` or `group_names`, or changed tokens in the store the users are hydrated from). - The stage-wide task sits in `tasks/stages/01_constructor.yml` after the `networks`, `design` and `service_provider` merges, after the `add_host` placements and after the token store, so the payload renders with the same `group_names`, `groups` and user records a role would see. Moving it earlier bakes stale values into the play. -- The carrier is parked again after every role in `tasks/utils/setup/role.yml`, after every nested application load in `tasks/utils/load/app.yml`, and after every persisted token in `roles/sys-token-store/tasks/write.yml`. The first two run on every host of the include, also on hosts the loaded role skips. A re-park whose key still matches serves the carried payload without rendering. +- The carrier is parked again after every role in `tasks/utils/setup/role.yml`, after every nested application load in `tasks/utils/load/app.yml`, and after every store write in `roles/sys-token-store/tasks/write_many.yml`, which `write.yml` runs for a single token. The first two run on every host of the include, also on hosts the loaded role skips. A re-park whose key still matches serves the carried payload without rendering. - The action renders with `application_id` removed from its variables, so the carried tree holds the play-scope result of every consumer-aligned `lookup('tls', ...)`, whichever role parks it. - The token store enters the key as the SHA-256 of exactly the tokens the users are hydrated from: the controller's tokens file until `sys-token-store` loads, the `sys_token_store_users` fact after. Both sources key alike when they hold the same tokens, so loading an unchanged store keeps the carrier, and a touch keeps the key. The carrier action opens the file before it computes the key, so an NFS client keys on revalidated attributes. - The action masks its own result: the fact is the whole rendered tree with every application's credentials, so it never reaches a log regardless of `MASK_CREDENTIALS_IN_LOGS`. diff --git a/plugins/action/applications_carrier.py b/plugins/action/applications_carrier.py index b85b9639f6..d7da126d30 100644 --- a/plugins/action/applications_carrier.py +++ b/plugins/action/applications_carrier.py @@ -25,7 +25,7 @@ rejected and the reader renders afresh:: ``tasks/utils/setup/role.yml`` parks one after every role, ``tasks/utils/load/app.yml`` one after every nested application load and -``roles/sys-token-store/tasks/write.yml`` one after every persisted token; +``roles/sys-token-store/tasks/write_many.yml`` one after every store write; where the key still matches they serve the carried payload without rendering. The result is masked by the action itself: the fact is the whole rendered diff --git a/roles/sys-token-store/README.md b/roles/sys-token-store/README.md index ea1eaa9d76..8a83ec3aba 100644 --- a/roles/sys-token-store/README.md +++ b/roles/sys-token-store/README.md @@ -98,9 +98,21 @@ This is the **canonical way** to store tokens. - Updates the token store file (idempotent) - Makes the token available via `lookup('users', '').tokens['']` -- Exports `sys_token_store_token` -Empty tokens are rejected explicitly. +Empty tokens are rejected explicitly. `write.yml` hands its token to `write_many.yml` as a one-entry list. + +### `write_many.yml` + +Persists several tokens with one store write and at most one re-render of the applications SPOT. + +**Input** + +- `sys_token_store_entries`: list of `{user_key, app, token}`; every `token` must be non-empty + +**Effects** + +- Merges every entry into the store in list order, so a later entry for the same user and app wins +- Writes the store file and re-renders the applications SPOT once, and only when the merged store differs from the loaded one --- @@ -126,6 +138,22 @@ After this, the token is available as: lookup('users', 'administrator').tokens['web-app-matomo'] ``` +### Persist several tokens at once + +```yaml +- include_role: + name: sys-token-store + tasks_from: write_many.yml + vars: + sys_token_store_entries: + - user_key: administrator + app: web-app-matomo + token: "{{ matomo_token_value }}" + - user_key: no-reply + app: web-app-stalwart + token: "{{ stalwart_bot_secret }}" +``` + --- ## What This Role Does *Not* Do diff --git a/roles/sys-token-store/tasks/write.yml b/roles/sys-token-store/tasks/write.yml index dff0ab7db0..fc51fe4ca0 100644 --- a/roles/sys-token-store/tasks/write.yml +++ b/roles/sys-token-store/tasks/write.yml @@ -4,74 +4,10 @@ # sys_token_store_app # sys_token_store_token # must be non-empty -- name: "✅ Ensure store exists" - ansible.builtin.import_tasks: 00_core.yml - -- name: "📐 Normalize incoming token" - ansible.builtin.set_fact: - sys_token_store_token_effective: "{{ sys_token_store_token | string | trim }}" - no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" - -- name: "🛑 Abort if token is empty (do not persist empties)" - ansible.builtin.assert: - that: - - sys_token_store_token_effective != '' - fail_msg: "sys_token_store_token is empty – refusing to persist empty token." - -- name: "🔨 Build merged structure" - ansible.builtin.set_fact: - sys_token_store_users_merged: >- - {{ - sys_token_store_users - | combine({ - sys_token_store_user_key: ( - sys_token_store_users.get(sys_token_store_user_key, {}) - | combine({ - 'tokens': ( - sys_token_store_users.get(sys_token_store_user_key, {}).get('tokens', {}) - | combine({ sys_token_store_app: sys_token_store_token_effective }) - ) - }, recursive=True - ) - ) - }, recursive=True - ) - }} - no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" - -- name: "💾 Persist store (idempotent, single-writer via STACK_HOST)" - ansible.builtin.copy: - dest: "{{ FILE_TOKENS }}" - owner: "{{ SYS_TOKEN_STORE_OWNER }}" - group: "{{ SYS_TOKEN_STORE_GROUP }}" - mode: "{{ SYS_TOKEN_STORE_FILE_MODE }}" - content: "{{ {'users': sys_token_store_users_merged} | to_nice_yaml(indent=2, width=9999) }}" - when: sys_token_store_users_merged != sys_token_store_users - delegate_to: "{{ STACK_HOST }}" - run_once: true - no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" - become: true - register: sys_token_store_persisted - -- name: "🪞 Mirror the store onto the controller, where every users lookup reads" - ansible.builtin.copy: - dest: "{{ FILE_TOKENS }}" - mode: "{{ SYS_TOKEN_STORE_FILE_MODE }}" - content: "{{ {'users': sys_token_store_users_merged} | to_nice_yaml(indent=2, width=9999) }}" - when: - - sys_token_store_users_merged != sys_token_store_users - - DEPLOYMENT_MODE == 'swarm' - delegate_to: localhost # nocheck: localhost-become FILE_TOKENS sits below the root-owned DIR_SECRETS - run_once: true - no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" - register: sys_token_store_mirrored - -- name: "🔄 Hand the written store to the lookups of the rest of the play" - ansible.builtin.set_fact: - sys_token_store_users: "{{ sys_token_store_users_merged }}" - when: sys_token_store_users_merged != sys_token_store_users - no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" - -- name: 🧊 Re-render the applications SPOT the new token invalidated - applications_carrier: {} - when: sys_token_store_persisted is changed or sys_token_store_mirrored is changed +- name: "💾 Persist the token as a one-entry batch" + ansible.builtin.import_tasks: write_many.yml + vars: + sys_token_store_entries: + - user_key: "{{ sys_token_store_user_key }}" + app: "{{ sys_token_store_app }}" + token: "{{ sys_token_store_token }}" diff --git a/roles/sys-token-store/tasks/write_many.yml b/roles/sys-token-store/tasks/write_many.yml new file mode 100644 index 0000000000..458e08225c --- /dev/null +++ b/roles/sys-token-store/tasks/write_many.yml @@ -0,0 +1,66 @@ +--- +# REQUIRED: +# sys_token_store_entries # list of {user_key, app, token}; every token must be non-empty + +- name: "✅ Ensure store exists" + ansible.builtin.import_tasks: 00_core.yml + +- name: "🛑 Abort if a token is empty (do not persist empties)" + ansible.builtin.assert: + that: + - sys_token_store_empty | length == 0 + fail_msg: "Refusing to persist the empty tokens of {{ sys_token_store_empty }}." + vars: + sys_token_store_empty: >- + {%- set empty = [] -%} + {%- for entry in sys_token_store_entries if (entry.token | string | trim) == '' -%} + {%- set _ = empty.append(entry.user_key ~ '/' ~ entry.app) -%} + {%- endfor -%} + {{ empty }} + +- name: "🔨 Build merged structure" + ansible.builtin.set_fact: + sys_token_store_users_merged: >- + {%- set layers = [] -%} + {%- for entry in sys_token_store_entries -%} + {%- set _ = layers.append({entry.user_key: {'tokens': {entry.app: entry.token | string | trim}}}) -%} + {%- endfor -%} + {{ sys_token_store_users | combine(layers, recursive=True) }} + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" + +- name: "💾 Persist store (idempotent, single-writer via STACK_HOST)" + ansible.builtin.copy: + dest: "{{ FILE_TOKENS }}" + owner: "{{ SYS_TOKEN_STORE_OWNER }}" + group: "{{ SYS_TOKEN_STORE_GROUP }}" + mode: "{{ SYS_TOKEN_STORE_FILE_MODE }}" + content: "{{ {'users': sys_token_store_users_merged} | to_nice_yaml(indent=2, width=9999) }}" + when: sys_token_store_users_merged != sys_token_store_users + delegate_to: "{{ STACK_HOST }}" + run_once: true + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" + become: true + register: sys_token_store_persisted + +- name: "🪞 Mirror the store onto the controller, where every users lookup reads" + ansible.builtin.copy: + dest: "{{ FILE_TOKENS }}" + mode: "{{ SYS_TOKEN_STORE_FILE_MODE }}" + content: "{{ {'users': sys_token_store_users_merged} | to_nice_yaml(indent=2, width=9999) }}" + when: + - sys_token_store_users_merged != sys_token_store_users + - DEPLOYMENT_MODE == 'swarm' + delegate_to: localhost # nocheck: localhost-become FILE_TOKENS sits below the root-owned DIR_SECRETS + run_once: true + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" + register: sys_token_store_mirrored + +- name: "🔄 Hand the written store to the lookups of the rest of the play" + ansible.builtin.set_fact: + sys_token_store_users: "{{ sys_token_store_users_merged }}" + when: sys_token_store_users_merged != sys_token_store_users + no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" + +- name: 🧊 Re-render the applications SPOT the new tokens invalidated + applications_carrier: {} + when: sys_token_store_persisted is changed or sys_token_store_mirrored is changed diff --git a/roles/web-app-stalwart/tasks/04_manage_user.yml b/roles/web-app-stalwart/tasks/04_manage_user.yml index 23e13ba3e5..89a0b20766 100644 --- a/roles/web-app-stalwart/tasks/04_manage_user.yml +++ b/roles/web-app-stalwart/tasks/04_manage_user.yml @@ -44,17 +44,23 @@ {%- set result = stalwart_jmap.methodResponses[0][1] -%} {%- set listed = {} -%} {%- for held in stalwart_jmap.methodResponses[1][1].list if held.domainId == stalwart_domain_id -%} - {%- set _ = listed.update({held.name | lower: held.id}) -%} + {%- set _ = listed.update({held.name | lower: held}) -%} {%- endfor -%} {%- set accounts = [] -%} {%- for account in stalwart_accounts -%} {%- set created = (result.created | default({}, true)).get('u' ~ loop.index0) or {} -%} {%- set refusal = (result.notCreated | default({}, true)).get('u' ~ loop.index0) or {} -%} - {%- set existing = refusal.get('type') == 'alreadyExists' -%} + {%- set held = listed.get(account.name | lower) or {} -%} + {%- set conflict = refusal.get('objectId') or {} -%} + {%- set credentials = held.get('credentials') or {} -%} + {%- set passwords = credentials | dict2items | selectattr('value.@type', 'equalto', 'Password') + | map(attribute='key') | list -%} {%- set _ = accounts.append(account | combine({ - 'id': ((refusal.get('objectId') or {}).get('id') if existing else created.get('id')) - or listed.get(account.name | lower, ''), - 'existing': existing, + 'id': created.get('id') or held.get('id', ''), + 'existing': refusal.get('type') == 'primaryKeyViolation' and conflict.get('object') == 'Account' + and held.get('id') is not none and conflict.get('id') == held.get('id'), + 'password_key': passwords | first if passwords + else (credentials | map('int') | max + 1 if credentials else 0), 'refusal': refusal})) -%} {%- endfor -%} {{ accounts }} @@ -68,8 +74,10 @@ vars: stalwart_account_refused: >- {%- set refused = {} -%} - {%- for account in stalwart_accounts if not account.id -%} - {%- if account.bot or not (STALWART_SSO_ENABLED | bool and account.refusal.get('type') == 'forbidden') -%} + {%- for account in stalwart_accounts -%} + {%- set external = STALWART_SSO_ENABLED | bool and account.refusal.get('type') == 'forbidden' -%} + {%- if (not account.id and (account.bot or not external)) + or (account.refusal and not (account.existing or external)) -%} {%- set _ = refused.update({account.address: account.refusal}) -%} {%- endif -%} {%- endfor -%} @@ -86,7 +94,7 @@ {%- set changes = {} -%} {%- for account in stalwart_accounts if account.existing -%} {%- set _ = changes.update({account.id: { - 'credentials': {'0': {'@type': 'Password', 'secret': account.secret}}}}) -%} + 'credentials/' ~ account.password_key ~ '/secret': account.secret}}) -%} {%- endfor -%} {{ {'using': ['urn:ietf:params:jmap:core', 'urn:stalwart:jmap'], 'methodCalls': [['x:Account/set', {'update': changes}, 'c0']]} | to_json }} @@ -108,16 +116,17 @@ - name: "💾 Persist the Stalwart submission credential of every bot" ansible.builtin.include_role: name: sys-token-store - tasks_from: write.yml + tasks_from: write_many.yml vars: - sys_token_store_user_key: "{{ stalwart_bot.key }}" - sys_token_store_app: "{{ application_id }}" - sys_token_store_token: "{{ stalwart_bot.secret }}" - loop: "{{ stalwart_accounts | selectattr('bot') | list }}" - loop_control: - loop_var: stalwart_bot - label: "{{ stalwart_bot.address }}" - when: not (STALWART_SSO_ENABLED | bool) + sys_token_store_entries: >- + {%- set entries = [] -%} + {%- for bot in stalwart_accounts if bot.bot -%} + {%- set _ = entries.append({'user_key': bot.key, 'app': application_id, 'token': bot.secret}) -%} + {%- endfor -%} + {{ entries }} + when: + - not (STALWART_SSO_ENABLED | bool) + - stalwart_accounts | selectattr('bot') | list | length > 0 no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" - name: "🔑 Keep an app password for every bot under SSO" diff --git a/roles/web-app-stalwart/tasks/04_manage_user/app_password.yml b/roles/web-app-stalwart/tasks/04_manage_user/app_password.yml index 1b4faff5aa..896dc8d747 100644 --- a/roles/web-app-stalwart/tasks/04_manage_user/app_password.yml +++ b/roles/web-app-stalwart/tasks/04_manage_user/app_password.yml @@ -62,15 +62,14 @@ - name: "💾 Persist the new app passwords" ansible.builtin.include_role: name: sys-token-store - tasks_from: write.yml + tasks_from: write_many.yml vars: - sys_token_store_user_key: "{{ stalwart_app_password_new.key }}" - sys_token_store_app: "{{ application_id }}" - sys_token_store_token: "{{ stalwart_app_password_new.secret }}" - loop: "{{ stalwart_app_password_minted }}" - loop_control: - loop_var: stalwart_app_password_new - label: "{{ stalwart_app_password_new.address }}" + sys_token_store_entries: >- + {%- set entries = [] -%} + {%- for minted in stalwart_app_password_minted -%} + {%- set _ = entries.append({'user_key': minted.key, 'app': application_id, 'token': minted.secret}) -%} + {%- endfor -%} + {{ entries }} no_log: "{{ MASK_CREDENTIALS_IN_LOGS | bool }}" - name: "🏷️ Retire the app passwords the new ones replace" diff --git a/roles/web-app-stalwart/tasks/jmap_call.yml b/roles/web-app-stalwart/tasks/jmap_call.yml index 2b7e281f4b..d76e74a1c2 100644 --- a/roles/web-app-stalwart/tasks/jmap_call.yml +++ b/roles/web-app-stalwart/tasks/jmap_call.yml @@ -1,7 +1,7 @@ --- # Reusable management JMAP call: in `stalwart_jmap_body` (JSON), out `stalwart_jmap`. # Param: stalwart_jmap_body - JSON request; a batch carries one method call per object owner, its call id naming that owner in a refusal. -# Param: stalwart_jmap_tolerate_rejected - true when the caller reads a set-level rejection itself (e.g. alreadyExists). +# Param: stalwart_jmap_tolerate_rejected - true when the caller reads a set-level rejection itself (e.g. primaryKeyViolation). - name: "🔧 Stalwart JMAP call" ansible.builtin.shell: cmd: >- diff --git a/tests/unit/python/roles/sys-token-store/test_write_many.py b/tests/unit/python/roles/sys-token-store/test_write_many.py new file mode 100644 index 0000000000..728eabea4d --- /dev/null +++ b/tests/unit/python/roles/sys-token-store/test_write_many.py @@ -0,0 +1,164 @@ +from __future__ import annotations + +import unittest + +from ansible.parsing.dataloader import DataLoader +from ansible.template import Templar + +from utils.cache.files import PROJECT_ROOT +from utils.cache.yaml import load_yaml_any +from utils.templating.ansible import _trust_as_template + +TASKS = PROJECT_ROOT / "roles" / "sys-token-store" / "tasks" +MERGE = "🔨 Build merged structure" +EMPTY = "🛑 Abort if a token is empty (do not persist empties)" +PERSIST = "💾 Persist store (idempotent, single-writer via STACK_HOST)" +MIRROR = "🪞 Mirror the store onto the controller, where every users lookup reads" +HAND_OVER = "🔄 Hand the written store to the lookups of the rest of the play" +RERENDER = "🧊 Re-render the applications SPOT the new tokens invalidated" + +USERS = { + "alice": {"tokens": {"web-app-a": "old-a"}, "note": "kept"}, + "carol": {"tokens": {"web-app-c": "token-c"}}, +} +SKIPPED = {"changed": False, "skipped": True} + + +def _tasks(name: str) -> list[dict]: + return load_yaml_any(str(TASKS / name), default_if_missing=[]) + + +def _task(name: str) -> dict: + return next(t for t in _tasks("write_many.yml") if t.get("name") == name) + + +def _render(template, **variables): + templar = Templar(loader=DataLoader(), variables=variables) + return templar.template(_trust_as_template(template)) + + +def _merged(entries: list[dict], users: dict = USERS) -> dict: + return _render( + _task(MERGE)["ansible.builtin.set_fact"]["sys_token_store_users_merged"], + sys_token_store_entries=entries, + sys_token_store_users=users, + ) + + +def _when(name: str, **variables) -> bool: + when = _task(name)["when"] + conditions = when if isinstance(when, list) else [when] + return _render( + "{{ ((" + ") and (".join(conditions) + ")) | bool }}", + DEPLOYMENT_MODE="swarm", + **variables, + ) + + +def _entry(user_key: str, app: str, token: str) -> dict: + return {"user_key": user_key, "app": app, "token": token} + + +class TestWriteMany(unittest.TestCase): + def test_every_entry_lands_in_one_merge(self): + merged = _merged( + [ + _entry("alice", "web-app-b", " new-b "), + _entry("alice", "web-app-a", "new-a"), + _entry("bob", "web-app-b", "token-b"), + ] + ) + self.assertEqual( + merged, + { + "alice": { + "tokens": {"web-app-a": "new-a", "web-app-b": "new-b"}, + "note": "kept", + }, + "bob": {"tokens": {"web-app-b": "token-b"}}, + "carol": {"tokens": {"web-app-c": "token-c"}}, + }, + ) + + def test_the_later_entry_of_one_user_and_app_wins(self): + merged = _merged( + [_entry("bob", "web-app-b", "first"), _entry("bob", "web-app-b", "second")] + ) + self.assertEqual(merged["bob"], {"tokens": {"web-app-b": "second"}}) + + def test_an_unchanged_store_skips_the_write_and_the_rerender(self): + merged = _merged( + [ + _entry("alice", "web-app-a", "old-a"), + _entry("carol", "web-app-c", "token-c"), + ] + ) + self.assertEqual(merged, USERS) + state = {"sys_token_store_users_merged": merged, "sys_token_store_users": USERS} + self.assertFalse(_when(PERSIST, **state)) + self.assertFalse(_when(MIRROR, **state)) + self.assertFalse(_when(HAND_OVER, **state)) + self.assertFalse( + _when( + RERENDER, + sys_token_store_persisted=SKIPPED, + sys_token_store_mirrored=SKIPPED, + ) + ) + + def test_a_changed_store_is_written_and_rerendered_once(self): + merged = _merged([_entry("bob", "web-app-b", "token-b")]) + state = {"sys_token_store_users_merged": merged, "sys_token_store_users": USERS} + self.assertTrue(_when(PERSIST, **state)) + self.assertTrue(_when(HAND_OVER, **state)) + self.assertTrue( + _when( + RERENDER, + sys_token_store_persisted={"changed": True}, + sys_token_store_mirrored=SKIPPED, + ) + ) + rerenders = [t for t in _tasks("write_many.yml") if "applications_carrier" in t] + self.assertEqual(len(rerenders), 1) + self.assertNotIn("loop", rerenders[0]) + + def test_the_store_has_one_writer(self): + persist = _task(PERSIST) + self.assertEqual(persist["delegate_to"], "{{ STACK_HOST }}") + self.assertIs(persist["run_once"], True) + mirror = _task(MIRROR) + self.assertEqual(mirror["delegate_to"], "localhost") + self.assertIs(mirror["run_once"], True) + + def test_an_empty_token_is_refused_by_name(self): + empty = _render( + _task(EMPTY)["vars"]["sys_token_store_empty"], + sys_token_store_entries=[ + _entry("alice", "web-app-a", "set"), + _entry("bob", "web-app-b", " "), + ], + ) + self.assertEqual(empty, ["bob/web-app-b"]) + + +class TestWriteIsAOneEntryBatch(unittest.TestCase): + def test_the_single_token_contract_feeds_write_many(self): + tasks = _tasks("write.yml") + self.assertEqual(len(tasks), 1) + self.assertEqual(tasks[0]["ansible.builtin.import_tasks"], "write_many.yml") + single = { + "sys_token_store_user_key": "administrator", + "sys_token_store_app": "web-app-matomo", + "sys_token_store_token": "secret-token", + } + entries = [ + {field: _render(value, **single) for field, value in entry.items()} + for entry in tasks[0]["vars"]["sys_token_store_entries"] + ] + self.assertEqual( + entries, [_entry("administrator", "web-app-matomo", "secret-token")] + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/unit/python/roles/web-app-stalwart/test_account_batch.py b/tests/unit/python/roles/web-app-stalwart/test_account_batch.py index c31ac7363d..c033c699ec 100644 --- a/tests/unit/python/roles/web-app-stalwart/test_account_batch.py +++ b/tests/unit/python/roles/web-app-stalwart/test_account_batch.py @@ -82,19 +82,42 @@ BASE = { "stalwart_app_password_stale": {}, } +PASSWORD = {"@type": "Password", "secret": "********"} +APP_PASSWORD = {"@type": "AppPassword", "description": DESCRIPTION} + + +def _violation(object_id: str, object_type: str = "Account") -> dict: + return { + "type": "primaryKeyViolation", + "properties": ["email"], + "objectId": {"object": object_type, "id": object_id}, + } + + +def _held(object_id: str, name: str, credentials: dict | None = None) -> dict: + return { + "id": object_id, + "name": name, + "domainId": "d1", + "credentials": {"0": PASSWORD} if credentials is None else credentials, + } + + CREATE_RESPONSE = [ [ "x:Account/set", { "created": {"u0": {"id": "a"}}, - "notCreated": { - "u1": {"type": "alreadyExists", "objectId": {"id": "b"}}, - "u2": {"type": "alreadyExists", "objectId": {"id": "c"}}, - }, + "notCreated": {"u1": _violation("b"), "u2": _violation("c")}, }, "c0", ] ] +HELD = ( + _held("a", "administrator"), + _held("b", "no-reply"), + _held("c", "contact"), +) def _resolved() -> list[dict]: @@ -106,7 +129,7 @@ def _resolved() -> list[dict]: ) -def _matched(response: list = CREATE_RESPONSE, listed: tuple = ()) -> list[dict]: +def _matched(response: list = CREATE_RESPONSE, listed: tuple = HELD) -> list[dict]: return _fact( "04_manage_user.yml", "🔍 Match every account to its Stalwart id", @@ -209,10 +232,73 @@ class TestAccountPhases(unittest.TestCase): def test_an_existing_account_without_an_id_fails(self): response = [ - ["x:Account/set", {"notCreated": {"u0": {"type": "alreadyExists"}}}, "c0"] + [ + "x:Account/set", + {"notCreated": {"u0": {"type": "primaryKeyViolation"}}}, + "c0", + ] + ] + refused = _refused(_matched(response, listed=())) + self.assertIn("administrator@infinito.test", refused) + + def test_a_primary_key_violation_on_the_listed_account_is_existing(self): + accounts = {a["key"]: a for a in _matched()} + self.assertEqual( + {key: (a["id"], a["existing"]) for key, a in accounts.items()}, + { + "administrator": ("a", False), + "no-reply": ("b", True), + "contact": ("c", True), + }, + ) + self.assertEqual(_refused(list(accounts.values())), {}) + + def test_a_violation_against_another_object_is_refused(self): + alias = _violation("a") + mailing_list = _violation("c", "MailingList") + response = [ + [ + "x:Account/set", + { + "created": {"u0": {"id": "a"}}, + "notCreated": {"u1": alias, "u2": mailing_list}, + }, + "c0", + ] + ] + accounts = _matched(response) + self.assertEqual([a["existing"] for a in accounts], [False, False, False]) + self.assertEqual( + _refused(accounts), + {"no-reply@infinito.test": alias, "contact@infinito.test": mailing_list}, + ) + + def test_an_unknown_refusal_of_a_listed_bot_fails_without_sso(self): + refusal = {"type": "invalidProperties", "properties": ["description"]} + response = [ + [ + "x:Account/set", + {"notCreated": {"u0": _violation("a"), "u1": refusal, "u2": refusal}}, + "c0", + ] ] refused = _refused(_matched(response)) - self.assertIn("administrator@infinito.test", refused) + self.assertEqual( + refused, + {"no-reply@infinito.test": refusal, "contact@infinito.test": refusal}, + ) + + def test_an_unknown_refusal_of_a_listed_bot_fails_under_sso(self): + refusal = {"type": "invalidProperties", "properties": ["description"]} + response = [ + [ + "x:Account/set", + {"notCreated": {"u0": EXTERNAL, "u1": refusal, "u2": EXTERNAL}}, + "c0", + ] + ] + refused = _refused(_matched(response), sso=True) + self.assertEqual(refused, {"no-reply@infinito.test": refusal}) def test_accounts_of_an_external_directory_take_their_listed_ids(self): listed = ( @@ -271,9 +357,65 @@ class TestAccountPhases(unittest.TestCase): ) method, arguments, _ = body["methodCalls"][0] self.assertEqual(method, "x:Account/set") - self.assertEqual(sorted(arguments["update"]), ["b", "c"]) self.assertEqual( - arguments["update"]["c"]["credentials"]["0"]["secret"], _sha("contact-pw") + arguments["update"], + { + "b": {"credentials/0/secret": _sha("no-reply-pw")}, + "c": {"credentials/0/secret": _sha("contact-pw")}, + }, + ) + + def _update_of_contact(self, credentials: dict) -> dict: + listed = (*HELD[:2], _held("c", "contact", credentials)) + body = _body( + "04_manage_user.yml", + "🔄 Send the passwords of the pre-existing accounts", + **BASE, + stalwart_accounts=_matched(listed=listed), + ) + return body["methodCalls"][0][1]["update"]["c"] + + def test_the_patch_keeps_the_other_credentials_of_the_account(self): + update = self._update_of_contact({"0": APP_PASSWORD, "3": PASSWORD}) + self.assertEqual(update, {"credentials/3/secret": _sha("contact-pw")}) + + def test_an_account_without_a_password_gets_one_at_a_free_key(self): + self.assertEqual( + self._update_of_contact({"0": APP_PASSWORD, "1": APP_PASSWORD}), + {"credentials/2/secret": _sha("contact-pw")}, + ) + self.assertEqual( + self._update_of_contact({}), {"credentials/0/secret": _sha("contact-pw")} + ) + + def test_one_store_write_carries_every_bot(self): + task = _task( + "04_manage_user.yml", + "💾 Persist the Stalwart submission credential of every bot", + ) + self.assertEqual( + task["ansible.builtin.include_role"]["tasks_from"], "write_many.yml" + ) + self.assertNotIn("loop", task) + entries = _render( + task["vars"]["sys_token_store_entries"], + **BASE, + stalwart_accounts=_matched(), + ) + self.assertEqual( + entries, + [ + { + "user_key": "no-reply", + "app": "web-app-stalwart", + "token": _sha("no-reply-pw"), + }, + { + "user_key": "contact", + "app": "web-app-stalwart", + "token": _sha("contact-pw"), + }, + ], ) def test_a_refused_update_fails_naming_the_account(self): @@ -493,6 +635,31 @@ class TestAppPasswordPhases(unittest.TestCase): ], ) + def test_the_minted_passwords_land_in_one_store_write(self): + task = _task( + "04_manage_user/app_password.yml", "💾 Persist the new app passwords" + ) + self.assertEqual( + task["ansible.builtin.include_role"]["tasks_from"], "write_many.yml" + ) + self.assertNotIn("loop", task) + minted = [ + {"key": "no-reply", "address": "no-reply@infinito.test", "secret": "p1"}, + {"key": "contact", "address": "contact@infinito.test", "secret": "p2"}, + ] + entries = _render( + task["vars"]["sys_token_store_entries"], + **BASE, + stalwart_app_password_minted=minted, + ) + self.assertEqual( + entries, + [ + {"user_key": "no-reply", "app": "web-app-stalwart", "token": "p1"}, + {"user_key": "contact", "app": "web-app-stalwart", "token": "p2"}, + ], + ) + def test_the_record_keeps_the_owners_of_an_earlier_chunk(self): stale = _fact( "04_manage_user/app_password.yml",