mirror of
https://github.com/kevinveenbirkenbach/computer-playbook.git
synced 2026-11-09 12:17:13 +00:00
feat(web-app-docs): merge infinito-sphinx and serve every release on demand
The Sphinx configuration, extensions and generators lived in the separate infinito-sphinx repository, whose image baked a snapshot of core at build time from a base image name core no longer publishes. Only core consumed it, and its generators read core's role layout, so the two repositories drifted without any CI covering the pair. The tooling now ships in roles/web-app-docs/files/, and web-app-sphinx is renamed to web-app-docs. The container is a small service instead of a static image: - it keeps a git mirror of the repository in the docs_sites volume; - /latest/ is the last commit of the default branch, rebuilt whenever a fetch finds a new one while the previous build stays online; - every release tag is selectable in the sidebar switcher and on /versions/, and is built the first time someone opens it, behind a progress bar; - replicas share the volume: the one holding builder.lock fetches and builds, the queue and the build states are files beside the sites, so another replica resumes the queue when the builder dies. The theme rewrites every page serially after the build, which took about 75 minutes for the 4544 pages; that pass now runs on the -j workers with the theme's own function. sphinx-apidoc skips tests/, because tests/utils/bare_bootstrap_probe.py installs an import blocker when autodoc imports it and killed the Sphinx workers. The roles overview is generated by a separate process, so autodoc resolves utils from the version being built, not from main. The role pages now show galaxy_info.description instead of the top-level key that never existed. The Sphinx packages join the dev extra, because the mirrored unit tests of the tooling import them. min_storage now budgets the built image, the latest site and a running build's scratch instead of the old image's size. The Playwright suite is split into front page, versions and on-demand build files; the last one selects an unbuilt tag in the overview and asserts the build gets queued. Operators must rename the web-app-sphinx inventory group to web-app-docs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kevinveenbirkenbachandClaude Opus 5 committed
1 parent
6de38921ca
commit
6eb9a59d14
81 files changed
+3712
-254
No files matched your search
@@ -86,6 +86,7 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag
|
||||
| [Docker Registry Cache](roles/svc-registry-cache/) | beta | Cluster-local pull-through cache for shared Docker Hub base-image pulls, cutting duplicate transfers and rate limits | [🛠️](roles/svc-registry-cache/README.md#quick-setup) | [Docker Swarm](roles/svc-swarm-node/) |
|
||||
| [Docker Swarm](roles/svc-swarm-node/) | beta | Container orchestration cluster for scheduling services across pooled manager and worker nodes | [🛠️](roles/svc-swarm-node/README.md#quick-setup) | |
|
||||
| [Docker Swarm Manager](roles/svc-swarm-manager/) | beta | Marker role tagging a host as the Docker Swarm manager node, with the init logic delegated to svc-swarm-node | [🛠️](roles/svc-swarm-manager/README.md#quick-setup) | [Docker Swarm](roles/svc-swarm-node/) |
|
||||
| [Documentation](roles/web-app-docs/) | beta | Infinito.Nexus documentation for the latest commit and every release, built on demand with Sphinx | [🌐](https://www.sphinx-doc.org/) [🛠️](roles/web-app-docs/README.md#quick-setup) | [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) |
|
||||
| [Elasticsearch](roles/svc-db-elasticsearch/) | beta | Central search and analytics engine with per-consumer index namespaces isolating tenants in a shared Docker deployment | [🌐](https://www.elastic.co/elasticsearch) [🛠️](roles/svc-db-elasticsearch/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Tor Onion Service](roles/svc-net-tor/) |
|
||||
| [ERPNext](roles/web-app-erpnext/) | beta | Self-hosted open-source ERP suite for finance, CRM, inventory, manufacturing, HR, sales, and projects | [🌐](https://erpnext.com/) [🎬](https://www.youtube.com/watch?v=JlqE-BfDEUY) [🛠️](roles/web-app-erpnext/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [MariaDB](roles/svc-db-mariadb/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Redis](roles/svc-db-redis/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) |
|
||||
| [EspoCRM](roles/web-app-espocrm/) | beta | Open-source CRM platform for managing sales, customer interactions, and business processes across teams | [🌐](https://www.espocrm.com/) [🎬](https://www.youtube.com/watch?v=TH6yJPTPLO0) [🛠️](roles/web-app-espocrm/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [MariaDB](roles/svc-db-mariadb/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) |
|
||||
@@ -171,7 +172,6 @@ Every invokable role in the tested lifecycle envelope, with its upstream homepag
|
||||
| [Simple Icons](roles/web-svc-simpleicons/) | beta | Containerized icon server for hosting the Simple Icons brand SVG and PNG collection for web and documentation use | [🌐](https://simpleicons.org/) [🛠️](roles/web-svc-simpleicons/README.md#quick-setup) | [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) |
|
||||
| [Snipe‑IT](roles/web-app-snipe-it/) | beta | Open-source IT asset management system for tracking hardware and software inventory | [🌐](https://snipeitapp.com/) [🎬](https://www.youtube.com/watch?v=q0V6FAPc1ls) [🛠️](roles/web-app-snipe-it/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Hermes Agent](roles/web-app-hermes/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [MariaDB](roles/svc-db-mariadb/), [Matomo](roles/web-app-matomo/), [Open WebUI](roles/web-app-openwebui/), [OpenClaw](roles/web-app-openclaw/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Redis](roles/svc-db-redis/), [SeaweedFS](roles/web-app-seaweedfs/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) |
|
||||
| [Social Home](roles/web-app-socialhome/) | alpha | Privacy-first social platform for households, federating peer-to-peer over WebRTC on an embedded SQLite store | [🌐](https://github.com/social-home-io/socialhome) [🛠️](roles/web-app-socialhome/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Coturn](roles/web-svc-coturn/), [Dashboard](roles/web-app-dashboard/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) |
|
||||
| [Sphinx](roles/web-app-sphinx/) | beta | Documentation generator that builds HTML from a source repository and serves it through a lightweight HTTP server | [🌐](https://www.sphinx-doc.org/) [🛠️](roles/web-app-sphinx/README.md#quick-setup) | [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Matomo](roles/web-app-matomo/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/) |
|
||||
| [Storage Optimizer](roles/svc-opt-ssd-hdd/) | beta | Tiering role placing Docker volumes on SSD or HDD by workload type, with symlinks keeping paths consistent | [🛠️](roles/svc-opt-ssd-hdd/README.md#quick-setup) | |
|
||||
| [SuiteCRM](roles/web-app-suitecrm/) | beta | Open-source CRM platform for sales, service, cases, and campaign management with workflow automation | [🌐](https://suitecrm.com/) [🎬](https://www.youtube.com/watch?v=pOAs_C3Fp_E) [🛠️](roles/web-app-suitecrm/README.md#quick-setup) | [Backup Docker Volumes](roles/svc-bkp-volume-2-local/), [Corporate Design](roles/web-svc-css/), [Dashboard](roles/web-app-dashboard/), [Keycloak](roles/web-app-keycloak/), [Mailu](roles/web-app-mailu/), [MariaDB](roles/svc-db-mariadb/), [Matomo](roles/web-app-matomo/), [OpenLDAP](roles/svc-db-openldap/), [Prometheus](roles/web-app-prometheus/), [Tor Onion Service](roles/svc-net-tor/), [Universal Logout](roles/web-svc-logout/) |
|
||||
| [Swapfile](roles/svc-opt-swapfile/) | beta | On-target automation for creating and activating swap space via a cloned swapfile script | [🛠️](roles/svc-opt-swapfile/README.md#quick-setup) | |
|
||||
|
||||
@@ -74,6 +74,7 @@ See [Docs README](artefact/files/docs/readme_md.md) for the purpose, scope, and
|
||||
|
||||
### Sphinx 📚
|
||||
|
||||
- The Sphinx configuration, extensions and generators live in [web-app-docs](../../roles/web-app-docs/). Deploy that role to build and serve the site.
|
||||
- The root `index.rst` uses a `:glob:` toctree (`docs/**`) to automatically include every documentation page.
|
||||
- Sphinx indexing SHOULD happen automatically through that root `:glob:` toctree.
|
||||
- You SHOULD avoid creating or maintaining manual per-directory indexes when automatic indexing already covers the same pages.
|
||||
|
||||
@@ -129,7 +129,7 @@ scenario; closing each item is a NOOP except when a stale env-key
|
||||
shows up; in that case remove it.
|
||||
|
||||
- [ ] [web-svc-cdn](../../roles/web-svc-cdn/), [web-svc-simpleicons](../../roles/web-svc-simpleicons/), [web-svc-xmpp](../../roles/web-svc-xmpp/), [web-svc-libretranslate](../../roles/web-svc-libretranslate/): backend services with no end-user UI.
|
||||
- [ ] [web-app-hugo](../../roles/web-app-hugo/), [web-app-sphinx](../../roles/web-app-sphinx/), [web-app-bridgy-fed](../../roles/web-app-bridgy-fed/), [web-app-mig](../../roles/web-app-mig/): static or pure-publishing content with no authenticated path.
|
||||
- [ ] [web-app-hugo](../../roles/web-app-hugo/), [web-app-docs](../../roles/web-app-docs/), [web-app-bridgy-fed](../../roles/web-app-bridgy-fed/), [web-app-mig](../../roles/web-app-mig/): static or pure-publishing content with no authenticated path.
|
||||
- [ ] [web-app-dashboard](../../roles/web-app-dashboard/), [web-app-matomo](../../roles/web-app-matomo/): own admin authentication is in scope of their own admin tests; biber-as-non-admin is not a meaningful surface here. Document and exempt.
|
||||
|
||||
### Verification
|
||||
|
||||
@@ -97,7 +97,7 @@ Notes:
|
||||
| **shopware** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-shopware/meta/services.yml) | ❌ | **shopware** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **shopware** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-shopware/meta/services.yml) | ❌ | ❌ | ❌ | **shopware** | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-shopware/meta/services.yml) | ❌ | [☑️](../../roles/web-app-shopware/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **shopware** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **shopware** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-shopware/meta/services.yml) | **shopware** | ❌ | — | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **shopware** | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-shopware/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **shopware** | [☑️](../../roles/web-app-shopware/meta/services.yml) | ❌ | ❌ | ❌ |
|
||||
| **snipe-it** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-snipe-it/meta/services.yml) | ❌ | **snipe-it** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **snipe-it** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-snipe-it/meta/services.yml) | ❌ | ❌ | ❌ | **snipe-it** | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-snipe-it/meta/services.yml) | ❌ | [☑️](../../roles/web-app-snipe-it/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **snipe-it** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **snipe-it** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-snipe-it/meta/services.yml) | **snipe-it** | ❌ | ❌ | — | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **snipe-it** | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-snipe-it/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **snipe-it** | [☑️](../../roles/web-app-snipe-it/meta/services.yml) | ❌ | ❌ | ❌ |
|
||||
| **socialhome** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-socialhome/meta/services.yml) | ❌ | **socialhome** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **socialhome** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-socialhome/meta/services.yml) | ❌ | ❌ | ❌ | **socialhome** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **socialhome** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **socialhome** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-socialhome/meta/services.yml) | **socialhome** | ❌ | ❌ | ❌ | — | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **socialhome** | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-socialhome/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | ❌ | **socialhome** | [☑️](../../roles/web-app-socialhome/meta/services.yml) | ❌ | ❌ | ❌ |
|
||||
| **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-sphinx/meta/services.yml) | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-sphinx/meta/services.yml) | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-sphinx/meta/services.yml) | ❌ | [☑️](../../roles/web-app-sphinx/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-sphinx/meta/services.yml) | **sphinx** | ❌ | ❌ | ❌ | ❌ | — | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-sphinx/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **sphinx** | [☑️](../../roles/web-app-sphinx/meta/services.yml) | ❌ | ❌ | ❌ |
|
||||
| **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | — | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **sphinx** | ❌ | ❌ | ❌ | ❌ |
|
||||
| **suitecrm** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-suitecrm/meta/services.yml) | ❌ | **suitecrm** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **suitecrm** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [✅](https://docs.suitecrm.com/8.x/admin/configuration/saml/8.7.0-saml-configuration/) | ❌ | ❌ | ❌ | **suitecrm** | ❌ | ❌ | ❌ | [✅](https://github.com/SuiteCRM/SuiteCRM) | ❌ | [☑️](../../roles/web-app-suitecrm/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **suitecrm** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **suitecrm** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-suitecrm/meta/services.yml) | **suitecrm** | ❌ | ❌ | ❌ | ❌ | ❌ | — | ❌ | ❌ | ❌ | ❌ | **suitecrm** | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-suitecrm/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **suitecrm** | [☑️](../../roles/web-app-suitecrm/meta/services.yml) | ❌ | ❌ | ❌ |
|
||||
| **taiga** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-taiga/meta/services.yml) | ❌ | **taiga** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **taiga** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [✅](https://github.com/taigaio/taiga-contrib-oidc-auth) | ❌ | ❌ | ❌ | **taiga** | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-taiga/meta/services.yml) | ❌ | [☑️](../../roles/web-app-taiga/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **taiga** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **taiga** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-taiga/meta/services.yml) | **taiga** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | — | ❌ | ❌ | ❌ | **taiga** | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-taiga/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **taiga** | [☑️](../../roles/web-app-taiga/meta/services.yml) | ❌ | ❌ | ❌ |
|
||||
| **wordpress** | ❌ | ❌ | [✅](https://wordpress.org/plugins/video-conferencing-with-bbb/) | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-wordpress/meta/services.yml) | ❌ | **wordpress** | [✅](https://wordpress.org/plugins/wp-discourse/) | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **wordpress** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [✅](https://wordpress.org/plugins/daggerhart-openid-connect-generic/) | ❌ | ❌ | [✅](https://wordpress.org/plugins/integration-for-listmonk-mailing-list-and-newsletter-manager/) | **wordpress** | [☑️](../../roles/web-app-wordpress/meta/services.yml) | ❌ | ❌ | [☑️](../../roles/web-app-wordpress/meta/services.yml) | [✅](https://wordpress.org/plugins/activitypub/) | [✅](https://wordpress.org/plugins/matomo/) | ❌ | ❌ | ❌ | ❌ | **wordpress** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | **wordpress** | ❌ | ❌ | [✅](https://wordpress.org/plugins/video-manager-for-peertube/) | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-wordpress/meta/services.yml) | **wordpress** | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | — | ❌ | ❌ | **wordpress** | ❌ | ❌ | ❌ | ❌ | ❌ | [☑️](../../roles/web-app-wordpress/meta/services.yml) | ❌ | ❌ | ❌ | ❌ | **wordpress** | [☑️](../../roles/web-app-wordpress/meta/services.yml) | ❌ | ❌ | ❌ |
|
||||
|
||||
@@ -501,7 +501,7 @@ These UIs expose identity or database administration rather than a bounded appli
|
||||
**`duplicate_owner` (1):** `web-app-litellm`.
|
||||
This role is only the UI of the separately classified LiteLLM service role, which owns any future MCP gateway contract.
|
||||
|
||||
**`no_remote_surface` (24):** `svc-ai-ollama`, `svc-ai-robot`, `web-app-chess`, `web-app-dashboard`, `web-app-fediwall`, `web-app-hugo`, `web-app-littlejs`, `web-app-mig`, `web-app-mini-qr`, `web-app-navigator`, `web-app-roulette-wheel`, `web-app-sphinx`, `web-opt-rdr-domains`, `web-opt-rdr-www`, `web-svc-asset`, `web-svc-cdn`, `web-svc-coturn`, `web-svc-css`, `web-svc-file`, `web-svc-html`, `web-svc-legal`, `web-svc-logout`, `web-svc-mirror`, `web-svc-simpleicons`.
|
||||
**`no_remote_surface` (24):** `svc-ai-ollama`, `svc-ai-robot`, `web-app-chess`, `web-app-dashboard`, `web-app-docs`, `web-app-fediwall`, `web-app-hugo`, `web-app-littlejs`, `web-app-mig`, `web-app-mini-qr`, `web-app-navigator`, `web-app-roulette-wheel`, `web-opt-rdr-domains`, `web-opt-rdr-www`, `web-svc-asset`, `web-svc-cdn`, `web-svc-coturn`, `web-svc-css`, `web-svc-file`, `web-svc-html`, `web-svc-legal`, `web-svc-logout`, `web-svc-mirror`, `web-svc-simpleicons`.
|
||||
These roles have no independently useful authenticated remote action contract at the pinned implementation.
|
||||
Static published content MAY later be consumed through one owning application's bounded `resource_readonly` adapter, but a new MCP sidecar that merely reimplements a static site's behavior does not count as integration.
|
||||
Model tool calling in Ollama does not by itself make Ollama an MCP client or server.
|
||||
|
||||
@@ -55,7 +55,7 @@ Excluded, with the reason recorded so the question is not reopened:
|
||||
| `web-app-openproject` | no AI surface found in the product |
|
||||
| `web-app-espocrm` | measured at tag 10.0.4: zero tree paths match `AI`/`Intelligence`/`OpenAI`/`LLM`/`GPT`/`prompt` while 116 match `Mail` on the same payload, so the search was live. The AI features live only in the closed-source Intelligence extension, which needs a purchased licence and is published in no `espocrm` repository. A version bump does not help: upstream documents the extension as requiring 10.0.3 or greater, so newer cores host it rather than absorb it |
|
||||
| `web-app-peertube` | measured at tag v8.2.4 across three independent surfaces: `config/production.yaml.example` `video_transcription:` carries only `enabled`/`engine`/`engine_path`/`model`/`model_path`/`timeout`/`remote_runners.enabled`; `support/docker/production/config/custom-environment-variables.yaml` exposes exactly the seven matching `PEERTUBE_VIDEO_TRANSCRIPTION_*` variables, none of them a URL or a key; `server/core/initializers/config.ts` agrees. Transcription is local Whisper, not an OpenAI-compatible endpoint |
|
||||
| infrastructure and static roles | `checkmk`, `prometheus`, `matomo`, `seaweedfs`, `minio`, `mailu`, `keycloak`, `lam`, `fusiondirectory`, `phpldapadmin`, `phpmyadmin`, `pgadmin`, `pihole`, `semaphore`, `jenkins`, `gitea`, `hugo`, `sphinx`, `yourls`, `mini-qr`, `littlejs`, `chess`, `roulette-wheel`, `navigator`, `mig`, `dashboard`, `fediwall`, `postmarks` carry no AI surface |
|
||||
| infrastructure and static roles | `checkmk`, `prometheus`, `matomo`, `seaweedfs`, `minio`, `mailu`, `keycloak`, `lam`, `fusiondirectory`, `phpldapadmin`, `phpmyadmin`, `pgadmin`, `pihole`, `semaphore`, `jenkins`, `gitea`, `hugo`, `docs`, `yourls`, `mini-qr`, `littlejs`, `chess`, `roulette-wheel`, `navigator`, `mig`, `dashboard`, `fediwall`, `postmarks` carry no AI surface |
|
||||
| fediverse and media roles | `mastodon`, `pixelfed`, `friendica`, `socialhome`, `bookwyrm`, `mobilizon`, `funkwhale`, `bluesky`, `bridgy-fed`, `jellyfin` ship no configurable AI surface |
|
||||
| remaining application roles | `akaunting`, `decidim`, `fider`, `listmonk`, `pretix`, `snipe-it`, `taiga`, `penpot`, `joomla`, `suitecrm`, `kix`, `opencloud`, `opentalk`, `jitsi`, `bigbluebutton`, `erpnext` show no first-party AI surface with a configurable endpoint; revisit when upstream adds one |
|
||||
|
||||
|
||||
@@ -39,6 +39,10 @@ dev = [
|
||||
"git-maintainer-tools",
|
||||
"pytest>=9.1",
|
||||
"pytest-xdist",
|
||||
"myst-parser",
|
||||
"pathspec",
|
||||
"sphinx",
|
||||
"sphinxawesome-theme",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
|
||||
@@ -50,7 +50,7 @@ flowchart LR
|
||||
dpt_web_app_confluence["web-app-confluence 🐳🐝"]
|
||||
dpt_web_app_decidim["web-app-decidim 🐳🐝"]
|
||||
dpt_web_app_discourse["web-app-discourse 🐳🐝"]
|
||||
dpt_web_app_erpnext["web-app-erpnext 🐳🐝"]
|
||||
dpt_web_app_docs["web-app-docs 🐳🐝"]
|
||||
dpt_more["..."]
|
||||
end
|
||||
dep_svc_net_tor -. "0..1" .-> svc_tor
|
||||
@@ -74,7 +74,7 @@ flowchart LR
|
||||
svc_sso -. "0..1" .-> dpt_web_app_confluence
|
||||
svc_sso -. "0..1" .-> dpt_web_app_decidim
|
||||
svc_sso -. "0..1" .-> dpt_web_app_discourse
|
||||
svc_sso -. "0..1" .-> dpt_web_app_erpnext
|
||||
svc_sso -. "0..1" .-> dpt_web_app_docs
|
||||
```
|
||||
|
||||
Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`.
|
||||
|
||||
@@ -2,14 +2,14 @@
|
||||
footer:
|
||||
children:
|
||||
|
||||
{% if "web-app-sphinx" in group_names %}
|
||||
{% if "web-app-docs" in group_names %}
|
||||
|
||||
- name: Documentation
|
||||
description: Access our comprehensive documentation and support resources to help you get the most out of the software.
|
||||
icon:
|
||||
class: fas fa-book
|
||||
url: {{ lookup('tls', 'web-app-sphinx', 'url.base') }}
|
||||
iframe: {{ lookup('config','web-app-sphinx', 'services.dashboard.enabled') }}
|
||||
url: {{ lookup('tls', 'web-app-docs', 'url.base') }}
|
||||
iframe: {{ lookup('config','web-app-docs', 'services.dashboard.enabled') }}
|
||||
|
||||
{% endif %}
|
||||
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
# Documentation
|
||||
|
||||
## Description
|
||||
|
||||
This role publishes the Infinito.Nexus documentation for the latest commit and for every release tag. It builds the repository's Markdown, reStructuredText and role READMEs into an HTML site with [Sphinx](https://www.sphinx-doc.org/) and the [Awesome Sphinx Theme](https://sphinxawesome.xyz/). There is no database.
|
||||
|
||||
## Overview
|
||||
|
||||
One `python:<version>-slim` container runs the service in [files/python/infinito_docs/](./files/python/infinito_docs/):
|
||||
|
||||
1. It keeps a git mirror of `services.docs.source_repository` in the `docs_sites` volume and fetches it every `services.docs.fetch_interval` seconds, or later when a build is still running.
|
||||
2. `latest` is the last commit of the default branch. It is built at start and rebuilt whenever a fetch finds a new commit; the previous build stays online until the new one replaces it.
|
||||
3. A release tag (`vX.Y.Z`) is built the first time someone opens it. The browser shows a progress bar and reloads once the build is done. Tags never change, so each tag is built only once.
|
||||
4. Builds run one at a time, each with `services.docs.build_jobs` parallel Sphinx processes.
|
||||
5. Every replica serves from the shared `docs_sites` volume. The replica holding `builder.lock` in that volume fetches and builds; the queue (`queue/`) and the build states (`states/`) are files beside the sites, so a new lock holder resumes the queue when the builder dies. Cross-node locking relies on NFSv4, the default of `svc-storage-nfs-client`.
|
||||
|
||||
| Path | Content |
|
||||
|---|---|
|
||||
| `/` | Redirect to `/latest/` |
|
||||
| `/<version>/...` | The built site, or its build progress |
|
||||
| `/versions/` | Every version with its build status and progress |
|
||||
| `/api/versions` | The same as JSON |
|
||||
|
||||
Every page carries a version switcher in its sidebar. Switching keeps the current page and falls back to the start page of a version that lacks it.
|
||||
|
||||
## Cosmos
|
||||
|
||||
The diagram places Documentation in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks).
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph deps [Dependencies]
|
||||
dep_svc_net_tor["svc-net-tor 🐳🐝"]
|
||||
dep_web_app_dashboard["web-app-dashboard 🐳🐝"]
|
||||
dep_web_app_keycloak["web-app-keycloak 🐳🐝"]
|
||||
dep_web_app_mailu["web-app-mailu 🐳🐝"]
|
||||
dep_web_app_matomo["web-app-matomo 🐳🐝"]
|
||||
dep_web_app_prometheus["web-app-prometheus 🐳🐝"]
|
||||
dep_web_svc_css["web-svc-css 💻"]
|
||||
end
|
||||
subgraph role [web-app-docs 🐳🐝]
|
||||
svc_logout["logout ❌"]
|
||||
svc_sso["sso ❌"]
|
||||
svc_dashboard["dashboard"]
|
||||
svc_matomo["matomo"]
|
||||
svc_docs["docs"]
|
||||
svc_css["css"]
|
||||
svc_email["email ❌"]
|
||||
svc_prometheus["prometheus"]
|
||||
svc_tor["tor"]
|
||||
end
|
||||
dep_svc_net_tor -. "0..1" .-> svc_tor
|
||||
dep_web_app_dashboard -. "0..1" .-> svc_dashboard
|
||||
dep_web_app_keycloak -- "0..0" --> svc_sso
|
||||
dep_web_app_mailu -- "0..0" --> svc_email
|
||||
dep_web_app_matomo -. "0..1" .-> svc_matomo
|
||||
dep_web_app_prometheus -. "0..1" .-> svc_prometheus
|
||||
dep_web_svc_css -. "0..1" .-> svc_css
|
||||
linkStyle 2,3 stroke:red;
|
||||
```
|
||||
|
||||
Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`.
|
||||
|
||||
## Features
|
||||
|
||||
- **Every release:** The latest commit is the default, each release tag is selectable and built on demand.
|
||||
- **Generated reference:** API pages via `sphinx-apidoc`, one page per role from its `meta/main.yml` and README, and an index of every YAML file.
|
||||
- **Folder navigation:** The sidebar mirrors the repository tree and highlights the current page and heading.
|
||||
|
||||
## Quick Setup
|
||||
|
||||
### Development
|
||||
|
||||
Clone, set up the workstation, and deploy Documentation onto the local stack:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/infinito-nexus/core.git
|
||||
cd core
|
||||
make onboard
|
||||
make compose-deploy mode=reinstall apps=web-app-docs full_cycle=false
|
||||
```
|
||||
|
||||
### Production
|
||||
|
||||
Run the published image to provision the inventory and deploy Documentation to a managed server (the mounted volume persists the inventory):
|
||||
|
||||
```bash
|
||||
APP=web-app-docs
|
||||
HOST=<your-server>
|
||||
DOMAIN=<your-domain>
|
||||
TLS_MODE=self_signed
|
||||
SSH_PUBLIC_KEY="<your-ssh-public-key>"
|
||||
|
||||
docker run --rm -it \
|
||||
-v "$PWD/inventories:/etc/infinito.nexus/inventories" \
|
||||
-e APP="$APP" -e HOST="$HOST" -e DOMAIN="$DOMAIN" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \
|
||||
ghcr.io/infinito-nexus/core/debian bash -c '
|
||||
INVENTORY=/etc/infinito.nexus/inventories/production
|
||||
infinito administration inventory provision "$INVENTORY" \
|
||||
--inventory-file "$INVENTORY/devices.yml" \
|
||||
--host "$HOST" \
|
||||
--include "$APP" \
|
||||
--vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" &&
|
||||
infinito administration deploy dedicated "$INVENTORY/devices.yml" \
|
||||
--password-file "$INVENTORY/.password" \
|
||||
--diff -vv'
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Override these keys of `services.docs` in the inventory:
|
||||
|
||||
| Key | Purpose |
|
||||
|---|---|
|
||||
| `source_repository` | Repository to document, e.g. a fork |
|
||||
| `fetch_interval` | Seconds between two fetches of new commits and tags |
|
||||
| `build_jobs` | Parallel Sphinx processes per build; raise `cpus` and `mem_limit` with it |
|
||||
|
||||
```yaml
|
||||
applications:
|
||||
web-app-docs:
|
||||
services:
|
||||
docs:
|
||||
source_repository: https://git.example.com/your-org/core.git
|
||||
build_jobs: 4
|
||||
cpus: "4"
|
||||
mem_limit: 8g
|
||||
```
|
||||
|
||||
Delete `sites/<version>` inside the `docs_sites` volume to rebuild a version on its next visit.
|
||||
|
||||
## Further Resources
|
||||
|
||||
- [Sphinx Official Website](https://www.sphinx-doc.org/)
|
||||
- [Awesome Sphinx Theme](https://sphinxawesome.xyz/)
|
||||
|
||||
## Persona contract opt-outs
|
||||
|
||||
The role publishes a public documentation site. [`meta/services.yml`](./meta/services.yml) pins `sso.enabled` and `sso.shared` to `false`, and the role ships no `meta/users.yml` and no account provisioning; [`templates/env.j2`](./templates/env.j2) only configures the build service. [`tasks/main.yml`](./tasks/main.yml) only wires the container and proxy stack and stages the Sphinx tooling into the build context.
|
||||
There is no account and no auth chain for the `biber` or `administrator` persona, so [`templates/playwright.env.j2`](./templates/playwright.env.j2) declares `PERSONA_BIBER_BLOCKED=true` and `PERSONA_ADMINISTRATOR_BLOCKED=true`. The `guest` persona and the baseline reachability and CSP assertions run unconditionally.
|
||||
|
||||
## Credits
|
||||
|
||||
Implemented by **Marko Pjevac, Kevin Veen-Birkenbach**.
|
||||
Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world).
|
||||
Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license).
|
||||
@@ -0,0 +1,29 @@
|
||||
ARG DOCS_IMAGE
|
||||
ARG DOCS_VERSION
|
||||
|
||||
FROM ${DOCS_IMAGE}:${DOCS_VERSION}
|
||||
|
||||
ARG DOCS_SOURCE_REPO
|
||||
|
||||
RUN rm -rf /var/lib/apt/lists/* \
|
||||
&& apt-get -o Acquire::Retries=3 update \
|
||||
&& apt-get -o Acquire::Retries=3 install -y --no-install-recommends git pandoc \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN pip install --no-cache-dir \
|
||||
myst-parser \
|
||||
pathspec \
|
||||
pyyaml \
|
||||
sphinx \
|
||||
sphinxawesome-theme \
|
||||
"infinito-nexus[deploy] @ git+${DOCS_SOURCE_REPO}"
|
||||
|
||||
COPY ./sphinx/python/ /app/
|
||||
COPY ./sphinx/javascript/ /app/infinito_docs/assets/js/
|
||||
|
||||
ENV PYTHONPATH=/app
|
||||
WORKDIR /app
|
||||
|
||||
EXPOSE 8000
|
||||
|
||||
CMD ["python", "-m", "infinito_docs.server"]
|
||||
@@ -0,0 +1,60 @@
|
||||
document.addEventListener("DOMContentLoaded", function () {
|
||||
function markAsCurrent(link) {
|
||||
let li = link.closest("li");
|
||||
while (li) {
|
||||
li.classList.add("current");
|
||||
if (li.__x && li.__x.$data) {
|
||||
li.__x.$data.expanded = true;
|
||||
}
|
||||
li = li.parentElement.closest("li");
|
||||
}
|
||||
}
|
||||
|
||||
function openCurrentSubmenus() {
|
||||
document.querySelectorAll(".current-index li.current").forEach(function (li) {
|
||||
li.querySelectorAll(":scope > [x-show]").forEach(function (elem) {
|
||||
if (elem.style.display === "none" || elem.style.display === "") {
|
||||
elem.style.display = "block";
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function processNav() {
|
||||
const currentHash = window.location.hash.trim();
|
||||
if (!currentHash) {
|
||||
return;
|
||||
}
|
||||
|
||||
document.querySelectorAll(".current-index a.reference.internal").forEach(function (link) {
|
||||
const href = (link.getAttribute("href") || "").trim();
|
||||
if (href.startsWith("#")) {
|
||||
if (href === currentHash) {
|
||||
document
|
||||
.querySelectorAll(".current-index a.reference.internal.current")
|
||||
.forEach(function (active) {
|
||||
active.classList.remove("current");
|
||||
});
|
||||
link.classList.add("current");
|
||||
markAsCurrent(link);
|
||||
}
|
||||
} else if (href.includes("#") && `#${href.split("#")[1].trim()}` === currentHash) {
|
||||
markAsCurrent(link);
|
||||
}
|
||||
});
|
||||
|
||||
openCurrentSubmenus();
|
||||
}
|
||||
|
||||
function initCurrentNav() {
|
||||
if (window.Alpine && typeof window.Alpine.nextTick === "function") {
|
||||
window.Alpine.nextTick(processNav);
|
||||
} else {
|
||||
processNav();
|
||||
}
|
||||
}
|
||||
|
||||
window.addEventListener("load", initCurrentNav);
|
||||
window.addEventListener("hashchange", initCurrentNav);
|
||||
window.initCurrentNav = initCurrentNav;
|
||||
});
|
||||
@@ -0,0 +1,93 @@
|
||||
(function () {
|
||||
const LABELS = {
|
||||
ready: "ready",
|
||||
missing: "not built yet",
|
||||
queued: "queued",
|
||||
building: "building",
|
||||
failed: "failed",
|
||||
};
|
||||
const POLL_MS = 2000;
|
||||
|
||||
function versionUrl(name, page) {
|
||||
return page ? `/${name}/${page}.html` : `/${name}/`;
|
||||
}
|
||||
|
||||
function describe(version) {
|
||||
const label = LABELS[version.state];
|
||||
return version.phase && version.state !== "ready" ? `${label}: ${version.phase}` : label;
|
||||
}
|
||||
|
||||
function optionLabel(version) {
|
||||
return version.built ? version.name : `${version.name} (${LABELS[version.state]})`;
|
||||
}
|
||||
|
||||
function fillSwitcher(select, versions) {
|
||||
const { current, page } = select.dataset;
|
||||
select.replaceChildren(
|
||||
...versions.map((version) => {
|
||||
const option = document.createElement("option");
|
||||
option.value = versionUrl(version.name, page);
|
||||
option.textContent = optionLabel(version);
|
||||
option.selected = version.name === current;
|
||||
return option;
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
function fillOverview(tbody, versions) {
|
||||
tbody.replaceChildren(
|
||||
...versions.map((version) => {
|
||||
const link = document.createElement("a");
|
||||
link.href = versionUrl(version.name);
|
||||
link.textContent = version.name;
|
||||
const bar = document.createElement("progress");
|
||||
bar.max = 100;
|
||||
bar.value = version.progress;
|
||||
const row = document.createElement("tr");
|
||||
[link, document.createTextNode(describe(version)), bar].forEach((child) => {
|
||||
const cell = document.createElement("td");
|
||||
cell.append(child);
|
||||
row.append(cell);
|
||||
});
|
||||
return row;
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
function followBuild(section, versions) {
|
||||
const version = versions.find((candidate) => candidate.name === section.dataset.build);
|
||||
if (!version) {
|
||||
return;
|
||||
}
|
||||
if (version.built) {
|
||||
window.location.reload();
|
||||
return;
|
||||
}
|
||||
section.querySelector("progress").value = version.progress;
|
||||
section.querySelector("[data-phase]").textContent = describe(version);
|
||||
section.querySelector("pre").textContent = version.log.join("\n");
|
||||
}
|
||||
|
||||
async function refresh() {
|
||||
const response = await fetch("/api/versions", { cache: "no-store" });
|
||||
const versions = await response.json();
|
||||
document.querySelectorAll("select.docs-version-switcher").forEach((select) => fillSwitcher(select, versions));
|
||||
document.querySelectorAll("tbody[data-versions]").forEach((tbody) => fillOverview(tbody, versions));
|
||||
document.querySelectorAll("section[data-build]").forEach((section) => followBuild(section, versions));
|
||||
}
|
||||
|
||||
function start() {
|
||||
document.querySelectorAll("select.docs-version-switcher").forEach((select) => {
|
||||
select.addEventListener("change", () => {
|
||||
window.location.href = select.value;
|
||||
});
|
||||
});
|
||||
refresh();
|
||||
if (document.querySelector("tbody[data-versions], section[data-build]")) {
|
||||
window.setInterval(refresh, POLL_MS);
|
||||
}
|
||||
}
|
||||
|
||||
window.docsVersions = { versionUrl, describe, optionLabel, fillSwitcher, fillOverview, followBuild, refresh };
|
||||
document.addEventListener("DOMContentLoaded", start);
|
||||
})();
|
||||
File renamed without changes.
@@ -0,0 +1,20 @@
|
||||
const { expect } = require("@playwright/test");
|
||||
const { resolveTimeout } = require("./timeouts");
|
||||
const { decodeDotenvQuotedValue, normalizeBaseUrl } = require("./personas");
|
||||
|
||||
const appBaseUrl = normalizeBaseUrl(process.env.APP_BASE_URL || "");
|
||||
const canonicalDomain = decodeDotenvQuotedValue(process.env.CANONICAL_DOMAIN || "");
|
||||
const RELEASE_TAG = /^v\d+\.\d+\.\d+$/;
|
||||
|
||||
async function fetchVersions(request) {
|
||||
const response = await request.get(`${appBaseUrl}/api/versions`, { timeout: resolveTimeout(30_000) });
|
||||
expect(response.status(), "Expected the versions api to answer").toBe(200);
|
||||
return response.json();
|
||||
}
|
||||
|
||||
async function versionState(request, name) {
|
||||
const version = (await fetchVersions(request)).find((candidate) => candidate.name === name);
|
||||
return version ? version.state : "unknown";
|
||||
}
|
||||
|
||||
module.exports = { appBaseUrl, canonicalDomain, RELEASE_TAG, fetchVersions, versionState };
|
||||
@@ -0,0 +1,42 @@
|
||||
const { test, expect } = require("@playwright/test");
|
||||
const { resolveTimeout } = require("./timeouts");
|
||||
const { runAdminFlow, runBiberFlow, runGuestFlow } = require("./personas");
|
||||
const shared = require("./_shared");
|
||||
|
||||
test.use({ ignoreHTTPSErrors: true });
|
||||
|
||||
test.beforeEach(async ({ page }) => {
|
||||
expect(shared.appBaseUrl, "APP_BASE_URL must be set").toBeTruthy();
|
||||
expect(shared.canonicalDomain, "CANONICAL_DOMAIN must be set").toBeTruthy();
|
||||
await page.context().clearCookies();
|
||||
});
|
||||
|
||||
require("./test-front-page").register(shared);
|
||||
require("./test-versions").register(shared);
|
||||
require("./test-on-demand-build").register(shared);
|
||||
|
||||
test("guest: public-landing → auth chain → never authenticated", async ({ page }) => {
|
||||
await runGuestFlow(page);
|
||||
});
|
||||
|
||||
test("biber: app → universal logout", async ({ page }) => {
|
||||
await runBiberFlow(page);
|
||||
});
|
||||
|
||||
test("administrator: app → universal logout", async ({ page }) => {
|
||||
await runAdminFlow(page, {
|
||||
adminInteraction: async (interactivePage) => {
|
||||
const link = interactivePage
|
||||
.getByRole("link", { name: /^(admin|build|configuration|projects)$/i })
|
||||
.first();
|
||||
if (await link.isVisible().catch(() => false)) {
|
||||
await link.click({ timeout: resolveTimeout(30_000) }).catch(() => {});
|
||||
await interactivePage.waitForLoadState("domcontentloaded", { timeout: resolveTimeout(30_000) }).catch(() => {});
|
||||
await expect(interactivePage.locator("body")).toContainText(
|
||||
/admin|build|configuration|projects|index/i,
|
||||
{ timeout: resolveTimeout(30_000) },
|
||||
);
|
||||
}
|
||||
},
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,28 @@
|
||||
const { test, expect } = require("@playwright/test");
|
||||
const { resolveTimeout } = require("./timeouts");
|
||||
const { expectHstsWhenTls, gotoOnion } = require("./personas");
|
||||
|
||||
exports.register = function (shared) {
|
||||
test("docs front page opens the latest commit under canonical domain with TLS", async ({ page }) => {
|
||||
const response = await gotoOnion(page, `${shared.appBaseUrl}/`);
|
||||
expect(response, "Expected docs response").toBeTruthy();
|
||||
expect(response.status(), "Expected docs front page status < 400").toBeLessThan(400);
|
||||
expect(
|
||||
response.url().includes(shared.canonicalDomain),
|
||||
`Expected canonical domain "${shared.canonicalDomain}" to back the docs URL`,
|
||||
).toBe(true);
|
||||
expect(new URL(response.url()).pathname, "Expected the front page to open the latest commit").toBe("/latest/");
|
||||
expectHstsWhenTls(response.headers(), shared.appBaseUrl, "docs");
|
||||
});
|
||||
|
||||
test("latest commit is either the built Sphinx site or its running build", async ({ request }) => {
|
||||
const response = await request.get(`${shared.appBaseUrl}/latest/`, { timeout: resolveTimeout(30_000) });
|
||||
const body = await response.text();
|
||||
expect([200, 202], "Expected the built site or the build page").toContain(response.status());
|
||||
const built = response.status() === 200;
|
||||
expect(body, "Expected the Sphinx site or the build page of the latest commit").toContain(
|
||||
built ? 'data-current="latest"' : 'data-build="latest"',
|
||||
);
|
||||
expect(body, "Expected the project title or a progress bar").toContain(built ? "Infinito.Nexus" : "<progress");
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,35 @@
|
||||
const { test, expect } = require("@playwright/test");
|
||||
const { resolveTimeout } = require("./timeouts");
|
||||
const { gotoOnion } = require("./personas");
|
||||
|
||||
exports.register = function (shared) {
|
||||
test("selecting an unbuilt version in the overview starts its build", async ({ page, request }) => {
|
||||
const target = (await shared.fetchVersions(request)).find(
|
||||
(version) => shared.RELEASE_TAG.test(version.name) && version.state === "missing",
|
||||
);
|
||||
expect(target, "Expected a release tag that is neither built nor requested yet").toBeTruthy();
|
||||
|
||||
await gotoOnion(page, `${shared.appBaseUrl}/versions/`);
|
||||
const link = page.getByRole("link", { name: target.name, exact: true });
|
||||
await expect(link, `Expected ${target.name} in the overview`).toBeVisible({ timeout: resolveTimeout(30_000) });
|
||||
|
||||
const [response] = await Promise.all([
|
||||
page.waitForResponse((candidate) => new URL(candidate.url()).pathname === `/${target.name}/`, {
|
||||
timeout: resolveTimeout(30_000),
|
||||
}),
|
||||
link.click(),
|
||||
]);
|
||||
expect(response.status(), "Expected the build page for an unbuilt version").toBe(202);
|
||||
|
||||
const build = page.locator(`section[data-build="${target.name}"]`);
|
||||
await expect(build, "Expected the build page of the selected version").toBeVisible({ timeout: resolveTimeout(30_000) });
|
||||
await expect(build.locator("progress"), "Expected a progress bar on the build page").toHaveCount(1);
|
||||
|
||||
await expect
|
||||
.poll(() => shared.versionState(request, target.name), {
|
||||
message: `Expected selecting ${target.name} to queue or start its build`,
|
||||
timeout: resolveTimeout(60_000),
|
||||
})
|
||||
.toMatch(/^(queued|building|ready)$/);
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,27 @@
|
||||
const { test, expect } = require("@playwright/test");
|
||||
const { resolveTimeout } = require("./timeouts");
|
||||
const { gotoOnion } = require("./personas");
|
||||
|
||||
exports.register = function (shared) {
|
||||
test("versions api offers the latest commit first and every release tag after it", async ({ request }) => {
|
||||
const names = (await shared.fetchVersions(request)).map((version) => version.name);
|
||||
expect(names[0], "Expected the latest commit to be the default version").toBe("latest");
|
||||
expect(names.length, "Expected at least one release tag besides the latest commit").toBeGreaterThan(1);
|
||||
expect(
|
||||
names.slice(1).filter((name) => !shared.RELEASE_TAG.test(name)),
|
||||
"Expected only release tags after the latest commit",
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
test("versions overview renders every version with a link and a progress bar", async ({ page, request }) => {
|
||||
const names = (await shared.fetchVersions(request)).map((version) => version.name);
|
||||
await gotoOnion(page, `${shared.appBaseUrl}/versions/`);
|
||||
const rows = page.locator("tbody[data-versions] tr");
|
||||
await expect(rows, "Expected one overview row per version").toHaveCount(names.length, {
|
||||
timeout: resolveTimeout(30_000),
|
||||
});
|
||||
await expect(rows.first().locator("a")).toHaveText("latest");
|
||||
await expect(rows.first().locator("a")).toHaveAttribute("href", "/latest/");
|
||||
await expect(page.locator("tbody[data-versions] progress")).toHaveCount(names.length);
|
||||
});
|
||||
};
|
||||
Whitespace-only changes.
@@ -0,0 +1,86 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from docutils import nodes
|
||||
from pygments.lexers.templates import DjangoLexer
|
||||
from sphinx.domains.python import PythonDomain
|
||||
from sphinx.highlighting import lexers
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.DEBUG if {"-v", "--verbose"} & set(sys.argv) else logging.INFO
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
CONF_DIR = Path(__file__).resolve().parent
|
||||
|
||||
templates_path = [str(CONF_DIR / "templates")]
|
||||
html_static_path = [str(CONF_DIR / "assets")]
|
||||
|
||||
project = "Infinito.Nexus - Cyber Master Infrastructure Solution"
|
||||
copyright = "2025, Kevin Veen-Birkenbach" # noqa: A001
|
||||
author = "Kevin Veen-Birkenbach"
|
||||
|
||||
lexers["jinja"] = DjangoLexer()
|
||||
lexers["j2"] = DjangoLexer()
|
||||
|
||||
exclude_patterns = ["docs/build", "venv", "venv/**"]
|
||||
|
||||
html_theme = "sphinxawesome_theme"
|
||||
html_sidebars = {"**": ["logo.html", "versions.html", "structure.html"]}
|
||||
html_context = {"docs_version": os.environ.get("DOCS_VERSION", "")}
|
||||
html_favicon = "assets/img/favicon.ico"
|
||||
html_theme_options = {
|
||||
"show_prev_next": False,
|
||||
"logo_light": "assets/img/logo.png",
|
||||
"logo_dark": "assets/img/logo.png",
|
||||
}
|
||||
|
||||
source_suffix = {
|
||||
".md": "markdown",
|
||||
".rst": "restructuredtext",
|
||||
".yml": "restructuredtext",
|
||||
".yaml": "restructuredtext",
|
||||
}
|
||||
|
||||
extensions = [
|
||||
"myst_parser",
|
||||
"infinito_docs.extensions.local_file_headings",
|
||||
"infinito_docs.extensions.local_subfolders",
|
||||
"infinito_docs.extensions.roles_overview",
|
||||
"infinito_docs.extensions.markdown_include",
|
||||
"infinito_docs.extensions.parallel_postprocess",
|
||||
"sphinx.ext.autodoc",
|
||||
"sphinx.ext.napoleon",
|
||||
]
|
||||
|
||||
autosummary_generate = True
|
||||
|
||||
myst_enable_extensions = ["colon_fence"]
|
||||
|
||||
|
||||
def replace_assets_in_doctree(app, doctree, docname):
|
||||
for node in doctree.findall(nodes.image):
|
||||
if "assets/" in node["uri"]:
|
||||
node["uri"] = node["uri"].replace("assets/", "_static/")
|
||||
logger.info("Replaced image URI in %s: %s", docname, node["uri"])
|
||||
|
||||
for node in doctree.findall(nodes.raw):
|
||||
if node.get("format") == "html" and "assets/" in node.astext():
|
||||
rewritten = node.astext().replace("assets/", "_static/")
|
||||
node.children = [nodes.raw("", rewritten, format="html")]
|
||||
logger.info("Replaced raw HTML assets in %s.", docname)
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.connect("doctree-resolved", replace_assets_in_doctree)
|
||||
|
||||
python_domain = app.registry.domains.get(PythonDomain.name)
|
||||
if python_domain is not None:
|
||||
directive = python_domain.directives.get("currentmodule")
|
||||
if directive is not None:
|
||||
directive.optional_arguments = 10
|
||||
return {"version": "1.0", "parallel_read_safe": True}
|
||||
Whitespace-only changes.
@@ -0,0 +1,54 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
|
||||
from .nav_utils import (
|
||||
DEFAULT_MAX_NAV_DEPTH,
|
||||
MAX_HEADING_LEVEL,
|
||||
extract_headings_from_file,
|
||||
group_headings,
|
||||
sort_tree,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def add_local_file_headings(app, pagename, templatename, context, doctree):
|
||||
directory = pagename.rpartition("/")[0]
|
||||
abs_dir = Path(app.srcdir) / directory
|
||||
if not abs_dir.is_dir():
|
||||
logger.warning("Directory %s not found for page %s.", abs_dir, pagename)
|
||||
context["local_md_headings"] = []
|
||||
return
|
||||
|
||||
files = [path.name for path in abs_dir.iterdir() if path.suffix in {".md", ".rst"}]
|
||||
if "index.rst" in {name.lower() for name in files}:
|
||||
files = [name for name in files if name.lower() != "readme.md"]
|
||||
|
||||
items = []
|
||||
for name in files:
|
||||
stem = Path(name).stem
|
||||
items.extend(
|
||||
{
|
||||
"level": heading["level"],
|
||||
"text": heading["text"],
|
||||
"link": f"{directory}/{stem}" if directory else stem,
|
||||
"anchor": heading["anchor"],
|
||||
"priority": 0 if stem.lower() == "index" else 1,
|
||||
"filename": stem,
|
||||
}
|
||||
for heading in extract_headings_from_file(
|
||||
abs_dir / name, max_level=MAX_HEADING_LEVEL
|
||||
)
|
||||
)
|
||||
|
||||
tree = group_headings(items)
|
||||
sort_tree(tree)
|
||||
context["local_md_headings"] = tree
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.add_config_value("local_nav_max_depth", DEFAULT_MAX_NAV_DEPTH, "env")
|
||||
app.connect("html-page-context", add_local_file_headings)
|
||||
return {"version": "0.1", "parallel_read_safe": True}
|
||||
@@ -0,0 +1,105 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
from functools import cache
|
||||
from pathlib import Path
|
||||
|
||||
from .nav_utils import MAX_HEADING_LEVEL, extract_headings_from_file
|
||||
|
||||
CANDIDATES = ("index.rst", "readme.md", "main.rst")
|
||||
|
||||
|
||||
def _join(base_url, name):
|
||||
return f"{base_url}/{name}" if base_url else name
|
||||
|
||||
|
||||
def _title(path, fallback):
|
||||
headings = extract_headings_from_file(path, max_level=MAX_HEADING_LEVEL)
|
||||
return headings[0]["text"] if headings else fallback
|
||||
|
||||
|
||||
def collect_folder_tree(dir_path, base_url):
|
||||
"""Return the navigation tree below ``dir_path``.
|
||||
|
||||
Args:
|
||||
dir_path: directory to walk.
|
||||
base_url: link prefix of ``dir_path`` inside the site.
|
||||
|
||||
Returns:
|
||||
``{"text", "link", "children", "filename"}``, or ``None`` for a hidden
|
||||
directory or one without an ``index.rst``, ``readme.md`` or
|
||||
``main.rst`` to title it.
|
||||
"""
|
||||
directory = Path(dir_path)
|
||||
if directory.name.startswith("."):
|
||||
return None
|
||||
|
||||
files = [
|
||||
path.name
|
||||
for path in directory.iterdir()
|
||||
if path.is_file() and path.suffix in {".md", ".rst"}
|
||||
]
|
||||
rep_file = next(
|
||||
(
|
||||
name
|
||||
for candidate in CANDIDATES
|
||||
for name in files
|
||||
if name.lower() == candidate
|
||||
),
|
||||
None,
|
||||
)
|
||||
if rep_file is None:
|
||||
return None
|
||||
|
||||
children = [
|
||||
{
|
||||
"level": 1,
|
||||
"text": _title(directory / name, name),
|
||||
"link": _join(base_url, Path(name).stem),
|
||||
"anchor": "",
|
||||
"priority": 1,
|
||||
"filename": name,
|
||||
}
|
||||
for name in sorted(files, key=str.lower)
|
||||
if name.lower() not in CANDIDATES
|
||||
]
|
||||
for sub in sorted(directory.iterdir(), key=lambda path: path.name.lower()):
|
||||
if sub.is_dir() and not sub.name.startswith("."):
|
||||
subtree = collect_folder_tree(sub, _join(base_url, sub.name))
|
||||
if subtree:
|
||||
children.append(subtree)
|
||||
|
||||
return {
|
||||
"text": _title(directory / rep_file, directory.name),
|
||||
"link": _join(base_url, Path(rep_file).stem),
|
||||
"children": children,
|
||||
"filename": directory.name,
|
||||
}
|
||||
|
||||
|
||||
def mark_current(node, active):
|
||||
link = node.get("link", "").rstrip("/")
|
||||
active = active.rstrip("/")
|
||||
is_current = bool(link) and (active == link or active.startswith(link + "/"))
|
||||
for child in node.get("children", []):
|
||||
if mark_current(child, active):
|
||||
is_current = True
|
||||
node["current"] = is_current
|
||||
return is_current
|
||||
|
||||
|
||||
@cache
|
||||
def _source_tree(srcdir):
|
||||
return collect_folder_tree(srcdir, "")
|
||||
|
||||
|
||||
def add_local_subfolders(app, pagename, templatename, context, doctree):
|
||||
folder_tree = copy.deepcopy(_source_tree(str(app.srcdir)))
|
||||
if folder_tree:
|
||||
mark_current(folder_tree, pagename)
|
||||
context["local_subfolders"] = [folder_tree] if folder_tree else []
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.connect("html-page-context", add_local_subfolders)
|
||||
return {"version": "0.1", "parallel_read_safe": True}
|
||||
@@ -0,0 +1,69 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from docutils import nodes
|
||||
from docutils.frontend import get_default_settings
|
||||
from docutils.parsers.rst import Directive
|
||||
from docutils.utils import new_document
|
||||
from myst_parser.parsers.sphinx_ import MystParser
|
||||
|
||||
|
||||
def _strip_title(doc):
|
||||
"""Drop the leading title of a parsed Markdown document in place.
|
||||
|
||||
Args:
|
||||
doc: docutils document produced by :class:`MystParser`.
|
||||
"""
|
||||
first = doc.children[0] if doc.children else None
|
||||
if not isinstance(first, nodes.section):
|
||||
return
|
||||
|
||||
if first.children and isinstance(first.children[0], nodes.title):
|
||||
if len(first.children) > 1:
|
||||
first.pop(0)
|
||||
else:
|
||||
first.children[0].clear()
|
||||
|
||||
if not any(
|
||||
isinstance(child, nodes.title) and child.astext().strip()
|
||||
for child in first.children
|
||||
):
|
||||
doc.children = list(first.children) + doc.children[1:]
|
||||
|
||||
|
||||
class MarkdownIncludeDirective(Directive):
|
||||
required_arguments = 1
|
||||
optional_arguments = 0
|
||||
final_argument_whitespace = True
|
||||
has_content = False
|
||||
|
||||
def _error(self, message):
|
||||
return self.state_machine.reporter.error(
|
||||
message,
|
||||
nodes.literal_block(self.block_text, self.block_text),
|
||||
line=self.lineno,
|
||||
)
|
||||
|
||||
def run(self):
|
||||
env = self.state.document.settings.env
|
||||
_, filename = env.relfn2path(self.arguments[0])
|
||||
path = Path(filename)
|
||||
if not path.exists():
|
||||
return [self._error(f"File not found: {filename}")]
|
||||
try:
|
||||
markdown = path.read_text(encoding="utf-8")
|
||||
except (OSError, UnicodeDecodeError) as exc:
|
||||
return [self._error(f"Error reading file {filename}: {exc}")]
|
||||
|
||||
settings = get_default_settings(MystParser)
|
||||
settings.env = env
|
||||
doc = new_document(filename, settings=settings)
|
||||
MystParser().parse(markdown, doc)
|
||||
_strip_title(doc)
|
||||
return doc.children
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.add_directive("markdown-include", MarkdownIncludeDirective)
|
||||
return {"version": "0.1", "parallel_read_safe": True}
|
||||
@@ -0,0 +1,99 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
from itertools import pairwise
|
||||
from pathlib import Path
|
||||
|
||||
DEFAULT_MAX_NAV_DEPTH = 4
|
||||
MAX_HEADING_LEVEL = 0
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_MD_HEADING = re.compile(r"^(#+)(.*?)$")
|
||||
_RST_UNDERLINE = re.compile(r"[-=~^+\"'`]+")
|
||||
|
||||
|
||||
def natural_sort_key(text):
|
||||
return [int(c) if c.isdigit() else c.lower() for c in re.split(r"(\d+)", text)]
|
||||
|
||||
|
||||
def _markdown_headings(lines, max_level):
|
||||
headings = []
|
||||
in_code_block = False
|
||||
for line in lines:
|
||||
if line.strip().startswith("```"):
|
||||
in_code_block = not in_code_block
|
||||
continue
|
||||
match = None if in_code_block else _MD_HEADING.match(line)
|
||||
if match and len(match.group(1)) <= max_level:
|
||||
text = match.group(2).strip()
|
||||
anchor = re.sub(r"[^a-z0-9\-]", "", re.sub(r"\s+", "-", text.lower()))
|
||||
headings.append(
|
||||
{"level": len(match.group(1)), "text": text, "anchor": anchor}
|
||||
)
|
||||
return headings
|
||||
|
||||
|
||||
def _rst_headings(lines):
|
||||
return [
|
||||
{"level": 1, "text": text.strip(), "anchor": ""}
|
||||
for text, underline in pairwise(lines)
|
||||
if len(underline) >= 3 and _RST_UNDERLINE.fullmatch(underline)
|
||||
]
|
||||
|
||||
|
||||
def extract_headings_from_file(filepath, max_level=MAX_HEADING_LEVEL):
|
||||
"""Return the headings of a Markdown or reStructuredText file.
|
||||
|
||||
Args:
|
||||
filepath: file to scan.
|
||||
max_level: deepest Markdown heading level to keep; ``0`` keeps all.
|
||||
|
||||
Returns:
|
||||
``{"level", "text", "anchor"}`` dicts in file order. An ``index.rst``
|
||||
without headings falls back to the ``README.md`` beside it.
|
||||
"""
|
||||
path = Path(filepath)
|
||||
try:
|
||||
lines = path.read_text(encoding="utf-8").splitlines()
|
||||
except (OSError, UnicodeDecodeError) as exc:
|
||||
logger.warning("Error reading %s: %s", path, exc)
|
||||
lines = []
|
||||
|
||||
suffix = path.suffix.lower()
|
||||
if suffix == ".md":
|
||||
headings = _markdown_headings(lines, max_level or 9999)
|
||||
elif suffix == ".rst":
|
||||
headings = _rst_headings(lines)
|
||||
else:
|
||||
headings = []
|
||||
|
||||
readme = path.with_name("README.md")
|
||||
if not headings and path.name.lower() == "index.rst" and readme.is_file():
|
||||
return extract_headings_from_file(readme, max_level)
|
||||
return headings
|
||||
|
||||
|
||||
def group_headings(headings):
|
||||
tree = []
|
||||
stack = []
|
||||
for heading in headings:
|
||||
heading["children"] = []
|
||||
while stack and stack[-1]["level"] >= heading["level"]:
|
||||
stack.pop()
|
||||
(stack[-1]["children"] if stack else tree).append(heading)
|
||||
stack.append(heading)
|
||||
return tree
|
||||
|
||||
|
||||
def sort_tree(tree):
|
||||
tree.sort(
|
||||
key=lambda x: (
|
||||
x.get("priority", 1),
|
||||
natural_sort_key(x.get("filename", x["text"])),
|
||||
)
|
||||
)
|
||||
for item in tree:
|
||||
if item.get("children"):
|
||||
sort_tree(item["children"])
|
||||
@@ -0,0 +1,47 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from multiprocessing import get_context
|
||||
|
||||
from sphinx.util.display import status_iterator
|
||||
from sphinxawesome_theme import postprocess
|
||||
|
||||
_STATE = {}
|
||||
|
||||
|
||||
def _postprocess_one(path):
|
||||
postprocess.modify_html(path, _STATE["app"])
|
||||
return path
|
||||
|
||||
|
||||
def parallel_post_process_html(app, exc):
|
||||
"""Run the theme's per-page HTML rewrite on ``-j`` forked workers.
|
||||
|
||||
Args:
|
||||
app: the Sphinx application; forked workers inherit it.
|
||||
exc: the exception that ended the build, if any.
|
||||
"""
|
||||
if exc is not None or app.builder.name not in {"html", "dirhtml"}:
|
||||
return
|
||||
files = [app.builder.get_outfilename(doc) for doc in app.env.awesome_changed_docs]
|
||||
_STATE["app"] = app
|
||||
with get_context("fork").Pool(max(app.parallel, 1)) as pool:
|
||||
for _ in status_iterator(
|
||||
pool.imap_unordered(_postprocess_one, files, chunksize=8),
|
||||
"postprocess html... ",
|
||||
"brown",
|
||||
len(files),
|
||||
app.verbosity,
|
||||
):
|
||||
pass
|
||||
|
||||
|
||||
def use_parallel_postprocess(app):
|
||||
for listener in list(app.events.listeners["build-finished"]):
|
||||
if listener.handler is postprocess.post_process_html:
|
||||
app.disconnect(listener.id)
|
||||
app.connect("build-finished", parallel_post_process_html)
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.connect("builder-inited", use_parallel_postprocess)
|
||||
return {"version": "0.1", "parallel_read_safe": True}
|
||||
@@ -0,0 +1,47 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from docutils import nodes
|
||||
from docutils.parsers.rst import Directive
|
||||
from sphinx.util import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
OVERVIEW_FILE = Path("generated") / "roles_overview.json"
|
||||
|
||||
|
||||
class RolesOverviewDirective(Directive):
|
||||
has_content = False
|
||||
|
||||
def run(self):
|
||||
overview = Path(self.state.document.settings.env.srcdir) / OVERVIEW_FILE
|
||||
if not overview.is_file():
|
||||
logger.warning("Roles overview not generated: %s", overview)
|
||||
return [
|
||||
self.state.document.reporter.error(
|
||||
"Roles overview not generated.", line=self.lineno
|
||||
)
|
||||
]
|
||||
|
||||
container = nodes.container()
|
||||
categories = json.loads(overview.read_text(encoding="utf-8"))
|
||||
for tag, roles in sorted(categories.items(), key=lambda item: item[0].lower()):
|
||||
category_section = nodes.section(ids=[nodes.make_id(tag)])
|
||||
category_section += nodes.title(text=tag)
|
||||
for role in sorted(roles, key=lambda role: role["name"].lower()):
|
||||
role_section = nodes.section(ids=[nodes.make_id(role["title"])])
|
||||
role_title = nodes.title()
|
||||
role_title += nodes.reference(text=role["title"], refuri=role["link"])
|
||||
role_section += role_title
|
||||
if role["description"]:
|
||||
role_section += nodes.paragraph(text=role["description"])
|
||||
category_section += role_section
|
||||
container += category_section
|
||||
return [container]
|
||||
|
||||
|
||||
def setup(app):
|
||||
app.add_directive("roles-overview", RolesOverviewDirective)
|
||||
return {"version": "0.1", "parallel_read_safe": True}
|
||||
Whitespace-only changes.
@@ -0,0 +1,82 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from utils.cache.yaml import load_yaml
|
||||
from utils.roles.mapping import ROLE_FILE_META_MAIN, ROLE_FILE_README
|
||||
|
||||
|
||||
def convert_md_to_rst(md_content):
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["pandoc", "-f", "markdown", "-t", "rst"],
|
||||
input=md_content.encode("utf-8"),
|
||||
capture_output=True,
|
||||
check=True,
|
||||
)
|
||||
except subprocess.CalledProcessError as exc:
|
||||
print("Error converting Markdown to reStructuredText:", exc)
|
||||
return md_content
|
||||
return result.stdout.decode("utf-8")
|
||||
|
||||
|
||||
def generate_ansible_roles_doc(roles_dir, output_dir):
|
||||
"""Write one ``<role>.rst`` per role that has a ``meta/main.yml``.
|
||||
|
||||
Args:
|
||||
roles_dir: directory holding one sub-directory per role.
|
||||
output_dir: directory the ``.rst`` files are written to.
|
||||
"""
|
||||
output = Path(output_dir)
|
||||
output.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
for role in Path(roles_dir).iterdir():
|
||||
meta_file = role / ROLE_FILE_META_MAIN
|
||||
if not meta_file.exists():
|
||||
continue
|
||||
galaxy_info = load_yaml(str(meta_file)).get("galaxy_info", {})
|
||||
lines = [
|
||||
f"{role.name.capitalize()} Role",
|
||||
"=" * (len(role.name) + 7),
|
||||
"",
|
||||
f"**Description:** {galaxy_info.get('description', 'No description available')}",
|
||||
"",
|
||||
"Variables",
|
||||
"---------",
|
||||
"",
|
||||
*(f"- **{key}**: {value}" for key, value in galaxy_info.items()),
|
||||
]
|
||||
text = "\n".join(lines) + "\n"
|
||||
|
||||
readme = role / ROLE_FILE_README
|
||||
if readme.exists():
|
||||
text += "\nREADME\n------\n\n" + convert_md_to_rst(
|
||||
readme.read_text(encoding="utf-8")
|
||||
)
|
||||
(output / f"{role.name}.rst").write_text(text, encoding="utf-8")
|
||||
|
||||
print(f"Ansible roles documentation has been generated in {output_dir}")
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Generate documentation for Ansible roles."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--roles-dir", required=True, help="Directory containing Ansible roles."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-dir",
|
||||
required=True,
|
||||
help="Directory where documentation will be saved.",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
generate_ansible_roles_doc(args.roles_dir, args.output_dir)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,59 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
from pathlib import Path, PurePosixPath
|
||||
|
||||
|
||||
def generate_ansible_roles_index(roles_dir, output_file, caption):
|
||||
"""Write a toctree listing every ``.rst`` file in ``roles_dir``.
|
||||
|
||||
Args:
|
||||
roles_dir: directory holding the generated role pages.
|
||||
output_file: index file to write; entries are relative to its directory.
|
||||
caption: page title and toctree caption.
|
||||
"""
|
||||
roles = Path(roles_dir).resolve()
|
||||
output = Path(output_file).resolve()
|
||||
if not roles.exists():
|
||||
print(f"Error: Directory {roles} does not exist.")
|
||||
return
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
entries = [
|
||||
f" {PurePosixPath(os.path.relpath(roles / name, start=output.parent)).with_suffix('')}"
|
||||
for name in sorted(
|
||||
path.name for path in roles.iterdir() if path.suffix == ".rst"
|
||||
)
|
||||
]
|
||||
lines = [
|
||||
caption,
|
||||
"===================",
|
||||
"",
|
||||
".. toctree::",
|
||||
" :maxdepth: 1",
|
||||
f" :caption: {caption}",
|
||||
"",
|
||||
*entries,
|
||||
]
|
||||
output.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
print(f"Index generated at {output}")
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(description="Generate an index for documentation.")
|
||||
parser.add_argument(
|
||||
"--roles-dir", required=True, help="Directory containing .rst files."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-file", required=True, help="Path to the output index.rst file."
|
||||
)
|
||||
parser.add_argument("--caption", required=True, help="The index title")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
generate_ansible_roles_index(args.roles_dir, args.output_file, args.caption)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,44 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
README = """\
|
||||
# Auto Generated Technical Documentation: {name}
|
||||
|
||||
This folder contains an auto-generated technical role documentation for Infinito.Nexus.
|
||||
"""
|
||||
|
||||
|
||||
def create_readme_in_subdirs(generated_dir):
|
||||
"""Give every directory below ``generated_dir`` a titled ``README.md``.
|
||||
|
||||
Args:
|
||||
generated_dir: root of the generated documentation tree.
|
||||
"""
|
||||
root = Path(generated_dir).resolve()
|
||||
if not root.exists():
|
||||
print(f"Error: Directory {root} does not exist.")
|
||||
return
|
||||
|
||||
for subdir in [path for path in root.rglob("*") if path.is_dir()]:
|
||||
readme = subdir / "README.md"
|
||||
readme.write_text(README.format(name=subdir.name), encoding="utf-8")
|
||||
print(f"README.md created at {readme}")
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Create README.md files in all subdirectories of the given directory."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--generated-dir", required=True, help="Path to the generated directory."
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
create_readme_in_subdirs(args.generated_dir)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,90 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
from utils.cache.yaml import load_yaml
|
||||
from utils.roles.mapping import ROLE_FILE_META_MAIN, ROLE_FILE_README
|
||||
|
||||
_README_TITLE = re.compile(r"^#\s+(.*)$")
|
||||
|
||||
|
||||
def _readme_title(readme, fallback):
|
||||
if not readme.exists():
|
||||
return fallback
|
||||
try:
|
||||
lines = readme.read_text(encoding="utf-8").splitlines()
|
||||
except (OSError, UnicodeDecodeError) as exc:
|
||||
print(f"Error reading {readme}: {exc}")
|
||||
return fallback
|
||||
return next(
|
||||
(
|
||||
match.group(1).strip()
|
||||
for line in lines
|
||||
if (match := _README_TITLE.match(line))
|
||||
),
|
||||
fallback,
|
||||
)
|
||||
|
||||
|
||||
def collect_roles_by_tag(roles_dir):
|
||||
"""Group every role below ``roles_dir`` by its galaxy tags.
|
||||
|
||||
Args:
|
||||
roles_dir: directory holding one sub-directory per role.
|
||||
|
||||
Returns:
|
||||
``{tag: [role, ...]}`` where each role is a dict with ``name``,
|
||||
``title``, ``description``, ``link`` and ``tags``. A role without
|
||||
tags lands under ``uncategorized``.
|
||||
"""
|
||||
categories = {}
|
||||
for role_path in Path(roles_dir).iterdir():
|
||||
if not role_path.is_dir() or role_path.name.startswith("."):
|
||||
continue
|
||||
meta_path = role_path / ROLE_FILE_META_MAIN
|
||||
if not meta_path.exists():
|
||||
print(f"{ROLE_FILE_META_MAIN} not found for role {role_path}")
|
||||
continue
|
||||
try:
|
||||
data = load_yaml(str(meta_path))
|
||||
except (OSError, TypeError, yaml.YAMLError) as exc:
|
||||
print(f"Error reading YAML file {meta_path}: {exc}")
|
||||
continue
|
||||
|
||||
galaxy_info = data.get("galaxy_info", {})
|
||||
tags = galaxy_info.get("galaxy_tags", []) or ["uncategorized"]
|
||||
entry = {
|
||||
"name": role_path.name,
|
||||
"title": _readme_title(role_path / ROLE_FILE_README, role_path.name),
|
||||
"description": galaxy_info.get("description", ""),
|
||||
"link": f"roles/{role_path.name}/{ROLE_FILE_README}",
|
||||
"tags": tags,
|
||||
}
|
||||
for tag in tags:
|
||||
categories.setdefault(tag, []).append(entry)
|
||||
return categories
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Write the roles overview the roles-overview directive renders."
|
||||
)
|
||||
parser.add_argument("--roles-dir", required=True, help="Directory of the roles.")
|
||||
parser.add_argument("--output-file", required=True, help="JSON file to write.")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
output = Path(args.output_file)
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
output.write_text(
|
||||
json.dumps(collect_roles_by_tag(args.roles_dir), indent=2), encoding="utf-8"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,74 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import pathspec
|
||||
|
||||
HEADER = """\
|
||||
YAML Files
|
||||
===========
|
||||
|
||||
This document lists all `.yaml` and `.yml` files found in the specified directory, excluding ignored files.
|
||||
|
||||
"""
|
||||
|
||||
|
||||
def load_gitignore_patterns(source_dir):
|
||||
gitignore = Path(source_dir) / ".gitignore"
|
||||
lines = (
|
||||
gitignore.read_text(encoding="utf-8").splitlines() if gitignore.exists() else []
|
||||
)
|
||||
return pathspec.GitIgnoreSpec.from_lines(lines)
|
||||
|
||||
|
||||
def generate_yaml_index(source_dir, output_file):
|
||||
"""Write an ``.rst`` page that literal-includes every non-ignored YAML file.
|
||||
|
||||
Args:
|
||||
source_dir: tree to scan; its top-level ``.gitignore`` filters the result.
|
||||
output_file: page to write; includes are relative to its directory.
|
||||
"""
|
||||
source = Path(source_dir)
|
||||
output = Path(output_file)
|
||||
spec = load_gitignore_patterns(source)
|
||||
yaml_files = sorted(
|
||||
(
|
||||
str(path)
|
||||
for path in source.rglob("*")
|
||||
if path.suffix in {".yml", ".yaml"}
|
||||
and path.is_file()
|
||||
and not spec.match_file(str(path.relative_to(source)))
|
||||
),
|
||||
)
|
||||
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
includes = "".join(
|
||||
f".. literalinclude:: {os.path.relpath(file, start=output.parent)}\n"
|
||||
" :language: yaml\n"
|
||||
" :linenos:\n\n"
|
||||
for file in yaml_files
|
||||
)
|
||||
output.write_text(HEADER + includes, encoding="utf-8")
|
||||
print(f"YAML index has been generated at {output_file}")
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Generate an index for YAML files while respecting .gitignore."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--source-dir", required=True, help="Directory containing YAML files."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output-file", required=True, help="Path to the output .rst file."
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
generate_yaml_index(args.source_dir, args.output_file)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,433 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tarfile
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
LATEST = "latest"
|
||||
LOG_TAIL = 40
|
||||
POLL_SECONDS = 2
|
||||
REFS_TTL_SECONDS = 10
|
||||
TAG = re.compile(r"^v(\d+)\.(\d+)\.(\d+)$")
|
||||
|
||||
_STEP = re.compile(
|
||||
r"^(reading sources|writing output|postprocess html)\.\.\. \[\s*(\d+)%\]"
|
||||
)
|
||||
_SPAN = {
|
||||
"reading sources": (10, 40),
|
||||
"writing output": (40, 60),
|
||||
"postprocess html": (60, 99),
|
||||
}
|
||||
|
||||
|
||||
def _generator(name, *args):
|
||||
return [
|
||||
sys.executable,
|
||||
"-P",
|
||||
"-m",
|
||||
f"infinito_docs.generators.{name}",
|
||||
*(str(arg) for arg in args),
|
||||
]
|
||||
|
||||
|
||||
def generate_commands(src):
|
||||
"""Return the commands that prepare ``src`` for ``sphinx-build``.
|
||||
|
||||
Args:
|
||||
src: checkout of the version to document.
|
||||
|
||||
Returns:
|
||||
argv lists, in the order they must run.
|
||||
"""
|
||||
generated = src / "generated"
|
||||
return [
|
||||
[
|
||||
"sphinx-apidoc",
|
||||
"-f",
|
||||
"-o",
|
||||
str(generated / "modules"),
|
||||
str(src),
|
||||
str(src / "tests"),
|
||||
],
|
||||
_generator(
|
||||
"yaml_index",
|
||||
"--source-dir",
|
||||
src,
|
||||
"--output-file",
|
||||
generated / "yaml_index.rst",
|
||||
),
|
||||
_generator(
|
||||
"ansible_roles",
|
||||
"--roles-dir",
|
||||
src / "roles",
|
||||
"--output-dir",
|
||||
generated / "roles",
|
||||
),
|
||||
_generator(
|
||||
"index",
|
||||
"--roles-dir",
|
||||
generated / "roles",
|
||||
"--output-file",
|
||||
src / "roles" / "ansible_role_glosar.rst",
|
||||
"--caption",
|
||||
"Ansible Role Glossary",
|
||||
),
|
||||
_generator(
|
||||
"roles_overview",
|
||||
"--roles-dir",
|
||||
src / "roles",
|
||||
"--output-file",
|
||||
generated / "roles_overview.json",
|
||||
),
|
||||
_generator("readmes", "--generated-dir", generated),
|
||||
]
|
||||
|
||||
|
||||
def progress_of(line, current):
|
||||
"""Return the overall build progress after one line of Sphinx output.
|
||||
|
||||
Args:
|
||||
line: a line of ``sphinx-build`` output.
|
||||
current: progress before this line, in percent.
|
||||
|
||||
Returns:
|
||||
The new progress in percent; it never moves backwards.
|
||||
"""
|
||||
match = _STEP.match(line)
|
||||
if not match:
|
||||
return current
|
||||
start, end = _SPAN[match.group(1)]
|
||||
return max(current, start + (end - start) * int(match.group(2)) // 100)
|
||||
|
||||
|
||||
def _write_json(path, payload):
|
||||
staging = path.with_name(f".{path.name}.{os.getpid()}.{threading.get_ident()}")
|
||||
staging.write_text(json.dumps(payload), encoding="utf-8")
|
||||
staging.replace(path)
|
||||
|
||||
|
||||
class Library:
|
||||
"""Mirror of the documented repository and the site of every version.
|
||||
|
||||
Args:
|
||||
repository: git URL of the documented repository.
|
||||
data_dir: shared volume holding mirror, queue, states, scratch and sites.
|
||||
jobs: parallel Sphinx processes per build.
|
||||
package_dir: the ``infinito_docs`` package directory.
|
||||
"""
|
||||
|
||||
def __init__(self, repository, data_dir, jobs, package_dir):
|
||||
data = Path(data_dir)
|
||||
self.repository = repository
|
||||
self.mirror = data / "repo.git"
|
||||
self.sites = data / "sites"
|
||||
self.queue = data / "queue"
|
||||
self.states = data / "states"
|
||||
self.scratch = data / "work"
|
||||
self.lock_file = data / "builder.lock"
|
||||
self.jobs = jobs
|
||||
self.package_dir = Path(package_dir)
|
||||
self._refs = ("", [])
|
||||
self._refs_read = 0.0
|
||||
self._lock = threading.Lock()
|
||||
self._lock_handle = None
|
||||
|
||||
def _git(self, *args):
|
||||
return subprocess.run(
|
||||
["git", "--git-dir", str(self.mirror), *args],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout
|
||||
|
||||
def refs(self):
|
||||
"""Return ``(head, tags)`` of the mirror.
|
||||
|
||||
Returns:
|
||||
The default branch's commit, empty before the first fetch, and the
|
||||
release tags newest first.
|
||||
"""
|
||||
with self._lock:
|
||||
if time.monotonic() - self._refs_read < REFS_TTL_SECONDS:
|
||||
return self._refs
|
||||
head, tags = "", []
|
||||
if (self.mirror / "HEAD").is_file():
|
||||
try:
|
||||
head = self._git("rev-parse", "HEAD").strip()
|
||||
tags = sorted(
|
||||
(t for t in self._git("tag", "--list").split() if TAG.match(t)),
|
||||
key=lambda tag: tuple(
|
||||
int(part) for part in TAG.match(tag).groups()
|
||||
),
|
||||
reverse=True,
|
||||
)
|
||||
except subprocess.CalledProcessError:
|
||||
head, tags = "", []
|
||||
with self._lock:
|
||||
self._refs, self._refs_read = (head, tags), time.monotonic()
|
||||
return head, tags
|
||||
|
||||
def _forget_refs(self):
|
||||
with self._lock:
|
||||
self._refs_read = 0.0
|
||||
|
||||
def versions(self):
|
||||
return [LATEST, *self.refs()[1]]
|
||||
|
||||
def built_ref(self, version):
|
||||
stamp = self.sites / version / "ref"
|
||||
return stamp.read_text(encoding="utf-8").strip() if stamp.is_file() else ""
|
||||
|
||||
def servable(self, version):
|
||||
return (self.sites / version / "html" / "index.html").is_file()
|
||||
|
||||
def _current(self, version, head):
|
||||
wanted = head if version == LATEST else version
|
||||
return self.servable(version) and self.built_ref(version) == wanted
|
||||
|
||||
def _state(self, version):
|
||||
path = self.states / f"{version}.json"
|
||||
try:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return {}
|
||||
|
||||
def _save_state(self, version, **state):
|
||||
self.states.mkdir(parents=True, exist_ok=True)
|
||||
_write_json(self.states / f"{version}.json", state)
|
||||
|
||||
def status(self):
|
||||
head, tags = self.refs()
|
||||
report = []
|
||||
for name in [LATEST, *tags]:
|
||||
built = self.servable(name)
|
||||
state = self._state(name)
|
||||
if (self.queue / name).exists():
|
||||
phase = "building" if state.get("state") == "building" else "queued"
|
||||
elif state.get("state") == "failed":
|
||||
phase = "failed"
|
||||
elif name == LATEST and not head and not built:
|
||||
phase = "queued"
|
||||
state = {"phase": "fetching"}
|
||||
else:
|
||||
phase = "ready" if built else "missing"
|
||||
progress = {"ready": 100, "building": state.get("progress", 0)}
|
||||
report.append(
|
||||
{
|
||||
"name": name,
|
||||
"built": built,
|
||||
"state": phase,
|
||||
"progress": progress.get(phase, 0),
|
||||
"phase": "" if phase == "ready" else state.get("phase", ""),
|
||||
"log": state.get("log", []) if phase == "failed" else [],
|
||||
}
|
||||
)
|
||||
return report
|
||||
|
||||
def request(self, version):
|
||||
"""Queue a build of ``version`` unless its site is current.
|
||||
|
||||
Args:
|
||||
version: ``latest`` or a release tag.
|
||||
"""
|
||||
head, _ = self.refs()
|
||||
if not head or self._current(version, head):
|
||||
return
|
||||
self.queue.mkdir(parents=True, exist_ok=True)
|
||||
(self.queue / version).touch(exist_ok=True)
|
||||
|
||||
def next_queued(self):
|
||||
if not self.queue.is_dir():
|
||||
return None
|
||||
waiting = sorted(
|
||||
self.queue.iterdir(), key=lambda marker: marker.stat().st_mtime
|
||||
)
|
||||
return waiting[0].name if waiting else None
|
||||
|
||||
def acquire_builder(self):
|
||||
"""Try to become the builder of all replicas without blocking.
|
||||
|
||||
Returns:
|
||||
``True`` while this process holds the builder lock.
|
||||
"""
|
||||
if self._lock_handle is not None:
|
||||
return True
|
||||
self.lock_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
handle = self.lock_file.open("a+")
|
||||
try:
|
||||
fcntl.lockf(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except OSError:
|
||||
handle.close()
|
||||
return False
|
||||
self._lock_handle = handle
|
||||
return True
|
||||
|
||||
def fetch(self):
|
||||
if (self.mirror / "HEAD").is_file():
|
||||
self._git("remote", "update", "--prune")
|
||||
else:
|
||||
subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"clone",
|
||||
"--mirror",
|
||||
"--quiet",
|
||||
self.repository,
|
||||
str(self.mirror),
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
self._forget_refs()
|
||||
self.request(LATEST)
|
||||
|
||||
def run_builder(self, interval):
|
||||
while not self.acquire_builder():
|
||||
time.sleep(POLL_SECONDS)
|
||||
next_fetch = 0.0
|
||||
while True:
|
||||
if time.monotonic() >= next_fetch:
|
||||
try:
|
||||
self.fetch()
|
||||
except (OSError, subprocess.CalledProcessError) as exc:
|
||||
print(f"fetch of {self.repository} failed: {exc}", file=sys.stderr)
|
||||
next_fetch = time.monotonic() + interval
|
||||
version = self.next_queued()
|
||||
if version is None:
|
||||
time.sleep(POLL_SECONDS)
|
||||
else:
|
||||
self.build(version)
|
||||
|
||||
def resolve(self, version, rest):
|
||||
"""Return the file a request for ``rest`` in ``version`` maps to.
|
||||
|
||||
Args:
|
||||
version: a built version.
|
||||
rest: the request path below the version, already unquoted.
|
||||
|
||||
Returns:
|
||||
The file inside the version's site, or ``None`` when the site has
|
||||
no such file or the path escapes it.
|
||||
"""
|
||||
root = (self.sites / version / "html").resolve()
|
||||
target = (root / rest).resolve()
|
||||
if target != root and root not in target.parents:
|
||||
return None
|
||||
if target.is_dir():
|
||||
target /= "index.html"
|
||||
return target if target.is_file() else None
|
||||
|
||||
def _run(self, version, state, command, env, cwd):
|
||||
with subprocess.Popen(
|
||||
command,
|
||||
cwd=cwd,
|
||||
env=env,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
) as process:
|
||||
for line in process.stdout:
|
||||
progress = progress_of(line, state["progress"])
|
||||
state["log"] = [*state["log"][-(LOG_TAIL - 1) :], line.rstrip()]
|
||||
if progress != state["progress"]:
|
||||
state["progress"] = progress
|
||||
self._save_state(version, **state)
|
||||
if process.returncode:
|
||||
raise subprocess.CalledProcessError(process.returncode, command)
|
||||
|
||||
def build(self, version):
|
||||
"""Build ``version`` into its site, replacing an older site atomically.
|
||||
|
||||
Args:
|
||||
version: ``latest`` or a release tag.
|
||||
"""
|
||||
self._forget_refs()
|
||||
head, tags = self.refs()
|
||||
if version != LATEST and version not in tags:
|
||||
(self.queue / version).unlink(missing_ok=True)
|
||||
return
|
||||
if self._current(version, head):
|
||||
(self.queue / version).unlink(missing_ok=True)
|
||||
return
|
||||
ref = head if version == LATEST else version
|
||||
work = self.scratch / version
|
||||
src, conf, out = work / "src", work / "conf", work / "out"
|
||||
tooling = str(self.package_dir.parent)
|
||||
state = {"state": "building", "phase": "checkout", "progress": 0, "log": []}
|
||||
try:
|
||||
self._save_state(version, **state)
|
||||
shutil.rmtree(work, ignore_errors=True)
|
||||
src.mkdir(parents=True)
|
||||
archive = work / "src.tar"
|
||||
self._git("archive", "--format=tar", "-o", str(archive), ref)
|
||||
with tarfile.open(archive) as tar:
|
||||
tar.extractall(src, filter="data")
|
||||
|
||||
shutil.copytree(self.package_dir, conf)
|
||||
if (src / "assets" / "img").is_dir():
|
||||
shutil.copytree(
|
||||
src / "assets" / "img", conf / "assets" / "img", dirs_exist_ok=True
|
||||
)
|
||||
|
||||
generators = generate_commands(src)
|
||||
env = {**os.environ, "PYTHONPATH": tooling}
|
||||
state["phase"] = "generate"
|
||||
for step, command in enumerate(generators, start=1):
|
||||
self._run(version, state, command, env, work)
|
||||
state["progress"] = 10 * step // len(generators)
|
||||
self._save_state(version, **state)
|
||||
|
||||
state["phase"] = "sphinx"
|
||||
sphinx_env = {
|
||||
**os.environ,
|
||||
"PYTHONPATH": os.pathsep.join([str(src), tooling]),
|
||||
"PYTHONUNBUFFERED": "1",
|
||||
"DOCS_VERSION": version,
|
||||
}
|
||||
self._run(
|
||||
version,
|
||||
state,
|
||||
[
|
||||
"sphinx-build",
|
||||
"-M",
|
||||
"html",
|
||||
str(src),
|
||||
str(out),
|
||||
"-c",
|
||||
str(conf),
|
||||
"-j",
|
||||
str(self.jobs),
|
||||
],
|
||||
sphinx_env,
|
||||
work,
|
||||
)
|
||||
self._publish(version, out / "html", ref)
|
||||
self._save_state(version, state="ready", phase="", progress=100, log=[])
|
||||
except (OSError, subprocess.CalledProcessError, tarfile.TarError) as exc:
|
||||
state["log"] = [*state["log"][-(LOG_TAIL - 1) :], str(exc)]
|
||||
self._save_state(version, **{**state, "state": "failed"})
|
||||
finally:
|
||||
(self.queue / version).unlink(missing_ok=True)
|
||||
shutil.rmtree(work, ignore_errors=True)
|
||||
|
||||
def _publish(self, version, html, ref):
|
||||
staging = self.sites / f".{version}.new"
|
||||
retired = self.sites / f".{version}.old"
|
||||
target = self.sites / version
|
||||
shutil.rmtree(staging, ignore_errors=True)
|
||||
shutil.rmtree(retired, ignore_errors=True)
|
||||
staging.mkdir(parents=True)
|
||||
html.rename(staging / "html")
|
||||
(staging / "ref").write_text(ref, encoding="utf-8")
|
||||
if target.exists():
|
||||
target.rename(retired)
|
||||
staging.rename(target)
|
||||
shutil.rmtree(retired, ignore_errors=True)
|
||||
@@ -0,0 +1,143 @@
|
||||
"""HTTP front of the documentation library.
|
||||
|
||||
Environment:
|
||||
DOCS_SOURCE_REPO: git URL of the documented repository.
|
||||
DOCS_DATA_DIR: volume holding the mirror and the site of every version.
|
||||
DOCS_BUILD_JOBS: parallel Sphinx processes per build.
|
||||
DOCS_FETCH_INTERVAL: seconds between two fetches of new commits and tags.
|
||||
DOCS_PORT: port to listen on.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import html
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
from functools import partial
|
||||
from http import HTTPStatus
|
||||
from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from urllib.parse import unquote, urlsplit
|
||||
|
||||
from infinito_docs.library import LATEST, Library
|
||||
|
||||
PACKAGE_DIR = Path(__file__).resolve().parent
|
||||
SCRIPT = PACKAGE_DIR / "assets" / "js" / "versions.js"
|
||||
|
||||
PAGE = """<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>{title}</title>
|
||||
<script src="/_docs/versions.js" defer></script>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
<h1>{title}</h1>
|
||||
{body}
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
"""
|
||||
|
||||
OVERVIEW = """<table>
|
||||
<thead><tr><th>Version</th><th>Status</th><th>Progress</th></tr></thead>
|
||||
<tbody data-versions></tbody>
|
||||
</table>
|
||||
"""
|
||||
|
||||
BUILDING = """<section data-build="{version}">
|
||||
<p>This version is not built yet. The build started and this page reloads once it is done.</p>
|
||||
<p data-phase>queued</p>
|
||||
<progress max="100" value="0" aria-label="Build progress of {version}"></progress>
|
||||
<pre></pre>
|
||||
</section>
|
||||
<p><a href="/versions/">All versions</a></p>
|
||||
"""
|
||||
|
||||
|
||||
class DocsHandler(SimpleHTTPRequestHandler):
|
||||
def __init__(self, *args, library, **kwargs):
|
||||
self.library = library
|
||||
super().__init__(*args, **kwargs)
|
||||
|
||||
def do_GET(self):
|
||||
self._route(head=False)
|
||||
|
||||
def do_HEAD(self):
|
||||
self._route(head=True)
|
||||
|
||||
def _send(self, status, content_type, body, head):
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", content_type)
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.send_header("Cache-Control", "no-store")
|
||||
self.end_headers()
|
||||
if not head:
|
||||
self.wfile.write(body)
|
||||
|
||||
def _redirect(self, location):
|
||||
self.send_response(HTTPStatus.FOUND)
|
||||
self.send_header("Location", location)
|
||||
self.send_header("Content-Length", "0")
|
||||
self.end_headers()
|
||||
|
||||
def _page(self, status, title, body, head):
|
||||
page = PAGE.format(title=html.escape(title), body=body)
|
||||
self._send(status, "text/html; charset=utf-8", page.encode("utf-8"), head)
|
||||
|
||||
def _file(self, path, head):
|
||||
self._send(HTTPStatus.OK, self.guess_type(str(path)), path.read_bytes(), head)
|
||||
|
||||
def _route(self, head):
|
||||
path = unquote(urlsplit(self.path).path)
|
||||
if path == "/":
|
||||
self._redirect(f"/{LATEST}/")
|
||||
elif path in {"/versions", "/versions/"}:
|
||||
self._page(HTTPStatus.OK, "Documentation versions", OVERVIEW, head)
|
||||
elif path == "/api/versions":
|
||||
body = json.dumps(self.library.status()).encode("utf-8")
|
||||
self._send(HTTPStatus.OK, "application/json", body, head)
|
||||
elif path == "/_docs/versions.js":
|
||||
self._file(SCRIPT, head)
|
||||
else:
|
||||
self._version(path, head)
|
||||
|
||||
def _version(self, path, head):
|
||||
version, slash, rest = path.lstrip("/").partition("/")
|
||||
if version not in self.library.versions():
|
||||
self.send_error(HTTPStatus.NOT_FOUND)
|
||||
elif not slash:
|
||||
self._redirect(f"/{version}/")
|
||||
elif not self.library.servable(version):
|
||||
self.library.request(version)
|
||||
body = BUILDING.format(version=html.escape(version))
|
||||
self._page(HTTPStatus.ACCEPTED, f"Building {version}", body, head)
|
||||
elif (target := self.library.resolve(version, rest)) is None:
|
||||
self._redirect(f"/{version}/")
|
||||
elif target.name == "index.html" and rest and not rest.endswith(("/", ".html")):
|
||||
self._redirect(f"/{version}/{rest}/")
|
||||
else:
|
||||
self._file(target, head)
|
||||
|
||||
|
||||
def main():
|
||||
library = Library(
|
||||
os.environ["DOCS_SOURCE_REPO"],
|
||||
os.environ["DOCS_DATA_DIR"],
|
||||
int(os.environ["DOCS_BUILD_JOBS"]),
|
||||
PACKAGE_DIR,
|
||||
)
|
||||
threading.Thread(
|
||||
target=library.run_builder,
|
||||
args=(int(os.environ["DOCS_FETCH_INTERVAL"]),),
|
||||
daemon=True,
|
||||
).start()
|
||||
handler = partial(DocsHandler, library=library)
|
||||
ThreadingHTTPServer(("", int(os.environ["DOCS_PORT"])), handler).serve_forever()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,3 @@
|
||||
<div class="sidebar-logo" style="text-align: center; margin-bottom: 1em;">
|
||||
<img src="{{ pathto("_static/img/logo.png", 1) }}" alt="Logo" style="max-width: 100%;">
|
||||
</div>
|
||||
@@ -0,0 +1,63 @@
|
||||
{% macro render_headings(headings, level=1) %}
|
||||
<ul class="toctree-l{{ level }}" style="list-style: none; padding-left: 0; overflow-x: auto; white-space: nowrap;">
|
||||
{% for item in headings %}
|
||||
<li class="toctree-l{{ level }}{% if item.current %} current{% endif %}"
|
||||
{% if item.children %}
|
||||
x-data="{ expanded: {{ 'true' if item.current else 'false' }} }"
|
||||
{% endif %}
|
||||
style="white-space: nowrap;">
|
||||
<div class="menu-item" style="display: inline-flex; align-items: center; justify-content: space-between; width: 100%; white-space: nowrap;">
|
||||
<div style="display: inline-flex; align-items: center; white-space: nowrap;">
|
||||
<a class="reference internal{% if item.children %} expandable{% endif %}{% if item.current and not item.children %} current{% endif %}"
|
||||
href="{{ pathto(item.link).replace('#', '') }}{% if item.anchor %}#{{ item.anchor }}{% endif %}"
|
||||
style="text-decoration: none; white-space: nowrap;">
|
||||
{{ item.text }}
|
||||
</a>
|
||||
</div>
|
||||
{% if item.children %}
|
||||
<button @click.prevent.stop="expanded = !expanded" type="button" class="toggle-button"
|
||||
style="background: none; border: none; padding: 0; margin-left: auto;">
|
||||
<span x-show="!expanded">
|
||||
<svg fill="currentColor" height="18px" stroke="none" viewBox="0 0 24 24" width="18px"
|
||||
xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M10 6L8.59 7.41 13.17 12l-4.58 4.59L10 18l6-6z"></path>
|
||||
</svg>
|
||||
</span>
|
||||
<span x-show="expanded">▼</span>
|
||||
</button>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% if item.children %}
|
||||
<div x-show="expanded">
|
||||
{{ render_headings(item.children, level+1) }}
|
||||
</div>
|
||||
{% endif %}
|
||||
</li>
|
||||
{% endfor %}
|
||||
</ul>
|
||||
{% endmacro %}
|
||||
|
||||
{% if local_md_headings or local_subfolders %}
|
||||
<div class="local-md-headings">
|
||||
{% if local_md_headings %}
|
||||
<div class="current-index" x-data x-init="typeof initCurrentNav === 'function' && initCurrentNav()">
|
||||
<p class="caption" role="heading">
|
||||
<span class="caption-text">Current Index</span>
|
||||
</p>
|
||||
{{ render_headings(local_md_headings) }}
|
||||
<br />
|
||||
</div>
|
||||
{% endif %}
|
||||
{% if local_subfolders %}
|
||||
<div class="full-index">
|
||||
<p class="caption" role="heading">
|
||||
<span class="caption-text">Full Index</span>
|
||||
</p>
|
||||
{{ render_headings(local_subfolders) }}
|
||||
<br />
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endif %}
|
||||
<script src="{{ pathto('_static/js/current-nav.js', 1) }}"></script>
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
<div class="docs-versions" style="margin-bottom: 1em;">
|
||||
<label for="docs-version-switcher">Version</label>
|
||||
<select id="docs-version-switcher" class="docs-version-switcher" data-current="{{ docs_version }}" data-page="{{ pagename }}">
|
||||
<option selected>{{ docs_version }}</option>
|
||||
</select>
|
||||
<a href="/versions/">All versions</a>
|
||||
</div>
|
||||
<script src="{{ pathto('_static/js/versions.js', 1) }}"></script>
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -1,7 +1,7 @@
|
||||
---
|
||||
galaxy_info:
|
||||
author: Marko Pjevac, Kevin Veen-Birkenbach
|
||||
description: Documentation generator that builds HTML from a source repository and serves it through a lightweight HTTP server
|
||||
description: Infinito.Nexus documentation for the latest commit and every release, built on demand with Sphinx
|
||||
license: Infinito.Nexus Community License (Non-Commercial)
|
||||
company: |
|
||||
Kevin Veen-Birkenbach
|
||||
File renamed without changes.
@@ -18,32 +18,37 @@ matomo:
|
||||
bond: 1
|
||||
enabled: "{{ 'web-app-matomo' in group_names }}"
|
||||
shared: "{{ 'web-app-matomo' in group_names }}"
|
||||
sphinx:
|
||||
docs:
|
||||
modes:
|
||||
compose:
|
||||
enabled: true
|
||||
swarm:
|
||||
enabled: true
|
||||
image: ghcr.io/kevinveenbirkenbach/infinito-sphinx
|
||||
version: v1.0.70
|
||||
min_storage: 1931MB
|
||||
image: python
|
||||
version: 3.13-slim
|
||||
custom: true
|
||||
source_repository: https://github.com/infinito-nexus/core.git
|
||||
fetch_interval: 900
|
||||
build_jobs: 2
|
||||
# nocheck: min-storage-headroom Reason: the image built on the python base (1.1 GB), the latest site (2.3 GB) and the scratch of a running build (2.4 GB); every further version opened adds about 2.3 GB
|
||||
min_storage: 6GB
|
||||
backup:
|
||||
no_stop_required: true
|
||||
name: sphinx
|
||||
name: docs
|
||||
ports:
|
||||
internal:
|
||||
http: 8000
|
||||
local:
|
||||
http: 8036
|
||||
healthcheck:
|
||||
flavor: curl
|
||||
flavor: http_status
|
||||
run_after:
|
||||
- web-app-matomo
|
||||
lifecycle: beta
|
||||
cpus: "0.2"
|
||||
mem_reservation: 128m
|
||||
mem_limit: 256m
|
||||
pids_limit: 256
|
||||
cpus: "2"
|
||||
mem_reservation: 512m
|
||||
mem_limit: 4g
|
||||
pids_limit: 512
|
||||
# nocheck: playwright-service-flag — css consumer, no Playwright surface (provider: web-svc-css)
|
||||
css:
|
||||
bond: 1
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
# Matrix-deploy variant overrides for web-app-sphinx.
|
||||
# Matrix-deploy variant overrides for web-app-docs.
|
||||
# See docs/contributing/artefact/files/role/variants.md for the file format.
|
||||
|
||||
# variant pinning every dynamic services.yml flag to true so the
|
||||
# matrix exercises the in-group_names branch of web-app-sphinx.
|
||||
# matrix exercises the in-group_names branch of web-app-docs.
|
||||
- services:
|
||||
dashboard:
|
||||
enabled: true
|
||||
@@ -0,0 +1,9 @@
|
||||
# nocheck: backup-service-flag the git mirror and every built site are regenerated on demand; the volume is backup-disabled
|
||||
---
|
||||
# nocheck: embedded-state-on-nfs the replicas coordinate through this volume: one holds the builder lock, the queue and build states are files beside the sites, and sites are published by atomic rename
|
||||
docs_sites:
|
||||
type: volume
|
||||
backup: false
|
||||
mounts:
|
||||
- service: docs
|
||||
target: /data
|
||||
@@ -0,0 +1,21 @@
|
||||
---
|
||||
- name: "🧩 load docker, proxy for '{{ application_id }}'"
|
||||
include_role:
|
||||
name: sys-stk-full
|
||||
vars:
|
||||
compose_handlers_flush: false
|
||||
|
||||
- name: "🎬 Stage the Sphinx tooling into the build context of '{{ application_id }}'"
|
||||
stack_host_copy:
|
||||
src: "{{ item }}/"
|
||||
dest: "{{ [DOCS_TOOLING_DIR, item] | path_join }}/"
|
||||
directory_mode: "0755"
|
||||
loop:
|
||||
- python
|
||||
- javascript
|
||||
notify:
|
||||
- compose-build
|
||||
- compose-up
|
||||
|
||||
- name: "🚿 Flush compose handlers for '{{ application_id }}'"
|
||||
ansible.builtin.meta: flush_handlers
|
||||
@@ -0,0 +1,19 @@
|
||||
{% include 'roles/sys-svc-compose/templates/base.yml.j2' %}
|
||||
{% set service_name = DOCS_SERVICE %}
|
||||
{{ service_name }}:
|
||||
{{ lookup('compose_only', 'container_name', DOCS_CONTAINER) }}
|
||||
{{ lookup('template', 'roles/sys-svc-container/templates/build.yml.j2') | indent(4) }}
|
||||
args:
|
||||
DOCS_IMAGE: "{{ DOCS_IMAGE }}"
|
||||
DOCS_VERSION: "{{ DOCS_VERSION }}"
|
||||
DOCS_SOURCE_REPO: "{{ DOCS_SOURCE_REPO }}"
|
||||
{{ lookup('container_image', application_id, 'docs') }}
|
||||
{{ lookup('container_ports', ['docs', 'http', DOCKER_BIND_HOST]) | indent(4) }}
|
||||
{{ lookup('container_volumes', service_name) | indent(4) }}
|
||||
{{ lookup('container_networks') | indent(4) }}
|
||||
{% include 'roles/sys-svc-container/templates/base.yml.j2' %}
|
||||
{{ lookup('container_healthcheck', service_name) | indent(4) }}
|
||||
|
||||
{{ lookup('compose_volumes') }}
|
||||
|
||||
{{ lookup('compose_networks') }}
|
||||
@@ -0,0 +1,7 @@
|
||||
{% include 'roles/sys-svc-container/templates/env.j2' %}
|
||||
|
||||
DOCS_SOURCE_REPO={{ DOCS_SOURCE_REPO }}
|
||||
DOCS_DATA_DIR={{ DOCS_DATA_DIR }}
|
||||
DOCS_BUILD_JOBS={{ DOCS_BUILD_JOBS }}
|
||||
DOCS_FETCH_INTERVAL={{ DOCS_FETCH_INTERVAL }}
|
||||
DOCS_PORT={{ DOCS_PORT }}
|
||||
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,15 @@
|
||||
---
|
||||
application_id: "web-app-docs"
|
||||
entity_name: "{{ application_id | get_entity_name }}"
|
||||
|
||||
DOCS_CONTAINER: "{{ lookup('config', application_id, 'services.' ~ entity_name ~ '.name') }}"
|
||||
DOCS_SERVICE: "{{ entity_name }}"
|
||||
|
||||
DOCS_IMAGE: "{{ lookup('config', application_id, 'services.docs.image') }}"
|
||||
DOCS_VERSION: "{{ lookup('config', application_id, 'services.docs.version') }}"
|
||||
DOCS_SOURCE_REPO: "{{ lookup('config', application_id, 'services.docs.source_repository') }}"
|
||||
DOCS_FETCH_INTERVAL: "{{ lookup('config', application_id, 'services.docs.fetch_interval') }}"
|
||||
DOCS_BUILD_JOBS: "{{ lookup('config', application_id, 'services.docs.build_jobs') }}"
|
||||
DOCS_PORT: "{{ lookup('config', application_id, 'services.docs.ports.internal.http') }}"
|
||||
DOCS_DATA_DIR: "/data"
|
||||
DOCS_TOOLING_DIR: "{{ [ lookup('container', application_id, 'directories.instance'), 'sphinx' ] | path_join }}"
|
||||
@@ -67,7 +67,7 @@ flowchart LR
|
||||
dpt_web_app_confluence["web-app-confluence 🐳🐝"]
|
||||
dpt_web_app_decidim["web-app-decidim 🐳🐝"]
|
||||
dpt_web_app_discourse["web-app-discourse 🐳🐝"]
|
||||
dpt_web_app_erpnext["web-app-erpnext 🐳🐝"]
|
||||
dpt_web_app_docs["web-app-docs 🐳🐝"]
|
||||
dpt_more["..."]
|
||||
end
|
||||
dep_svc_bkp_volume_2_local -. "0..1" .-> svc_container_backup
|
||||
@@ -92,8 +92,8 @@ flowchart LR
|
||||
svc_mailu -- "0..0" --> dpt_web_app_confluence
|
||||
svc_mailu -. "0..1" .-> dpt_web_app_decidim
|
||||
svc_mailu -. "0..1" .-> dpt_web_app_discourse
|
||||
svc_mailu -. "0..1" .-> dpt_web_app_erpnext
|
||||
linkStyle 17,19 stroke:red;
|
||||
svc_mailu -- "0..0" --> dpt_web_app_docs
|
||||
linkStyle 17,19,22 stroke:red;
|
||||
```
|
||||
|
||||
Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`.
|
||||
|
||||
@@ -1,127 +0,0 @@
|
||||
# Sphinx
|
||||
|
||||
## Description
|
||||
|
||||
Sphinx is a powerful documentation generator originally created for Python projects and now widely used to build and maintain comprehensive documentation for all kinds of software projects. This role automates the process of building and deploying Sphinx documentation using Docker, ensuring reproducible builds and seamless updates.
|
||||
|
||||
## Overview
|
||||
|
||||
This Docker Compose deployment leverages Ansible to automatically pull your source repository, build the documentation using Sphinx, and serve the generated HTML through a lightweight HTTP server. The entire process is containerized, which guarantees a consistent and isolated environment regardless of the host system. By default it uses [Infinito.Nexus Sphinx](https://s.infinito.nexus/code-sphinx) to build the docs.
|
||||
|
||||
## Cosmos
|
||||
|
||||
The diagram places Sphinx in the Infinito.Nexus cosmos: the components it deploys (capabilities), the central services it consumes (dependencies), and its outward reach (federation and bridged external networks).
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph deps [Dependencies]
|
||||
dep_svc_net_tor["svc-net-tor 🐳🐝"]
|
||||
dep_web_app_dashboard["web-app-dashboard 🐳🐝"]
|
||||
dep_web_app_keycloak["web-app-keycloak 🐳🐝"]
|
||||
dep_web_app_mailu["web-app-mailu 🐳🐝"]
|
||||
dep_web_app_matomo["web-app-matomo 🐳🐝"]
|
||||
dep_web_app_prometheus["web-app-prometheus 🐳🐝"]
|
||||
dep_web_svc_css["web-svc-css 💻"]
|
||||
end
|
||||
subgraph role [web-app-sphinx 🐳🐝]
|
||||
svc_logout["logout ❌"]
|
||||
svc_sso["sso ❌"]
|
||||
svc_dashboard["dashboard"]
|
||||
svc_matomo["matomo"]
|
||||
svc_sphinx["sphinx"]
|
||||
svc_css["css"]
|
||||
svc_email["email ❌"]
|
||||
svc_prometheus["prometheus"]
|
||||
svc_tor["tor"]
|
||||
end
|
||||
dep_svc_net_tor -. "0..1" .-> svc_tor
|
||||
dep_web_app_dashboard -. "0..1" .-> svc_dashboard
|
||||
dep_web_app_keycloak -- "0..0" --> svc_sso
|
||||
dep_web_app_mailu -- "0..0" --> svc_email
|
||||
dep_web_app_matomo -. "0..1" .-> svc_matomo
|
||||
dep_web_app_prometheus -. "0..1" .-> svc_prometheus
|
||||
dep_web_svc_css -. "0..1" .-> svc_css
|
||||
linkStyle 2,3 stroke:red;
|
||||
```
|
||||
|
||||
Solid `1:1` edges are fixed relationships; dashed `0..1` edges are conditional (enabled only in matching deployments); red `0..0` edges are turned off in this role. Node markers show the role's deploy modes (💻 host, 🐳 compose, 🐝 swarm); ❌ marks a service that is explicitly turned off, and ⚙️ an Ansible role dependency declared in `meta/main.yml`.
|
||||
|
||||
## Purpose
|
||||
|
||||
By automating the Sphinx build process and containerized deployment, this role minimizes manual intervention and helps you ensure that your documentation is always up-to-date with the latest changes in your codebase. It is ideal for continuous integration environments and for projects that require frequent documentation updates.
|
||||
|
||||
## Features
|
||||
|
||||
- **Automated Sphinx Build:**
|
||||
The role automatically triggers a Sphinx build using a Makefile. This build process compiles source files into clean, navigable HTML documentation.
|
||||
|
||||
- **Dynamic and Reproducible Builds:**
|
||||
Source code is dynamically pulled from your repository before each build. This ensures that every documentation update accurately reflects the latest version of your project.
|
||||
|
||||
- **Docker Compose Integration:**
|
||||
By harnessing Docker Compose, the role deploys the Sphinx build and serving process in an isolated, reproducible container environment, which simplifies the setup and improves portability.
|
||||
|
||||
- **Customizable Configuration:**
|
||||
Variables allow you to define source and output directories, build parameters, and other settings. This flexibility enables you to tailor the documentation build process to your project’s specific needs.
|
||||
|
||||
- **Reliable Serving of Documentation:**
|
||||
Once the build is complete, the generated HTML is served through a lightweight HTTP server. Health checks are in place to ensure the service is running reliably.
|
||||
|
||||
- **Consistent Deployment Workflow:**
|
||||
The entire process (from pulling the repository to serving the final output) is automated. This reduces human error and makes it easy to update or roll back documentation builds.
|
||||
|
||||
## Quick Setup
|
||||
|
||||
### Development
|
||||
|
||||
Clone, set up the workstation, and deploy Sphinx onto the local stack:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/infinito-nexus/core.git
|
||||
cd core
|
||||
make onboard
|
||||
make compose-deploy mode=reinstall apps=web-app-sphinx full_cycle=false
|
||||
```
|
||||
|
||||
### Production
|
||||
|
||||
Run the published image to provision the inventory and deploy Sphinx to a managed server (the mounted volume persists the inventory):
|
||||
|
||||
```bash
|
||||
APP=web-app-sphinx
|
||||
HOST=<your-server>
|
||||
DOMAIN=<your-domain>
|
||||
TLS_MODE=self_signed
|
||||
SSH_PUBLIC_KEY="<your-ssh-public-key>"
|
||||
|
||||
docker run --rm -it \
|
||||
-v "$PWD/inventories:/etc/infinito.nexus/inventories" \
|
||||
-e APP="$APP" -e HOST="$HOST" -e DOMAIN="$DOMAIN" -e TLS_MODE="$TLS_MODE" -e SSH_PUBLIC_KEY="$SSH_PUBLIC_KEY" \
|
||||
ghcr.io/infinito-nexus/core/debian bash -c '
|
||||
INVENTORY=/etc/infinito.nexus/inventories/production
|
||||
infinito administration inventory provision "$INVENTORY" \
|
||||
--inventory-file "$INVENTORY/devices.yml" \
|
||||
--host "$HOST" \
|
||||
--include "$APP" \
|
||||
--vars "{\"TLS_MODE\": \"$TLS_MODE\", \"DOMAIN_PRIMARY\": \"$DOMAIN\", \"users\": {\"administrator\": {\"authorized_keys\": [\"$SSH_PUBLIC_KEY\"]}}}" &&
|
||||
infinito administration deploy dedicated "$INVENTORY/devices.yml" \
|
||||
--password-file "$INVENTORY/.password" \
|
||||
--diff -vv'
|
||||
```
|
||||
|
||||
## Further Resources
|
||||
|
||||
For more information about Sphinx and its capabilities, please visit the [Sphinx Official Website](https://www.sphinx-doc.org/).
|
||||
|
||||
For detailed Sphinx documentation, see the [Sphinx Documentation](https://www.sphinx-doc.org/en/master/).
|
||||
|
||||
## Persona contract opt-outs
|
||||
|
||||
The role publishes a pre-built static documentation site. [`meta/services.yml`](./meta/services.yml) pins `sso.enabled` and `sso.shared` to `false`, and the role ships no `templates/env.j2`, no `meta/users.yml` and no account provisioning — [`tasks/main.yml`](./tasks/main.yml) only wires the container and proxy stack.
|
||||
There is no account and no auth chain for the `biber` or `administrator` persona, so [`templates/playwright.env.j2`](./templates/playwright.env.j2) declares `PERSONA_BIBER_BLOCKED=true` and `PERSONA_ADMINISTRATOR_BLOCKED=true`. The `guest` persona and the baseline reachability and CSP assertions run unconditionally.
|
||||
|
||||
## Credits
|
||||
|
||||
Implemented by **Marko Pjevac, Kevin Veen-Birkenbach**.
|
||||
Part of the [Infinito.Nexus Project](https://s.infinito.nexus/code) and maintained by [Kevin Veen-Birkenbach](https://www.veen.world).
|
||||
Licensed under the [Infinito.Nexus Community License (Non-Commercial)](https://s.infinito.nexus/license).
|
||||
@@ -1,74 +0,0 @@
|
||||
const { test, expect } = require("@playwright/test");
|
||||
const { resolveTimeout } = require("./timeouts");
|
||||
|
||||
const { decodeDotenvQuotedValue, normalizeBaseUrl, runAdminFlow, runBiberFlow, runGuestFlow , expectHstsWhenTls, gotoOnion } = require("./personas");
|
||||
test.use({ ignoreHTTPSErrors: true });
|
||||
|
||||
const appBaseUrl = normalizeBaseUrl(process.env.APP_BASE_URL || "");
|
||||
const canonicalDomain = decodeDotenvQuotedValue(process.env.CANONICAL_DOMAIN || "");
|
||||
|
||||
test.beforeEach(async ({ page }) => {
|
||||
expect(appBaseUrl, "APP_BASE_URL must be set").toBeTruthy();
|
||||
expect(canonicalDomain, "CANONICAL_DOMAIN must be set").toBeTruthy();
|
||||
await page.context().clearCookies();
|
||||
});
|
||||
|
||||
test("sphinx front page is served under canonical domain with TLS", async ({ page }) => {
|
||||
const response = await gotoOnion(page, `${appBaseUrl}/`);
|
||||
expect(response, "Expected sphinx response").toBeTruthy();
|
||||
expect(response.status(), "Expected sphinx front page status < 400").toBeLessThan(400);
|
||||
expect(
|
||||
response.url().includes(canonicalDomain),
|
||||
`Expected canonical domain "${canonicalDomain}" to back the sphinx URL`
|
||||
).toBe(true);
|
||||
const headers = response.headers();
|
||||
expectHstsWhenTls(headers, appBaseUrl, "sphinx");
|
||||
});
|
||||
|
||||
test("sphinx returns HTML content under canonical domain", async ({ request }) => {
|
||||
// Initial deploy serves an empty docs root (the docs are populated by a
|
||||
// separate Sphinx-build job, not by the deploy itself), so the front
|
||||
// page resolves to an Index-of listing rather than a Sphinx-rendered
|
||||
// page. Validate only what the deploy MUST guarantee at this stage:
|
||||
// an HTML response under the canonical domain. Theme/chrome assertions
|
||||
// belong in a follow-up suite once a docs-build step is part of the
|
||||
// role contract.
|
||||
const response = await request.get(`${appBaseUrl}/`, { timeout: resolveTimeout(30_000) });
|
||||
expect(response.status(), "Expected sphinx front page status < 400").toBeLessThan(400);
|
||||
const contentType = response.headers()["content-type"] || "";
|
||||
expect(
|
||||
contentType.includes("text/html"),
|
||||
`Expected HTML content-type, got "${contentType}"`
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
// Persona scenarios.
|
||||
// Bodies live in the shared helper roles/test-e2e-playwright/files/personas.js
|
||||
// so every role's persona flow stays consistent.
|
||||
|
||||
test("guest: public-landing → auth chain → never authenticated", async ({ page }) => {
|
||||
await runGuestFlow(page);
|
||||
});
|
||||
|
||||
test("biber: app → universal logout", async ({ page }) => {
|
||||
await runBiberFlow(page);
|
||||
});
|
||||
|
||||
test("administrator: app → universal logout", async ({ page }) => {
|
||||
await runAdminFlow(page, {
|
||||
adminInteraction: async (interactivePage) => {
|
||||
// web-app-sphinx admin-only interaction: open a management surface.
|
||||
const link = interactivePage
|
||||
.getByRole("link", { name: /^(admin|build|configuration|projects)$/i })
|
||||
.first();
|
||||
if (await link.isVisible().catch(() => false)) {
|
||||
await link.click({ timeout: resolveTimeout(30_000) }).catch(() => {});
|
||||
await interactivePage.waitForLoadState("domcontentloaded", { timeout: resolveTimeout(30_000) }).catch(() => {});
|
||||
await expect(interactivePage.locator("body")).toContainText(
|
||||
/admin|build|configuration|projects|index/i,
|
||||
{ timeout: resolveTimeout(30_000) },
|
||||
);
|
||||
}
|
||||
},
|
||||
});
|
||||
});
|
||||
@@ -1,6 +0,0 @@
|
||||
---
|
||||
- name: "🧩 load docker, proxy for '{{ application_id }}'"
|
||||
include_role:
|
||||
name: sys-stk-full
|
||||
vars:
|
||||
application_id: "web-app-sphinx"
|
||||
@@ -1,13 +0,0 @@
|
||||
{% include 'roles/sys-svc-compose/templates/base.yml.j2' %}
|
||||
{% set service_name = 'sphinx' %}
|
||||
{{ service_name }}:
|
||||
{% include 'roles/sys-svc-container/templates/base.yml.j2' %}
|
||||
{{ lookup('container_image', application_id, 'sphinx') }}
|
||||
{{ lookup('compose_only', 'container_name', SPHINX_CONTAINER) }}
|
||||
{{ lookup('container_ports', ['sphinx', 'http', DOCKER_BIND_HOST]) | indent(4) }}
|
||||
{{ lookup('container_healthcheck', service_name) | indent(4) }}
|
||||
{{ lookup('container_networks') | indent(4) }}
|
||||
|
||||
{{ lookup('compose_volumes') }}
|
||||
|
||||
{{ lookup('compose_networks') }}
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
application_id: "web-app-sphinx"
|
||||
entity_name: "{{ application_id | get_entity_name }}"
|
||||
|
||||
SPHINX_CONTAINER: "{{ lookup('config', application_id, 'services.' ~ entity_name ~ '.name') }}"
|
||||
@@ -13,10 +13,9 @@ function getExportName(slug) {
|
||||
.join('');
|
||||
}
|
||||
|
||||
// Root: redirect to documentation (only if web-app-sphinx is available)
|
||||
app.get('/', (req, res) => {
|
||||
{% if 'web-app-sphinx' in group_names %}
|
||||
res.redirect('{{ lookup("tls", "web-app-sphinx", "url.base") }}{{ application_id | rel_role_path_by_application_id }}/README.html');
|
||||
{% if 'web-app-docs' in group_names %}
|
||||
res.redirect('{{ lookup("tls", "web-app-docs", "url.base") }}{{ application_id | rel_role_path_by_application_id }}/README.html');
|
||||
{% else %}
|
||||
res
|
||||
.status(200)
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
const test = require("node:test");
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs");
|
||||
const path = require("node:path");
|
||||
const vm = require("node:vm");
|
||||
|
||||
const SOURCE = path.join(
|
||||
__dirname,
|
||||
"..", "..", "..", "..", "..", "..",
|
||||
"roles", "web-app-docs", "files", "javascript", "current-nav.js",
|
||||
);
|
||||
|
||||
function node(tag, parent = null, href = null) {
|
||||
const classes = new Set();
|
||||
const el = {
|
||||
tag,
|
||||
parentElement: parent,
|
||||
style: { display: "" },
|
||||
children: [],
|
||||
classList: {
|
||||
add: (name) => classes.add(name),
|
||||
remove: (name) => classes.delete(name),
|
||||
contains: (name) => classes.has(name),
|
||||
},
|
||||
getAttribute: (name) => (name === "href" ? href : null),
|
||||
closest(selector) {
|
||||
let current = el;
|
||||
while (current && current.tag !== selector) {
|
||||
current = current.parentElement;
|
||||
}
|
||||
return current;
|
||||
},
|
||||
querySelectorAll: () => el.children,
|
||||
};
|
||||
return el;
|
||||
}
|
||||
|
||||
function page(hash, hrefs, alreadyCurrent = []) {
|
||||
const outer = node("li", node("ul"));
|
||||
const submenu = node("div", outer);
|
||||
outer.children.push(submenu);
|
||||
const inner = node("li", node("ul", outer));
|
||||
const links = hrefs.map((href) => node("a", inner, href));
|
||||
alreadyCurrent.forEach((index) => links[index].classList.add("current"));
|
||||
const items = [outer, inner];
|
||||
|
||||
let ready = null;
|
||||
const document = {
|
||||
addEventListener: (type, handler) => {
|
||||
if (type === "DOMContentLoaded") {
|
||||
ready = handler;
|
||||
}
|
||||
},
|
||||
querySelectorAll(selector) {
|
||||
if (selector === ".current-index a.reference.internal") {
|
||||
return links;
|
||||
}
|
||||
if (selector === ".current-index a.reference.internal.current") {
|
||||
return links.filter((link) => link.classList.contains("current"));
|
||||
}
|
||||
if (selector === ".current-index li.current") {
|
||||
return items.filter((item) => item.classList.contains("current"));
|
||||
}
|
||||
return [];
|
||||
},
|
||||
};
|
||||
const window = { location: { hash }, addEventListener() {} };
|
||||
|
||||
vm.runInNewContext(fs.readFileSync(SOURCE, "utf8"), { document, window });
|
||||
ready();
|
||||
window.initCurrentNav();
|
||||
return { outer, inner, submenu, links };
|
||||
}
|
||||
|
||||
test("a hash-only link matching the location hash becomes the only current link", () => {
|
||||
const { outer, inner, submenu, links } = page("#setup", ["#other", "#setup"], [0]);
|
||||
|
||||
assert.equal(links[1].classList.contains("current"), true);
|
||||
assert.equal(links[0].classList.contains("current"), false);
|
||||
assert.equal(inner.classList.contains("current"), true);
|
||||
assert.equal(outer.classList.contains("current"), true);
|
||||
assert.equal(submenu.style.display, "block");
|
||||
});
|
||||
|
||||
test("a page link carrying the location hash marks its list items current", () => {
|
||||
const { outer, inner, links } = page("#setup", ["guide.html#setup"]);
|
||||
|
||||
assert.equal(inner.classList.contains("current"), true);
|
||||
assert.equal(outer.classList.contains("current"), true);
|
||||
assert.equal(links[0].classList.contains("current"), false);
|
||||
});
|
||||
|
||||
test("without a location hash nothing is marked", () => {
|
||||
const { outer, inner, submenu } = page("", ["#setup"]);
|
||||
|
||||
assert.equal(inner.classList.contains("current"), false);
|
||||
assert.equal(outer.classList.contains("current"), false);
|
||||
assert.equal(submenu.style.display, "");
|
||||
});
|
||||
@@ -0,0 +1,153 @@
|
||||
const test = require("node:test");
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs");
|
||||
const path = require("node:path");
|
||||
const vm = require("node:vm");
|
||||
|
||||
const SOURCE = path.join(
|
||||
__dirname,
|
||||
"..", "..", "..", "..", "..", "..",
|
||||
"roles", "web-app-docs", "files", "javascript", "versions.js",
|
||||
);
|
||||
|
||||
const VERSIONS = [
|
||||
{ name: "latest", built: true, state: "building", progress: 42, phase: "sphinx", log: [] },
|
||||
{ name: "v14.0.0", built: true, state: "ready", progress: 100, phase: "", log: [] },
|
||||
{ name: "v13.0.0", built: false, state: "failed", progress: 12, phase: "sphinx", log: ["a", "b"] },
|
||||
{ name: "v11.6.0", built: false, state: "missing", progress: 0, phase: "", log: [] },
|
||||
];
|
||||
|
||||
function element(tag, fields = {}) {
|
||||
return {
|
||||
tag,
|
||||
children: [],
|
||||
dataset: {},
|
||||
value: "",
|
||||
textContent: "",
|
||||
listeners: {},
|
||||
...fields,
|
||||
append(...nodes) {
|
||||
this.children.push(...nodes);
|
||||
},
|
||||
replaceChildren(...nodes) {
|
||||
this.children = nodes;
|
||||
},
|
||||
addEventListener(type, handler) {
|
||||
this.listeners[type] = handler;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function load(selectors = {}) {
|
||||
let ready = null;
|
||||
const window = {
|
||||
reloaded: 0,
|
||||
intervals: [],
|
||||
location: {
|
||||
href: "",
|
||||
reload() {
|
||||
window.reloaded += 1;
|
||||
},
|
||||
},
|
||||
setInterval(handler, ms) {
|
||||
window.intervals.push(ms);
|
||||
},
|
||||
};
|
||||
const document = {
|
||||
addEventListener: (type, handler) => {
|
||||
if (type === "DOMContentLoaded") {
|
||||
ready = handler;
|
||||
}
|
||||
},
|
||||
createElement: (tag) => element(tag),
|
||||
createTextNode: (text) => ({ text }),
|
||||
querySelectorAll: (selector) => selectors[selector] || [],
|
||||
querySelector: (selector) =>
|
||||
selector
|
||||
.split(",")
|
||||
.map((part) => (selectors[part.trim()] || [])[0])
|
||||
.find(Boolean) || null,
|
||||
};
|
||||
const fetch = async (url) => ({ json: async () => (url === "/api/versions" ? VERSIONS : null) });
|
||||
vm.runInNewContext(fs.readFileSync(SOURCE, "utf8"), { document, window, fetch });
|
||||
return { api: window.docsVersions, window, start: () => ready() };
|
||||
}
|
||||
|
||||
test("urls keep the page when switching versions", () => {
|
||||
const { api } = load();
|
||||
assert.equal(api.versionUrl("v14.0.0", "docs/index"), "/v14.0.0/docs/index.html");
|
||||
assert.equal(api.versionUrl("latest"), "/latest/");
|
||||
});
|
||||
|
||||
test("labels tell built, building and failed versions apart", () => {
|
||||
const { api } = load();
|
||||
assert.equal(api.optionLabel(VERSIONS[1]), "v14.0.0");
|
||||
assert.equal(api.optionLabel(VERSIONS[3]), "v11.6.0 (not built yet)");
|
||||
assert.equal(api.describe(VERSIONS[0]), "building: sphinx");
|
||||
assert.equal(api.describe(VERSIONS[2]), "failed: sphinx");
|
||||
assert.equal(api.describe(VERSIONS[1]), "ready");
|
||||
});
|
||||
|
||||
test("switcher lists every version and selects the current one", () => {
|
||||
const { api } = load();
|
||||
const select = element("select", { dataset: { current: "v14.0.0", page: "roles/index" } });
|
||||
|
||||
api.fillSwitcher(select, VERSIONS);
|
||||
|
||||
assert.deepEqual(
|
||||
select.children.map((option) => [option.value, option.selected]),
|
||||
[
|
||||
["/latest/roles/index.html", false],
|
||||
["/v14.0.0/roles/index.html", true],
|
||||
["/v13.0.0/roles/index.html", false],
|
||||
["/v11.6.0/roles/index.html", false],
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
test("overview renders one row with link, status and bar per version", () => {
|
||||
const { api } = load();
|
||||
const tbody = element("tbody");
|
||||
|
||||
api.fillOverview(tbody, VERSIONS);
|
||||
|
||||
const cells = tbody.children[0].children.map((cell) => cell.children[0]);
|
||||
assert.equal(tbody.children.length, 4);
|
||||
assert.equal(cells[0].href, "/latest/");
|
||||
assert.equal(cells[1].text, "building: sphinx");
|
||||
assert.equal(cells[2].value, 42);
|
||||
});
|
||||
|
||||
test("build page shows progress and the failure log, reloads once built", () => {
|
||||
const { api, window } = load();
|
||||
const bar = element("progress");
|
||||
const phase = element("p");
|
||||
const log = element("pre");
|
||||
const parts = { progress: bar, "[data-phase]": phase, pre: log };
|
||||
const section = element("section", { dataset: { build: "v13.0.0" }, querySelector: (s) => parts[s] });
|
||||
|
||||
api.followBuild(section, VERSIONS);
|
||||
assert.deepEqual([bar.value, phase.textContent, log.textContent], [12, "failed: sphinx", "a\nb"]);
|
||||
assert.equal(window.reloaded, 0);
|
||||
|
||||
section.dataset.build = "v14.0.0";
|
||||
api.followBuild(section, VERSIONS);
|
||||
assert.equal(window.reloaded, 1);
|
||||
});
|
||||
|
||||
test("start wires the switcher and polls only on overview and build pages", async () => {
|
||||
const select = element("select", { dataset: { current: "latest", page: "index" } });
|
||||
const sidebarOnly = load({ "select.docs-version-switcher": [select] });
|
||||
sidebarOnly.start();
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
assert.equal(select.children.length, 4);
|
||||
assert.deepEqual(sidebarOnly.window.intervals, []);
|
||||
select.value = "/v14.0.0/index.html";
|
||||
select.listeners.change();
|
||||
assert.equal(sidebarOnly.window.location.href, "/v14.0.0/index.html");
|
||||
|
||||
const overview = load({ "tbody[data-versions]": [element("tbody")] });
|
||||
overview.start();
|
||||
assert.deepEqual(overview.window.intervals, [2000]);
|
||||
});
|
||||
Whitespace-only changes.
Whitespace-only changes.
@@ -0,0 +1,3 @@
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT: Path = Path(__file__).resolve().parents[7]
|
||||
@@ -0,0 +1,3 @@
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT: Path = Path(__file__).resolve().parents[8]
|
||||
+70
@@ -0,0 +1,70 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from types import SimpleNamespace
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
local_file_headings = importlib.import_module(
|
||||
"infinito_docs.extensions.local_file_headings"
|
||||
)
|
||||
|
||||
|
||||
def _flatten(items):
|
||||
for item in items:
|
||||
yield item
|
||||
yield from _flatten(item.get("children", []))
|
||||
|
||||
|
||||
class TestLocalFileHeadings(unittest.TestCase):
|
||||
def test_collects_the_headings_of_the_page_directory(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
docs = Path(td) / "docs"
|
||||
docs.mkdir()
|
||||
(docs / "index.rst").write_text("Index\n=====\n", encoding="utf-8")
|
||||
(docs / "readme.md").write_text("# Readme\n", encoding="utf-8")
|
||||
(docs / "a.md").write_text("# A\n## A1\n", encoding="utf-8")
|
||||
context = {}
|
||||
|
||||
local_file_headings.add_local_file_headings(
|
||||
SimpleNamespace(srcdir=td), "docs/index", "page.html", context, None
|
||||
)
|
||||
|
||||
items = {item["text"]: item for item in _flatten(context["local_md_headings"])}
|
||||
self.assertEqual(set(items), {"Index", "A", "A1"})
|
||||
self.assertEqual(items["A1"]["link"], "docs/a")
|
||||
self.assertEqual(items["A1"]["anchor"], "a1")
|
||||
self.assertEqual(context["local_md_headings"][0]["text"], "Index")
|
||||
|
||||
def test_top_level_page_links_without_a_directory_prefix(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
(Path(td) / "guide.md").write_text("# Guide\n", encoding="utf-8")
|
||||
context = {}
|
||||
|
||||
local_file_headings.add_local_file_headings(
|
||||
SimpleNamespace(srcdir=td), "guide", "page.html", context, None
|
||||
)
|
||||
|
||||
self.assertEqual(context["local_md_headings"][0]["link"], "guide")
|
||||
|
||||
def test_missing_page_directory_yields_no_headings(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
context = {}
|
||||
|
||||
local_file_headings.add_local_file_headings(
|
||||
SimpleNamespace(srcdir=td), "gone/index", "page.html", context, None
|
||||
)
|
||||
|
||||
self.assertEqual(context["local_md_headings"], [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+114
@@ -0,0 +1,114 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from types import SimpleNamespace
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
local_subfolders = importlib.import_module("infinito_docs.extensions.local_subfolders")
|
||||
|
||||
|
||||
def _tree(root: Path) -> None:
|
||||
(root / "index.rst").write_text("Root\n====\n", encoding="utf-8")
|
||||
(root / "a.md").write_text("# A\n", encoding="utf-8")
|
||||
(root / "sub").mkdir()
|
||||
(root / "sub" / "readme.md").write_text("# Sub\n", encoding="utf-8")
|
||||
(root / "sub" / "x.rst").write_text("X\n===\n", encoding="utf-8")
|
||||
(root / "untitled").mkdir()
|
||||
(root / "untitled" / "loose.md").write_text("# Loose\n", encoding="utf-8")
|
||||
(root / ".hidden").mkdir()
|
||||
(root / ".hidden" / "index.rst").write_text("Hidden\n======\n", encoding="utf-8")
|
||||
|
||||
|
||||
class TestLocalSubfolders(unittest.TestCase):
|
||||
def test_tree_titles_folders_by_their_representative_file(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
_tree(Path(td))
|
||||
|
||||
tree = local_subfolders.collect_folder_tree(td, "")
|
||||
|
||||
self.assertEqual((tree["text"], tree["link"]), ("Root", "index"))
|
||||
self.assertEqual(
|
||||
[(child["text"], child["link"]) for child in tree["children"]],
|
||||
[("A", "a"), ("Sub", "sub/readme")],
|
||||
)
|
||||
self.assertEqual(
|
||||
[
|
||||
(child["text"], child["link"])
|
||||
for child in tree["children"][1]["children"]
|
||||
],
|
||||
[("X", "sub/x")],
|
||||
)
|
||||
|
||||
def test_folder_without_representative_file_is_skipped(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
(Path(td) / "a.md").write_text("# A\n", encoding="utf-8")
|
||||
|
||||
self.assertIsNone(local_subfolders.collect_folder_tree(td, ""))
|
||||
|
||||
def test_mark_current_propagates_to_ancestors_only(self) -> None:
|
||||
node = {
|
||||
"link": "index",
|
||||
"children": [
|
||||
{"link": "child/index", "children": []},
|
||||
{"link": "other", "children": []},
|
||||
],
|
||||
}
|
||||
|
||||
local_subfolders.mark_current(node, "child/index")
|
||||
|
||||
self.assertEqual(
|
||||
[
|
||||
node["current"],
|
||||
node["children"][0]["current"],
|
||||
node["children"][1]["current"],
|
||||
],
|
||||
[True, True, False],
|
||||
)
|
||||
|
||||
def test_page_context_receives_the_marked_tree(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
_tree(Path(td))
|
||||
context = {}
|
||||
|
||||
local_subfolders.add_local_subfolders(
|
||||
SimpleNamespace(srcdir=td), "sub/x", "page.html", context, None
|
||||
)
|
||||
|
||||
(root,) = context["local_subfolders"]
|
||||
self.assertTrue(root["current"])
|
||||
self.assertTrue(root["children"][1]["children"][0]["current"])
|
||||
self.assertFalse(root["children"][0]["current"])
|
||||
|
||||
def test_tree_is_walked_once_and_pages_keep_their_own_flags(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
_tree(Path(td))
|
||||
app = SimpleNamespace(srcdir=td)
|
||||
first, second = {}, {}
|
||||
before = local_subfolders._source_tree.cache_info()
|
||||
|
||||
local_subfolders.add_local_subfolders(
|
||||
app, "sub/x", "page.html", first, None
|
||||
)
|
||||
local_subfolders.add_local_subfolders(app, "a", "page.html", second, None)
|
||||
|
||||
after = local_subfolders._source_tree.cache_info()
|
||||
|
||||
self.assertEqual(
|
||||
(after.misses - before.misses, after.hits - before.hits), (1, 1)
|
||||
)
|
||||
self.assertTrue(first["local_subfolders"][0]["children"][1]["current"])
|
||||
self.assertFalse(second["local_subfolders"][0]["children"][1]["current"])
|
||||
self.assertTrue(second["local_subfolders"][0]["children"][0]["current"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+101
@@ -0,0 +1,101 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from types import SimpleNamespace
|
||||
|
||||
from docutils import nodes
|
||||
from docutils.utils import new_document
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
markdown_include = importlib.import_module("infinito_docs.extensions.markdown_include")
|
||||
|
||||
|
||||
class _Reporter:
|
||||
def error(self, message, *args, **kwargs):
|
||||
return ("ERROR", message)
|
||||
|
||||
|
||||
class _Env:
|
||||
def __init__(self, base: Path) -> None:
|
||||
self._base = base
|
||||
|
||||
def relfn2path(self, name):
|
||||
return name, str(self._base / name)
|
||||
|
||||
|
||||
class _App:
|
||||
def __init__(self) -> None:
|
||||
self.added = {}
|
||||
|
||||
def add_directive(self, name, directive) -> None:
|
||||
self.added[name] = directive
|
||||
|
||||
|
||||
def _section(*children):
|
||||
section = nodes.section()
|
||||
section += list(children)
|
||||
doc = new_document("page.md")
|
||||
doc += section
|
||||
return doc
|
||||
|
||||
|
||||
class TestMarkdownInclude(unittest.TestCase):
|
||||
def test_setup_registers_the_directive(self) -> None:
|
||||
app = _App()
|
||||
markdown_include.setup(app)
|
||||
self.assertIs(
|
||||
app.added["markdown-include"], markdown_include.MarkdownIncludeDirective
|
||||
)
|
||||
|
||||
def test_missing_file_reports_an_error(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
directive = markdown_include.MarkdownIncludeDirective(
|
||||
name="markdown-include",
|
||||
arguments=["nope.md"],
|
||||
options={},
|
||||
content=[],
|
||||
lineno=1,
|
||||
content_offset=0,
|
||||
block_text=".. markdown-include:: nope.md",
|
||||
state=SimpleNamespace(
|
||||
document=SimpleNamespace(
|
||||
settings=SimpleNamespace(env=_Env(Path(td)))
|
||||
)
|
||||
),
|
||||
state_machine=SimpleNamespace(reporter=_Reporter()),
|
||||
)
|
||||
|
||||
(result,) = directive.run()
|
||||
|
||||
self.assertEqual(result[0], "ERROR")
|
||||
self.assertIn("File not found", result[1])
|
||||
|
||||
def test_title_is_dropped_and_its_section_unwrapped(self) -> None:
|
||||
paragraph = nodes.paragraph(text="Body")
|
||||
doc = _section(nodes.title(text="README"), paragraph)
|
||||
|
||||
markdown_include._strip_title(doc)
|
||||
|
||||
self.assertEqual(doc.children, [paragraph])
|
||||
|
||||
def test_lone_title_is_emptied_and_unwrapped(self) -> None:
|
||||
title = nodes.title(text="README")
|
||||
doc = _section(title)
|
||||
|
||||
markdown_include._strip_title(doc)
|
||||
|
||||
self.assertEqual(doc.children, [title])
|
||||
self.assertEqual(title.astext(), "")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,79 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
nav_utils = importlib.import_module("infinito_docs.extensions.nav_utils")
|
||||
|
||||
|
||||
class TestNavUtils(unittest.TestCase):
|
||||
def test_natural_sort_orders_numbers_by_value(self) -> None:
|
||||
self.assertLess(
|
||||
nav_utils.natural_sort_key("file2"), nav_utils.natural_sort_key("file10")
|
||||
)
|
||||
|
||||
def test_markdown_headings_skip_fenced_code(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
page = Path(td) / "x.md"
|
||||
page.write_text(
|
||||
"# Title\n```\n# Not a heading\n```\n## Sub Page\n", encoding="utf-8"
|
||||
)
|
||||
|
||||
headings = nav_utils.extract_headings_from_file(page)
|
||||
|
||||
self.assertEqual(
|
||||
[(h["level"], h["text"], h["anchor"]) for h in headings],
|
||||
[(1, "Title", "title"), (2, "Sub Page", "sub-page")],
|
||||
)
|
||||
|
||||
def test_max_level_drops_deeper_markdown_headings(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
page = Path(td) / "x.md"
|
||||
page.write_text("# One\n## Two\n### Three\n", encoding="utf-8")
|
||||
|
||||
headings = nav_utils.extract_headings_from_file(page, max_level=2)
|
||||
|
||||
self.assertEqual([h["text"] for h in headings], ["One", "Two"])
|
||||
|
||||
def test_rst_heading_needs_an_underline_of_three(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
page = Path(td) / "x.rst"
|
||||
page.write_text("Header\n======\n\nShort\n--\n", encoding="utf-8")
|
||||
|
||||
headings = nav_utils.extract_headings_from_file(page)
|
||||
|
||||
self.assertEqual([h["text"] for h in headings], ["Header"])
|
||||
|
||||
def test_index_without_headings_falls_back_to_readme(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
(Path(td) / "index.rst").write_text(".. toctree::\n", encoding="utf-8")
|
||||
(Path(td) / "README.md").write_text("# Folder\n", encoding="utf-8")
|
||||
|
||||
headings = nav_utils.extract_headings_from_file(Path(td) / "index.rst")
|
||||
|
||||
self.assertEqual([h["text"] for h in headings], ["Folder"])
|
||||
|
||||
def test_group_nests_by_level_and_sort_puts_priority_first(self) -> None:
|
||||
headings = [
|
||||
{"level": 1, "text": "B", "anchor": "b", "priority": 1, "filename": "b"},
|
||||
{"level": 2, "text": "B1", "anchor": "b1", "priority": 1, "filename": "b"},
|
||||
{"level": 1, "text": "A", "anchor": "a", "priority": 0, "filename": "z"},
|
||||
]
|
||||
tree = nav_utils.group_headings(headings)
|
||||
nav_utils.sort_tree(tree)
|
||||
|
||||
self.assertEqual([item["text"] for item in tree], ["A", "B"])
|
||||
self.assertEqual([item["text"] for item in tree[1]["children"]], ["B1"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+99
@@ -0,0 +1,99 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
from utils.cache.files import read_text
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
parallel_postprocess = importlib.import_module(
|
||||
"infinito_docs.extensions.parallel_postprocess"
|
||||
)
|
||||
|
||||
|
||||
class _App:
|
||||
def __init__(self, listeners=()) -> None:
|
||||
self.connected = []
|
||||
self.disconnected = []
|
||||
self.events = SimpleNamespace(listeners={"build-finished": list(listeners)})
|
||||
|
||||
def connect(self, event, handler) -> None:
|
||||
self.connected.append((event, handler))
|
||||
|
||||
def disconnect(self, listener_id) -> None:
|
||||
self.disconnected.append(listener_id)
|
||||
|
||||
|
||||
def _mark_done(path, app) -> None:
|
||||
Path(path).write_text(f"done by {app.builder.name}", encoding="utf-8")
|
||||
|
||||
|
||||
class TestParallelPostprocess(unittest.TestCase):
|
||||
def test_setup_swaps_the_postprocess_once_the_builder_exists(self) -> None:
|
||||
app = _App()
|
||||
self.assertTrue(parallel_postprocess.setup(app)["parallel_read_safe"])
|
||||
self.assertEqual(
|
||||
app.connected,
|
||||
[("builder-inited", parallel_postprocess.use_parallel_postprocess)],
|
||||
)
|
||||
|
||||
def test_theme_postprocess_is_replaced_by_the_parallel_one(self) -> None:
|
||||
theme = SimpleNamespace(
|
||||
id=7, handler=parallel_postprocess.postprocess.post_process_html
|
||||
)
|
||||
other = SimpleNamespace(id=8, handler=print)
|
||||
app = _App([theme, other])
|
||||
|
||||
parallel_postprocess.use_parallel_postprocess(app)
|
||||
|
||||
self.assertEqual(app.disconnected, [7])
|
||||
self.assertEqual(
|
||||
app.connected,
|
||||
[("build-finished", parallel_postprocess.parallel_post_process_html)],
|
||||
)
|
||||
|
||||
def test_every_changed_page_is_rewritten_and_failed_builds_are_left(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
pages = [Path(td) / f"{doc}.html" for doc in ("a", "b", "c")]
|
||||
for page in pages:
|
||||
page.write_text("raw", encoding="utf-8")
|
||||
app = SimpleNamespace(
|
||||
builder=SimpleNamespace(
|
||||
name="html",
|
||||
get_outfilename=lambda doc: str(Path(td) / f"{doc}.html"),
|
||||
),
|
||||
env=SimpleNamespace(awesome_changed_docs=["a", "b", "c"]),
|
||||
parallel=2,
|
||||
verbosity=0,
|
||||
)
|
||||
|
||||
with patch.object(
|
||||
parallel_postprocess.postprocess, "modify_html", _mark_done
|
||||
):
|
||||
parallel_postprocess.parallel_post_process_html(
|
||||
app, RuntimeError("build failed")
|
||||
)
|
||||
untouched = [read_text(str(page)) for page in pages]
|
||||
parallel_postprocess.parallel_post_process_html(app, None)
|
||||
|
||||
rewritten = [
|
||||
page.read_text(encoding="utf-8") # nocheck: cache-read
|
||||
for page in pages
|
||||
]
|
||||
|
||||
self.assertEqual(untouched, ["raw"] * 3)
|
||||
self.assertEqual(rewritten, ["done by html"] * 3)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+97
@@ -0,0 +1,97 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import json
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from types import SimpleNamespace
|
||||
|
||||
from docutils import nodes
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
roles_overview = importlib.import_module("infinito_docs.extensions.roles_overview")
|
||||
|
||||
|
||||
class _Reporter:
|
||||
def error(self, message, *args, **kwargs):
|
||||
return ("ERROR", message)
|
||||
|
||||
|
||||
def _directive(srcdir: str):
|
||||
return roles_overview.RolesOverviewDirective(
|
||||
name="roles-overview",
|
||||
arguments=[],
|
||||
options={},
|
||||
content=[],
|
||||
lineno=1,
|
||||
content_offset=0,
|
||||
block_text=".. roles-overview::",
|
||||
state=SimpleNamespace(
|
||||
document=SimpleNamespace(
|
||||
settings=SimpleNamespace(env=SimpleNamespace(srcdir=srcdir)),
|
||||
reporter=_Reporter(),
|
||||
)
|
||||
),
|
||||
state_machine=SimpleNamespace(reporter=_Reporter()),
|
||||
)
|
||||
|
||||
|
||||
def _role(name, title, description):
|
||||
return {
|
||||
"name": name,
|
||||
"title": title,
|
||||
"description": description,
|
||||
"link": f"roles/{name}/README.md",
|
||||
"tags": [],
|
||||
}
|
||||
|
||||
|
||||
class TestRolesOverview(unittest.TestCase):
|
||||
def test_categories_and_roles_render_sorted_with_links(self) -> None:
|
||||
overview = {
|
||||
"wiki": [_role("web-app-demo", "Demo Title", "Demo role")],
|
||||
"Docs": [
|
||||
_role("web-app-zeta", "Zeta", ""),
|
||||
_role("web-app-demo", "Demo Title", "Demo role"),
|
||||
],
|
||||
}
|
||||
with TemporaryDirectory() as td:
|
||||
target = Path(td) / roles_overview.OVERVIEW_FILE
|
||||
target.parent.mkdir(parents=True)
|
||||
target.write_text(json.dumps(overview), encoding="utf-8")
|
||||
|
||||
(container,) = _directive(td).run()
|
||||
|
||||
sections = {
|
||||
section[0].astext(): [role[0].astext() for role in section[1:]]
|
||||
for section in container.children
|
||||
}
|
||||
self.assertEqual(
|
||||
sections, {"Docs": ["Demo Title", "Zeta"], "wiki": ["Demo Title"]}
|
||||
)
|
||||
self.assertEqual(
|
||||
[section[0].astext() for section in container.children], ["Docs", "wiki"]
|
||||
)
|
||||
reference = next(iter(container.findall(nodes.reference)))
|
||||
self.assertEqual(reference["refuri"], "roles/web-app-demo/README.md")
|
||||
self.assertEqual(
|
||||
[paragraph.astext() for paragraph in container.findall(nodes.paragraph)],
|
||||
["Demo role", "Demo role"],
|
||||
)
|
||||
|
||||
def test_missing_overview_reports_an_error(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
(result,) = _directive(td).run()
|
||||
|
||||
self.assertEqual(result, ("ERROR", "Roles overview not generated."))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,3 @@
|
||||
from pathlib import Path
|
||||
|
||||
PROJECT_ROOT: Path = Path(__file__).resolve().parents[8]
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from utils.cache.files import read_text
|
||||
from utils.roles.mapping import ROLE_FILE_META_MAIN, ROLE_FILE_README
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
ansible_roles = importlib.import_module("infinito_docs.generators.ansible_roles")
|
||||
|
||||
|
||||
class TestAnsibleRoles(unittest.TestCase):
|
||||
@patch.object(ansible_roles.subprocess, "run")
|
||||
def test_markdown_is_converted_through_pandoc(self, run: MagicMock) -> None:
|
||||
run.return_value = MagicMock(stdout=b"RST\n")
|
||||
|
||||
self.assertEqual(ansible_roles.convert_md_to_rst("# Title\n"), "RST\n")
|
||||
self.assertEqual(
|
||||
run.call_args.args[0], ["pandoc", "-f", "markdown", "-t", "rst"]
|
||||
)
|
||||
|
||||
@patch.object(ansible_roles.subprocess, "run")
|
||||
def test_failed_conversion_keeps_the_markdown(self, run: MagicMock) -> None:
|
||||
run.side_effect = subprocess.CalledProcessError(1, ["pandoc"])
|
||||
|
||||
self.assertEqual(ansible_roles.convert_md_to_rst("# Title\n"), "# Title\n")
|
||||
|
||||
@patch.object(ansible_roles.subprocess, "run")
|
||||
def test_one_page_per_role_with_meta(self, run: MagicMock) -> None:
|
||||
run.return_value = MagicMock(stdout=b"Converted README\n")
|
||||
|
||||
with TemporaryDirectory() as td:
|
||||
roles = Path(td) / "roles"
|
||||
meta_file = roles / "web-app-demo" / ROLE_FILE_META_MAIN
|
||||
meta_file.parent.mkdir(parents=True)
|
||||
meta_file.write_text(
|
||||
"galaxy_info:\n description: Demo role\n license: MIT\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
(roles / "web-app-demo" / ROLE_FILE_README).write_text(
|
||||
"# Demo\n", encoding="utf-8"
|
||||
)
|
||||
(roles / "no-meta").mkdir()
|
||||
|
||||
ansible_roles.generate_ansible_roles_doc(roles, Path(td) / "out")
|
||||
|
||||
self.assertEqual(
|
||||
sorted(p.name for p in (Path(td) / "out").iterdir()),
|
||||
["web-app-demo.rst"],
|
||||
)
|
||||
page = read_text(str(Path(td) / "out" / "web-app-demo.rst"))
|
||||
|
||||
self.assertTrue(page.startswith("Web-app-demo Role\n" + "=" * 19 + "\n\n"))
|
||||
self.assertIn("**Description:** Demo role\n", page)
|
||||
self.assertIn("- **license**: MIT\n", page)
|
||||
self.assertTrue(page.endswith("\nREADME\n------\n\nConverted README\n"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,50 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
from utils.cache.files import read_text
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
index = importlib.import_module("infinito_docs.generators.index")
|
||||
|
||||
|
||||
class TestIndex(unittest.TestCase):
|
||||
def test_toctree_lists_every_page_relative_to_the_index(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
pages = Path(td) / "generated" / "roles"
|
||||
pages.mkdir(parents=True)
|
||||
(pages / "b.rst").write_text("b", encoding="utf-8")
|
||||
(pages / "a.rst").write_text("a", encoding="utf-8")
|
||||
(pages / "README.md").write_text("# skip", encoding="utf-8")
|
||||
output = Path(td) / "roles" / "ansible_role_glosar.rst"
|
||||
|
||||
index.generate_ansible_roles_index(pages, output, "Ansible Role Glossary")
|
||||
text = read_text(str(output))
|
||||
|
||||
self.assertEqual(
|
||||
text,
|
||||
"Ansible Role Glossary\n===================\n\n"
|
||||
".. toctree::\n :maxdepth: 1\n :caption: Ansible Role Glossary\n\n"
|
||||
" ../generated/roles/a\n ../generated/roles/b\n",
|
||||
)
|
||||
|
||||
def test_missing_pages_directory_writes_nothing(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
output = Path(td) / "index.rst"
|
||||
|
||||
index.generate_ansible_roles_index(Path(td) / "absent", output, "Caption")
|
||||
|
||||
self.assertFalse(output.exists())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,46 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
from utils.cache.files import read_text
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
readmes = importlib.import_module("infinito_docs.generators.readmes")
|
||||
|
||||
|
||||
class TestReadmes(unittest.TestCase):
|
||||
def test_every_nested_directory_gets_a_titled_readme(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
generated = Path(td) / "generated"
|
||||
(generated / "a").mkdir(parents=True)
|
||||
(generated / "b" / "c").mkdir(parents=True)
|
||||
|
||||
readmes.create_readme_in_subdirs(generated)
|
||||
|
||||
titles = {
|
||||
name: read_text(str(generated / name / "README.md")).splitlines()[0]
|
||||
for name in ("a", "b", "b/c")
|
||||
}
|
||||
self.assertFalse((generated / "README.md").exists())
|
||||
|
||||
self.assertEqual(
|
||||
titles,
|
||||
{
|
||||
"a": "# Auto Generated Technical Documentation: a",
|
||||
"b": "# Auto Generated Technical Documentation: b",
|
||||
"b/c": "# Auto Generated Technical Documentation: c",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import json
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
from utils.cache.files import read_text
|
||||
from utils.roles.mapping import ROLE_FILE_META_MAIN, ROLE_FILE_README
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
roles_overview = importlib.import_module("infinito_docs.generators.roles_overview")
|
||||
|
||||
|
||||
def _role(roles_dir: Path, name: str, meta: str, readme: str | None = None) -> None:
|
||||
meta_file = roles_dir / name / ROLE_FILE_META_MAIN
|
||||
meta_file.parent.mkdir(parents=True)
|
||||
meta_file.write_text(meta, encoding="utf-8")
|
||||
if readme is not None:
|
||||
(roles_dir / name / ROLE_FILE_README).write_text(readme, encoding="utf-8")
|
||||
|
||||
|
||||
class TestRolesOverviewGenerator(unittest.TestCase):
|
||||
def test_roles_are_grouped_by_tag_and_titled_from_their_readme(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
roles = Path(td) / "roles"
|
||||
_role(
|
||||
roles,
|
||||
"web-app-demo",
|
||||
"galaxy_info:\n description: Demo role\n galaxy_tags: [wiki, docs]\n",
|
||||
"Intro\n# Demo Title\n\nText\n",
|
||||
)
|
||||
_role(roles, "web-app-bare", "galaxy_info:\n description: Bare\n")
|
||||
(roles / "no-meta").mkdir()
|
||||
(roles / ".hidden").mkdir()
|
||||
output = Path(td) / "out" / "overview.json"
|
||||
|
||||
roles_overview.main(
|
||||
["--roles-dir", str(roles), "--output-file", str(output)]
|
||||
)
|
||||
categories = json.loads(read_text(str(output)))
|
||||
|
||||
self.assertEqual(
|
||||
{
|
||||
tag: [role["title"] for role in roles]
|
||||
for tag, roles in categories.items()
|
||||
},
|
||||
{
|
||||
"wiki": ["Demo Title"],
|
||||
"docs": ["Demo Title"],
|
||||
"uncategorized": ["web-app-bare"],
|
||||
},
|
||||
)
|
||||
self.assertEqual(
|
||||
categories["wiki"][0],
|
||||
{
|
||||
"name": "web-app-demo",
|
||||
"title": "Demo Title",
|
||||
"description": "Demo role",
|
||||
"link": "roles/web-app-demo/README.md",
|
||||
"tags": ["wiki", "docs"],
|
||||
},
|
||||
)
|
||||
|
||||
def test_unreadable_meta_is_skipped(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
roles = Path(td) / "roles"
|
||||
_role(roles, "web-app-broken", "galaxy_info: [unclosed\n")
|
||||
_role(roles, "web-app-list", "- not a mapping\n")
|
||||
|
||||
self.assertEqual(roles_overview.collect_roles_by_tag(roles), {})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+54
@@ -0,0 +1,54 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
|
||||
from utils.cache.files import read_text
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
yaml_index = importlib.import_module("infinito_docs.generators.yaml_index")
|
||||
|
||||
|
||||
class TestYamlIndex(unittest.TestCase):
|
||||
def test_includes_every_yaml_file_the_gitignore_keeps(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
source = Path(td) / "repo"
|
||||
(source / "roles" / "x").mkdir(parents=True)
|
||||
(source / "build").mkdir()
|
||||
(source / ".gitignore").write_text(
|
||||
"ignored.yml\nbuild/*\n", encoding="utf-8"
|
||||
)
|
||||
(source / "ok.yaml").write_text("a: 1\n", encoding="utf-8")
|
||||
(source / "roles" / "x" / "main.yml").write_text("b: 2\n", encoding="utf-8")
|
||||
(source / "ignored.yml").write_text("c: 3\n", encoding="utf-8")
|
||||
(source / "build" / "cache.yml").write_text("d: 4\n", encoding="utf-8")
|
||||
(source / "notes.txt").write_text("e", encoding="utf-8")
|
||||
output = Path(td) / "out" / "yaml_index.rst"
|
||||
|
||||
yaml_index.generate_yaml_index(source, output)
|
||||
text = read_text(str(output))
|
||||
|
||||
self.assertTrue(text.startswith(yaml_index.HEADER))
|
||||
self.assertEqual(
|
||||
[
|
||||
line
|
||||
for line in text.splitlines()
|
||||
if line.startswith(".. literalinclude::")
|
||||
],
|
||||
[
|
||||
".. literalinclude:: ../repo/ok.yaml",
|
||||
".. literalinclude:: ../repo/roles/x/main.yml",
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,74 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
from docutils import nodes
|
||||
from docutils.utils import new_document
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
with patch.object(sys, "argv", ["sphinx-build"]):
|
||||
conf = importlib.import_module("infinito_docs.conf")
|
||||
|
||||
|
||||
class _App:
|
||||
def __init__(self) -> None:
|
||||
self.connected = []
|
||||
self.registry = SimpleNamespace(domains={})
|
||||
|
||||
def connect(self, event, handler) -> None:
|
||||
self.connected.append((event, handler))
|
||||
|
||||
|
||||
class TestConf(unittest.TestCase):
|
||||
def test_every_local_extension_resolves(self) -> None:
|
||||
local = [name for name in conf.extensions if name.startswith("infinito_docs.")]
|
||||
self.assertEqual(len(local), 5)
|
||||
for name in local:
|
||||
self.assertTrue(hasattr(importlib.import_module(name), "setup"), name)
|
||||
|
||||
def test_templates_and_theme_point_at_shipped_files(self) -> None:
|
||||
self.assertEqual(conf.html_theme, "sphinxawesome_theme")
|
||||
for template in conf.html_sidebars["**"]:
|
||||
self.assertTrue(Path(conf.templates_path[0], template).is_file(), template)
|
||||
|
||||
def test_every_documented_suffix_is_parsed(self) -> None:
|
||||
self.assertEqual(set(conf.source_suffix), {".md", ".rst", ".yml", ".yaml"})
|
||||
|
||||
def test_asset_references_are_rewritten_to_static(self) -> None:
|
||||
doctree = new_document("index")
|
||||
image = nodes.image(uri="assets/img/logo.png")
|
||||
raw = nodes.raw("", '<img src="assets/img/favicon.ico">', format="html")
|
||||
doctree += [image, raw]
|
||||
|
||||
conf.replace_assets_in_doctree(None, doctree, "index")
|
||||
|
||||
self.assertEqual(image["uri"], "_static/img/logo.png")
|
||||
rewritten = next(iter(doctree.findall(nodes.raw))).astext()
|
||||
self.assertIn("_static/img/favicon.ico", rewritten)
|
||||
|
||||
def test_setup_hooks_the_asset_rewrite(self) -> None:
|
||||
app = _App()
|
||||
self.assertTrue(conf.setup(app)["parallel_read_safe"])
|
||||
self.assertIn(
|
||||
("doctree-resolved", conf.replace_assets_in_doctree), app.connected
|
||||
)
|
||||
|
||||
def test_built_version_comes_from_the_environment(self) -> None:
|
||||
with patch.dict("os.environ", {"DOCS_VERSION": "v1.2.3"}):
|
||||
reloaded = importlib.reload(conf)
|
||||
self.assertEqual(reloaded.html_context, {"docs_version": "v1.2.3"})
|
||||
self.assertIn("versions.html", reloaded.html_sidebars["**"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,288 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from unittest.mock import patch
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
library = importlib.import_module("infinito_docs.library")
|
||||
|
||||
FAKE_SPHINX = """#!/usr/bin/env python3
|
||||
import os, pathlib, shutil, sys
|
||||
src, out = pathlib.Path(sys.argv[3]), pathlib.Path(sys.argv[4])
|
||||
environment = f"{os.environ.get('PYTHONUNBUFFERED')} {os.environ.get('DOCS_VERSION')}"
|
||||
print("reading sources... [ 50%] index")
|
||||
print("writing output... [100%] index")
|
||||
if (src / "BROKEN").exists():
|
||||
print("Sphinx error: broken source")
|
||||
sys.exit(2)
|
||||
(out / "html" / "docs").mkdir(parents=True)
|
||||
shutil.copy(src / "VERSION", out / "html" / "version.txt")
|
||||
(out / "html" / "index.html").write_text("root")
|
||||
(out / "html" / "docs" / "index.html").write_text("docs")
|
||||
(out / "html" / "environment.txt").write_text(environment)
|
||||
"""
|
||||
|
||||
LOCK_PROBE = """
|
||||
import fcntl, sys
|
||||
with open(sys.argv[1], "a+") as handle:
|
||||
try:
|
||||
fcntl.lockf(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except OSError:
|
||||
sys.exit(1)
|
||||
"""
|
||||
|
||||
|
||||
def _git(repo: Path, *args: str) -> str:
|
||||
return subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"-C",
|
||||
str(repo),
|
||||
"-c",
|
||||
"user.name=t",
|
||||
"-c",
|
||||
"user.email=t@t",
|
||||
"-c",
|
||||
"commit.gpgSign=false",
|
||||
"-c",
|
||||
"tag.gpgSign=false",
|
||||
*args,
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
).stdout.strip()
|
||||
|
||||
|
||||
def _commit(repo: Path, version: str, *tags: str) -> None:
|
||||
(repo / "VERSION").write_text(version, encoding="utf-8")
|
||||
_git(repo, "add", "-A")
|
||||
_git(repo, "commit", "-q", "-m", version)
|
||||
for tag in tags:
|
||||
_git(repo, "tag", tag)
|
||||
|
||||
|
||||
class TestLibrary(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self._tmp = TemporaryDirectory()
|
||||
root = Path(self._tmp.name)
|
||||
self.repo = root / "repo"
|
||||
self.repo.mkdir()
|
||||
_git(self.repo, "init", "-q", "-b", "main")
|
||||
_commit(self.repo, "one", "v1.0.0", "v0.7.0-test")
|
||||
_commit(self.repo, "two", "v1.2.0", "latest")
|
||||
_commit(self.repo, "three")
|
||||
|
||||
self.package = root / "package" / "infinito_docs"
|
||||
self.package.mkdir(parents=True)
|
||||
(self.package / "conf.py").write_text("", encoding="utf-8")
|
||||
|
||||
bin_dir = root / "bin"
|
||||
bin_dir.mkdir()
|
||||
(bin_dir / "sphinx-build").write_text(FAKE_SPHINX, encoding="utf-8")
|
||||
(bin_dir / "sphinx-build").chmod(0o755)
|
||||
path = f"{bin_dir}{os.pathsep}{os.environ['PATH']}"
|
||||
self._env = patch.dict(os.environ, {"PATH": path})
|
||||
self._env.start()
|
||||
self._commands = patch.object(
|
||||
library, "generate_commands", lambda src: [[sys.executable, "-c", "pass"]]
|
||||
)
|
||||
self._commands.start()
|
||||
self._ttl = patch.object(library, "REFS_TTL_SECONDS", 0)
|
||||
self._ttl.start()
|
||||
|
||||
self.data = root / "data"
|
||||
self.library = self._replica()
|
||||
self.library.fetch()
|
||||
|
||||
def tearDown(self) -> None:
|
||||
self._ttl.stop()
|
||||
self._commands.stop()
|
||||
self._env.stop()
|
||||
self._tmp.cleanup()
|
||||
|
||||
def _replica(self):
|
||||
return library.Library(str(self.repo), self.data, 1, self.package)
|
||||
|
||||
def _site(self, version: str, name: str) -> str:
|
||||
page = self.library.sites / version / "html" / name
|
||||
return page.read_text(encoding="utf-8") # nocheck: cache-read
|
||||
|
||||
def _state(self, replica, name: str) -> dict:
|
||||
(entry,) = [e for e in replica.status() if e["name"] == name]
|
||||
return entry
|
||||
|
||||
def test_fetch_lists_latest_then_release_tags_and_queues_latest(self) -> None:
|
||||
self.assertEqual(self.library.versions(), ["latest", "v1.2.0", "v1.0.0"])
|
||||
self.assertEqual(self.library.refs()[0], _git(self.repo, "rev-parse", "HEAD"))
|
||||
self.assertEqual(self.library.next_queued(), "latest")
|
||||
self.assertEqual(self._state(self.library, "latest")["state"], "queued")
|
||||
|
||||
def test_latest_is_built_from_the_last_commit(self) -> None:
|
||||
self.library.build("latest")
|
||||
|
||||
self.assertEqual(self._site("latest", "version.txt"), "three")
|
||||
self.assertEqual(self._site("latest", "environment.txt"), "1 latest")
|
||||
self.assertEqual(self.library.built_ref("latest"), self.library.refs()[0])
|
||||
entry = self._state(self.library, "latest")
|
||||
self.assertEqual((entry["state"], entry["progress"]), ("ready", 100))
|
||||
self.assertEqual(self._state(self.library, "v1.0.0")["state"], "missing")
|
||||
self.assertIsNone(self.library.next_queued())
|
||||
self.assertFalse(list(self.library.scratch.iterdir()))
|
||||
|
||||
def test_tag_is_built_from_its_own_commit_and_only_once(self) -> None:
|
||||
self.library.request("v1.0.0")
|
||||
self.library.build("v1.0.0")
|
||||
|
||||
self.assertEqual(self._site("v1.0.0", "version.txt"), "one")
|
||||
self.library.request("v1.0.0")
|
||||
self.assertFalse((self.library.queue / "v1.0.0").exists())
|
||||
|
||||
def test_new_commit_requeues_latest_while_the_old_site_stays_served(self) -> None:
|
||||
self.library.build("latest")
|
||||
_commit(self.repo, "four")
|
||||
|
||||
self.library.fetch()
|
||||
|
||||
entry = self._state(self.library, "latest")
|
||||
self.assertEqual((entry["state"], entry["built"]), ("queued", True))
|
||||
self.assertEqual(self._site("latest", "version.txt"), "three")
|
||||
self.library.build("latest")
|
||||
self.assertEqual(self._site("latest", "version.txt"), "four")
|
||||
|
||||
def test_failed_build_keeps_the_log_and_publishes_nothing(self) -> None:
|
||||
(self.repo / "BROKEN").write_text("x", encoding="utf-8")
|
||||
_commit(self.repo, "broken")
|
||||
self.library.fetch()
|
||||
|
||||
self.library.build("latest")
|
||||
|
||||
entry = self._state(self.library, "latest")
|
||||
self.assertEqual(entry["state"], "failed")
|
||||
self.assertIn("Sphinx error: broken source", entry["log"])
|
||||
self.assertFalse(self.library.servable("latest"))
|
||||
self.library.request("latest")
|
||||
self.assertEqual(self._state(self.library, "latest")["state"], "queued")
|
||||
|
||||
def test_unknown_version_is_dropped_from_the_queue(self) -> None:
|
||||
self.library.request("v9.9.9")
|
||||
self.library.build("v9.9.9")
|
||||
|
||||
self.assertFalse((self.library.sites / "v9.9.9").exists())
|
||||
self.assertFalse((self.library.queue / "v9.9.9").exists())
|
||||
|
||||
def test_queue_is_served_oldest_request_first(self) -> None:
|
||||
self.library.request("v1.0.0")
|
||||
self.library.request("v1.2.0")
|
||||
os.utime(self.library.queue / "latest", (3000, 3000))
|
||||
os.utime(self.library.queue / "v1.0.0", (2000, 2000))
|
||||
os.utime(self.library.queue / "v1.2.0", (1000, 1000))
|
||||
|
||||
self.assertEqual(self.library.next_queued(), "v1.2.0")
|
||||
|
||||
def test_replicas_share_queue_states_and_sites(self) -> None:
|
||||
other = self._replica()
|
||||
|
||||
other.request("v1.0.0")
|
||||
self.assertEqual(self._state(self.library, "v1.0.0")["state"], "queued")
|
||||
self.library.build("v1.0.0")
|
||||
|
||||
self.assertEqual(self._state(other, "v1.0.0")["state"], "ready")
|
||||
html = (self.data / "sites" / "v1.0.0" / "html").resolve()
|
||||
self.assertEqual(other.resolve("v1.0.0", "docs"), html / "docs" / "index.html")
|
||||
|
||||
def test_only_one_process_holds_the_builder_lock(self) -> None:
|
||||
def other_process_gets_lock() -> bool:
|
||||
probe = [sys.executable, "-c", LOCK_PROBE, str(self.library.lock_file)]
|
||||
return subprocess.run(probe, check=False).returncode == 0
|
||||
|
||||
self.assertTrue(other_process_gets_lock())
|
||||
self.assertTrue(self.library.acquire_builder())
|
||||
self.assertTrue(self.library.acquire_builder())
|
||||
self.assertFalse(other_process_gets_lock())
|
||||
|
||||
def test_resolve_stays_inside_the_site(self) -> None:
|
||||
self.library.build("latest")
|
||||
html = (self.library.sites / "latest" / "html").resolve()
|
||||
|
||||
self.assertEqual(self.library.resolve("latest", ""), html / "index.html")
|
||||
self.assertEqual(
|
||||
self.library.resolve("latest", "docs"), html / "docs" / "index.html"
|
||||
)
|
||||
self.assertIsNone(self.library.resolve("latest", "missing.html"))
|
||||
self.assertIsNone(self.library.resolve("latest", "../ref"))
|
||||
self.assertIsNone(self.library.resolve("latest", "/etc/passwd"))
|
||||
|
||||
|
||||
class TestBeforeTheFirstFetch(unittest.TestCase):
|
||||
def test_latest_is_offered_as_fetching_and_nothing_is_queued(self) -> None:
|
||||
with TemporaryDirectory() as td:
|
||||
shelf = library.Library("unused", Path(td), 1, Path(td))
|
||||
|
||||
shelf.request("latest")
|
||||
|
||||
self.assertEqual(shelf.versions(), ["latest"])
|
||||
(entry,) = shelf.status()
|
||||
self.assertEqual(
|
||||
(entry["name"], entry["state"], entry["phase"]),
|
||||
("latest", "queued", "fetching"),
|
||||
)
|
||||
self.assertIsNone(shelf.next_queued())
|
||||
|
||||
|
||||
class TestProgress(unittest.TestCase):
|
||||
def test_sphinx_phases_map_onto_one_rising_bar(self) -> None:
|
||||
lines = [
|
||||
"Running Sphinx v9.1.0",
|
||||
"reading sources... [ 50%] a .. b",
|
||||
"writing output... [100%] c",
|
||||
"reading sources... [100%] late line",
|
||||
"postprocess html... [ 50%] /out/html/x.html",
|
||||
]
|
||||
progress, seen = 0, []
|
||||
for line in lines:
|
||||
progress = library.progress_of(line, progress)
|
||||
seen.append(progress)
|
||||
|
||||
self.assertEqual(seen, [0, 25, 60, 60, 79])
|
||||
|
||||
def test_generators_run_in_order_without_the_checkout_on_sys_path(self) -> None:
|
||||
commands = library.generate_commands(Path("/w/src"))
|
||||
|
||||
self.assertEqual(
|
||||
commands[0],
|
||||
[
|
||||
"sphinx-apidoc",
|
||||
"-f",
|
||||
"-o",
|
||||
"/w/src/generated/modules",
|
||||
"/w/src",
|
||||
"/w/src/tests",
|
||||
],
|
||||
)
|
||||
self.assertEqual(
|
||||
[command[3] for command in commands[1:]],
|
||||
[
|
||||
"infinito_docs.generators.yaml_index",
|
||||
"infinito_docs.generators.ansible_roles",
|
||||
"infinito_docs.generators.index",
|
||||
"infinito_docs.generators.roles_overview",
|
||||
"infinito_docs.generators.readmes",
|
||||
],
|
||||
)
|
||||
self.assertTrue(all(command[1] == "-P" for command in commands[1:]))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,131 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import json
|
||||
import sys
|
||||
import threading
|
||||
import unittest
|
||||
from functools import partial
|
||||
from http.client import HTTPConnection
|
||||
from http.server import ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from tempfile import TemporaryDirectory
|
||||
from unittest.mock import patch
|
||||
|
||||
from . import PROJECT_ROOT
|
||||
|
||||
_TOOLING = str(PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "python")
|
||||
if _TOOLING not in sys.path:
|
||||
sys.path.insert(0, _TOOLING)
|
||||
|
||||
library = importlib.import_module("infinito_docs.library")
|
||||
server = importlib.import_module("infinito_docs.server")
|
||||
|
||||
SCRIPT = (
|
||||
PROJECT_ROOT / "roles" / "web-app-docs" / "files" / "javascript" / "versions.js"
|
||||
)
|
||||
|
||||
|
||||
class TestServer(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls) -> None:
|
||||
cls._tmp = TemporaryDirectory()
|
||||
data = Path(cls._tmp.name)
|
||||
cls.library = library.Library("unused", data, 1, data)
|
||||
cls.library.refs = lambda: ("abc123", ["v1.0.0"])
|
||||
site = data / "sites" / "latest"
|
||||
(site / "html" / "docs").mkdir(parents=True)
|
||||
(site / "html" / "index.html").write_text("root", encoding="utf-8")
|
||||
(site / "html" / "page.html").write_text("page", encoding="utf-8")
|
||||
(site / "html" / "docs" / "index.html").write_text("docs", encoding="utf-8")
|
||||
(site / "ref").write_text("abc123", encoding="utf-8")
|
||||
(data / "secret.txt").write_text("secret", encoding="utf-8")
|
||||
|
||||
cls._script = patch.object(server, "SCRIPT", SCRIPT)
|
||||
cls._script.start()
|
||||
cls.httpd = ThreadingHTTPServer(
|
||||
("127.0.0.1", 0), partial(server.DocsHandler, library=cls.library)
|
||||
)
|
||||
threading.Thread(target=cls.httpd.serve_forever, daemon=True).start()
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls) -> None:
|
||||
cls.httpd.shutdown()
|
||||
cls.httpd.server_close()
|
||||
cls._script.stop()
|
||||
cls._tmp.cleanup()
|
||||
|
||||
def _request(self, path, method="GET"):
|
||||
connection = HTTPConnection(
|
||||
"127.0.0.1", self.httpd.server_address[1], timeout=5
|
||||
)
|
||||
connection.request(method, path)
|
||||
response = connection.getresponse()
|
||||
body = response.read().decode("utf-8")
|
||||
connection.close()
|
||||
return response.status, response.getheader("Location"), body
|
||||
|
||||
def test_root_redirects_to_the_latest_commit(self) -> None:
|
||||
self.assertEqual(self._request("/")[:2], (302, "/latest/"))
|
||||
|
||||
def test_version_without_slash_gains_one(self) -> None:
|
||||
self.assertEqual(self._request("/latest")[:2], (302, "/latest/"))
|
||||
|
||||
def test_built_files_are_served(self) -> None:
|
||||
self.assertEqual(self._request("/latest/"), (200, None, "root"))
|
||||
self.assertEqual(self._request("/latest/page.html"), (200, None, "page"))
|
||||
self.assertEqual(self._request("/latest/docs/"), (200, None, "docs"))
|
||||
|
||||
def test_directory_without_slash_gains_one(self) -> None:
|
||||
self.assertEqual(self._request("/latest/docs")[:2], (302, "/latest/docs/"))
|
||||
|
||||
def test_page_missing_in_a_version_falls_back_to_its_start_page(self) -> None:
|
||||
self.assertEqual(self._request("/latest/gone.html")[:2], (302, "/latest/"))
|
||||
|
||||
def test_paths_cannot_leave_the_site(self) -> None:
|
||||
self.assertTrue(
|
||||
(self.library.sites / "latest" / "html" / "../../../secret.txt")
|
||||
.resolve()
|
||||
.is_file()
|
||||
)
|
||||
for path in (
|
||||
"/latest/../../../secret.txt",
|
||||
"/latest/%2e%2e/%2e%2e/%2e%2e/secret.txt",
|
||||
):
|
||||
status, location, body = self._request(path)
|
||||
self.assertEqual((status, location), (302, "/latest/"), path)
|
||||
self.assertNotIn("secret", body)
|
||||
|
||||
def test_unknown_version_is_not_found(self) -> None:
|
||||
self.assertEqual(self._request("/v9.9.9/")[0], 404)
|
||||
|
||||
def test_unbuilt_version_starts_its_build_and_shows_progress(self) -> None:
|
||||
status, _, body = self._request("/v1.0.0/page.html")
|
||||
|
||||
self.assertEqual(status, 202)
|
||||
self.assertIn('data-build="v1.0.0"', body)
|
||||
self.assertIn("<progress", body)
|
||||
self.assertTrue((self.library.queue / "v1.0.0").exists())
|
||||
|
||||
def test_api_reports_every_version(self) -> None:
|
||||
status, _, body = self._request("/api/versions")
|
||||
|
||||
self.assertEqual(status, 200)
|
||||
self.assertEqual(
|
||||
[(entry["name"], entry["built"]) for entry in json.loads(body)],
|
||||
[("latest", True), ("v1.0.0", False)],
|
||||
)
|
||||
|
||||
def test_overview_and_its_script_are_served(self) -> None:
|
||||
status, _, body = self._request("/versions/")
|
||||
self.assertEqual(status, 200)
|
||||
self.assertIn("<tbody data-versions>", body)
|
||||
self.assertIn('src="/_docs/versions.js"', body)
|
||||
self.assertIn("docsVersions", self._request("/_docs/versions.js")[2])
|
||||
|
||||
def test_head_sends_no_body(self) -> None:
|
||||
self.assertEqual(self._request("/latest/", "HEAD"), (200, None, ""))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user