Swarm Nextcloud#3 on main 748189e890 (job 117182318941) wedged the
OpenProject schema: the convergence check gave up after 74 attempts
while the seeder kept exiting 1, first with
`relation "ar_internal_metadata" already exists` from its structure
load, then with `relation "announcements" already exists` on every
`db:migrate`.
The seeder (`docker/prod/seeder`) loads `db/structure.sql` through
psql, statement by statement, when `\dt` finds no relation. In swarm,
`depends_on` is ignored, so web and worker start alongside it and poll
`await_migrations.sh`. That gate called
`rails db:abort_if_pending_migrations`, which runs
`migration_context.open`, i.e. `Migrator.new`, whose constructor
executes `schema_migration.create_table` and
`internal_metadata.create_table` (Rails 8.0.4,
activerecord/lib/active_record/migration.rb:1430-1431). The seeder
saw an empty database at 05:33:20; by 05:33:38 the gate had created
`ar_internal_metadata`, the structure load aborted at line 255 with
everything alphabetically before it already committed, and every
later attempt took the migrate path into tables that already existed.
Probe with `rails db:migrate:status` instead. It aborts while
`schema_migrations` is missing and otherwise only reads the recorded
versions (database_tasks.rb:303, migration.rb:1318); the gate treats
any `down` row as pending.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Swarm ignores depends_on, so web and worker start while the seeder is still migrating and die on ActiveRecord::PendingMigrationError. The artifact records 12 failed attempts across the cluster before any replica survived, and all three survivors landed on the node with the fewest failures - one against five and seven - which is how the drain drill lost the worker task it asserts on.
The app services now start through files/shell/await_migrations.sh, which polls a read-only gate and then execs the upstream entrypoint. The gate never migrates, so replicas cannot race each other the way an `until db:migrate` loop does. The seeder keeps its job: the artifact shows it applying 465 migration lines and then seeding, so the wait is aimed at work that really happens during the deploy.
The wait is bounded because the gate is ambiguous. Measured against openproject/openproject:17: `bundle exec rails db:abort_if_pending_migrations` takes 11 s and exits 1 both for pending migrations and for an unreachable database (ActiveRecord::DatabaseConnectionError). An unbounded loop would spin forever whenever the database is down. Past the deadline the entrypoint starts regardless, so the worst case stays the crash loop that already exists, never a service that refuses to come up.
The container path is declared once, in meta/volumes.yml; the second mount takes it by YAML anchor and compose reads it back through lookup(volume). Both were verified through the projects own loader, not only yaml.safe_load.
Verified by running the script in that image with no database and no workdir override: it polls, reports the deadline and starts the entrypoint. Not verified: the success branch, since no migrated database was available, and no live deploy.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kevinveenbirkenbachandClaude Opus 5.5