Preflight check reference
Every check a node runs before it starts: its id, which nodes run it, what it prints when it fails, what to do, and whether it refuses in production.
This page is written from the code of this Release, so it says what this Release does.
A check is Passed, a Warning or Failed. A failed check makes the node refuse to start in production and is only a warning in development. Preflight explains how to run the checks without starting a node and how to read the table they print. The wording below is the code's own English; a message names the setting it is about, and this page writes the name as <NAME> or <store> where every node words it the same way.
What a check says
| Check | Nodes | What the node prints | What to do | In production | In development |
|---|---|---|---|---|---|
env.TRANSPORTER | every node | TRANSPORTER is not set | Set TRANSPORTER to the broker every node of this site shares (nats://<host>:4222, or TCP on a single host): without it this node cannot see the others. | Refuses to start | Warning, the node starts |
env.TRANSPORTER | every node | TRANSPORTER names a transporter this release does not know ('carrier-pigeon') | Set TRANSPORTER to one of: tcp, nats, redis, amqp, amqp10, mqtt, kafka, stan, fake, tcp://, nats://, redis://, rediss://, amqp://, amqps://, mqtt://, mqtts://, kafka://, stan://. | Refuses to start | Warning, the node starts |
env.NAMESPACE | every node | NAMESPACE is not set | Set NAMESPACE to the namespace every node of this site shares (usually awhere): nodes in different namespaces do not see each other. | Refuses to start | Warning, the node starts |
env.NAMESPACE | every node | NAMESPACE has characters a namespace cannot (got 'a namespace!') | Use letters, digits, '.', '_' and '-' only in NAMESPACE. | Refuses to start | Warning, the node starts |
env.<NAME> | core-services, background-services, user-provider-ldap | <NAME> is not set | Set <NAME> in the node's env file: <why>. | Refuses to start | Warning, the node starts |
env.<NAME> | core-services, authorization-server, item-services, file-services, logging-services, tile-services, realtime-services, user-provider-ldap | <NAME> is not a port number between 1 and 65535 (got 'abc')<NAME> is not an absolute http(s) URL (got 'not-a-url')<NAME> is not a number of at least 1 (got 'abc')<NAME> is not a number of at least 0 (got 'abc')<NAME> is not a time zone name such as Asia/Bangkok (got 'Mars/Base')<NAME> is not a number from 0 to 1 (got 'abc') | Correct <NAME> in the node's env file: <why>. | Refuses to start | Warning, the node starts |
env.<NAME> | core-services | <NAME> is not an absolute http(s) URL (got 'not-a-url') | Correct <NAME> in the node's env file: <why>. | Warning, the node starts | Warning, the node starts |
db.<store>.cert | core-services, authorization-server, item-services, file-services, logging-services, tile-services, tile-indexer, realtime-services, background-services | <STORE>_CERT is set, which the MongoDB driver cannot take (it wants a file path, not a PEM string) | Remove <STORE>_CERT and add tlsCertificateKeyFile=<path to the PEM file mounted in the container> to <STORE>_URL. | Refuses to start | Warning, the node starts |
db.<store>.config | core-services, authorization-server, item-services, file-services, logging-services, tile-services, tile-indexer, realtime-services, background-services | <STORE>_URL is not set, so <store> has no database to open | Set <STORE>_URL=mongodb://<user>:<password>@<host>:27017/<database> in the node's env file (<why>). | Refuses to start | Warning, the node starts |
db.<store>.config | logging-services | LOG_STORE_URL is not set, so <store> has no database to open | Set LOG_STORE_URL=mongodb://<user>:<password>@<host>:27017/<database> in the node's env file (<why>). | Refuses to start | Warning, the node starts |
db.<store>.config | core-services, authorization-server, item-services, file-services, tile-services, realtime-services, background-services | no database provider implements 'postgres:' (this release knows 'mongodb:' and 'mongodb+srv:') | Use a mongodb:// or mongodb+srv:// URL in <STORE>_URL. | Refuses to start | Warning, the node starts |
db.<store>.config | logging-services | no database provider implements 'postgres:' (this release knows 'mongodb:' and 'mongodb+srv:') | Use a mongodb:// or mongodb+srv:// URL in LOG_STORE_URL. | Refuses to start | Warning, the node starts |
db.<store>.config | core-services, authorization-server, item-services, file-services, logging-services, tile-services, tile-indexer, realtime-services, background-services | <STORE>_URL is not set; <STORE>_HOSTS, <STORE>_NAME, <STORE>_USER no longer configure anything, so <store> has no database to open | Write <STORE>_URL=mongodb://<user>:<password>@<hosts>/<NAME>?authSource=admin from those values (the authSource is needed when the user lives in the admin database) and delete <STORE>_HOSTS, <STORE>_NAME, <STORE>_USER (<why>). | Refuses to start | Warning, the node starts |
db.<store>.split-keys | core-services, authorization-server, item-services, file-services, logging-services, tile-services, tile-indexer, realtime-services, background-services | <STORE>_HOSTS, <STORE>_NAME, <STORE>_USER are set beside <STORE>_URL and ignored | Delete <STORE>_HOSTS, <STORE>_NAME, <STORE>_USER from the node's env file: only <STORE>_URL configures <store>. | Warning, the node starts | Warning, the node starts |
db.<store>.reachable | core-services, authorization-server, item-services, file-services, logging-services, tile-services, tile-indexer, realtime-services, background-services | cannot connect to mongodb://db.example.com:27017/awhere: connect ECONNREFUSED 192.0.2.1:27017cannot connect to mongodb://files.example.com:27017/awhere: connect ECONNREFUSED 192.0.2.1:27017 | Check <STORE>_URL: the host and port, the credentials and authSource, that MongoDB is running, and that this container can reach it. | Refuses to start | Warning, the node starts |
db.<store>.reachable | logging-services | cannot connect to mongodb://db.example.com:27017/awhere: connect ECONNREFUSED 192.0.2.1:27017 | Check LOG_STORE_URL: the host and port, the credentials and authSource, that MongoDB is running, and that this container can reach it. | Refuses to start | Warning, the node starts |
db.<store>.reachable | core-services, authorization-server, item-services, file-services, logging-services, tile-services, tile-indexer, realtime-services, background-services | connected to mongodb://db.example.com:27017/awhere but cannot read it: not authorized on awhere to execute commandconnected to mongodb://files.example.com:27017/awhere but cannot read it: not authorized on awhere to execute command | Give the user in <STORE>_URL the read role on its database, and check authSource. | Refuses to start | Warning, the node starts |
db.<store>.reachable | logging-services | connected to mongodb://db.example.com:27017/awhere but cannot read it: not authorized on awhere to execute command | Give the user in LOG_STORE_URL the read role on its database, and check authSource. | Refuses to start | Warning, the node starts |
db.<store>.version | core-services, authorization-server, item-services, file-services, logging-services, tile-services, tile-indexer, realtime-services, background-services | MongoDB 4.0.25 is older than 4.2, the oldest this release's driver talks to MongoDB at mongodb://db.example.com:27017/awhere is older than 4.2, the oldest this release's driver talks to (Server at db.example.com:27017 reports maximum wire version 7, but this version of the Node.js Driver requires at least 8 (MongoDB 4.2)) | Upgrade MongoDB to 4.2 or newer (6.0 and 8.x are the versions this release is tested on). | Refuses to start | Warning, the node starts |
db.<store>.replica-set | tile-indexer | content-store is a standalone MongoDB, and its change stream on the content database needs a replica set | Start mongod with --replSet <name> and run rs.initiate() (a single-node replica set is enough), then add replicaSet=<name> to CONTENT_STORE_URL. | Refuses to start | Warning, the node starts |
file-store.provider | core-services, authorization-server, item-services, file-services, background-services | no file provider is registered for 's3:' (registered: 'mongodb:', 'mongodb+srv:') | Use a mongodb:// URL in FILE_STORE_URL (GridFS is the only file provider this release has), or unset it to use the content store. | Refuses to start | Warning, the node starts |
transporter.reachable | every node | cannot reach NATS at nats.example.com:4222: connect ECONNREFUSED 192.0.2.2:4222 | Start NATS where TRANSPORTER points, or correct TRANSPORTER; check the host name resolves from this container and port 4222 is open. | Refuses to start | Warning, the node starts |
transporter.module | every node | the 'nats' package does not resolve from moleculer, which the TRANSPORTER=nats transporter needs | This image is missing a package it needs, so it is not a sound image of its Release. Do not repair it: use the images of a published Release, and report this check id and the Release to aWhere. | Refuses to start | Warning, the node starts |
cacher.reachable | every node | cannot reach Redis at cache.example.com:6379: connect ECONNREFUSED 192.0.2.3:6379 | Start Redis where CACHER points, or correct CACHER; check the host resolves from this container and the password is right. | Refuses to start | Warning, the node starts |
cacher.module | every node | the 'ioredis' package does not resolve from moleculer, which the Redis cacher needs | This image is missing a package it needs, so it is not a sound image of its Release. Do not repair it: use the images of a published Release, and report this check id and the Release to aWhere. | Refuses to start | Warning, the node starts |
jetstream | core-services, authorization-server, item-services, file-services, background-services | there is no NATS to ask for JetStream (no ACTIVITY_CHANNEL_URL, and TRANSPORTER is not nats://); the node starts and records activity once JetStream answers | Set ACTIVITY_CHANNEL_URL=nats://<host>:4222, or use a nats:// TRANSPORTER: the audit log is delivered over JetStream. | Warning, the node starts | Warning, the node starts |
jetstream | logging-services | there is no NATS to ask for JetStream (no ACTIVITY_CHANNEL_URL, and TRANSPORTER is not nats://) | Set ACTIVITY_CHANNEL_URL=nats://<host>:4222, or use a nats:// TRANSPORTER: the audit log is delivered over JetStream. | Refuses to start | Warning, the node starts |
jetstream | core-services, authorization-server, item-services, file-services, background-services | cannot reach NATS at nats.example.com:4222: connect ECONNREFUSED 192.0.2.2:4222; the node starts and records activity once JetStream answers NATS at nats.example.com:4222 answers, but JetStream is not enabled (the server has no JetStream); the node starts and records activity once JetStream answers | Run NATS with JetStream enabled: nats-server -js -sd <store directory> (the stream is created by the first node that starts). | Warning, the node starts | Warning, the node starts |
jetstream | logging-services | cannot reach NATS at nats.example.com:4222: connect ECONNREFUSED 192.0.2.2:4222 NATS at nats.example.com:4222 answers, but JetStream is not enabled (the server has no JetStream) | Run NATS with JetStream enabled: nats-server -js -sd <store directory> (the stream is created by the first node that starts). | Refuses to start | Warning, the node starts |
jetstream | core-services, authorization-server, item-services, file-services, logging-services, background-services | JetStream answers at nats.example.com:4222, but 3 activity event(s) gave up and sit in the dead-letter stream | Look at the stream <namespace>_activity_dead (nats stream view), fix what made the consumer fail, then replay or purge it. | Warning, the node starts | Warning, the node starts |
redis.<id> | background-services | cannot reach Redis at redis.example.com:6379: connect ECONNREFUSED 192.0.2.3:6379 | Start Redis where <SETTING> points, or correct <SETTING>; check the host resolves from this container and the password is right. (<why>) | Refuses to start | Warning, the node starts |
tmp.writable | background-services | the temp directory /tmp is not writable by this process | Mount a writable directory at /tmp (a read-only root filesystem needs --tmpfs /tmp), or set TMPDIR to one: <why>. | Refuses to start | Warning, the node starts |
tmp.free | background-services | only 100 MiB is free in /tmp, and <why> needs at least 256 MiB | Free space on the filesystem holding /tmp, or point TMPDIR at a larger volume. | Refuses to start | Warning, the node starts |
tmp.free | background-services | 512 MiB is free in /tmp; <why> may need more than that | Free space on the filesystem holding /tmp, or point TMPDIR at a larger volume (1.0 GiB or more is comfortable). | Warning, the node starts | Warning, the node starts |
path.<id> | authorization-server | <path> is not writable by this process | Make <path> writable by the container's user (and do not mount the image read-only): <why>. | Refuses to start | Warning, the node starts |
path.<id> | authorization-server | <path> does not exist, so it cannot be checked (not running from the image?) | Run this from the image's working directory: <why>. | Warning, the node starts | Warning, the node starts |
release | every node | this image has no Release number (APP_VERSION is unset): it was not built by CI | Deploy an image built from a tag (vX.Y.Z), or pass --build-arg APP_VERSION=<X.Y.Z> when building by hand. | Warning, the node starts | Passes |
migrations.<db>.provider | core-services, file-services, logging-services, background-services | database datastore is on the postgres: Provider: Migrations and their ledger run on the MongoDB driver only (postgres://db.example.com/awhere) | Point datastore's connection string at a MongoDB server (mongodb:// or mongodb+srv://). | Refuses to start | Refuses to start |
migrations.<db>.ledger | core-services, file-services, logging-services, background-services | cannot read the ledger of datastore: connect ECONNREFUSED 192.0.2.1:27017 | Fix db.datastore.reachable first; the ledger is read with the same connection. | Refuses to start | Warning, the node starts |
migrations.<db>.contract | core-services, file-services, logging-services, background-services | database datastore has Migration 2026-11-01-example-drop-old, applied by release 5.1.0 on 2026-10-01; this node is release 5.0.0 and cannot read data after it. Run release 5.1.0 or newer. | Deploy release 5.1.0 or newer of this node (every node of a site runs the same Release), then start it again. Nothing in the database was changed. To go back to an older Release the site needs a restore from before that Migration. | Refuses to start | Refuses to start |
migrations.<db>.upgrade-stop | core-services, file-services, logging-services, background-services | database datastore has data but has not applied Migration 2026-02-01-example-old-shape, which release 5.0.0 no longer carries (release 4.9.0 was the last that did); upgrade to release 4.9.0 first and let its Migrations finish, then this one. | Release 4.9.0 is an Upgrade stop: deploy it, wait until none of its Migrations is running or failed (npm run preflight with that release lists them), then deploy release 5.0.0. | Refuses to start | Refuses to start |
migrations.<db>.failed | core-services, file-services, logging-services, background-services | Migration(s) failed earlier and boot retries them: 2026-10-01-example-first (the database stopped answering) | Fix the cause in the message, then start the node: it resumes from the saved progress. | Warning, the node starts | Warning, the node starts |
checkpoint | core-services, file-services, logging-services, background-services | the Migration checkpoint failed to run: boom | Fix the database error above, then start the node again. | Refuses to start | Refuses to start |
The checks each node runs
The ids a node reports when everything is well, in the order it reports them. release is a warning at most. The migrations.* checks appear on the nodes that run Migrations, once for each database they run them on; .contract, .upgrade-stop and .failed appear only when they apply.
core-services
env.TRANSPORTER, env.NAMESPACE, env.PORT, env.PLATFORM_URL, env.MANUAL_URL, db.datastore.cert, db.datastore.config, db.datastore.reachable, db.datastore.version, db.content-store.cert, db.content-store.config, db.content-store.reachable, db.content-store.version, file-store.provider, db.file-store.reachable, db.file-store.version, transporter.reachable, transporter.module, cacher.reachable, cacher.module, jetstream, release, migrations.datastore.provider, migrations.datastore.ledger
authorization-server
env.TRANSPORTER, env.NAMESPACE, env.PORT, db.datastore.cert, db.datastore.config, db.datastore.reachable, db.datastore.version, db.session-store.cert, db.session-store.config, db.session-store.reachable, db.session-store.version, file-store.provider, db.file-store.reachable, db.file-store.version, transporter.reachable, transporter.module, cacher.reachable, cacher.module, jetstream, path.client-build, release
item-services
env.TRANSPORTER, env.NAMESPACE, env.PORT, db.datastore.cert, db.datastore.config, db.datastore.reachable, db.datastore.version, db.content-store.cert, db.content-store.config, db.content-store.reachable, db.content-store.version, file-store.provider, db.file-store.reachable, db.file-store.version, transporter.reachable, transporter.module, cacher.reachable, cacher.module, jetstream, release
file-services
env.TRANSPORTER, env.NAMESPACE, env.PORT, env.THUMBNAIL_MAX_BYTES, db.datastore.cert, db.datastore.config, db.datastore.reachable, db.datastore.version, db.content-store.cert, db.content-store.config, db.content-store.reachable, db.content-store.version, file-store.provider, db.file-store.reachable, db.file-store.version, transporter.reachable, transporter.module, cacher.reachable, cacher.module, jetstream, release, migrations.datastore.provider, migrations.datastore.ledger
logging-services
env.TRANSPORTER, env.NAMESPACE, env.PORT, env.AUDIT_RETENTION_DAYS, env.USAGE_TIMEZONE, db.datastore.cert, db.datastore.config, db.datastore.reachable, db.datastore.version, db.log-store.config, db.log-store.reachable, db.log-store.version, transporter.reachable, transporter.module, cacher.reachable, cacher.module, jetstream, release, migrations.log-store.provider, migrations.log-store.ledger
tile-services
env.TRANSPORTER, env.NAMESPACE, env.PORT, env.TILE_QUERY_LIMIT, env.TILE_SIMPLIFY, env.TILE_SLICE_CACHE_TTL_SECONDS, env.TILE_GZIP_LEVEL, env.TILE_MAX_QUEUED, env.TILE_MAX_HEAP_RATIO, db.datastore.cert, db.datastore.config, db.datastore.reachable, db.datastore.version, db.content-store.cert, db.content-store.config, db.content-store.reachable, db.content-store.version, transporter.reachable, transporter.module, cacher.reachable, cacher.module, release
tile-indexer
env.TRANSPORTER, env.NAMESPACE, db.content-store.cert, db.content-store.config, db.content-store.reachable, db.content-store.version, db.content-store.replica-set, transporter.reachable, transporter.module, cacher.reachable, cacher.module, release
realtime-services
env.TRANSPORTER, env.NAMESPACE, env.PORT, db.datastore.cert, db.datastore.config, db.datastore.reachable, db.datastore.version, db.content-store.cert, db.content-store.config, db.content-store.reachable, db.content-store.version, transporter.reachable, transporter.module, cacher.reachable, cacher.module, release
background-services
env.TRANSPORTER, env.NAMESPACE, env.TASK_MANAGER_REDIS_HOST, db.datastore.cert, db.datastore.config, db.datastore.reachable, db.datastore.version, db.content-store.cert, db.content-store.config, db.content-store.reachable, db.content-store.version, file-store.provider, db.file-store.reachable, db.file-store.version, transporter.reachable, transporter.module, cacher.reachable, cacher.module, jetstream, redis.task-manager, tmp.writable, tmp.free, release, migrations.content-store.provider, migrations.content-store.ledger
user-provider-ldap
env.TRANSPORTER, env.NAMESPACE, env.LDAP_DOMAIN, env.LDAP_BASE, env.LDAP_USER, env.LDAP_PASS, env.LDAP_TIMEOUT_MS, env.LDAP_CONNECT_TIMEOUT_MS, env.LDAP_HEALTHCHECK_INTERVAL_SECONDS, transporter.reachable, transporter.module, cacher.reachable, cacher.module, release