Going to production
A queue that works on your laptop needs a few more settings before it runs real jobs on several pods. Go through this list once per service. Each item says what to check and links to the page with the details.
Checklist
- A durable, shared store. The built-in
MemoryStoreloses every job when the process exits and cannot be shared between processes. This release ships no durable store, so production needs a store adapter for your database that implements the store contract and passes the conformance kit. See Store contract and Writing your own store. - No
acknowledgeVolatileStorein production. It only silences the warning that the store is not durable. If you see that warning in production, the queue is on the wrong store. See Worker options: running. - One namespace per app and environment. Set
namespaceon every queue and producer, the same on the worker and producer pods. See Queue options: store and identity. - Shutdown that fits your orchestrator. Set
signals: ['SIGTERM']or callclose()yourself, and makeshutdownTimeoutplusshutdownAbortGrace, plus about a second, fit inside the pod's grace period. See Graceful shutdown. - Health endpoints. Serve
liveandreadyfromhealth()on two endpoints. To fail fast when the store is down at start-up, callawait queue.ready()first. Keep the probe timeout above thehealth()timeout. See Health and metrics. - Alerts and an
errorlistener. Setalertsrules for queue depth, waiting age and failure rate, and listen foralertanderroron every queue and worker. WatchDEAD_LETTERS_HIGHinhealth().warnings. See Alerts and Events: store and pods. - Logging into your own logger. Pass
logger, for examplepinoLogger(pino()). SureQ logs errors, alert changes and one-off advice, never each job, so use thecompleted,failedandretryingevents for job logs. See Logging. - Concurrency and leases. Pick
concurrencyfrom what one pod and the services it calls can handle. Give long jobs a longerleaseDuration, orleaseDurationByNameper job name. See Worker options. - Time limits. Set
timeouton jobs that can hang, and passctx.signalto your I/O. See Worker options: time limits. - Payload limits. Keep job data small: store large data elsewhere and put its id in the job. Set
maxPayloadBytesandmaxResultBytesto what you expect, so a bad caller fails fast. See Queue options: adding jobs and Queue options: results and errors. - Handlers safe to run twice. SureQ runs each job at least once, so a job can run again after a crash or a lost lease. Pass
job.idempotencyKeyto outside APIs. See Core concepts: at least once. - Retention and dead letters. Check how long finished jobs and dead letters are kept, and decide who redrives dead letters. See Queue options: retention and Dead letters and redrive.
- The store contract is experimental until 1.1, so a minor release may change it. Pin SureQ and your store adapter together, and read the CHANGELOG before you upgrade. A mismatch fails at start-up with
StoreContractVersionError. See Contract version.
Common questions
Can I run one pod with a MemoryStore in production? Only if losing every waiting job on a restart or a deploy is fine. For most services it is not.
Do producer pods need all of this? No. A producer runs no workers, so shutdown, concurrency, leases, alerts and retention are set on the worker pods. It still needs the shared store, the same namespace and the payload limits. See Producer-only pods.
Where are the defaults? In the reference pages linked above. Start with Queue options.