Processing jobs
A worker claims jobs from a queue and runs your handler on each one. You start one with queue.process(handler, options).
This page runs three jobs, two at a time, and stops the worker when the queue is empty.
Example
import { setTimeout as sleep } from 'node:timers/promises'
import { Queue } from '@sureq/core'
type Email = { to: string; subject: string }
const queue = new Queue<Email, { sentAt: number }>('emails')
const worker = queue.process(
async (job, ctx) => {
console.log('start', job.data.to, 'attempt', ctx.attempt)
// Pass ctx.signal on, so the work stops when the job must stop.
await sleep(100, undefined, { signal: ctx.signal })
return { sentAt: Date.now() }
},
{ concurrency: 2 },
)
worker.on('completed', ({ job }) => console.log('done', job.data.to))
worker.on('error', ({ error }) => console.error('worker error', error))
const drained = new Promise<void>((resolve) => queue.once('drained', () => resolve()))
for (const to of ['ada@example.com', 'alan@example.com', 'grace@example.com']) {
await queue.add({ to, subject: 'Welcome' })
}
await drained
console.log(worker.state, (await queue.getCounts()).completed)
await queue.close()
console.log(worker.state)Output:
start ada@example.com attempt 1
start alan@example.com attempt 1
done ada@example.com
start grace@example.com attempt 1
done alan@example.com
done grace@example.com
running 3
stopped- The handler gets the
joband actxfor this attempt. What it returns is the job's result. What it throws fails the attempt. concurrency: 2lets two handlers run at once, so the third job starts only when a slot is free.ctx.signalaborts when the attempt must stop: a timeout, a cancel, a lost lease or a shutdown. Hand it to your I/O.- Listen for
erroron every worker. It reports problems that are not a job failure, such as a store error. queue.close()stops the worker. Its state goes fromrunningtostopped.
Common questions
Can one worker run several kinds of job? Yes. Give each job a name, and pass process an object with one handler per name. See Named jobs.
Should I listen on the worker or on the queue? Either. The queue passes on its workers' job events, so queue.on('completed', ...) hears all of them. See Events.
Can a job run twice? Yes. SureQ runs each job at least once, never exactly once. Pass job.idempotencyKey to outside APIs so a second run does no harm.
How do I start a worker later? Pass autorun: false, then call worker.run().
What happens to a job whose name has no handler? No worker claims it. A worker asks the store only for the job names it has handlers for, and process(handler) handles unnamed jobs only. So the job stays waiting, with no error or event, until a worker with a handler for that name runs. See Queue: running jobs.