All 15 Background Jobs & Queues questions
More in Background Jobs & Queues
What is the difference between a delayed job and a scheduled (repeatable) job in a Node.js job queue? How does BullMQ (built on Redis) implement a job queue in Node.js, and what are its core building blocks? What is job idempotency, and why is it critical when designing background job processing? What is a cron job, and how would you schedule recurring background tasks in a Node.js application using node-cron or BullMQ's repeatable jobs? How would you track and expose the progress of a long-running background job to the client that requested it? What are the trade-offs of using an in-memory job queue (like a simple array-based queue) versus a persistent, Redis-backed queue? What is a 'stalled' job in BullMQ, and what causes it? What is a dead-letter queue (DLQ), and how should a Node.js job-processing system use one? How do you implement job prioritization in a Node.js queue system, and when is it necessary? What is the difference between a message queue (like RabbitMQ) and a job queue (like BullMQ), and when would you choose one over the other?
Advertisement
Back to results for “Profiling”
Node.js › Background Jobs & Queues

What is job concurrency in a worker process, and how do you configure it safely in BullMQ?

Intermediate

Answer

Job concurrency controls how many jobs a single worker process handles simultaneously -- BullMQ workers are inherently concurrent for I/O-bound job handlers (since Node's event loop can juggle multiple in-flight async operations), configurable via the 'concurrency' option, but setting it too high risks exhausting downstream resources (database connections, memory, external API rate limits) that all concurrently running jobs compete for.
const worker = new Worker('emails', async (job) => {
  await sendEmail(job.data);
}, {
  connection,
  concurrency: 10, // up to 10 email jobs processed simultaneously by this worker
});
Real-world example An email-sending worker initially set its concurrency to 50, quickly exhausting its email provider's connection pool and triggering rate-limit errors; lowering it to 10 (matched to the provider's documented concurrent-connection limit) resolved the errors while still processing the backlog quickly.

Common follow-ups: How do you determine the right concurrency value for a given downstream dependency's actual capacity?;How does concurrency interact with horizontal scaling -- running more worker processes versus raising concurrency within one?

Caching with Redis;Performance Optimization & Profiling
All 15 Background Jobs & Queues questions