Message Queues & DLQs

Message queues are the async buffer of the backend — the producer enqueues the work (L222), the consumer processes it (L222), with the at-least-once delivery (L255) and the idempotent consumers (L255), the ordering (FIFO or best-effort, L245), and the dead-letter queue (L232) that catches the poison jobs with the alert and the replay (L245) — the async home of the AI work (L222).

The one-sentence interview answer: "Message queues are the async buffer (L245). The producer — the API, the webhook (L220) — enqueues the work (L222): the model call (L145), the workflow (L217), the agent run (L200). The consumer — the worker (L222) — dequeues and processes (L245). The design has four parts (L245). The delivery semantics: at-least-once (L255) — the message can be redelivered, so the consumer is idempotent (L255). The ordering: FIFO — the strict order for the dependent work (L245) — or best-effort for the independent (L245). The DLQ: the poison jobs — the retries exhausted (L232) — land in the dead-letter queue (L232) with the alert and the replay path (L245). And the visibility: the in-flight message with a timeout (L245) — the crashed consumer's work returns to the queue (L245). The queue is the async home of the AI work (L222)."

Think of the message queue as the conveyor belt between the workshop's front desk and the back room. The front desk (the producer, L222) drops the work orders onto the belt (the queue, L245); the back room (the consumers, L222) picks them up and does the work (L245). The belt has rules: an order is marked "in progress" when picked up (the visibility timeout, L245) — if the worker drops it (a crash), the order goes back on the belt (the redelivery, L255); the orders that keep failing are moved to the repair bin (the DLQ, L232) where the supervisor (the alert, L208) looks at them. Some belts are strictly ordered (FIFO, L245) — order #2 never overtakes order #1; others are best-effort (L245). The workshop works because the belt buffers, the failures redeliver, and the poison is binned (L245).

   the conveyor belt (the queue, L245)
   ┌────────────────────────────────────────────────────────┐
   │ the front desk drops (the producer, L222)              │
   │ the back room picks (the consumer, L222)               │
   │ in-progress with a timeout (the visibility, L245)      │
   │ the poison → the repair bin (the DLQ, L232)            │
   └────────────────────────────────────────────────────────┘

The mental model is the conveyor belt: the buffer, the redelivery, and the repair bin (L245).

   the work is enqueued (L222)
        │
        ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 1 · THE PRODUCER (L222)                                  │
   │     the API or the webhook enqueues the job (L220)       │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 2 · THE CONSUMER (L222)                                  │
   │     the worker dequeues, processes (L245)                │
   │     the visibility timeout — the crash redelivers (L245) │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 3 · THE OUTCOMES (L232)                                  │
   │     success → ack (L245)                                 │
   │     failure → retry (L169) → exhausted → the DLQ (L232)  │
   └──────────────────────────────────────────────────────────┘
                      ▼
   the DLQ's alert + the replay (L245) — the poison reviewed (L232)

The flow is the belt: produce → consume → ack/retry/DLQ (L245).

• The delivery (L255). The queue delivers at-least-once (L255): the message can be redelivered after a crash (L245). The consumer is idempotent (L255) — the redelivered message's side effects are deduplicated (L255).
• The ordering (L245). FIFO queues — the strict order for the dependent work (L245): the events that must process in order (L248). Best-effort — the independent work, the default (L245).
• The visibility (L245). The in-flight message has a visibility timeout (L245): if the consumer doesn't ack in time (L245) — a crash — the message returns to the queue (L245).
• The DLQ (L232). The poison jobs — the retries exhausted (L169) — land in the dead-letter queue (L232): the alert (L208) and the replay path (L245). The DLQ is the failures' catcher (L232).
• The AI shape (L222). The queued work: the model calls (L145), the workflows (L217), the agent runs (L200), and the embedding batches (L181) — the slow, failure-prone AI work (L245).

• The model calls (L145). The generation requests enqueued (L222) — the workers call the model (L145), the response posted via the webhook (L220).
• The workflows (L217). The workflow runs enqueued (L222) — the automation's engine room (L230).
• The agent runs (L200). The long agent loops (L200) as the queued jobs (L245) — the checkpoints (L207) in the job's state (L245).
• The embedding batches (L181). The ingestion's embedding jobs (L181) queued (L245) — the batch processed by the workers (L222).
• Anything slow (L260). The queue (L245) is the L260 platform's async layer (L260) — the buffer under the AI work (L222).

The through-line: the queue is the belt — the buffer, the redelivery, and the repair bin, carrying the AI work off the request path (L245).

Say it in four moves:

1. The queue. "The producer enqueues (L222); the consumer processes (L245)."
2. The semantics. "At-least-once (L255) — the consumer is idempotent (L255)."
3. The ordering. "FIFO for the dependent (L245); best-effort for the independent (L245)."
4. The DLQ. "The poison jobs (L232) — the alert (L208) and the replay (L245)."

• The queue is the request path's shield (L222). The senior answer enqueues the AI work (L222) — the model call (L145) off the request (L151) — the queue as the buffer (L245).
• The idempotency is the at-least-once's pair (L255). The redelivery (L255) is safe because the consumer deduplicates (L255) — the delivery and the consumer are one design (L245).
• The FIFO is the ordering's price (L245). The strict order (L245) costs the throughput (L245) — the senior answer uses the FIFO only for the dependent work (L245).
• The DLQ is the failures' catcher (L232). The poison (L232), the alert (L208), and the replay (L245) — the queue's production story (L245).
• The visibility is the crash's recovery (L245). The in-flight timeout (L245) — the crashed consumer's work returns (L245), the job is not lost (L255).

• The model in the request (L222). The AI work awaited inline (L151) — the queue bypassed (L245).
• No DLQ (L232). The poison retries forever (L232) — the queue blocked (L245).
• Non-idempotent consumers (L255). The redelivered message double-processes (L255) — the dedupe missing (L245).
• The FIFO for everything (L245). The strict order for the independent (L245) — the throughput paid for nothing (L245).
• No visibility timeout (L245). The crashed consumer's work stuck in-flight (L245) — the message lost (L255).
• The queue as the truth (L245). The message's state unpersisted (L207) — the durable state (L207) missing (L245).

• Enqueue the AI work (L222) — the model calls (L145), the workflows (L217), the agents (L200).
• Make the consumers idempotent (L255) — the redelivery safe (L255).
• Choose the ordering by the work (L245) — FIFO for the dependent, best-effort for the rest (L245).
• Set the visibility timeout (L245) — the crash's recovery (L245).
• Include the DLQ (L232) — the poison, the alert (L208), the replay (L245).
• Persist the job's state (L207) — the queue is the transport, not the truth (L245).

Q: What's a message queue?

A: The async buffer (L245). The producer — the API, the webhook (L220) — enqueues the work (L222); the consumer — the worker (L222) — dequeues and processes (L245). The design: the at-least-once delivery (L255) with the idempotent consumers (L255), the ordering (FIFO or best-effort, L245), the visibility timeout (L245), and the DLQ (L232).

Q: What does at-least-once mean?

A: The delivery guarantee (L255): the message can be redelivered — after a crash, after a timeout (L245). The consequence: the consumer must be idempotent (L255) — the redelivered message's side effects are deduplicated (L255). At-least-once is the queue's default (L255); the idempotency is the consumer's response (L245).

Q: What's the dead-letter queue?

A: The failures' catcher (L232). The poison jobs — the retries exhausted (L169) — are moved to the DLQ (L232): the alert goes to the human (L208), and the replay path (L245) re-runs the fixed job (L232). Without the DLQ, the poison retries forever and blocks the queue (L232). The DLQ is what makes the queue production (L245).

Q: Why not await the model in the request?

A: Because the AI work is slow (L145) and failure-prone (L211). A model call in the request makes the user wait the seconds (L151) — and a failure retries against the user's request (L168). The queue moves the work off the request path (L222): the request enqueues and returns (L151), and the worker processes with the retries (L169), the visibility (L245), and the DLQ (L232).

• What's the queue's flow (L245)?
• What does at-least-once require (L255)?
• When is FIFO right (L245)?
• What's the visibility timeout (L245)?
• What lands in the DLQ (L232)?

The senior read: the right column is the belt — the AI work carried off the request path (L245).

// The message queue: produce, consume, DLQ (L245, L232).
// THE PRODUCER (L222) — the request path stays fast (L151).
export async function POST(req) {
  const job = { id: crypto.randomUUID(), type: 'generate', payload: body };
  await queue.send(job);                             // the belt (L245)
  return { jobId: job.id };                          // fast (L151)
}

// THE CONSUMER (L222) — the worker.
async function worker() {
  while (true) {
    const msg = await queue.receive();               // the pick-up (L245)
    if (!msg) continue;
    try {
      // IDEMPOTENCY (L255) — the redelivery is safe (L255).
      if (await alreadyProcessed(msg.id)) { await queue.ack(msg); continue; }

      const result = await process(msg);             // the model call (L145)
      await markProcessed(msg.id);                   // the dedupe key (L255)
      await queue.ack(msg);                          // done (L245)
    } catch (e) {
      // THE DLQ (L232) — the retries exhausted → the poison bin (L245).
      if (msg.attempts >= MAX_RETRIES) {
        await queue.deadLetter(msg, e);              // the DLQ (L232)
        await alert({ job: msg.id, error: e });      // the supervisor (L208)
      } else {
        await queue.retry(msg, { delay: backoff(msg.attempts) });  // L169
      }
    }
  }
}

What the reader must SEE — the belt's four parts:

  queue.send(job)          → the producer (L222)
  alreadyProcessed(msg.id) → the idempotency (L255)
  queue.ack / retry        → the outcomes (L245, L169)
  queue.deadLetter + alert → the DLQ (L232, L208)

  The belt buffers, the failures redeliver, the poison is binned.

• The enqueue is the latency lever (L151). The request enqueues and returns (L222) — the queue's send is the fast path (L245).
• The workers are the throughput (L222). The consumers scale (L222) — the queue absorbs the bursts (L151).
• The DLQ is the failure's cost (L150). The poison stored (L232) — cheap, with the alert (L208) and the review (L245).
• The visibility is the reliability (L245). The in-flight timeout (L245) — the crashed work returns (L245), the job not lost (L255).

• The message queue = the async buffer (L245): the producer (L222), the consumer (L222).
• The delivery: at-least-once (L255) — the idempotent consumers (L255).
• The ordering: FIFO (L245) or best-effort (L245) — by the work (L245).
• The visibility: the in-flight timeout (L245).
• The DLQ: the poison (L232), the alert (L208), the replay (L245).
• The AI shape: the model calls (L145), the workflows (L217), the agents (L200).

MESSAGE QUEUES & DLQS = the async buffer of the backend

THE FLOW (L245)
  the producer enqueues (L222) — the API, the webhook (L220)
  the consumer processes (L222) — the workers (L245)
  the outcomes: ack (L245) · retry (L169) · dead-letter (L232)

THE SEMANTICS (L255)
  at-least-once delivery (L255) — the message can redeliver
  the consumer is idempotent (L255) — the dedupe by the message ID
  the retry is safe because the redelivery is a no-op (L255)

THE ORDERING (L245)
  FIFO — the strict order for the dependent work (L245)
  best-effort — the independent work, the default (L245)

THE VISIBILITY (L245)
  the in-flight message with a timeout (L245)
  the crashed consumer's work returns to the queue (L245)

THE DLQ (L232)
  the poison — the retries exhausted (L169) — the dead-letter (L232)
  the alert (L208) + the replay path (L245)
  the DLQ is what makes the queue production (L245)

INTERVIEW, 4 MOVES
  1 flow    "produce, consume, ack/retry/DLQ (L245)"
  2 semantics "at-least-once (L255), idempotent consumers (L255)"
  3 ordering "FIFO for the dependent, best-effort for the rest (L245)"
  4 DLQ     "the poison, the alert, the replay (L232, L245)"

You now hold the async layer: the producer that enqueues, the workers that process, the at-least-once semantics with idempotent consumers, and the DLQ that catches the poison. The AI work now rides the belt — off the request path, safe against the failures (L245).

Next: the managed belt — Amazon SQS (L246), the queue service behind the serverless AI workloads.