Amazon SQS

Amazon SQS is the managed message queue (L246) — the standard queue (best-effort ordering, high throughput, L246) and the FIFO queue (strict order, exactly-once groups, L246), with the visibility timeout (L245), the at-least-once delivery (L255) and the idempotent consumers (L255), and the DLQ (L232) — the managed belt behind the serverless AI workloads (L283).

The one-sentence interview answer: "SQS is the managed queue (L246). Two queue types. The standard queue — best-effort ordering, high throughput, at-least-once delivery (L246): the default for the independent work (L245). The FIFO queue — strict order, exactly-once processing per message group (L246): for the dependent work (L245), at the cost of the throughput (L246). The design: the visibility timeout (L245) — the in-flight message's clock; the redelivery (L255) — the crashed consumer's message returns, so the consumers are idempotent (L255); and the DLQ (L232) — the poison jobs with the alert (L208). The polling: the long polling (L246) keeps the workers efficient — the Lambda (L266) or the worker (L222) pulls the batch, processes, and acks (L245). SQS is the managed belt behind the serverless AI workloads (L283)."

Think of SQS as a managed conveyor belt with two modes and a repair bin. The belt's default mode (the standard queue, L246) is fast but not strictly ordered — the orders might arrive slightly out of sequence (best-effort, L246); it's fine for the independent jobs (L245). The strict mode (the FIFO queue, L246) guarantees the order — order #2 never overtakes #1 — for the dependent jobs (L245), at the cost of speed (L246). The belt marks an order in-progress when a worker picks it (the visibility timeout, L245); if the worker drops it, the order returns (the redelivery, L255). The orders that keep failing go to the repair bin (the DLQ, L232), where the supervisor (the alert, L208) reviews them. The belt is managed — you don't build it, you just use it (L246).

   the managed belt (SQS, L246)
   ┌────────────────────────────────────────────────────────┐
   │ standard — best-effort, fast (L246)                    │
   │ FIFO — strict order, exactly-once (L246)               │
   │ in-progress with the visibility (L245)                 │
   │ the poison → the DLQ (L232)                            │
   └────────────────────────────────────────────────────────┘

The mental model is the managed belt: two modes, the visibility, and the repair bin — no building required (L246).

   the work is enqueued (L222)
        │
        ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 1 · THE SEND (L246)                                      │
   │     the producer sends to the queue (L246)               │
   │     standard or FIFO by the work (L246)                  │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 2 · THE RECEIVE (L245)                                   │
   │     the worker long-polls (L246), receives the batch     │
   │     (L245) — the visibility timeout starts (L245)        │
   └──────────────────┬───────────────────────────────────────┘
                      ▼
   ┌──────────────────────────────────────────────────────────┐
   │ 3 · THE PROCESS + DELETE (L245)                          │
   │     the work done → the message deleted (the ack, L245)  │
   │     the failure → the retry → the DLQ (L232)             │
   └──────────────────────────────────────────────────────────┘

The flow is the belt: send → receive → process + delete — with the visibility and the DLQ (L246).

• The standard queue (L246). Best-effort ordering, high throughput, at-least-once (L246): the default for the independent work (L245). The standard queue may deliver out of order and more than once (L246) — the consumers are idempotent (L255).
• The FIFO queue (L246). Strict order per message group, exactly-once processing (L246): for the dependent work (L245), at the throughput's cost (L246). The FIFO's message group ID is the ordering key (L246).
• The delivery (L245, L255). The visibility timeout (L245): the in-flight message's clock — the un-acked message returns (L245). The at-least-once (L255) and the idempotent consumers (L255).
• The DLQ (L232). The redrive policy (L246): the poison messages move to the DLQ (L232) after the max receives (L246) — with the alert (L208) and the replay (L245).
• The polling (L246). The long polling (L246) — the worker's receive waits for the messages (L246), keeping the Lambda (L266) and the worker (L222) efficient (L283).

• The serverless AI (L283). The Lambda (L266) triggered by the SQS queue (L246) — the API Gateway (L267) enqueues, the Lambda processes the model call (L145).
• The ingestion batches (L181). The embedding jobs (L181) in the standard queue (L246) — the workers process in parallel (L222).
• The event processing (L248). The domain events (L248) in the FIFO queue (L246) — the per-aggregate order (L248).
• The workflows (L217). The automation's engine room (L222) on SQS (L246) — the L230 platform's belt (L260).
• Anything queued on AWS (L283). SQS (L246) is the managed belt (L246) — the standard for the independent, the FIFO for the ordered (L246).

The through-line: SQS is the managed belt — the two queue types, the visibility, and the DLQ, behind the serverless AI workloads (L246).

Say it in four moves:

1. The types. "Standard — best-effort, fast (L246). FIFO — strict order (L246)."
2. The delivery. "At-least-once (L255), the visibility (L245), the idempotent consumers (L255)."
3. The DLQ. "The redrive policy (L246) — the poison to the DLQ (L232), the alert (L208)."
4. The polling. "The long polling (L246) — the Lambda (L266) and the workers efficient (L283)."

• The queue type follows the work (L246). The senior answer picks by the dependency (L245): the independent → the standard (L246), the dependent → the FIFO (L246).
• The consumers are idempotent, always (L255). Even the FIFO's exactly-once (L246) has the consumer's failure window (L245) — the idempotency (L255) is the constant (L246).
• The DLQ is the ops story (L232). The redrive policy (L246), the alert (L208), and the replay (L245) — the queue's production posture (L246).
• The long polling is the serverless pattern (L283). The Lambda (L266) with the long polling (L246) — the event source mapping (L266) — the serverless belt (L283).
• The visibility is the crash's recovery (L245). The in-flight clock (L245) — the crashed consumer's work returns (L245), sized to the job's duration (L246).

• The queue type by habit (L246). The FIFO for the independent (L246) — the throughput paid for nothing (L245).
• Non-idempotent consumers (L255). The redelivery double-processes (L255) — even on the FIFO (L246).
• No DLQ (L232). The poison retries forever (L232) — the redrive policy (L246) missing.
• The visibility too short (L245). The slow job's message returns mid-work (L245) — the timeout mis-sized (L246).
• The short polling (L246). The empty receives hammer (L246) — the long polling (L246) missing.
• The queue as the truth (L245). The job's state unpersisted (L207) — the durable state (L207) missing (L246).

• Pick the type by the work (L246) — the standard for the independent (L245), the FIFO for the ordered (L246).
• Make the consumers idempotent (L255) — the redelivery safe (L246).
• Configure the DLQ (L246) — the redrive policy (L246), the alert (L208).
• Size the visibility (L245) — to the job's duration (L246).
• Use the long polling (L246) — the workers efficient (L283).
• Persist the state (L207) — the queue is the transport, not the truth (L245).

Q: What is SQS?

A: The managed message queue (L246). Two types: the standard queue — best-effort ordering, high throughput, at-least-once (L246); and the FIFO queue — strict order, exactly-once per message group (L246). The design: the visibility timeout (L245), the idempotent consumers (L255), the DLQ (L232), and the long polling (L246).

Q: Standard or FIFO?

A: The work's dependency decides (L246). The independent work — the embedding jobs, the parallel calls (L181) — the standard queue (L246): high throughput, best-effort (L246). The dependent work — the per-aggregate events (L248) — the FIFO queue (L246): strict order, exactly-once per group (L246), at the throughput's cost (L245). The queue type follows the work (L246).

Q: How does the visibility timeout work?

A: The in-flight message's clock (L245). When the worker receives a message, the visibility timeout starts (L245): if the worker doesn't delete the message in time (the ack, L245) — a crash — the message becomes visible again and is redelivered (L245). The timeout is sized to the job's duration (L246) — too short, the slow job's message returns mid-work (L245).

Q: How do you handle failures?

A: The retry and the DLQ (L232). The message's receive count grows with each redelivery (L246); the redrive policy (L246) moves the message to the DLQ after the max receives (L246). The alert (L208) tells the humans, and the replay path (L245) re-runs the fixed message (L232). The DLQ is the queue's failure catcher (L246).

• What are the queue types (L246)?
• When is the FIFO right (L246)?
• What's the visibility timeout (L245)?
• How does the redrive policy work (L246)?
• How does the Lambda consume the queue (L283)?

The senior read: the columns are the trade — the throughput vs the order, chosen by the work (L246).

// Amazon SQS: send, receive, DLQ (L246, L283).
import { SQSClient, SendMessageCommand, ReceiveMessageCommand, DeleteMessageCommand } from '@aws-sdk/client-sqs';

const sqs = new SQSClient({ region: 'us-east-1' });

// THE PRODUCER (L222) — the enqueue (L246).
await sqs.send(new SendMessageCommand({
  QueueUrl: process.env.AI_JOBS_QUEUE,
  MessageBody: JSON.stringify(job),
  // FIFO: MessageGroupId: job.tenant,          // the ordering key (L246)
  // FIFO: MessageDeduplicationId: job.id,      // the dedupe (L246)
}));

// THE CONSUMER (L222) — the long-polling worker (L246, L283).
async function worker() {
  const { Messages } = await sqs.send(new ReceiveMessageCommand({
    QueueUrl: process.env.AI_JOBS_QUEUE,
    MaxNumberOfMessages: 10,
    WaitTimeSeconds: 20,                        // the LONG polling (L246)
    VisibilityTimeout: 300,                     // the in-flight clock (L245)
  }));

  for (const msg of Messages ?? []) {
    try {
      if (await alreadyProcessed(msg.MessageId)) {         // the idempotency (L255)
        await sqs.send(new DeleteMessageCommand({ QueueUrl, ReceiptHandle: msg.ReceiptHandle }));
        continue;
      }
      await processJob(JSON.parse(msg.Body));   // the model call (L145)
      await sqs.send(new DeleteMessageCommand({ QueueUrl, ReceiptHandle: msg.ReceiptHandle }));  // the ack (L245)
    } catch (e) {
      // The redrive policy (L246) moves the poison to the DLQ (L232).
      await alert({ messageId: msg.MessageId, error: e });   // L208
    }
  }
}

What the reader must SEE — the managed belt:

  SendMessage            → the producer (L222, L246)
  WaitTimeSeconds: 20    → the long polling (L246)
  VisibilityTimeout      → the in-flight clock (L245)
  alreadyProcessed       → the idempotency (L255)
  DeleteMessage          → the ack (L245)
  the redrive → the DLQ  → the poison catcher (L232)

  The belt is managed — send, receive, delete; the failures redrive.

• The long polling is the efficiency (L246). The 20-second wait (L246) — the empty receives eliminated (L246), the Lambda (L266) cost controlled (L283).
• The visibility is the job's clock (L245). Sized to the job (L246) — the slow job's redelivery avoided (L245).
• The FIFO is the throughput's cost (L246). The strict order (L246) — the lower throughput (L246), chosen for the dependent work (L245).
• The DLQ is the ops cost (L150). The poison stored (L232) — cheap, with the alert (L208) and the replay (L245).

• SQS = the managed queue (L246) — the belt, no building.
• The types: standard (L246) vs FIFO (L246) — by the work (L245).
• The delivery: at-least-once (L255), the visibility (L245), the idempotency (L255).
• The DLQ: the redrive policy (L246), the alert (L208), the replay (L245).
• The polling: the long polling (L246) — the serverless pattern (L283).
• The truth: the queue is the transport, not the truth (L245).

AMAZON SQS = the managed message queue — the belt, no building

THE TYPES (L246)
  standard  best-effort ordering · high throughput · at-least-once (L246)
            the independent work (L245)
  FIFO      strict order, exactly-once per message group (L246)
            the dependent work (L245) · the throughput's cost (L246)

THE DELIVERY (L245, L255)
  the visibility timeout (L245) — the in-flight clock
  the crash → the redelivery (L245)
  at-least-once (L255) — the consumers idempotent (L255)
  even the FIFO's consumers need the idempotency (L255)

THE DLQ (L232)
  the redrive policy (L246) — the poison after the max receives (L246)
  the alert (L208) · the replay path (L245)

THE POLLING (L246)
  the long polling (L246) — the empty receives eliminated (L246)
  the Lambda (L266) and the workers efficient (L283)

THE RULES
  the queue type follows the work (L246)
  the queue is the transport, not the truth (L245)
  the state persists in the job (L207)

INTERVIEW, 4 MOVES
  1 types   "standard vs FIFO (L246)"
  2 delivery "at-least-once (L255), visibility (L245)"
  3 DLQ     "the redrive policy (L246), the alert (L208)"
  4 polling "the long polling (L246) — serverless (L283)"

You now hold the AWS belt: the standard and the FIFO queues, the visibility and the redelivery, the idempotent consumers, and the DLQ's redrive — with the long polling keeping the serverless workers efficient. The AI work now rides a managed belt — no building required (L246).

Next: the fan-out — SNS & Pub/Sub (L247), broadcasting events to the systems that care.