Skip to content

Lesson 265·M24 Cloud & AWS for AI

S3

Open

Learning Objectives

By the end of this lesson you should be able to:

  • Explain the buckets: the containers (L265)
  • Explain the keys: the object names (L265)
  • Explain the storage classes: the tiers (L265)
  • Explain the security: the bucket policies and the IAM (L262)
  • Explain the AI shape: the documents, the datasets, the artifacts (L265)

One-Line Definition

S3 is the object store behind documents, datasets, and model artifacts (L265) — the buckets (the global-name containers, L265), the keys (the object names, L265), the storage classes (the tiers from the hot Standard to the cold Glacier, L265), and the security (the bucket policies and the IAM, L262) — with the versioning and the lifecycle (L265) that make the data durable and cheap (L265).

The one-sentence interview answer: "S3 is AWS's object store (L265). The container is the bucket — the globally unique name (L265); the object is identified by its key — the name inside the bucket (L265). The durability is the headline: 11 nines, replicated across the AZs (L261) in the region (L265). The cost is the storage classes (L265): the Standard for the hot data, the Infrequent Access for the cold, the Glacier for the archive (L265) — and the lifecycle rules (L265) move the objects between the tiers automatically (L265). The security is layered (L262): the bucket policies — what the bucket allows; the IAM (L262) — who may act; and the encryption (L275) for the data at rest (L265). The AI shape: the documents and the datasets (L265) feed the RAG ingestion (L280); the model artifacts (L265) are the training's inputs and outputs (L365); the static frontend (L272) is served from a bucket (L265) — the object store behind everything (L265)."

Mental Model

Think of S3 as the giant warehouse with the conveyor belts. The warehouse (the bucket, L265) has a global name — you can find it from anywhere (L265). The boxes (the objects, L265) are identified by their labels (the keys, L265): docs/2026/report.pdf (L265). The warehouse is organized by the shelves' climate (the storage classes, L265): the front shelves (the Standard, L265) for the frequent pickups, the back shelves (the Infrequent Access, L265) for the occasional, and the deep storage (the Glacier, L265) for the archive — with the automatic shifting (the lifecycle, L265) between them. And the warehouse has the guards (the bucket policies and the IAM, L262): who may take a box, and from which shelf (L262). The warehouse works because the labels are clear, the shelves are tiered, and the guards are scoped (L265).

   the warehouse (S3, L265)
   ┌────────────────────────────────────────────────────────┐
   │ the buckets (L265) — the global-name containers        │
   │ the keys (L265) — the object labels                    │
   │ the classes (L265) — Standard → IA → Glacier           │
   │ the guards (L262) — the bucket policies + the IAM      │
   │ the versioning + the lifecycle (L265)                  │
   └────────────────────────────────────────────────────────┘

The mental model is the warehouse: the global name, the labeled boxes, the tiered shelves, and the guards (L265).

Visual Flow — One Object's Life

   the object (L265)
        │
        ▼
   ┌──────────────────── THE PUT (L265) ──────────────────────────────┐
   │  the key: docs/2026/report.pdf (L265)                           │
   │  the class: Standard (the hot data, L265)                       │
   │  the security: the IAM (L262) + the bucket policy (L265)        │
   │  the encryption: at rest (L275)                                 │
   └──────────────────────────┬───────────────────────────────────────┘
                              ▼
   ┌──────────────────── THE LIFECYCLE (L265) ────────────────────────┐
   │  the rule: after 30 days → Infrequent Access (L265)             │
   │  after 90 days → Glacier (L265)                                 │
   │  after 365 days → the expiry (L265)                             │
   └──────────────────────────┬───────────────────────────────────────┘
                              ▼
   ┌──────────────────── THE READ (L265) ────────────────────────────┐
   │  the ingestion (L280) reads it · the frontend (L272) serves it  │
   │  the versioning keeps the history (L265)                        │
   └──────────────────────────────────────────────────────────────────┘

The flow is the object's life: put → tier → read, guarded and versioned (L265).

How It Works — The Store, Part by Part

  • The buckets (L265). The containers with globally unique names (L265). The bucket is the security and the cost boundary (L265): the policy on the bucket, the tier on the object (L265).
  • The keys (L265). The object names inside the bucket (L265). The key is the layout's design (L265): tenant/{id}/docs/... — the isolation (L320) and the lifecycle (L265) by the prefix (L265).
  • The storage classes (L265). The tiers: the Standard (the hot data), the Infrequent Access (the cold, with a retrieval fee), the Glacier (the archive, minutes to hours to retrieve) (L265). The lifecycle rules (L265) move the objects automatically (L265).
  • The security (L262). The bucket policies — what the bucket allows (L265); the IAM (L262) — who may act; the encryption (L275) — at rest and in transit (L265). The public bucket is the breach (L265).
  • The versioning (L265). The object's history (L265) — the accidental overwrite undone (L265), the audit's evidence (L322).

Note

The bucket policy and the IAM are different layers (L262, L265). The IAM (L262) says who may act — the Lambda's role (L262) may read the bucket (L265). The bucket policy says what the bucket allows — the public read for the frontend (L272), the deny for the rest (L265). The senior design uses both: the IAM for the identities (L262), the bucket policy for the bucket's boundary (L265) — the L172 baseline, storage-shaped (L265).

Real Project Usage

  • A RAG pipeline (L280). The documents and the datasets in the buckets (L265), the ingestion (L280) reading them, the vector store (L268) indexing them.
  • A training pipeline (L365). The datasets and the checkpoints in the buckets (L265) — the training (L365) reads and writes the artifacts (L265).
  • A static frontend (L272). The built site in a bucket (L265), served through CloudFront (L272).
  • A data lake (L265). The raw data in the buckets (L265), the analytics and the audit (L322) reading it.
  • Anything AWS (L265). The object store is the data's home (L265) — the durable, tiered, secure storage (L265).

The through-line: the object store is the data's home — durable, tiered, and guarded (L265).

Interview Explanation

Say it in four moves:

  1. The bucket. "The globally named container (L265)."
  2. The keys. "The object names — the layout's design (L265)."
  3. The classes. "Standard → Infrequent → Glacier, moved by the lifecycle (L265)."
  4. The security. "The IAM (L262) + the bucket policy + the encryption (L265)."

Senior-Level Insights

  • The key layout is the isolation (L320). The senior answer designs the keys (L265) around the tenant (L320): tenant/{id}/... — the isolation (L320), the lifecycle (L265), and the audit (L322) by the prefix (L265).
  • The storage class is the cost design (L285). The tier per object (L265) and the lifecycle (L265) — the hot data fast, the cold data cheap (L285) — the S3 bill is the tier's design (L265).
  • The versioning is the recovery (L265). The object's history (L265) — the accidental overwrite undone (L265), the audit's evidence (L322).
  • The public bucket is the breach (L265). The senior answer assumes the bucket is private (L265) and the access is explicit (L262) — the L312 data leakage, storage-shaped (L265).
  • The S3 is the RAG's source (L280). The documents (L265) feed the ingestion (L280) — the S3 event (L276) triggers the pipeline (L265).

Common Mistakes

  • The public bucket (L265). The data readable by the internet (L265) — the L312 leakage (L265).
  • The single class (L265). Everything in the Standard (L265) — the cold data's cost (L285) ignored.
  • The keys without the layout (L265). The flat keys (L265) — the isolation (L320) and the lifecycle (L265) impossible.
  • The versioning off (L265). The overwrite unrecoverable (L265) — the recovery (L304) lost.
  • The credentials in the app (L265). The keys in the code (L265) — the roles (L262) are the access (L262).

Best Practices

  • Keep the buckets private (L265) — the access explicit (L262).
  • Design the keys (L265) — the tenant (L320), the prefix, the lifecycle (L265).
  • Tier by the lifecycle (L265) — the hot fast, the cold cheap (L285).
  • Turn on the versioning (L265) — the recovery (L304) and the audit (L322).
  • Encrypt at rest (L275) — the data at rest protected (L265).

Interview Questions

Q: Walk me through S3.

A: The object store (L265). The buckets — the globally named containers (L265). The keys — the object names (L265). The storage classes — Standard, Infrequent Access, Glacier (L265). The security — the IAM (L262), the bucket policies, and the encryption (L275).

Q: How do you store AI documents?

A: In a private bucket, with a designed key layout (L265): tenant/{id}/docs/... (L320). The lifecycle moves the old documents to the colder tiers (L265); the versioning keeps the history (L265); and the S3 event (L276) triggers the ingestion (L280) when a document lands (L265).

Q: What are the storage classes?

A: The tiers (L265): the Standard for the hot data, the Infrequent Access for the cold (with a retrieval fee), the Glacier for the archive (minutes to hours to retrieve) (L265). The lifecycle rules (L265) move the objects between them automatically (L265).

Q: How do you secure a bucket?

A: Layered (L265). The bucket is private by default (L265). The IAM (L262) scopes who may act — the Lambda's role (L262) reads the bucket and nothing else (L262). The bucket policy defines the bucket's boundary (L265). And the encryption (L275) protects the data at rest (L265).

Follow-Up Questions

  • What's a bucket (L265)?
  • What's a key (L265)?
  • What are the storage classes (L265)?
  • How do you secure a bucket (L265)?
  • How does the lifecycle work (L265)?

Comparison Table — The Storage Classes

Class (L265)Use (L265)Cost (L285)Retrieval (L265)
Standardthe hot data — the active documents (L265)highest (L285)instant (L265)
Infrequent Accessthe cold data — the monthly reads (L265)lower + a fee (L285)instant, fee (L265)
Glacierthe archive — the compliance (L371)lowest (L285)minutes to hours (L265)

The senior read: the tier is the cost design — the hot data fast, the cold data cheap (L285).

Code Example — The Object's Life

// The object's life, declared (L265).
// THE PUT (L265) — the document lands in the private bucket (L265).
// The key layout (L265): the tenant (L320), the prefix, the name.
const key = 'tenant/42/docs/2026/report.pdf';

// THE INGESTION (L280) — the S3 event (L276) triggers the pipeline (L265).
// The bucket notification: on the object-created event, invoke the
// ingestion Lambda (L266) — the RAG pipeline starts (L280).

// THE LIFECYCLE (L265) — the tiers, automatic (L265).
const lifecycle = {
  rules: [
    { prefix: 'tenant/42/docs/', after: 30,  to: 'IA' },      // the cold (L265)
    { prefix: 'tenant/42/docs/', after: 90,  to: 'Glacier' }, // the archive (L265)
    { prefix: 'tenant/42/docs/', after: 365, to: 'expire' },  // the expiry (L265)
  ],
};

// THE READ (L265) — the IAM (L262) scopes the access (L265).
// The Lambda's role (L262): s3:GetObject on the one bucket (L265).
What the reader must SEE — the object's life, declared:

  tenant/42/docs/...  → the key layout — the isolation (L265, L320)
  the S3 event → the ingestion Lambda → the RAG pipeline (L280)
  30d → IA · 90d → Glacier · 365d → expire → the lifecycle (L265)
  s3:GetObject on the one bucket → the IAM scope (L262)

  Private, tiered, versioned, and triggered — the data's home (L265).

Tip

The pair that defines S3: the designed key (the isolation and the lifecycle, L265) and the private-by-default access (the IAM + the bucket policy, L262). Design the keys, keep the bucket private — the object store behind everything (L265).

Performance Notes

  • The S3 is the durability (L265). The 11 nines (L265) — the data survives (L265).
  • The class is the latency (L265). The Standard is instant (L265); the Glacier is minutes to hours (L265) — the retrieval latency is the class's (L265).
  • The prefix is the throughput (L265). The key layout (L265) spreads the partitions (L265) — the hot prefixes are the throughput's limit (L265).
  • The lifecycle is the cost (L285). The tier per object (L285) — the S3 bill is the tier's design (L265).

Debugging Scenarios

SymptomFirst check (L265)The lever
The document is publicThe bucket policy (L265)The private-by-default (L265)
The access deniedThe IAM (L262)The role's policy (L262)
The overwrite lostThe versioning (L265)The versioning on (L265)
The bill is highThe storage classes (L285)The lifecycle (L265)
The ingestion didn't runThe S3 event (L276)The bucket notification (L265)

Quick Revision Notes

  • S3 = the object store (L265): the buckets, the keys, the classes, the security.
  • The buckets: the globally named containers (L265).
  • The keys: the object names — the layout's design (L265).
  • The classes: Standard → IA → Glacier, via the lifecycle (L265).
  • The security: the IAM (L262) + the bucket policy + the encryption (L275).

Cheat Sheet

S3 = the object store behind documents, datasets, and artifacts

THE BUCKETS (L265)
  the globally unique names · the security + cost boundary

THE KEYS (L265)
  the object names — the layout's design (L320)
  tenant/{id}/docs/... — the isolation, the lifecycle, the audit

THE CLASSES (L265)
  Standard — the hot data · IA — the cold · Glacier — the archive
  the lifecycle: 30d → IA · 90d → Glacier · 365d → expire

THE SECURITY (L265)
  private by default · the IAM (L262) · the bucket policy
  the encryption at rest (L275)

THE AI SHAPE (L265)
  the documents → the ingestion (L280) via the S3 event (L276)
  the datasets + artifacts → the training (L365)
  the static frontend → the bucket + CloudFront (L272)

INTERVIEW, 4 MOVES
  1 bucket  "the globally named container (L265)"
  2 keys    "the object names — the layout (L265)"
  3 classes "Standard → IA → Glacier, via the lifecycle (L265)"
  4 security "IAM + bucket policy + encryption (L262, L265)"

Key Takeaways

Recap

  • S3 is the object store behind documents, datasets, and model artifacts (L265): the buckets (L265), the keys (L265), the storage classes (L265), and the security (L262)
  • The buckets (L265) are the globally named containers — the security and cost boundary; the keys (L265) are the object names — the layout's design (the tenant prefix, L320)
  • The storage classes (L265) are the tiers — Standard, Infrequent Access, Glacier — moved automatically by the lifecycle (L265)
  • The security is layered (L265): the private-by-default bucket (L265), the scoped IAM (L262), the bucket policy (L265), and the encryption at rest (L275)
  • The versioning (L265) keeps the history — the recovery (L304) and the audit (L322)
  • The AI shape (L265): the documents feed the ingestion (L280) via the S3 event (L276), the datasets and the artifacts feed the training (L365), and the static frontend is served from a bucket through CloudFront (L272)

Check your understanding

Answer each one out loud. Then rate yourself honestly.

0/8
1

What's a bucket (L265)?

2

What's a key (L265)?

3

What are the storage classes (L265)?

4

How do you secure a bucket (L265)?

5

How does the lifecycle work (L265)?

6

What's the versioning for (L265)?

7

How does the S3 event trigger the RAG pipeline (L276, L280)?

8

What is the object store's AI shape (L265)?

A Closing Note — The Warehouse, Stocked

You now hold the storage: the buckets, the keys, the storage classes, and the security — with the lifecycle keeping the bill sane and the versioning keeping the history. The compute has a store — and the data has a home (L265).

Next: the serverless compute — Lambda (L266).