Streaming UI

The streaming UI is the product layer that renders the model's stream as it arrives — incremental markdown, visible tool progress, real cancellation, and clear states — turning the parts-and-states of L161 into the felt-quality of a conversation.

The one-sentence interview answer: "The streaming UI is where the stream becomes the product: text parts render as incremental markdown (L161), tool-call parts show pending → executed progress (L164), a Stop button aborts the upstream call (L145), and the states — streaming, tool-running, done, errored — are always explicit. The user never waits in silence, never wonders what's happening, and never sees a truncated answer treated as complete (L149)."

Think of the streaming UI as watching a skilled person think out loud — instead of a messenger who disappears and returns with the whole answer.

   Non-streaming UI                Streaming UI (this lesson)
   ┌────────────────────┐          ┌──────────────────────────┐
   │ [spinner] … 3s …    │          │ "A closure is a function │  ← streaming
   │ [spinner] … 8s …    │          │  that remembers…"        │
   │ [the whole answer]  │          │                          │
   └────────────────────┘          │ [using get_stock…] ✓     │  ← tool progress (L164)
                                    │ "$212.40 — up 1.2%."     │
   feels broken after 2s            │ [Stop]                   │  ← cancellation (L145)
                                    └──────────────────────────┘
                                        feels like a conversation

The mental model is progress replaces waiting: every moment of the stream is visible, every state is explained, and the user is never left with a spinner and a prayer.

   user sends
        │
        ▼
   ┌─────────────────────────────┐
   │ streaming                   │ ← text parts render as markdown (L161)
   │   │                         │
   │   ▼ (tool-call part)        │
   │ tool-running                │ ← "using get_stock…" (L164)
   │   │                         │
   │   ▼ (tool-result part)      │
   │ streaming (continues)       │ ← the answer resumes
   │   │                         │
   │   ▼ (finish part)           │
   ├─────────────────────────────┤
   │ done          stop          │ → the answer stands
   │ done          length        │ → "continued?" (L149) — never silent
   │ errored       error         │ → recovery + partial kept (L168)
   └─────────────────────────────┘
        any state → [Stop] aborts the upstream call (L145)

The state machine is L161's, made visible: every state has a rendering, and the two end states that need honesty — length and error — are designed UX, not surprises.

• Render incremental markdown. Text parts (L161) arrive as deltas; the UI accumulates them and renders markdown. The tricky part is incremental correctness: the markdown is incomplete mid-stream — a code fence is half-open, a list is mid-item. The renderer must handle partial markdown without flashing broken formatting (a common approach: render the raw text progressively and parse only at safe points, or accept minor transient formatting flicker — the senior trade-off).
• Show tool progress (L164). A tool-call part renders as "using get_stock…", and the tool-result part flips it to the result — pending → executed, as part of the conversation. The user sees why the model paused.
• Make cancellation real (L145). The Stop button calls abort() on the client; the server aborts the upstream provider call. Cancellation that only hides the text (while the provider keeps billing tokens) is a fake Stop.
• Handle the end states honestly. finish: stop → done. finish: length (L149) → the answer was truncated — render "continued?" rather than a normal-looking half-answer. error (L168) → recovery with the partial kept.

• Chat products. Incremental markdown, tool progress, Stop — the ChatGPT/Claude/Gemini feel, built from L161's parts.
• Copilots and assistants. The streaming UI shows the tool loop (L164) as visible steps — the user watches the assistant check, run, and answer.
• Agents (L200). Tool-call parts become agent steps; the UI renders the sequence — a primitive agent observability (L213) that doubles as user trust.
• Structured output (L163). streamObject parts render a typed result as it forms — a form that fills in, validated live by the schema (L143).
• Any human-facing AI output. Summaries, code, reports — rendered as they're produced, with a Stop that works.

The through-line: the streaming UI is where AI products earn their feel — progress instead of waiting, explanation instead of mystery, and honesty about limits.

Say it in four moves:

1. The frame. "The streaming UI renders the stream as it arrives — text, tool progress, cancellation, and honest end states — turning L161's parts into the product's feel."
2. The rendering. "Text parts render as incremental markdown; tool-call parts show pending → executed (L164); a Stop aborts the upstream call (L145)."
3. The honesty. "The end states tell the truth: length renders 'continued?' not a silent cut (L149), and error shows recovery with the partial kept (L168)."
4. The why. "Progress replaces waiting — the user never stares at a spinner, never wonders what's happening, and never trusts a truncated answer as complete."

• The streaming UI is the product layer of latency (L145, L151). TTFT is engineering; the streaming UI is what the user feels of it. A 300ms TTFT rendered as a spinner feels worse than a 1s TTFT that streams. The UI is a latency lever.
• Tool progress is trust (L164, L213). Showing "checking the database…" explains the pause; hiding it makes the model feel unpredictable. The pending → executed states are the seed of agent observability (L213) and user confidence.
• Cancellation is a cost control (L145, L150). A Stop that aborts upstream stops token billing mid-stream (L149) — it's not just UX, it's cost. A fake Stop is a hidden cost leak.
• Incremental markdown is a rendering trade (L151). Perfect markdown needs the full text; streaming needs incremental. The senior answer names the trade — progressive rendering with transient formatting flicker, or plain-text-until-stable — and picks per product.
• The state machine is the testable core (L341). A UI that's a pure function of the stream's parts is testable: feed parts, assert states. The streaming UI's quality is an eval (L343), not a vibe.

• Rendering when done. Waiting for the whole stream (L145) — the spinner that kills the feel, and the demo's tell.
• Fake Stop. Hiding the text but not aborting the upstream call (L145) — tokens keep billing (L150), and the user's "stop" was a lie.
• Tool calls invisible. No pending state (L164) — the pause is unexplained, the model feels random.
• length treated as done. The truncated answer, normal-looking (L149) — the honesty rule, broken.
• Markdown flicker, unfixed. Mid-stream code fences flashing broken — a rendering trade handled badly (or ignored).
• No errored state. A mid-stream failure as a blank UI (L168) — the stream died and the user got nothing.

• Stream everything a human reads (L145) — render the parts as they arrive, never wait for the whole.
• Model the states explicitly (L161): streaming, tool-running, done, errored — each with a rendering.
• Show tool progress (L164): "using X…" → the result, as conversation.
• Make Stop real (L145): abort the upstream call, and surface it as a cost control (L150).
• Handle length honestly (L149): "continued?" — never a silent cut.
• Handle error with recovery (L168): keep the partial, offer retry.
• Decide the markdown trade deliberately — progressive rendering, accepted flicker, or plain-text-until-stable.

Q: What makes a streaming UI good?

A: Progress replaces waiting. Text renders incrementally (L161), tool calls show pending → executed (L164), Stop actually aborts the upstream call (L145), and the end states tell the truth — length renders "continued?" (L149), error keeps the partial and offers recovery (L168). The user never stares at a spinner and never trusts a truncated answer as complete.

Q: How do you render markdown that's still streaming?

A: It's a trade (L151). The markdown is incomplete mid-stream — a code fence is half-open. I either render progressively and accept transient formatting flicker, or render plain text until a safe parse point. The senior answer names the trade and picks deliberately per product — perfect markdown needs the full text; streaming needs incremental.

Q: Why is a real Stop button important?

A: Two reasons. UX — the user must be able to stop a generation they don't want. And cost (L150) — a Stop that only hides the text keeps billing tokens mid-stream (L149). A fake Stop is a hidden cost leak; a real one aborts the upstream call.

Q: How do the states make the UI honest?

A: Every state has a rendering and tells the truth. Streaming shows the answer forming. Tool-running shows why the model paused (L164). Done means done. length means "this was cut off — continue?" (L149). Error means "this failed — keep the partial, retry" (L168). A demo treats length and error as done; a product tells the truth.

• How does the streaming UI compose with structured output (L163)?
• How do agent steps render from tool-call parts (L200, L213)?
• What's the markdown trade, and how do you decide (L151)?
• How does cancellation interact with the token budget (L145, L149)?
• How do you test the streaming UI's states (L341)?

The senior read: the table is the product spec — the streaming UI is the difference between "the AI is loading" and "the AI is thinking, with me watching."

// The streaming UI: parts, states, Stop, and honest end states.
import { useChat } from 'ai/react';

export function Chat() {
  const {
    messages, input, handleInputChange, handleSubmit,
    isLoading, stop, error,          // ← stop (L145) · error state (L168)
  } = useChat();

  // The markdown trade (L151): render incrementally; the message
  // body is the accumulated deltas, parsed at safe points.
  return (
    <div className="chat">
      {messages.map((m) => (
        <MessageBubble key={m.id} message={m} />
      ))}

      {/* THE STATES — every one has a rendering (L161) */}
      {isLoading && <StreamingIndicator />}         {/* streaming / tool-running */}
      {error && (                                    {/* errored — honest (L168) */}
        <div className="error">
          The stream failed. <button onClick={retry}>Retry</button>
          <button onClick={clearError}>Keep partial</button>
        </div>
      )}

      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
        {isLoading ? (
          <button type="button" onClick={stop}>Stop</button>   {/* real abort (L145) */}
        ) : (
          <button type="submit">Send</button>
        )}
      </form>
    </div>
  );
}

What the reader must SEE — the states are the design:

  isLoading → StreamingIndicator (text streaming / tool-running)
  error     → recovery, partial kept (L168)
  Stop      → real abort of the upstream call (L145)
  length    → handled in the message renderer as "continued?" (L149)

  The UI is a state machine with a rendering for every state.

• The streaming UI is the felt layer of TTFT (L151). Rendering deltas as they arrive makes the first token feel like the start of the answer — the UI is a latency lever, not just a renderer.
• Incremental markdown costs render passes (L151). Re-parsing the growing text each delta can be hot; throttle parsing or render plain text between safe points.
• Real Stop saves tokens (L150). Aborting upstream stops billing mid-stream (L149) — cancellation is a cost control with a UI face.
• Tool progress covers execution latency (L164, L151). The pending state turns a tool round trip into visible progress instead of dead air.

• The streaming UI = progress replaces waiting — L161's parts, made visible.
• Render incremental markdown (a deliberate trade, L151); show tool progress (L164).
• Real Stop (L145) aborts upstream — a cost control (L150), not just UX.
• Honest end states: length → "continued?" (L149); error → recovery + partial (L168).
• The UI is a state machine with a rendering for every state (L161).
• The streaming UI is the product layer of latency (L145, L151).

STREAMING UI = the stream, made felt (L161 → product)

FOUR JOBS
  render incremental markdown   (the trade, L151)
  show tool progress            pending → executed (L164)
  make Stop real                abort upstream (L145) — a cost control (L150)
  handle end states honestly    length → "continued?" (L149) · error → recover (L168)

THE STATES (every one has a rendering)
  streaming      the answer forming
  tool-running   "using get_stock…" (L164)
  done           the answer stands
  errored        recovery + partial kept (L168)

THE MARKDOWN TRADE (L151)
  progressive + transient flicker   OR   plain text until a safe point
  name it, pick deliberately

RULES
  never render when done — stream it (L145)
  never fake the Stop (L145, L150)
  never treat length or error as "done" (L149, L168)
  the state machine is testable (L341)

INTERVIEW, 4 MOVES
  1 frame    "progress replaces waiting"
  2 rendering "incremental markdown + tool progress (L164)"
  3 honesty  "length → 'continued?' · error → recover (L149, L168)"
  4 control  "real Stop — a cost control (L145, L150)"

You now hold the layer that makes AI products feel alive: incremental markdown, tool progress, a real Stop, and honest end states. It's the difference between "the AI is loading" and "the AI is thinking, with me watching" — and it's the layer the next lessons slot around: structured generation in the stream (L163), the full tool loop (L164), and the state model that holds it all (L165).