The LLM Layer: One Message Format, Every Surface (Part 10)

Series: Inside the DeepSeek Harness — Part 10 of 16

The LLM layer is where Part 6’s invariant — model-visible means logged — gets its hardest test. Every message the UI renders, every message the durable log stores, and every message that actually crosses the wire to a model provider needs to be the same object, or the whole “the log is the source of truth” claim quietly stops being true. This post covers how dsh keeps that promise, and ends on the single cleverest mechanism in this series so far: a retry counter that isn’t stored anywhere except the log itself.

One Message type, three consumers

packages/llm/llm/src/message.ts defines exactly one Message shape — {id, role, content: ContentBlock[], source} — immutable and frozen, and it’s the type shared by delivery (what the UI shows), durable history (what the log stores), and model requests (what actually gets sent). ContentBlockMap is merge-extensible the same way SessionEventMap was in Part 6 (text | reasoning | image | tool-call | tool-result); a new modality only truly belongs once the adapter, the UI, compaction, and durable replay all agree on how to handle it.

The chunk protocol: one fold, everywhere

Adapters translate provider-specific streaming formats into one canonical StreamChunk protocol — a closed discriminated union (block-start | text-delta | reasoning-delta | tool-call-delta | block-end | usage | finish) with a switch that ends in assertNever, so adding a new chunk kind without updating every consumer is a compile error, not a runtime surprise. One shared BlockAssembler (packages/llm/llm/src/assembler.ts) folds those chunks into a frozen AssistantMessage — every adapter feeds the same assembler, rather than each provider integration writing its own accumulation logic.

Two failure paths are sanctioned, and only two: a thrown error (transport/protocol failure) or a finish { kind: 'error' | 'aborted', failure } chunk (an in-band provider error). Both normalize to the same LlmFailure shape before reaching a consumer — LlmRuntime.stream() never lets a provider’s raw throw leak through unnormalized. One specific rule is worth calling out because it came from a real production fix, not a first-pass design: an empty completion is treated as a retryable error, not a silent success. Both shipping adapters map a contentless stop finish to EMPTY_RESPONSE, retried by default — documented directly against a real, dated incident note (2026-07-24-empty-model-response-is-retryable.md). Before that fix, a model that returned nothing would have looked, to the rest of the system, exactly like a model that had nothing more to say.

Retry: the state isn’t in memory — it’s derived from the log

This is the mechanism worth reading this whole post for. The retry policy’s defaults are ordinary — maxRetries: 2, initialDelayMs: 500, maxDelayMs: 10_000, jitterRatio: 0.1, retryable on EMPTY_RESPONSE | RATE_LIMIT | SERVER | TIMEOUT | TRANSPORT (packages/llm/llm/src/retry-policy.ts:14-24). What’s not ordinary is how the retry count itself is tracked.

There’s no counter variable held in memory across retries. Instead, on every agent/request-error, the retry handler scans the durable session log — agent.session.events.findLast(...) — for a prior llm/retry event matching the current turn, step, provider, and a stable JSON fingerprint of the resolved policy called policyKey. That scan is the count. If it finds two prior llm/retry events for this exact turn/step/provider/policy combination, this is attempt three, and the max-retries check applies against that derived number — nowhere else.

Change the retry configuration — a different maxRetries, a different backoff, a reordered list of retryable codes — and policyKey changes with it, which means the retry count silently resets, because as far as the log is concerned it’s now a different policy being tracked. That’s not a bug; it’s the natural consequence of deriving state from a fingerprinted key instead of storing it separately from what it describes. The llm/retry event itself is durably appended before the cancellable backoff delay even starts — so a crash mid-wait still leaves an accurate record behind, ready to be replayed correctly on restart.

There’s also an always mode beyond the normal bounded retry — unbounded retry until success, cancellation, or disposal — implemented as a downstream-first fallback: it awaits whatever the normal retry decision would have been, and only synthesizes its own infinite retry if the normal path declined. Backoff itself is min(initialDelayMs * 2^min(retry-1, 1024), maxDelayMs), jittered into [1-jitterRatio, 1+jitterRatio] — the 1024 cap on the exponent exists purely to avoid numeric overflow and is otherwise irrelevant, since maxDelayMs already caps the real value long before the exponent gets anywhere near that size.

Multi-provider details: one adapter, many routes — safely

llm-deepseek is the simple case: one adapter, one route, direct fetch()-based SSE. llm-pi-ai is the more interesting one — a single adapter instance can own many provider routes at once, built from a profiles() map, and it ships in a deliberately dormant posture: zero routes registered until settings actually supply provider profiles, then it calls .replace(routes) for an atomic swap. Provider-level baseUrl on pi-ai is display metadata only — actual routing happens through the library’s own internal resolution, not the harness-visible URL. Each resolution produces one immutable snapshot, which is exactly what prevents a hot-reload from ever combining one adapter generation’s capability result with another’s mid-request — a request either sees the whole old world or the whole new one, never a mix.

Coming up in this series

(All 16 parts are live today — no daily drip.)

Part 16 shipped after this series’ initial 15 posts, once Agent Presets and Code Mode landed — it reads best right after Part 4, which is why its row sits there instead of at the end.

Next: Part 11 zooms out to how dsh composes agents from agents — subagent provider kinds, the difference between a one-shot delegation and a genuinely continuable background conversation, and a workflow engine that never lets a script hang forever.

References

• Source citations pulled directly from the repo as of commit 47f9438 (master, Aug 13, 2026 — same commit cited throughout this series): packages/llm/llm/src/message.ts, packages/llm/llm/src/assembler.ts (BlockAssembler), packages/llm/llm/src/retry-policy.ts (defaults), packages/llm/llm-retry/src/index.ts (log-replay retry counting), packages/llm/llm-pi-ai/src/provider.ts
• .agents/notes/implemented/bug-fix/2026-07-24-empty-model-response-is-retryable.md — the real incident behind the empty-completion retry rule
• .agents/notes/implemented/architecture/2026-06-21-mandatory-app-attribution-headers.md
• docs/subsystems/llm-streaming.md and docs/cookbook/adding-an-llm-adapter.md
• Part 1 through Part 9 of this series