Code With Wilson

You're given an array and asked to find all unique triplets `(a, b, c)` where `a + b + c = 0`. Naive: three nested loops, O(n³). NeetCode's optimal: **sort the array, then for each `a`, run two pointers on the rest looking for `b + c = -a`**. O(n²) total. The clever parts are (1) sorting first to enable two pointers and (2) carefully skipping duplicates at three levels to avoid duplicate triplets in the output. 3Sum is **sort + two-pointer**, with three de-duplication skips. NeetCode's canonical approach is the keystone of the Two Pointers category. **The pattern composes: sorting enables two-pointer, fixing one element reduces 3Sum to 2Sum, and de-duplication relies on duplicates being adjacent post-sort.** **The pattern transfer: any K-sum problem reduces to "fix K-2 elements, two-pointer the remaining two."** This is one of the most leverage-able compositions in LeetCode. ---

Whenever a problem asks for the longest (or shortest) **subarray or substring** with some property, the answer is **sliding window**. Two pointers — `left` and `right` — define a window over the input. The right pointer expands the window; when the property breaks, the left pointer shrinks until the property holds again. NeetCode's canonical approach uses a **hashset** to track unique characters and shrinks the window from the left until the duplicate is gone. Longest Substring Without Repeating Characters is the **canonical sliding window**. Two pointers, a hashset, one pass. The window grows from the right and shrinks from the left when the uniqueness property breaks. **O(n) time by amortization** — each character is touched at most twice. NeetCode's `while`-loop shrink is the part that makes the algorithm correct on multi-char duplicates. **The pattern transfer: any time the problem says "longest" or "shortest" plus "substring" or "subarray" with some property, reach for sliding window. The data structure tracking 'what's in the window' depends on the property.** ---

You're given a list of stock prices and you need to find the maximum profit from buying once and selling once on a later day. The brute force is O(n²) — try every (buy, sell) pair. The optimization: as you walk the array, **maintain the minimum price seen so far**. At each day, the best possible profit *if you sold today* is `current_price - min_so_far`. Track the maximum across all days. **One pass, two running variables, done.** Best Time to Buy and Sell Stock is the **prototype of the running-state pattern**. Maintain a small number of running variables that summarize "everything seen so far," update them in one pass, return the final state. **NeetCode's canonical solution: single-pass with `min_price` and `max_profit`. O(n) time, O(1) space.** **The pattern transfer: any time the question is "best result over a left-extending range of the array," the answer is running state in a single pass.** This is the gateway pattern to dynamic programming. ---

Whenever a problem involves **matching pairs that nest** — parentheses, function calls, undo/redo, HTML tags — the answer is a stack. The stack remembers what's still "open" so you can match the next "close" against the most recent open. **The pattern is "last opened, first closed" = LIFO = stack.** Get this once and you transfer it to the next dozen stack problems instantly. Valid Parentheses is the **introduction to the Stack pattern**. Stack remembers the most recent unclosed open; LIFO matches the nesting structure of brackets. The close-to-open hashmap collapses the comparison into one line. **Memorize NeetCode's approach as the canonical implementation — it's the template you'll modify on every other stack problem.** **The pattern transfer: any time the problem involves matching pairs with nesting, undo/redo, or "find the most recent thing of type X," reach for a stack.** ---

A palindrome reads the same forwards and backwards. The naive way is to reverse the string and compare. The optimal way is to walk two pointers from the ends inward, comparing characters. The two-pointer pattern is the single most common alternative to hashmap when the data has natural "structure from both ends" — symmetry, sortedness, monotonicity. **Two pointers is what you reach for when the problem has structure but doesn't need memory.** Valid Palindrome is **the introduction to the two-pointer pattern**. Two pointers from both ends, moving inward, comparing as they go, with skip-logic for irrelevant characters. Time stays O(n); space drops to O(1). The pattern applies whenever data has symmetric or monotonic structure. **The pattern transfer: two-pointer is the alternative to hashmap when the problem has structure (sortedness, symmetry, monotonicity) instead of needing memory of past elements. Hashmap asks "have I seen this?"; two-pointer asks "what's at the other end?".** Knowing both, and knowing when to reach for which, covers the majority of array and string problems. ---

This problem looks like it should be solved with division (compute total product, divide by each element). But the problem explicitly *forbids* division. That constraint forces you to invent the prefix/suffix product pattern: for each index, the answer is "everything to my left, multiplied by everything to my right." Build those two products in two linear passes — done. Product of Array Except Self introduces the **prefix/suffix array** pattern. Whenever the answer at each position depends on "everything to the left" or "everything to the right," build those two products (or sums, or maxes) in linear passes. The space-optimized version reuses the output array, giving O(1) extra space. **The pattern transfer: this is the same skeleton as prefix sums, prefix maxes, prefix XORs, and the running-buy-low/sell-high trick in Best Time to Buy and Sell Stock. The data structure is implicit (no hashmap, no heap) — it's just two running variables and careful indexing.** ---

Top K Frequent reduces to a two-step pipeline: (1) count occurrences with a hashmap, (2) extract the top k counts using a selection algorithm. The whole interesting decision is *how* you do step 2 — heap, full sort, or bucket sort. **Bucket sort is the optimal answer; heap is the standard interview answer; full sort is the brute force.** Knowing all three and articulating the tradeoff is the senior move. Top K Frequent is **count + select**. Counting is the hashmap pattern. Selection has three flavors: full sort (brute force), heap (standard), bucket sort (optimal when frequency range is bounded by n). Recognizing all three, picking the right one, and articulating the choice is the skill being tested. **The pattern transfer: any "top-k" problem decomposes into "produce candidates" plus "select top-k." The selection step is a heap problem unless something about the data structure (bounded keys, sorted input) opens a faster path.** ---

Longest Consecutive Sequence asks for the longest run of consecutive integers in an unsorted array, in O(n) time. Sorting takes O(n log n), so we can't sort. The trick: **start a run only from a number that has no predecessor in the set**. That single observation collapses what looks like an O(n²) problem into a clean linear scan. Longest Consecutive Sequence is the **start-of-run** problem. The hashset gives O(1) membership; the trick is recognizing that you should only expand from numbers without predecessors. That single observation collapses an apparent O(n²) into a clean O(n). **The pattern transfer: when computing something over runs/groups in unordered data, find a way to identify each group exactly once — usually by checking whether you're at the "start" or "canonical representative" of that group.** This generalization shows up in flood-fill (visit each component from one seed), union-find (each component has one root), and many graph problems. ---

Contains Duplicate is the degenerate case of Two Sum: instead of "have I seen the complement?", the question is just "have I seen *this*?" The answer is the foundation of every membership-test problem in LeetCode. **One pass, hashmap, return early on collision.** That's it. Contains Duplicate is the **membership test**. The hashset answers "have I seen this?" in O(1). Once you have that hammer, every "have I seen X?" question becomes a one-pass linear scan. **The pattern transfer: every time the question reduces to "is this thing in some collection of things I've seen?", the answer is hashset (or hashmap if you also need extra info).** Contains Duplicate is the cleanest possible expression of this; harder problems vary the question being asked. ---

Group Anagrams demonstrates the upgrade from "hashmap as memory" to "hashmap as group-by." The whole problem becomes trivial once you see the move: compute a canonical signature for each string, use that signature as a dictionary key, and append every string with that signature to the same bucket. **The hashmap key is the real algorithm.** Group Anagrams is the canonical demonstration of "hashmap as group-by." The whole problem reduces to choosing a good canonical signature and bucketing. **The interesting question is not "how do I solve this" but "what's the signature."** Once you've answered that, the algorithm is two lines. **The pattern transfer: any time the problem says "group items that share property X," the answer is "compute a canonical signature for X, use it as a hashmap key."** Anagrams use sorted-string or count-tuple. Other problems use other signatures — but the structure is identical. ---

Anagram problems are about **canonical form**: two strings are anagrams if and only if their character counts are identical. The whole problem reduces to "build the count, compare the counts." Once you see that, every anagram-family problem (Valid Anagram, Group Anagrams, Find All Anagrams in a String) is the same move. Anagrams reduce to **comparing frequency tables**. Sorting is the brute force; counting is the optimization. The fixed-size-array trick exploits the bounded-alphabet constraint to drop the space complexity from O(k) to O(1). **The pattern transfer: any time two collections must contain "the same things with the same multiplicities," count them and compare counts.** This is a different flavor of the broader hashmap pattern — Two Sum used hashmap for *complement lookup*, Valid Anagram uses hashmap for *frequency comparison*. Same data structure, different question being asked of memory. ---

Two Sum is not interesting because it's hard — it's interesting because it's the canonical demonstration of the single most common optimization move in all of LeetCode: trade space for time using a hashmap. This episode walks through the six-phase framework explicitly, applied to Two Sum, so the framework itself gets reinforced. Six topics: (1) Phase 1 understand the problem — restate, examples by hand, name constraints; (2) Phase 2 recognize the pattern — 'find a pair that sums to a target' → hashmap pattern; (3) Phase 3 brute force first always — O(n²) baseline, name the waste (rescanning); (4) Phase 4 optimize — hashmap for O(1) complement lookup, check before store, O(n) time and space; (5) Phase 5 implement and test — narrate decisions, walk examples, name edge cases (duplicates, negatives); (6) Phase 6 communicate — narration is continuous, complexity statement is the closer. Single takeaway: Two Sum is a unit test for your framework, not a problem to memorize. The hashmap-as-memory pattern transfers to Group Anagrams, Contains Duplicate, Longest Consecutive Sequence, Valid Anagram, and dozens more. Master the process, not the problem.

What ML systems actually look like at frontier scale, why traditional systems thinking breaks, and where the engineering leverage lives. Six topics: (1) the scale of modern ML infrastructure — 1e26 FLOPs, billion-dollar starting prices, nuclear-power-plant data centers; (2) why ML systems are fundamentally different — absurdly simple programs, absurdly high performance demands (50% MFU), monoculture architecture, monopsony customers; (3) the evolution of ML programming models from Caffe protobufs through TensorFlow 1 to PyTorch eager-with-graph-capture; (4) where GPU runtime actually goes — compute, memory bandwidth, and overhead, with operator fusion as the foundational optimization; (5) why compilers fail in practice — the FMA NaN bug and the FlexAttention pivot to better programming models over smarter compilers; (6) distributed ML and the three parallelism dimensions, plus fault tolerance scaling nonlinearly with cluster size. Single takeaway: programming models are durable, optimizations are fragile — the right interface beats the cleverest implementation.

The framework matters more than any individual problem. Most beginners fail LeetCode by jumping to code; strong candidates do six distinct things in sequence and only touch the keyboard at phase 5. Walk-through of each phase: Phase 1 understand the problem (restate, examples, constraints); Phase 2 recognize the pattern (15-20 patterns cover 95% of interview problems); Phase 3 brute force first always (anchors the optimization narrative); Phase 4 optimize (trade space for time, sort first, two pointers, better data structure); Phase 5 implement and test (narrate decisions, walk examples, check edges); Phase 6 communicate (continuous, not at the end). With a 45-minute interview budget across the phases. Internalize one process, not 150 problems.

Capstone topic for the AI engineering discipline. An AI product is not 'an LLM call' — it's a seven-layer system where most production failures happen outside the model. Covers: the seven-layer architecture (frontend, orchestration, retrieval, guardrails, model, output filter, telemetry); observability and distributed tracing for multi-step agents; explicit vs implicit user feedback; the feedback flywheel from telemetry to eval set to improvement; A/B testing under output stochasticity (compare distributions, not means); shadow vs canary deployment; human-in-the-loop patterns and confidence-threshold escalation. The framework that turns 'we built a demo' into 'we shipped and continuously improve a product.'

A frontier LLM at $5 per million tokens looks cheap on the pricing page. Then the arithmetic: one million DAUs issuing one 2,000-token query each is two billion tokens per day, $10K per day, $3.6M per year — and that's just inference, before retrieval, embeddings, evals, or human review. **Inference optimization is serving the same intelligence at materially lower cost without breaking the quality bar.** The mental model is a triangle — **latency, cost, quality** — and you optimize for two and accept the third as a constraint imposed by the use case. **There is no free lunch; only well-chosen tradeoffs.** This brief covers five topics. Each topic has between two and three points. --- Inference optimization is **production economics engineering**. The latency-cost-quality triangle has no free lunch — pick the bound, pick the optimizations. **KV caching is the foundational technique** (Wilson's Horace He insight: memory bandwidth, not compute, is the war). Continuous batching is the modern serving default. Quantization, distillation, and speculative decoding are individually large wins. Model routing on top can cut cost in half on mixed workloads. The team that knows this toolbox cold ships AI products with positive unit economics; the team that doesn't, doesn't. ---

When you build with foundation models, the model itself is largely a fixed input — you pick GPT, Claude, or Gemini, and you don't retrain weights. The variables you actually control are **the data going in**: training data if you fine-tune, the retrieval corpus if you do RAG, the prompt examples if you do in-context learning, the eval data you use to measure whether anything works. Every one of those is a dataset. **Dataset engineering is the craft of building, cleaning, labeling, versioning, and continuously refreshing those datasets** so the system you ship today still works six months from now. This brief covers four topics. Each topic has between two and four points. --- Dataset engineering is **a continuous engineering discipline, not a one-time project kickoff task**. The dataset lifecycle has five phases, each with specific failure modes. Three data flavors (supervised, preference, synthetic) serve different training methods. Quality crushes quantity for fine-tuning. Distribution drift is the silent killer that production monitoring catches. **The team that treats data work as the work ships better products than the team that treats it as setup.** ---

Synthesis of the 105 RAG interview questions in the dedicated RAG Q-bank. Goes much deeper than Chapter 6 — into the engineering details interviewers actually probe for production RAG systems. Covers: the 4-step pipeline (index, retrieve, augment, generate), chunking trade-offs (size, overlap, semantic chunking), embedding choice (generic vs domain-tuned, dimensions vs storage), hybrid retrieval (dense + BM25), re-ranking with cross-encoders, retrieval metrics (recall@K, MRR, NDCG), production failure modes (lost in the middle, distributional drift, latency budgets), and HyDE / query rewriting techniques. Most-shipped production AI pattern in 2025-2026. Knowing RAG mechanics deeply distinguishes 'I've heard of it' from 'I've shipped it.' Pairs with Chip Huyen Chapter 6.

Synthesis of the 33 AI Agents questions in the AI Engineer interview Q-bank. Goes deeper than Chapter 6 — into the production tradeoffs interviewers actually probe. Covers: the chatbot/RAG/agent distinction, the reason-act-observe loop (ReAct), planning patterns (ReAct vs Plan-and-Execute vs Reflexion), tool definitions and function calling, Anthropic tool use and MCP, memory layering (short-term context vs long-term vector store), multi-agent orchestration patterns. The most-asked design question in 2025-2026 AI Engineer interviews. The differentiator: knowing WHEN NOT to use an agent — when simpler RAG or just a prompt would do. Includes specific failure modes (infinite loops, runaway cost, lost-in-the-middle, bad tool definitions) and how interviewers probe for them. Pairs with Chip Huyen Chapter 6 (RAG and Agents).

Fine-tuning means teaching a model new **behaviors** by updating its weights. Different from prompting (no weight changes — steer in-context) and RAG (no weight changes — inject context at inference). **The hierarchy is prompting → RAG → fine-tuning, in that order**, because each step costs more, takes longer, and is harder to reverse than the last. Fine-tuning is the most powerful tool and the most dangerous: a bad fine-tune can degrade general capability, bake in biases, and force a full rollback. **Use it when behavior — not knowledge — is the gap.** This brief covers four topics. Each topic has between three and four points. --- Fine-tuning is **expensive, irreversible, and powerful**. The senior move is to exhaust prompting and RAG first, fine-tune only when behavior is the gap (not knowledge), use PEFT/LoRA/QLoRA to keep costs bounded, prefer DPO over RLHF when preference tuning, invest in 1,000 carefully curated examples over 100K noisy ones, and *always* run before/after evals on target task + general capabilities + safety. ---

Two ideas, one chapter, same root limitation. A foundation model is **frozen at training time**, has a **bounded context window**, and **cannot act on the world**. **RAG (Retrieval-Augmented Generation)** injects fresh, private, or domain-specific knowledge at runtime by retrieving relevant chunks into the prompt — knowledge without retraining. **Agents** extend the model's reach by letting it call tools (search, calculators, APIs, code execution, other models) in a loop — capability without retraining. Both are scaffolding around the LLM: the LLM is the reasoning engine, RAG is its memory, tools are its hands. This brief covers five topics. Each topic has between two and four points. --- RAG and agents are scaffolding around the LLM. **The model is the reasoning engine; RAG is its memory; tools are its hands.** Get the four-stage RAG pipeline cold. Know when to reach for an agent vs when RAG alone suffices (most "agent" questions are actually RAG questions). Pick your agent loop deliberately. And always cap iterations and spend. ---

Prompt engineering is the practice of treating prompts as **software artifacts** rather than chat messages. A prompt has a structure, a version, an associated eval set, a cost profile, and a failure mode catalog. You refactor prompts, diff them, roll them back. The shift from "prompting" to "prompt engineering" happens the moment you stop tweaking strings in a playground and start measuring delta against a held-out eval set on every change. Once prompts are code, every software discipline applies: version control, testing, review, observability. This brief covers five topics. Each topic has between two and three points. --- Prompt engineering is **prompts-as-code**: structured turns, version control, eval-driven iteration, careful few-shot calibration, surgical use of CoT. Context engineering is **what-the-model-sees-as-engineering**: retrieval quality, chunk count, conversation compression. Production AI engineers do both — and recognize that context engineering is the higher-leverage discipline once a baseline prompt is in place. ---

The shift from Chapter 3 to Chapter 4 is the shift from evaluating a *model* to evaluating a *system*. A model is a single function scored on a benchmark. A system is a pipeline of components, hit by real traffic, under joint constraints: latency, dollar cost, quality, safety — plus the variance of all four under load. **System eval treats those as a joint optimization, not a stack of independent metrics.** You're no longer asking "is this model good"; you're asking "does this product, on this user population, at this price, at this latency, meet the bar I committed to in the contract." This brief covers four topics. Each topic has between two and four points. --- System eval is **joint optimization across components and constraints**, not a sequence of independent metric checks. Score each component, score the full pipeline, optimize for the cost-latency-quality Pareto frontier, calibrate human raters before trusting their data, and gate online deployment with offline eval plus shadow plus canary plus A/B. **The team with this discipline ships faster than the team with a better model and worse infrastructure.** ---

Software testing assumes a deterministic spec — given input X, the function must return Y. LLM evaluation lives in a different universe: the same prompt produces different outputs across runs, two different outputs can both be correct, and "correct" often depends on context the test harness never saw. So eval is **not** "did the function return the right value?" It's **"across a representative distribution of real inputs, does the system produce outputs that satisfy a fuzzy quality bar consistently enough to ship?"** That reframe — from spec-checking to distributional quality measurement — is the whole game. This brief covers four topics. Each topic has between two and four points. --- Eval is **distributional quality measurement**, not spec checking. The discipline is: triangulate across multiple imperfect metrics, maintain golden + production-derived datasets, run eval as gated CI, and only A/B test what passes offline. Teams with mature eval pipelines have a moat that doesn't disappear when the next better model ships. ---

Synthesis of the 49 LLM Fundamentals questions in the AI Engineer interview Q-bank. Goes deeper than Ch 2 — into the math interviewers actually probe. Covers: Transformer architecture, the attention mechanism (Q/K/V, multi-head, scaled dot-product, causal masking, GQA, MHA, Flash Attention), tokenization (BPE, WordPiece, SentencePiece), positional encoding (RoPE), embeddings, KV cache, context window, temperature/top-p/top-k sampling, autoregressive vs masked LM, encoder-only vs decoder-only vs encoder-decoder, MoE (Mixture of Experts), distillation, RLHF. This is the deep technical layer that distinguishes 'I've used LLMs' from 'I understand LLMs.' Common interview traps flagged: scaled dot-product normalization rationale, why causal masking preserves training-objective integrity, RLHF doesn't teach facts, KV cache vs weights distinction.

A foundation model is the result of three ingredients combined in sequence: **a giant pile of data**, **an architecture that can learn patterns in that data efficiently**, and **a training process that runs for weeks on thousands of GPUs**. After pre-training, the raw model is shaped further with **post-training** to make it follow instructions and stay safe. To use the model in production, you also choose **how it samples its next token** — and those sampling knobs change what kind of output you get. This brief covers five topics. Each topic has between two and four points. The structure walks through the model end-to-end: data in, architecture, scaling, sampling, post-training out. --- A foundation model is **data + architecture + scale + post-training**, in that order. Each layer determines a different aspect of model behavior. **As an AI engineer, you don't change any of these directly — but you reason about all of them every time you pick a model, set a sampling parameter, or decide whether to fine-tune.** The mental model is "what knob am I actually turning, and what does it do?" ---

For most of the last decade, building anything intelligent meant collecting data, training a model, and shipping it. Foundation models broke that loop. Now most teams **consume** rather than **train** — and the new craft is everything wrapped around the API call. That craft, AI engineering, is a different discipline from ML engineering, with different skills, different bottlenecks, and different career paths. This chapter draws the distinction and lays out the stack. This brief covers five topics. Each topic has between two and four points. Each point is followed by why it matters and how it connects to the rest of the AI engineering picture. --- AI engineering exists as a discipline because foundation models turned the model layer into a vendor service, which moved the engineering frontier up the stack into orchestration, evaluation, retrieval, and application design. **The role is about consuming intelligence well, not producing it.** Get this framing right and the rest of the book is operating manuals; get it wrong and you'll keep pitching yourself for a job that no longer exists. ---