Skip to content
S26.E0924 – An AI Said the Function Didn't Exist. It Did. Four Errors in Four Hours. episode artwork

Sep 25, 2026 47 min

S26.E0924 – An AI Said the Function Didn't Exist. It Did. Four Errors in Four Hours.

from Runtime Reality Hosted by Kenneth Henseler

Claude Opus 5 made four confident, plausible, completely wrong claims during a single development session. None were caught by better reasoning. Each was caught by reading a primary source at the moment the answer felt already known. The client was World of Warcraft: Forever, build 69913 — a beta where most legacy Lua APIs have been removed and the survivors were kept inconsistently. The training data describes a game that no longer exists. Error 1 — An API documented as absent. The project's own notes recorded "zero occurrences in this client's surface" as established fact. A three-line type() check found it immediately. Two independent sources of truth were blind in different ways: the API index covers the C API, and a runtime global walk sees functions and C_ namespaces. A Lua table inside a plain global falls through both. Error 2 — A migration that would have destroyed what it was upgrading. Wiping stored data on a schema mismatch reads as reasonable defensive coding. The capability probe had just written its results into that table. Error 3 — The textbook-correct answer that broke every message. ASCII 31, the unit separator, is the character the standard provides for exactly this purpose. The transport library rejects the range it sits in and throws on every send. Error 4 — A rescue plan that had it backwards. Two saved files, and the one visible on screen was empty on disk. SavedVariables flush at reload or logout, so the file lags memory by one reload. The pattern. Not one was caught by reasoning — every error was arrived at by sound reasoning from its inputs. Not by general caution either; being careful is a disposition, it does not tell you which file to open. Measured in this episode: build 69913 · ASCII 31 · CBOR 386 bytes, Deflate 293, Base64 392 — encoded output larger than the input, paying for a chunk boundary rather than for compression · two chunks at 240 · race ID 96 · C_Housing.GetMaxHouseLevel, documented and present at runtime as one of 63 functions, and still access-violating the client. Honest caveat: roughly a third of the addon's fields were never exercised. Sharing has never been tested with a second player; guild and title display have never rendered real data. An episode about verification that overstated its own completeness would be self-refuting. Chapters 00:00 You record it as established fact 02:44 The thesis: plausible is not verified 04:01 The client: build 69913, and what it broke 07:49 Error 1 — two sources of truth, both blind 13:20 Error 2 — the migration that ate its own evidence 25:22 Error 3 — ASCII 31, the textbook answer that failed 30:04 The crash model: why pcall cannot save you 34:07 Error 4 — the rescue plan that had it backwards 39:08 What this means for AI-assisted engineering 44:13 What was never verified Links Source, tooling and commit history — github.com/justadakaje/wow-addons Measured findings for build 69913 — docs/facts-build-69913.md Unedited session, Part 1 — youtu.be/guSsG1I_XX0 Unedited session, Part 2 — youtu.be/lAFS9JJeLPATHE 31-MINUTE CUT — youtu.be/_RP-CQRNKKg Tool used: Claude Code, Opus 5. The "wow-api" MCP server verified API calls against a custom index — correctness, not speed.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

S26.E0924 – An AI Said the Function Didn't Exist. It Did. Four Errors in Four Hours.

0:00 47:15

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Runtime Reality?

This episode is 47 minutes long.

When was this Runtime Reality episode published?

This episode was published on September 25, 2026.

Can I download this Runtime Reality episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.