S1E02. The Ruler and the Thing
from The Reliability Gap
Gordon forwards a headline saying AI can now work on its own for a full working day. Gordon and Hartwell, the furniture chain where Shelley runs operations, are invented for the show. Rob traces the headline to METR, a nonprofit research group that measures AI rather than selling it, and explains what they call a time horizon: the length of task, timed on skilled people, that a system gets right about half the time. Using METR's Frontier Risk Report and its online tracker, you'll hear what happens to that figure when the work has to be right eight times out of ten, why METR itself says it is not a measure of how long a system can act alone, where METR's own ruler stops, and why programming tasks say little about refunds. Coming up, a basketball player asked for ten free throws in a row.
Embed this episode
No transcript yet.