“Cyborg evals” by Eye You, frmsaul

The low-background steel problem Modern steel is slightly radioactive. We did a lot of atomic testing in the 40s and 50s, and now our atmosphere has some amount of radioactive particles, which make their way into steel during production. This is mostly fine, but some scientific instruments require steel that is not radioactive. In order to get such steel, people typically scavenge for pre-1950s steel (such as the sunken Imperial German Navy). Here's a photo of the SMS Hindenburg being salvaged. You might see an analogy here between pre-war steel and pre-LLM data. Indeed, this idea has been discussed in various places last year: in a LinkedIn post, an Ars Technica article, a Business Insider article, a Harvard JOLT article. It's a good analogy, but not the one we wish to make. Instead of talking about data, let's talk about humans. Modern software engineers are radioactive. Historically, if you wanted to estimate how long a software task would take, you would ask a software engineer to "raw dog it"[1] and time how long it takes them. This is mostly impossible now, because almost all software engineers are highly dependent on LLMs. An engineer today who tries to do a [...] ---Outline:(00:09) The low-background steel problem(02:53) Cyborg evals(05:22) What kind of tasks would be good for cyborg evals?(08:53) The Cyborg Gap The original text contained 5 footnotes which were omitted from this narration. --- First published: April 30th, 2026 Source: https://www.lesswrong.com/posts/zctBgvzxamFThgc3T/cyborg-evals --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.