Measuring Anything, Before the LLMs

Douglas Hubbard invented Applied Information Economics and wrote How to Measure Anything. In 2016 I asked him how you would measure something everyone agrees is unmeasurable. He asked me how I would measure collaboration, and I gave him the answer everybody gives: count the messages.Ten years later that is still the answer, it is still wrong, and now it is on a dashboard with the word AI at the top of it.This episode is an interview from my archive, recut with a new introduction and close. It predates ChatGPT, copilots, and any AI budget anyone had to defend to a board, which is the reason to play it now rather than a caveat about it. Nothing in it needed updating.In this episode:* Why there is no such thing as a statistically significant sample size* Why you have more data than you think and need less than you think* How to define what you are measuring by the decision it changes, not the thing you can count* Why refusing to price a human life just means pricing it badly and in secret* Why you are worse at confidence than you think, and how half a day fixes itDouglas Hubbard: howtomeasureanything.comAudio only this week. Full transcript and every surface: https://sigsub.show/episodes/ep-007/Signals & Subtractions is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit sigsub.substack.com