What Else Did You Let It Hear?
from What Are We Waiting For! Podcast Hosted by ConcordiaPax
Someone films a birthday in a backyard. Candles, a crooked paper crown, a dog that won’t stay out of frame. Thirty seconds of video, posted that evening, watched by a few dozen relatives. Far overhead, a small propeller plane crosses the sky. Nobody at the party notices it. The phone’s microphone does. In the soundtrack, under the singing, there’s a faint hum that slides slowly downward in pitch as the plane passes, the same way a siren drops as it goes by. That hum is enough to work out, to within a few hundred meters, where the backyard is. The sound behind the sound That isn’t a hypothetical. Recently, a YouTube creator called Colsto published a video titled “I discovered a sound that leaks your exact location (and it’s everywhere).” The method is elegant. Propeller planes give off a low tone, and as a plane flies past, the Doppler effect bends that tone downward along a curve. The shape of the curve depends on exactly where the listener was standing relative to the plane’s path. Public flight-tracking data records where every plane was at every second. Match the curve in the audio to the flights that day, and the recording points back to a spot on the map. Developers quickly rebuilt it as open tools. One of them, dopplertrack, found its test recording’s location to within 43 meters after a blind search across a region the size of a small state. Another project reported errors of 44 and 357 meters on two real recordings. Both teams are careful to call their work experimental, and both need to know roughly when the recording was made. Still, the point stands: a sound nobody heard just gave away where everyone was. We’re used to thinking about what our cameras capture. We almost never think about what our microphones catch in the background. Machines that listen to everything At the end of October, NVIDIA released Nemotron 3.5 Transcribe, an open-weight model that turns speech into text in 40 languages. It translates 31 of them to or from English. In 13 languages it can sort out who is talking, keeping track of up to eight speakers across a recording up to an hour long. You can steer it in plain language, too: ask for speaker labels, ask for a translation, ask questions about the audio, or give it a list of tricky words to listen for, like the names of medications in a doctor’s note. Alongside it came Granary v2, a public map of more than 460,000 hours of speech across 48 languages for training systems like it. This is genuinely useful. Meetings get minutes. Lectures get captions. A family can transcribe a grandparent’s stories before they’re gone. But put it next to the airplane hum, and the picture shifts a little. The machines aren’t just getting better at hearing what we say. They’re getting better at hearing everything, including the parts we didn’t mean to say at all. Whose voice counts There’s a quieter question underneath all of this: when a machine listens, whose speech does it actually understand? The Open ASR Leaderboard, a public scoreboard for speech-recognition models hosted on Hugging Face, just added two new test sets: conversational Egyptian English and Nigerian English, provided by a company called Liva AI. Results can now be split into “Inner Circle” English (the US, UK, Australia, and Canada) and the Outer and Expanding Circles, which include India, Nigeria, and Egypt. Two details stand out. First, the new sets are private, kept away from model builders on purpose so nobody can quietly train on the test and inflate their scores. The maintainers even have a word for that problem: “benchmaxxing.” Second, the sets are small, about an hour and a half each. Tens of millions of people speak Nigerian English. The benchmark that checks whether machines can understand them is shorter than a movie. It’s a real step, and it deserves credit. It’s also a reminder of how long “understanding speech” quietly meant understanding a narrow slice of it. Machines that speak, quietly The last story runs the other direction. On November 5, a company called Gradium released Phonon, a family of text-to-speech models small enough to run on a phone, a laptop, a single-board computer, or a browser tab. The models are around 65 to 75 megabytes. That’s smaller than a lot of phone games. In streaming tests on an ordinary processor, the smallest version starts speaking in about 28 milliseconds, faster than you can blink. What caught our attention wasn’t the speed, though. It was one line in the announcement: Phonon “needs no server and no network, and what you type stays with you.” That’s the inverse of the airplane problem. Most voice technology works by sending your words somewhere else to be processed. Phonon keeps them at home. The English-only version is released under a license that allows commercial use. It can also clone a voice from a short recording, and the company frames that around the speaker’s consent, which is exactly where the hard questions about voice cloning live. The oldest alarm system Hearing is one of our oldest senses, and it was built as a threat detector long before it carried language. Your ears never close. Even asleep, some part of you is listening for the snap of a branch. For most of human history, sound was how danger announced itself. What’s new is that the listening now runs in both directions. We’re surrounded by microphones that never sleep either, attached to systems that can transcribe forty languages, separate a crowded room into eight voices, and pull a location out of a hum no person noticed. Meanwhile, the tools that could keep our voices local and private are suddenly small enough to fit in a pocket. Here’s the shadow side, said plainly. Better listening helps the deaf student, the overwhelmed note-taker, the family archivist. The same capability helps anyone who wants to know where you were and who you were with. And a leaderboard that can only test 90 minutes of Nigerian English is a sign that the people building these systems still mostly hear themselves. None of that is a reason to fear the microphone. It’s a reason to start asking a question we’ve never had to ask before. Not “what did I say?” but “what else did I let it hear?” The next time you post a video, listen to the background. Something in it is talking. Have you ever noticed something in the background of a photo or video after you posted it, something you never meant to share? ConcordiaPax is developing the Exotic Intelligence Platform, modular intelligence infrastructure built around human agency, transparency, and cooperative design. This is a field note from inside the build, part of a series documenting it. This Substack is reader-supported. To receive new posts and support my work, consider becoming a free or paid subscriber. Get full access to ConcordiaPax at concordiapax.substack.com/subscribe
Embed this episode
No transcript yet.