This is Hidden Brain, I'm Shankar Vedanta. At some point in our lives, many of us realize that the way we hear our own voice isn't the way others hear our voice. Shae had that realization as a child helping out with the family business, Adeli, in South West Virginia. We did a lot of business with Adeli, a lot of call-in orders, and I had to answer the phone with the name of the business.
Often, when the phone rang, the same thing happened over and over again. I vividly remember always being confused for my mother. They would always say, oh, hey, Judy. You know, and either start into their questions or whatever they were looking to speak to my mother about.
This case of mistaken identity became a running joke. You know, it was ha ha, they thought that he was Judy. Shae didn't correct the callers. That's because Shae didn't mind being mistaken for Judy.
It was just comforting to me because it felt natural. Shae was raised as a boy. But now, decades later, Shae identifies as a transgender woman. We're not using her legal name at her request, because it's a man's name.
Shae's experience at Adeli became a template for the rest of her life. She listened to her voice, and she listened to the way others heard her voice. There was always a gap between the two. She first tried to sound more masculine to fit in with the way the world saw her.
So I would consciously make an effort to try to talk a little deeper. It was, you know, I practiced it. One way she practiced was to sing along with the doors, tool, and nine-inch nails. Attended to sort of go towards heavier music, you know, it's raspy, deep.
Yelling almost voices. Sounding more masculine became second nature, but it wore on her. My entire life, I have been playing the role of a boy. And it is exhausting.
It truly is. Years and years passed like this. A divorce, a second wife, two kids, and a cancer scallator. Shae began to reconsider how she wanted to sound.
Instead of trying to sound more masculine, she now started to try to sound more feminine. Even prior to accepting that I was trans, before I could put a label on what I was, I consciously made an effort to not sound as masculine. And that started in my early 30s. Once again, she used music as a way to practice.
I would always sing in a car alone, and I would attempt Britney Spears. It seems silly, but I would, when I'm singing in the car, I would sort of turn my head towards my driver's side window because it would reflect the sound back to me a little more loudly so that I could hear the pitch and tone of my voice. And I would try to make my voice sound at a higher pitch without it sounding like I was trying. What did you hear?
Did you hear the voice that you wanted to hear? No, no, I've never, I don't know that I have ever actually been able to hear the same voice that I hear come from me. She has spent a lifetime being dissatisfied with the way she sounds. She viscerally knows something the rest of us often forget.
Our voices shape who we are. They shape how other people think of us. This week on Hidden Brain, we look at the relationship between our voices and our identities. Voices about who you are, our voice signals, things about our personality.
Plus, how technology might help people with vocal impairments find voices that reflect who they are. Once you close your eyes and let your mind relax, it doesn't take long to escape to that beautiful pitch. And the ethical quandaries that arise when we can create personalized, customized voices. This is you, they can make us say anything now really anything.
Ah, the faint scent of factor 50. It's time to dust off your It's Not Burnt, It's Well Done, Abram. You're ready to tackle the garden. And suddenly, it's the busiest room in the house.
It's summer and the gold play outside summer sale is now on at Home Value with great value on barbecues, lawn mores, garden furniture and more from now until June 4th or while stock's last. With 88 stores nationwide, call into your local home value or shop online at homevalue.ie. Home value trusted local experts. The bank holiday is finally here.
Enjoy summer savers at Central with great offers like Centra Fresh Hour Stripper on stage 2 pack only 8 euro offer in Sunday. Selected Pringles 165 Grand 1 75 each plus Diet Coke and Coke Zero 12 Compact only 6 euro each plus deposit. Bank Holiday Value Sorted Centra Live Every Day. Jackie Kirk used to love the sound of her voice.
She spent her 20s in San Francisco. And like many young people living in the big city, she enjoyed going out with her friends. She danced to electronic music at clubs and drapid bars. She was outgoing and she was a flirt.
I have to admit, I've always enjoyed flirting. And so I was quite the flirted. It was sort of a fun activity to help past the day when you're doing something mundane. For Jackie, flirting was also a demonstration of her confidence.
It reinforces my identity. How I felt about myself. Fun, I'm somebody that people are attracted to. Not just physically or sexually, but a person who people like.
Jackie liked to be liked. She liked being someone people wanted to be around. When she thinks about who she was back then, she refers to that person as Voice One Jackie. Voice One Jackie was a really fun, loving, always joking.
Pretty carefree. For years Jackie had been doing backpacking trips with her then boyfriend. All of our gear on our back, 50 pounds, 60 pounds, et cetera. Even carrying bottles of wine in the back pack.
She'd been doing these backpacking trips for several years. But during the trip to a national forest in California, Jackie came up short. I couldn't go one step further. I, you know, and being young you'd think, what could it be?
There's nothing wrong with me. I'm 20, whatever, 20, something years old. And I noticed, kind of really short of breath and it was a real struggle. I finally made it, of course, but it was really slow and a real struggle to make it back.
Hiking became too taxing for her, so she cut back and switched to ballet. But one day, during the series of relevés, a more aerobically challenging dance move, Jackie felt lightheaded and dizzy. It was serious. I had a seizure and, you know, all of the medical follow-up led me to discover that I had a lung disease.
She was diagnosed with idiopathic pulmonary hypertension. It's a rare progressive disease where the blood vessels in the lungs shrink and oxygen is not distributed properly. There's no cure for it except for lung transplant. In 2008, at 32, Jackie received a double lung transplant.
The surgery was successful. Within weeks, she was out of the hospital and back in the dance studio. In 2010, she left San Francisco to explore Latin America and Europe. But her body began to reject her new lungs.
Before long, Jackie was back in the hospital, this time in Switzerland, waiting for another lung transplant. I was in January, January of 2013, and I was asleep still for a month and a half. When she woke up, she found her surgery had been successful, except for one very important thing. I couldn't speak.
I couldn't speak. During the operation, the medical staff had used a ventilator to help her breathe. So I was intubated. That means they basically had a tube that they put inside your mouth, and it goes down your throat and they send that down into your lungs.
And during this intubation, the tube was rigid enough to cause some damage to my vocal cords. Jackie began speech therapy, and within a few weeks, she slowly regained the ability to speak. But the voice that came out of her mouth, it wasn't her voice. My voice changed.
It's raspy or it's broken a bit. Ever since, speaking has been hard work. Yeah, I really have to push my vocal cords. I feel it.
It's actually a physical effort. I'm actually squeezing the vocal cords as hard as I can to make the loudest sound possible to get to be heard. And it's very tiring. The harder it became to produce sound, the more self-conscious she became about the way she sounded.
I feel less confident. I'm aware of how people might perceive me. So I'm a little more shy. I don't approach people like I used to.
Jackie believes the change in her voice has led to a dramatic change in her personality. For much of her life, her voice was a manifestation of her confidence. I used to go to clubs quite a bit, you know, but you know, when you have a normal voice, you can still talk to people in those environments where it's kind of loud and noisy or bars to meet friends or to flirt like I like doing. But you know, those places now, you know, I don't really go to anymore.
Jackie, a woman who once described herself as carefree and outgoing, who took pride in her ability to flirt, became withdrawn, reserved. You know, I have tons of scars all over my body and that plays on my confidence as well. But in public life, people can't see those scars and I feel like my voice is that, you know, that scar they can hear. You know, they know something's wrong and that they know, oh, maybe she's weak, maybe she's sick.
Just by hearing my voice, it's the signal. Our voices communicate so much more than mere information. They communicate our feelings, our temperament, our identity. When we come back, how scientists are weaving this insight into custom-built voices.
I can't wait for my friends to hear my new voice. At the ID Electrical, we know getting a new appliance is genuinely exciting. A fridge with more space, thrilling. A washing machine that's actually quiet, even better.
This weekend only, at DID Electrical, buy selected Samsung Large Appliances and get 10% back on a DID Rewards card. Plus enjoy 5-year-old delivery and half-price installation on all large appliances right now. DID Electrical, for everything you. Ah, the faint scent of Factor 50.
It's time to dust off your It's Not Burnt, It's Well Done, apron. You're ready to tackle the garden. And suddenly, it's the busiest room in the house. It's summer, and the go play outside summer sale is now on at Home Value, with great value on barbecues, lawnmores, garden furniture and more.
From now until June 4th or Wild Stocks last. With 88 stores nationwide, come into your local Home Value or shop online at HomeValue.ie. Home Value, trusted local experts. The Bank Holiday is finally here.
Enjoy summer savers at Central with great offers, like Central Fresh Art Stripper on stage 2 pack only 8-year-old offer ends Sunday. Selector Pringles 165 Grand 1.75 each. Plus Diet Coke and Coke Zero 12 pack only 6-year-old each plus deposit. Bank Holiday Value Sorted.
Central. Live every day. Scientists have been trying for more than two centuries to analyze the human voice, decode its components and recreate it. An early success came from a man named Homer Dudley.
He developed an organ-like machine that he called the Vodr. It worked using special keys and a foot pedal, and was capable of creating about 20 different electronic buzzes and sounds. When those sounds were combined, they formed words. The Vodr fascinated people at the 1939 World's Fair in New York.
The Vodr was an early example of electronic speech, but it was cumbersome to operate and require special training. Over the next 40 years, speech scientists continued studying the components of the human voice. They eventually developed methods to mathematically map the acoustic patterns and phonetic properties of natural speech, vowels, syllable constructions, consonants. By the 80s, speech synthesis was no longer the staff of science demonstrations at shows and fairs.
Text to speech systems are beginning to be applied in many ways, including aid for the handicapped, medical aid, and teaching devices. The first kind of aid to be considered as a talking aid for the locally handicapped. The research of Dennis Klatt at MIT paved the way for the voices we might be familiar with today, many of them used in assistive communication devices. I am beautiful Betty, the standard female voice.
I am very large person with the B-voice. I am the standard male voice, perfect all. This last one, by the way, became famous after Stephen Hawking adopted it. Speech technology has come a long way in the years since Homer Dudley unveiled the Vodr.
But in many ways, synthetic voices still sounded synthetic. They didn't convey all the information that's packed into the human voice. Voice is identity, right? Voice is about who you are, our voice signals, how old we are, our voice signals, our gender, our voice signals, you know, things about our personality.
Rubal Patel is a speech scientist at Northeastern University. Perhaps more than many people, she has thought a lot about the human voice. When she misses her mother, for instance, Rubal has a special technique to evoke her presence. That's right.
My parents now live in LA and I live here in Boston. And oftentimes, I find myself imitating my mom, you know, say, oh, baby, how are you? How are you? You know, or something like that.
I'll imitate her the way she might say something. I might say that the same way to my daughter or something like that. But what I'm evoking is my mother's voice, primarily to feel the closeness of her here. In 2002, Rubal took these ideas with her to Denmark, where she was scheduled to speak at a conference for researchers and patients.
I was presenting some of my early work showing that individuals with very severe speech disorder still have the ability to make sound. And those sounds have some communicative content and some information that could be used. After her presentation, she walked out to the exhibit hall. And that's where she noticed something.
Lots of people were using devices that produced synthetic voices. What was odd was that many of the voices didn't seem to match the people using them. At that point back in 2002, we had very limited synthetic voices of options available. And so I heard a little girl or a young girl using a device to talk with an adult male voice and having a conversation with another person, a middle-aged man, who also was using the same voice.
And so they're using different devices, but their voices were identical. She had just presented on the idea that our individual voices carry something unique about us. So why was this not reflected in these synthetic voices? Why are we giving them the same black box to speak through?
There's got to be something that we can do that we can harness the quality of the voices that they have, and imprint those or use that to give them a prosthetic voice that somehow reflects who they are and not just the same voice for everyone. Could a synthetic voice capture the richness of natural human speech? Rubal launched a company to answer this question. It's called Vocal ID, and it uses machine learning and other artificial intelligence technologies to create personalized voices.
So what synthetic speech is, is taking recordings of anyone, and then taking those recordings and building a model of the voice quality of the enunciation abilities, right? You aren't necessarily analyzing it from a top down saying, well, this person has a high-pitched voice, this person has a low-pitched voice. You're taking the recordings as basically the raw ingredients to feed to a machine learning algorithm or set of algorithms really. And those are learning the patterns of the clarity of the person's S.
How that sound is changed in the different phonetic environments, the voice quality aspects of it. All of these are learned by the machine. It's really re-emulating the human voice by the machine. In other words, the idea is to build a model of how a person sounds.
To do so, you use a vast range of examples of that person's speech. Then you use the model to produce spoken language that incorporates all the idiosyncrasies and texture of that person's voice. One of Rubal's early clients was a young girl, Mae of Flak. Mae was born with cerebral palsy.
She's in a beautiful family where she's two older sisters. Rubal's goal was to give Maeve her own unique synthetic voice, one that could express not just her words, but her identity. The first step was to record her. So those sounds were the kinds of sounds that are unique to Maeve.
Those are the sounds that she makes, where if, you know, in a classroom with several other kids who also have communication disabilities, when she makes that sound, you know what's Maeve speaking. So we harnessed those sounds of Maeve's to create her unique voice for her. Then Rubal turned to Maeve's older sisters, Erin and Maegan, who volunteered to record their voices, so they could be blended with Maeve's. A scream is my guilty pleasure.
That man ran fast. Erin and Maegan read hundreds of sentences and phrases and uploaded them to our website for Rubal. Like a painter mixing a palette, Rubal took elements of Maeve's voice and mixed them with those of her sisters and other vocal donors to create what she calls a bespoke voice. I can't wait for my friends to hear my new voice.
My parents are really happy. I'm not addicted to Fortnite. I want to meet Taylor Swift. So we're hearing Maeve at this age in terms of her sound as well as her siblings' recordings being combined and being produced through this speech in this ascension.
It's possible that Maeve may decide as she gets older that her voice needs to age with her. She'll need a new bespoke voice at that point. The same technology can also be used to preserve a person's existing voice. Sometimes this is done when a person faces the prospect of losing his voice.
These could be individuals who are losing their voice to degenerative conditions that slowly their voice is changing, such as ALS or Parkinson's disease. And then those who the trauma is actually far more pronounced for individuals with something like head and neck cancer where they learn that they're going to have their voice box removed within a couple of weeks. Lonnie Blanchard confronted this traumatic news in 2018. Doctors had diagnosed him with cancer and said surgery was the only option.
Lonnie had to have his tongue removed. Here he is, speaking to the BBC. Now that I know I'm going to lose my voice, I've got to get some sayings down on a person's recorder to get what I would normally say to my wife and kids. Every time I go to do that, I draw up a blank.
By the time Lonnie started working with RuPaul, he only had a few weeks to bank his voice. We helped him get set up in terms of the microphone he would need and things like that. RuPaul worked with Lonnie to build a database of sound samples before his surgery. He recorded sentences that gave RuPaul and her colleagues the different kinds of sounds they would need to build a new voice.
I wish we could get acquainted. I'm going to be a teacher when I grow up. After Lonnie banned his voice, RuPaul used the recordings to create a personalized voice for him. The difference between his voice and May's voice is that RuPaul didn't need to blend voices from donors.
Lonnie was his own donor. Those voice samples then are cleaned up and annotated by machine actually and then used to feed into the algorithms we have to create a synthetic voice. Similar to May, Lonnie uses an assistive device, in his case, an iPad. He can type out what he wants to say and hear his voice speaking to his family.
Once you close your eyes and let your mind relax, it doesn't take long to escape to the beautiful beach. It's really empowering. It's continued to be a way that he can connect to family members and feel part that part of him is not fully lost. While most of us will never have the experience of losing our voices and having to obtain synthetic voices as replacements.
Increasingly, many of us are coming into contact with these voices. Hey Siri, hey Google. How many ounces are in a cup? One cup is eight few ounces.
Hey Siri, set a timer. For how long? Fifty-six minutes. Okay, sure.
Fifty-six minutes. Starting now. Alexa, keep playing music. For how long?
It's a station you might like. Synthetic voices are already changing our lives and it's likely we're going to become even more reliant on them. In May 2018, Google revealed a new program it was working on. CEO Sundar Pichai presented it to an audience of software developers.
The technology is called Google Duplex. It allows you to make a restaurant reservation through a voice assistant. Hi, I'd like to reserve a table for Wednesday the 7th. For 7 people?
It's for four people. For people when? Wednesday? At 6pm.
Oh, actually we leave here for like a five people. For people you can come. How long is the wait usually to be seated? For when tomorrow or weekend?
For next Wednesday, the 7th. Oh, no, it's not too easy. You can come for 4 people, okay? Oh, I got you.
Thanks. The audience is laughing and applauding because the man making the call isn't a man, but a machine. It didn't seem like it was a robotic voice. The robotic voices were used to are the voices like when you are in a parking garage and you hear the, you know, please place your ticket to the strike facing to the right.
Very, very canned sort of speech. This was far more sophisticated and much more like you and I talk with hesitations and pauses and ums and us. You think it's a human on the other hand. Now, of course, one of the things about that voice that Google had is that it did seem like a convincing voice, but if you need to convince me that that voice is not just a human voice, but a particular human voice, you need to convince me that this is not just anyone calling for a restaurant reservation, but it's Barack Obama calling for a restaurant reservation.
Presumably not the bar is much much higher. That's right. It is. Barack Obama though does have a ton of this audio on the internet and there's a lot more audio to make, you know, his voice than there is my voice, for example.
And so, yeah, but it's absolutely possible to learn. If you have long enough, you can learn anybody's voice and if you have enough data. It's not hard to see how bad actors could misuse this, create havoc in people's lives, trouble at companies, political misinformation. That's absolutely, I mean, we're seeing deep fakes in video.
You know, President Obama's face with being manipulated and the audio coming out, you know, people creating these fake media in video and you're also seeing an audio. That's exactly why the security aspects of what we're doing are trying to detect. Is that fake audio? Or is that real?
Is part of that fake audio or is part of that real? Right? So, this isn't completely sci-fi. It isn't so far away.
It isn't necessarily 20, you know, 2028. It's probably 20, 20. So we've got to get our defenses up in terms of questioning where that the authenticity of audio just as we do video. In 2017, a Canadian company called Liobird showed how audio deep fakes might work in politics.
This is huge. They can make us say anything, not really anything. The good news is that they will offer the technology to anyone. This is huge.
How does their technology work? Hey, guys, I think that they use deep learning and artificial neural networks. By 2019, deep fake audio technology had gotten even better. Shortly after critics panned the final season of Game of Thrones, a YouTube channel called Eating Things with Famous People put out this tongue-in-cheek video showing the supposed remorse of the lead character, Jon Snow.
Time for some apologies. I'm sorry we were arrested for time. I'm sorry we didn't learn anything from the ending. I've lost.
I have more lines in this video than I had in the last season. I'm sorry we wrote this in like six days or something. Now, let's burn the script at season eight. I just forget it forever.
Spoofing a TV show is one thing, but imagine such high quality deep fakes occurring in a more high-stakes setting. Voices are increasingly being used by financial institutions to authenticate the identities of consumers. Recently, Rubel worked with a bank to assess how vulnerable it was to vocal hacking. We tested their authentication system by creating synthetic samples or synthetic voices of particular individuals who were enrolled in their authentication system, and we tried to test those voices against the system to see if we could get through with the synthetic voices.
We were not able to do that for every single voice, but we were able to do it for some voices. It just starts to show that there is a vulnerability in this technology. How would you guard against it? There are many ways to guard against it.
One is you can classify the difference between is the audio signal line listening to, is it synthetic or is it human? As the synthetic voices become better and better sounding, that will be a more difficult decision to make. It is something that if we can proactively solve, I think, or at least start to address. We're going to be way ahead of the curve than if we're trying to clean up our mess after the fact.
Despite the potential risks of these new technologies, Rubel is also optimistic. Voice synthesis tools have the potential to allow people to craft the voice they hear on the outside, so that it matches the identity they feel on the inside. Ideally, in the future, these decisions are made by the end-user themselves. Like, oh, I actually want that to be sound a little breathier.
I'd love that to sound a little bit more confident. I mean, how does that translate to the acoustics? We don't quite know yet, but that's actually, I think, where when we can finally give the control of what the voice sounds like to the individual, I mean, that's the holy grail. The first kind of age would be considered as a talking ape or the locally handicapped.
I am beautiful Betty, the standard female voice. I am the standard male voice, perfect call. I'm sad because there was no ice cream in the freezer. This guy is clear when the stars are clean.
I can't wait for my friends to hear my voice. One of the same few answers. You are listening to a machine. This is huge, how does their technology work?
Hey guys, I think that they use deep learning and artificial neural networks. Hi, I'd like to take a look at the table for Wednesday, the E7. This week's show was produced by Thomas Liu. It was edited by Tara Boyle and Reyna Cohen.
Our team includes Parth Shah, Jenny Schmidt and Laura Querel. Special thanks to Brent Bachman, Greg Sauer and K-Bahn Jones. Our unsung hero this week is Rebecca Rolf. She's part of NPR's team looking at our changing interactions with smart speakers.
She helped us record some of the smart devices you heard in this week's episode. Thanks Rebecca. If you like this episode, please share it with a friend. We're always looking for new people to discover hidden brain.
I'm Shankar Vedantam and this is NPR. At DID Electrical, we know getting a new appliance is genuinely exciting. A fridge with more space? Thrilling.
A washing machine that's actually quiet? Even better. This weekend only, at DID Electrical, buy selected Samsung large appliances and get 10% back on a DID rewards card. Plus enjoy 5 euro delivery and half price installation on all large appliances right now.
DID Electrical for everything you. The faint scent of factor 50. It's time to dust off your It's Not Burnt. It's well done, apron.
You're ready to tackle the garden. And suddenly, it's the busiest room in the house. It's summer. And the go play outside summer sale is now on at Home Value with great value on barbecues, lawnmores, garden furniture and more.
From now until June 4th or while stocks last. With 88 stores nationwide, call into your local home value or shoponline at homevalue.ie. Home Value. Trusted Local Experts.