Skip to content
Putting Our Assumptions to the Test episode artwork

Mar 7, 2022 52 min

Putting Our Assumptions to the Test

from Hidden Brain Hosted by Hidden Brain Media

Do you ever stop to wonder if the way you see the world is how the world really is? Economist Abhijit Banerjee has spent a lifetime asking himself this question. His answer: Our world views often don't reflect reality. The only way to get more accurate is to think like a scientist — even when you're not looking through a microscope. If you like this show, please check out our new podcast, My Unsung Hero! And if you’d like to support our work, you can do so at support.hiddenbrain.org.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Putting Our Assumptions to the Test

0:00 52:14
of matches

Transcript Auto-generated, may contain errors

This is Hidden Brain. I'm Shankar Vedantam. Human beings are always trying to make sense of the world. When things happen to us, or to our communities, or in our nations, we understand those events through the lens of culture, through ideology, or through the prism of our own perspectives.

You see a homeless man on the street, or a tycoon getting on a yacht. Why is one so poor, and the other so rich? Many of us have ready answers to these questions, because we've spent years or decades perfecting our preferred stories. But what would happen if we just...

stopped? If we told ourselves, you know, I might not actually have a very good handle on the world. My theories are just that. Theories.

This week on Hidden Brain, the story of a kid from Calcutta, who became a Nobel Prize winner by asking a deceptively simple question. How do you know that's true? At D.I.D. Electrical, we know getting a new appliance is genuinely exciting.

A fridge with more space? Thrilling. A washing machine that's actually quiet? Even better.

This weekend only, at D.I.D. Electrical, buy selected Samsung large appliances and get 10% back on a D.I.D. rewards card. Plus enjoy 5-euro delivery and half-price installation on all large appliances right now.

D.I.D. Electrical, for everything you. The bank holiday is finally here. Enjoy summer savers at Centra with great offers.

Like Centra Fresh Air Strip Runs Day 2-pack only 8-euro. Offer ends Sunday. Selected Pringles, 165 gram, 175 each. Plus Diet Coke and Coke, 0-12 compact.

Only 6-euro each plus deposit. Bank holiday value sorted. Centra, live every day. Okay, Frank, back up.

Keep coming. Go to room, go to room. Frank just got himself a new Kia EV2. He loves everything about the car.

How it looks, how it drives, its class-leading range, and its highly impressive spec. Back, back, back, back, back, back, back, back. In fact, the only thing Frank can't get used to is how much bigger it feels than it actually is. Go to room, go to room.

Discover the EV2 for yourself at your nearest Kia dealer. Kia, movement that inspires. In 1937, Abhichit Banachie's grandfather struck a real estate deal in Calcutta. It turned out to be a very bad deal.

Oh, my grandfather thought he was being very clever when he built his house. He thought, well, I'm getting this plot cheap, not quite getting the idea that there might be a reason why it was getting cheap. The plot of land turned out to be right on the edge of a slum. But as people see throughout this episode, things that sound like bad ideas can sometimes turn out well, and seemingly good ideas can turn out poorly.

Often, the only way to really find out how things will unfold is to watch and see what happens. Yeah, I think my grandfather, he loved the idea of doing something dramatic and with a flourish, and this was one of his flourishes, and it turned out, in many ways, it was a great idea, but not for the reasons he thought. Now, I can't see for sure that Abhijit Banerjee would not have won the 2019 Nobel Prize in Economics if his grandfather had not bought that plot of land next to a slum. But it definitely helped.

Here's why. When he was a child, Abhijit would go out onto the flat roof of the house. He discovered two things. One, it was the perfect place to launch a kite.

Because we literally were the last house, and the sidelines were for a mile. There was nothing blocking us. For flying a kite, it was the best place in the world. Like, you know, it was open, you know, there was no place your kite would get stuck.

The second gift of the house was that precisely because it was an ideal location to launch a kite, it became the neighborhood hangout for kids. And not just the kids who lived in the houses up the street, but the kids who lived in the tenements down in the slum. They were better at flying kites. We would fly kites together on our terrace.

They were better at it. And since flying kites is something that is very competitive, and you know, you would really want a strong team, it was nice to know them. To the middle-class families who lived in the houses, the people who lived in the slum might have sparked sympathy or perhaps judgment. But Abhijit, he regularly found himself in awe of his poorer claimants.

I was scared of playing marbles with them. I love playing marbles. In front of our house, actually, you know, you would dig a little hole in the ground and then you would play marbles and they were so much better at it than me. You know, for me, it was always a little bit, I was half admiring of their savvy.

They were more savvy than me and, you know, they would use swear words much more. I mean, I have a very developed vocabulary in that domain, as it turns out. I've been coached by experts. But these kids, they would use words that I wouldn't dare use.

Seeing their cool, their bravado, Abhijit experienced a strange emotion. This was the Huckleberry Finn moment for me. I mean, absolutely. There is a wisdom that for me was very clear that I didn't have.

I was also, for me, I played cricket with them and they were better at cricket than me. In fact, it was very interesting. They had somehow the idea that as someone from a middle-class family, I should be better at cricket and I was clearly palpably worse and they found that a little baffling, in fact. It's not like Abhijit was unaware that his friends from the slum struggled with hardship.

The contrast between his life and theirs was obvious. And it was very clear to me and my brother that, you know, there was this other life where the kids didn't go to school where, you know, you might be wearing a pant where it was kind of tied together with a rope and a shirt that was stitched together with safety pins and it sort of created a sense of somehow there was there was a other world out there and it's not that we were 10-year-old social scientists. We weren't. I think we took it as a given that poor people live that way.

But his close ties with these kids meant he did not see them as objects of pity or scorn. In the years that followed, these experiences as a child would help shape the work that led up to his Nobel Prize. Another early insight that shaped his perspective on the world came from an incident that unfolded at his school. In what will surely be heartening news to any student with less than stellar grades, our future Nobel laureate turns out to have been a really bad student.

So I did badly. I was everything. I was so not engaged with the whole school business that it was not. For several years, I was made to sit at the teacher's table because otherwise I would be looking out the window and paying no attention.

It was an extreme case of someone who had no interest in the school system. Arbitif's teachers were puzzled. He was the son of two accomplished economists. What could explain his underperformance?

If you think this is a child of two professors, he has to be talented. Therefore, the explanation for my deficiency was talent. That I'm so bored with what we got in class that I needed to be promoted. So instead of demoting me, I was promoted one grade.

In the middle of the year, I was taken from one grade and put in the next higher grade with the theory that once I'm stimulated enough, I'll do well. There was no reason to think that. But given that my parents were academics, the theory was that I must be able to do better. And so I was placed in this more advanced class.

I did equally badly there. In a sense, they were right. It was almost unrelated to whichever class they put me in, I would do equally badly. The teachers who thought Abhijit should be doing better academically were doing something we all do all the time.

When we look out at the world, we draw inferences from the events we see. We connect dots that may or may not be connected. We tell stories. These stories are powerful because they're often generated unconsciously.

The fact that the kid of two smart parents was doing badly in school didn't make sense to Abhijit's teachers. So they developed a theory to get the pieces to fit together. It was a story that satisfied Abhijit's parents even if they did very little to change the trajectory of his schooling. In time, Abhijit came to sense that such stories obscure the truth even if they reassure us that we understand the world.

This penchant for connecting the dots, for storytelling, abounded among the well-educated friends who came by his parents' home. Calcutta, or Kolkata as it's called today, is in a region of India called West Bengal. It's known for the passion with which people debate big ideas. Bengalis, at least in that era, were famous for armchair theorizing.

And I often would have this reaction, yeah, but how do you know that? Maybe just being surrounded by people who are extremely generous with their intuitions and conclusions was a way to be a little more skeptical. I think it was mostly reacting to my very voluble, emphatic environment I grew up in. People were very talkative and they all had strong views.

The Calcutta of Abhijit's youth was a hotbed of political arguments. By the time he was a teenager, Abhijit found himself pushing back against the sweeping theories of his friends and neighbors. I think the one strong influence on me was actually a reaction to, I think, left-wing ideology of a particularly Marxist kind. And I hit that very early.

This is Calcutta in the 1960s and 70s. Marxist ideology is everywhere and it's sort of the dominant ideological framework. And people had such quick answers to questions. And I must say, I think of myself very much as a leftist, but I did find that the speed at which people jumped to conclusions and sort of assumed that something was a result of some capitalist conspiracy or something, I found that very unsatisfying.

When Estrella Bengalis came up with stories about why the world was the way it was or why things worked the way they did, Abhijit often asked them to slow down. How did they know their theories were true? What was the evidence that backed up their claims? I think that was one thing that made me more empirical, I would say.

I like the idea of how things actually work at a more granular level. I think it's also a good antidote against very broad sweep theorizing. So I remember arguing with people when I was 15 or 16 and we were discussing social issues. And for me, I understand things and yes, you tell me this happened, but how did that capitalist make that conspiracy work?

How did they implement it? Later in his career, after he became an economist at MIT, Abhijit would describe what he came to do as the academic equivalent of plumbing. Where an architect might be interested in the broad sweep of a building, how the overall design comes together, a plumber is interested in whether the pipes are connected and the joints have leaks. Abhijit was always less interested in the big conclusion than in how things work at a granular level.

As a teenager, he came to be known among his friends as the guy who asked irritating questions. It was a very common trope of our conversations, me saying, why do you assume there is a reason for everything? I remember saying that sentence a lot. The fact that things happen doesn't mean there's a good reason for it.

And were these interjections received warmly or were you seen as being the fly in the ointment here of basically asking people how they came to the conclusions they did? I think I was seen as being slightly annoying. Sometimes people would have this exasperated look. He's making that point again.

Abhijit decided to follow his parents' career path and became an economist. As he pursued his training, he found that many people in the field shared the same tendencies as the voluble Bengali neighbors of his youth. People spent a lot of time arguing about different theories. Liberal economists clashed with conservative economists.

Labor economists had models of why the economy worked the way it did and they clashed with other economists whose frame was the market or monetary policy. Experts disagreed about how the world worked and what policies made the most sense. Theory was so powerful. Even when I was a graduate student, theory was key.

This is in the 1980s and I indeed followed the herd and studied theory as a result of that. It was very much the idea that we know what the fundamentals of human behavior are so if you put them together you get the answer. The empirical work was often shallow and shallow for the reason that doing experiments was seen as too challenging but also because people thought look it's a waste of time. I think there was a lot of confidence in the theory.

Despite his natural predilection to be wary of sweeping theories, Abhijit fell in line with his peers. For years, he followed the norms of his field. He too spent time coming up with intricate models of how the world worked, models that involved complex mathematics and very little has today decided to award the Svelius Rixbank Prize in Economic Sciences in memory of Alfred Nobel for 2019 jointly to Abhijit Panerjee, Hestad DiFlo and Michael Kramer for their experimental approach to alleviating global poverty. When we come back, how Abhijit found his way back to being the annoying teenager of his youth and discovered his real calling.

You're listening to Hidden Brain. I'm Shankar Vedanta. At D.I.D. Electrical, we know getting a new appliance is genuinely exciting.

A fridge with more space? Thrilling. A washing machine that's actually quiet? Even better.

This weekend only at D.I.D. Electrical buy selected Samsung large appliances and get 10% back on a D.I.D. Rewards card. Plus enjoy 5 euro delivery and half price installation on all large appliances right now.

D.I.D. Electrical for everything you. The bank holiday is finally here. Enjoy summer savers at Centra with great offers.

Buy Centra, fresh air, strip-roin, stay two-pack, only 8 euro. Offer end Sunday. Selected Pringles, 165 gram, 175 each, plus Diet Coke and Coke zero 12 compact, only 6 euro each, plus deposit. Bank holiday value sorted.

Centra, live every day. This is Hidden Brain. I'm Shankar Ghedanthan. In the field of economics, as in many other academic disciplines, researchers spend a lot of time theorizing about why the world works the way it does.

A libertarian economist, for example, might locate the roots of poverty in excessive government regulations that shackle entrepreneurs. A Marxist economist, on the other hand, might locate the roots of poverty in an unequal capitalist playing field. If you set the libertarian and the Marxist economists down in a room to chat, they'll argue for hours about which theory is right. You see the same love for theorizing an argument outside of academia too.

If you turn on cable TV any night of the week, you'll see progressive and conservative commentators explaining why the world works the way it does and how their model offers the best path to fixing problems. Abhijit Panerjee began his career as an economist, developing and refining intricate theories. That's what he did if he was smart. Job security and accolades usually followed.

Right from the start though, Abhijit worried that many competing theories were just stories. They connected the dots, often in superficial ways, much like his elementary school teachers who assumed he was failing as a student because he was actually an excellent student. His frustrations were particularly acute when it came to questions of poverty. Why are some people poor while others are not?

I never found it honestly very easy to have a clear-cut theory of poverty. It was always a little bit opaque to me and that's part of why maybe I found the economics particularly in that domain particularly unsatisfying. It seemed to me that it was a theory that really had very little in it. It was just, you know, either it was the kind theory which is that the poor don't have enough to eat so they're handicapped and we should think of them as being handicapped people who therefore won't be able to do very much for themselves and the less kind theory which is that people get what they deserve and unproductive people are poor and that's a very straightforward application of economics 101 which is that that's why they're poor.

I never found that particularly satisfying and I figured that the idea that it should come down to either not enough food bad genes or whatever. It seems to me to be extraordinarily shallow. The sense in which growing up, observing the talents of this set of people who I was playing with, it did inform me. Abhijit kept asking, quietly at first and increasingly more loudly, the question he had asked his Bengali neighbors as a youth.

How do you know that's true? Neither liberal nor conservative theories swirling among economists capture the complexity of the kids Abhijit knew from the slum. I did feel that they were often smart, interesting, funny people with gifts of their own. And I think that sense maybe is important.

As Abhijit pondered the disconnect between theories of poverty and his own experiences in Calcutta, he reflected that there might be lessons to learn from another academic discipline. Medicine had experienced something of a revolution over the previous 150 years. For a long time, medicine had been the domain of theoreticians. People in different countries developed theories about why people felt sick and how to cure illness.

Aristotle came up with the notion that the body had four humors or bodily fluids that needed to be kept in balance. When they were out of balance, people felt sick. You know, the way medicine was often made was people would have a broad theory of how the body functions and, you know, for example, humors and then we are going to counteract the humor. If it's heat, I'm going to counteract the heat.

And so it was often based on very analogic reasoning and the theory was often pretty shallow, but the driver of a lot of medical care was often not particularly well-founded theory of how the body works and therefore how that counteracts it. And I think the big revolution comes basically, people start to do observational studies. I think the famous one is cholera. And you start to see that, you know, in figuring out the roots of cholera or malaria, both of those, to figure out, you know, that malaria is caused by mosquitoes and cholera is caused by water.

Both of those were, I think, deep insights. Ronald Ross won the Nobel Prize for figuring that out. He did it in Calcutta, figured out how malaria gets transmitted. Those were still observational studies, but they meant that you started to keep track of data and you tried to make sure that the data wasn't contaminated by other things.

About 100 years ago, some medical scientists came up with an even more radical idea. Instead of arguing over which theory of disease was correct, why not simply run experiments to determine the truth? And in starting in the 1920s, basically, there's a set of biostatisticians who start to argue that you can just do experiments. You give, if you want to see whether the medicine works or not, pick people at random, give some the medicine, the others not the medicine, and you'll see whether it works or not.

As simple as that. Once that idea was there, it was hard to challenge it. And it changes the nature of medicine. Take, for example, the development of an effective treatment for AIDS.

It involved combining several different drugs into a single cocktail. The AIDS cocktail was, in a sense, just off-the-cuff reasoning. It was not deeply theorized. It was like a set of things that might work.

People put them together, and they tried it, and it worked. And it saved them millions of lives. The idea that we won't be able to theorize something, but we might have to solve the problem nonetheless, became very powerful in medicine. And that's just the idea that, you know, we try it.

It works. It's great. If it doesn't, we'll try something else. And it's often guided by relatively little theory.

Where doctors with different models of human illness may once have argued for years about which theory was right, the new approach said, spend less time arguing about why things work the way they do, and more time studying the granular details of how things work. Less architecting, more plumbing. You want to know if vaccine A works? Test it.

You want to find out if acupuncture is effective against diabetes? Run an experiment. Instead of arguing about whether bad parenting causes mental illness, give people different forms of psychotherapy or medication, and compare which one makes patients better. Obviously it's had to ask himself what would happen if he were to bring the same insight to economics.

Instead of putting grand theories on a pedestal, what if he ran controlled experiments and looked at the results? He found a partner, an NGO, working in a remote district in northern India. And he decided to study a question that had an obviously correct answer. The goal of this first experiment was not to generate any new insights, but to simply understand how to run a field experiment.

And so I wasn't trying to do anything deep. The idea was, take the obvious. The obvious is, the educational theory was very clear. Teacher-student ratios matter.

This was, you know, how schools advertise themselves. We have seven students per teacher or whatever. They talk of, not just of economics, education specialists was very much that. So we were all completely confident that if you double the number of teachers, we'll get better test scores.

Abhijit found the simplest setting for the experiment. In a rural area in the state of Rajasthan, many poor schools had just one teacher. Abhijit took a set of 40 schools. In 20 of them, picked at random, he doubled the number of teachers to two.

In the other 20, nothing was changed. We went from one teacher's schools to two teacher's schools. If you increase the number of teachers by 10%, that's a big intervention. It's expensive.

We had doubled it. So we thought this would be a slam dunk. Then the results came in. In schools with two teachers, test scores of students changed by nada.

Nothing. There was zero effect. I think our first reaction was, we're doing something wrong. Let's go measure better.

And so we did, I think, extremely expensive and painful measurement and found nothing. And at that point, for both of the NGO and us, it was really this aha moment, okay? There's something strange happening. We have to sort of go back to the drawing board, understand what's missing in that story.

Because it was a common sense of both economics and education that this will work. If anything works, this will work. And so it really changed my life. Because in fact, there were nuances there, none of which mattered.

There was an attempt to hire a woman teacher. The idea was that the girls would benefit more. And so we looked for all those things. I mean, we had the theory, which is that woman teacher, good for girls.

We had all the nice hypotheses which were laid out. And then it was just that. Did you find out sort of what actually was happening? Why?

I mean, even now, I'm sort of gobsmacked that this didn't have the effect that it should have had. I mean, doubling the number of teachers in school, by definition, almost, I think, should make sense. I mean, even if you ask me today, I will tell you, of course, it's going to improve the test scores of students. Did you ever figure out why the result turned out the way it did?

We did the same thing in Mumbai, in a later experiment, in the early 2000s, where we cut the class size from 40 to 20. And it had no effects on the test scores. So I think we're pretty convinced it was right. And then eventually, I think we came up with a story.

I can't tell you that it's necessarily right, but it fits with many other things. And the story is very simple. The way the Indian education system is constructed, the idea is there is content that needs to be delivered to the child every day. There's a syllabus, you follow that.

And then that's the best you can do. The fact that the child may or may not get it, these are one of the poorest people in India. These are female literacy in this area was 2%. So this is really an extremely underprivileged area.

Parents were in no position to support their children often. So if the children fall behind, they fall behind. And so I think what's happening is that the teacher was just teaching any child who managed to stay with the teacher, but the rest of the children were behind. And it really wasn't, there was no mechanism for integrating them.

You know, today's material is we'll learn these kinds of words. If you don't follow, that's too bad for you. We have actually lots of evidence suggesting that that's the right explanation. So therefore, it doesn't matter how many, that if you double the number of teachers, you know, there are only two or three children in that whole group who were able to follow.

The rest of them were lost in any case. They were bewildered looking at their hands. I could see that physically, I could see what was wrong, which is that the kids were often just, you know, so shell-shocked by the whole process that they weren't really even trying to engage. They were just staring into space.

I mean, one of the things that jumps out at me is, of course, you know, when we have these models in our heads about how the world is supposed to work, and then somebody comes along and provides you with data showing that the world doesn't work that way, you know, our first reaction might be to dismiss the data, but our second reaction would actually be to double down on the model. I mean, in some ways, you know, when I'm thinking about, you know, the experience that you had as a small child where, you know, you did badly in class and you got promoted, you know, one conclusion you could draw from that is, all right, so he's still disengaged in this class, this higher class. That must mean that he is still bored because this new class is also not up to par with how smart young Abhijit is. Let's promote him to the next class.

In other words, you can, when you rely on sort of your intuitions about how the world works, you can make the same mistake over and over again because you can interpret the results you're seeing in a way that conforms to your model. I think you're exactly right. People could have said that this just means that these kids are unteachable. I remember being in one conversation, I won't name the organization, a different one, where they said that, you know, basically, you know, these kids are so unteachable that you have to start at the beginning and, you know, reconstruct them in a sense to be able to learn.

And I remember saying, no, these kids are just being taught badly. And if they were taught better, they'll learn. And I think the experience of the next 20 years of our work on education has been entirely that, which is that if you actually focus on teaching the kids they want to learn, in two months, they make more progress than they make over the previous two years. And that's very much the conclusion.

But you could have easily taken an essentialist understanding, which is, you know, this case is hopeless. Yeah. Or you could have said, you know, yes, we increase the size of the class to the teachers by 100 percent. But what you really had to do was you had to increase it by 200 percent or 400 percent or 800 percent.

You could have doubled down on the same approach because your fundamental model is correct. That's also true. But that was too expensive. So that one didn't come up.

The one that did come up is how do you know these kids can be taught? Maybe they are so malnutrited that their brain has collapsed or something. This is something that keeps coming back, sadly. So in another set of studies that you did, you looked at the effectiveness of different strategies in getting small children vaccinated.

And if I remember correctly, in this particular area that you were working in, the vaccination rates were very low. They were like 2 percent or something. And there were sort of competing theories about why the vaccination rates were so low. And the NGOs and the government had sort of different theories about what happened.

Paint me a picture of where this was, when this was, and what the different theories were for the very low vaccination rate. This was also in rural Liverpool district. The NGO we were working with was the same one. When the vaccination conversation started, they convened a meeting of local NGOs.

And it was interesting. The local NGOs had the view that this is because the government's supply system doesn't work. The, you know, the government is incompetent. And, you know, when you want to get vaccinated, you can't get vaccinated.

The government, of course, has the opposite view, which is that these people, they're primitive. They have traditional beliefs. And therefore, you can't get them to get vaccinated because they don't want to get vaccinated. And our, so the NGO's interest was, in a sense, the one we were working with.

They were also interested primarily in showing that if you could have reliable supply of vaccines, that you can get vaccinated in a predictable way, then that's going to solve the problem. So the first intervention we had was exactly that. This NGO worked with the government. The government gave them the vaccines.

And they announced a day on which they would show up in each village. And on that day, they did arrive extremely punctually. And the vaccines got delivered. You know, anybody who came could get vaccinated.

So everything on the supply side worked. That raised it to, eventually, it was 6 percent in the privileges, which had the traditional, the status quo. It went to 17 percent, not to 100. Now, what we had done, partly just out of, again, just to see, let's see if it works.

Not particularly if I had a strong intuition. It was to say, OK, why don't we give them a small reward for getting vaccinated? And it was really one kilo of lentils, two and a half pounds of lentils. And that was, people's view was, this is useless.

But it was really very much the idea that this is some, you know, strange thing that academics think of. But, you know, that raised the vaccination rate to 36 percent. So that had an enormous effect. And I think we've repeated that particular exercise many times.

And every time we find the same thing, which is a small payment that makes the occasion memorable, is very important in getting people to get vaccinated. Otherwise, they just get drowned in, there's so many things happening in their lives, there's so much, so many challenges in being poor, it's easy to get distracted and forget to get vaccinated. So one of the points that you've made in the study is that you can call it an incentive, you can call it a bribe, you can call it a nudge. You know, you're basically giving people something that's unrelated to the vaccine, the benefit of the vaccine, you're giving them something unrelated to get them to take the vaccine.

And you made the point that in some ways, the reason that this is actually attractive to people, given that the gift is actually so small, is it changes what people are thinking about, perhaps on the day of the vaccination. What do you mean by that? You could always say, I'll do it next month. You know, you're busy, you have another child, he's four, and he needs to be taken care of.

And if you want to go get the younger child vaccinated, you have to bring him, and he's going to not particularly want to be sitting there, he's going to run around and chase after him. There's all kinds of good reasons to want to postpone. What the lentil does is it just says, okay, fine, okay, but if I do it, get it done today, I'm going to get that nice thing. And that's going to be, you know, a little bit of sweetness in my life.

I really think that that's the right psychology. I mean, I talked to some of these people who came for it. It's not that they would get rich because they got the lentils. It's just that it sort of makes that moment, when you are deciding between many compulsions, it's one, one compulsion that is very well defined.

It's, you know, I could go next month, next month, maybe I won't get this, this, it's right now, I'm going to get the lentils. Notice that the intervention that had the biggest effect on vaccination rates was not shaped by some grand theory about why poor people don't get vaccinated. It wasn't driven by the belief that incompetent government officials and unreliable supply chains were the source of the problem. It also wasn't connected with the even broader theory that poor people in rural Udaipur district had such primitive beliefs that they would refuse to get vaccinated.

It was just an experiment. Let's try this and see if it works. It was plumbing. The faucet has a leak.

Let's install a new washer and see if it fixes the problem. In other work along the same lines, for example, Abhijit and his colleagues have found that cash transfers to poor people do not prompt them to become lazy or drop out of the workforce, as many models in economics might predict. Many people use the money to live more comfortably. But Abhijit has also found that when you focus on the mundane and the practical, something surprising happens.

Once you discover that a new washer fixes the leak in the faucet, the solution tells you what your problem was. Once you see that increasing the number of teachers doesn't automatically increase test scores, it allows you to come up with a new theory. Some kids are being left behind in class, and to help them, you have to change the way you teach. Once you see that two pounds of lentils can dramatically increase your vaccination rate, you're less likely to reach for pat explanations about incompetent governments and primitive villagers.

This is exactly what has happened in medicine. The experimental evidence of what works and what doesn't work has turned out to be a powerful engine to improve our understanding of how the body functions. The experiments improve the theories. When we come back, more Abhijit's forays into control experiments and what the debate between theory and experiment can teach us about the human capacity for insight and humility.

You're listening. to Hidden Brain. I'm Shankar Vedanta. In 2019, Abitut Banerjee, Esther Duflo, and Michael Kramer were awarded the Nobel Prize in Economics for their experimental approach to alleviating global poverty.

Over the past two decades, they have not only come up with a number of discoveries, they have revolutionized the practice of economics itself. Abitut now works with teams in dozens of countries. He and his colleagues have now run hundreds of experiments and inspired thousands more. Increasingly, policymakers are taking a leaf from Abitut's book.

Instead of falling back on path ideological answers to questions of public policy, smart leaders are saying, Want to know which policy is the best policy in a given state or country? Don't argue about it on cable TV, just run an experiment and find out what works best. Take the question of how to fight malaria in poor countries. Everyone knew that when people in areas infested with malaria use bed nets, fewer people fall sick, fewer kids die.

The question policymakers confronted was, should you just give away bed nets for free? Would people value the nets less if they were free? Would they start using bed nets as fishing nets? The debate was in some ways a classic economics in the sense that on one side, yes, there were people who were saying, look, you know, if you signal to people that these things are free, they won't value them.

So even if you think it's entirely appropriate to give them free, don't give it free. So it was a debate between people. Nobody really wanted to charge the full price. I think there were crazies who did, but this was a debate among people who were all willing to subsidize it.

But just some people thought that, you know, if you make it fully free, then people won't value it. Now you could spend hours arguing the question. You could make a case for why charging a nominal amount for bed nets would actually increase usage and save lives. You could also make a case for why giving them away for free would save the most lives.

Some of our colleagues ran an experiment where different groups of people received either free bed nets or cheap bed nets. The researchers then carefully tracked their use. The result, there was no correlation between how much someone paid for the net and whether they used it. And then I think now, essentially everybody's giving away bed nets free.

And I think that's a great thing, but it came out, I think, until that moment where the evidence showed that the usage was unaffected by the price, I think everybody had that doubt. You know, there's all these stories about, you know, the condoms that were used as balloons and the bed nets being used as fishing nets. And those stories keep coming back. I think anecdotes were trumping evidence till that point.

The evidence really shut down the anecdotes. I think that was critical. So I just want to stay with this idea for a second because, of course, when we see anecdotes in the world, when we come by information about somebody who's using their malaria-fighting bed net as a fishing net or, you know, using a condom as a balloon, you know, it does lend itself to your coming up with a story. I mean, this is what we do.

When we look at events in the real world, we're constantly coming up with stories to explain why the events are the way they are. So it's not as if the people coming up with the stories are necessarily malicious or they're trying to draw the wrong conclusions about the world. Many of them, in fact, are deeply well-intentioned and want the best things for everyone else. It's just that the act of storytelling itself runs the risk that you're extrapolating from too little data to too sweeping a conclusion.

It's even worse than that, I think. I think the act of storytelling really pivots on narratives like these. They're wonderful, no? When you say, you know, see, there was this program and then the kids were playing with these strange balloons and when you looked at it, there were just little kids playing with condoms filled with water.

It's such a great story. So in fact, it's even better, even more seductive than, we need to have a narrative. These are wonderful narratives and therefore they're particularly compelling. We keep passing them on because, after all, if the story was that they didn't play with the condom, that's not an interesting story to tell.

So, you know, in a sense, these extreme examples are very, very seductive and they tend to over-inform us all the time. You know, I'm thinking about an underrated study for a second just to complement what you were saying. The state of California at one point decided to make salaries of state employees public and this was partly designed to, with a view of basically saying sunshine and transparency are always good things and one of the things they were trying to combat was the gender wage gap and they basically said by making salaries public and transparent we can reduce pay disparities between men and women. this is a fundamentally good thing to do.

The economist Emmanuel Saez basically analyzed employees of the University of California system and he found that as a result of this pay transparency some 20 to 40% of people working for the University of California system were now thinking of leaving their jobs because now they looked over their shoulders they saw that people who were doing work that were similar to theirs were now being paid more than them and now instead of being an engine for equity and transparency it became an engine for resentment and frustration and people wanting to leave. And again, I think it's an example of how the stories that we tell about the world sometimes have unintended consequences and sometimes no matter how many unintended consequences we see we can't help but constantly keep coming up with stories. Again, I think that's exactly right but in particular I think the words like transparency how could you be against it? It's transparent.

It's clear. There is a valorization that's built into these words and I think that's extremely compelling often and so I think to say that I'm against transparency makes me sound sleazy but in fact I am against transparency and that's a critical problem in bureaucracy. People want to have a paper trail and that's often the reason why we're happy to say bureaucracies are inefficient but in fact bureaucracies are often inefficient precisely because we want them to be transparent and that turns out to be one of the drivers of a lot of red tape. So I'm often skeptical of transparency.

Our values can be a powerful driver of storytelling that obscures the truth. Of course increasing the ratio of teachers to students is going to improve task force. Of course transparency is always a good thing. Another driver of this sort of storytelling partisanship.

It's so easy for us to see that the theories of our opponents are fanciful and misguided. It's so much harder to be skeptical of our own stories. I'm as guilty as anyone. I'm seduced by my own stories.

That's why I think the methodologies are useful. They're useful because in some sense while privately I hold on to many theories I think the fact that we are now in a place where people are able to challenge you and say do you have any evidence for it I think it does change. I don't know that we'll ever be not seduced by stories but I think the insistence of what's sometimes called the credibility revolution in economics. I think that's the discipline of saying well you have no evidence for it.

I think that does help the public conversation. I think we are in a better place now. So let's just stay with this idea for just a second because it's an important point. It is the case that randomized controlled trials can tell you something about the world that you didn't know.

It is true that the data can challenge your preconceived notions but as you point out there is a way of doing randomized controlled trials that in some ways predetermines the answers that you actually want to get or in some ways figures out how you're conducting the experiment to guide the experiment in a certain direction. So the point I'm trying to make is that the mere act a certain honesty in terms of actually approaching the experiment with an open mind with some humility in order to be able to generate those answers because if you actually again if you allow your preconceived notions and theories to guide how you're setting up the experiment in the first place you can very easily set up the experiment so that you always find eventually what you want to find. Or at least find the wrong answer even if you don't find it you may limit the set of possibilities so that you still learn something but you'll often learn in a very limited way. And I don't think there is any obvious good antidote to the other one you said which is ask yourself why do you believe that this is the right set of possibilities and I think we try but it's hard because again it's easy to be seduced by your own narrative as you said before.

I'm wondering as someone who's won the Nobel Prize in economics you are widely seen as an expert and people must defer to your opinions in a variety of settings academic settings social settings community settings do you sometimes feel like you are at risk yourself of falling prey to some of the models that you have challenged because now people sort of say well you know Abhijit Banerjee Nobel Prize winner clearly he must know the correct answer. Oh yeah yeah I call it the oracular status yeah you're the oracle now you can just speak you know absolutely it's extremely dangerous and tempting sometimes because you know in the end on some things I feel that I have an opinion I might as well say it and I can't say that I always resist I do try very hard to tell myself I need to resist shut up shut up shut up do I always manage? We've sort of circled around the same question I think multiple times which is sort of the importance of humility sort of what I think I'm hearing over and over again sort of the trajectory of your life's work you know Richard Feynman said the first rule is that you must not fool yourself and you are the easiest person to fool so I mean we've known this for a very long time but it feels like in practice this is actually really hard to do and it is really hard to do partly because I think the idea that most answers come with enormous uncertainty it's very hard to convey uncertainty is very hard to convey you can say I think my specific belief is 7 but it could be 11 or 4 or 3 and then people don't hear the 11 or 3 they just hear the 7 it's very hard to convey uncertainty when I speak should I be silent therefore? that's a very hard dilemma I don't know that any of us know our way around it because we know that it will be over interpreted if I say anything it's going to be over interpreted now that I have a Nobel Prize even more over interpreted so it's just you know so you sort of hold back and say well should I shut up?

but then is it okay to shut up in a context where lives are at stake? it's really and I don't know that there is a good resolution to that but you're right in saying that it's extremely fundamental tension in any academic's life I think in mine especially Abhijit Panerjee is an economist at MIT along with Esther Duflo and Michael Kramer he was awarded the 2019 Nobel Prize in Economics Abhijit thank you for joining me today on Hidden Brain thank you very much for having me Hidden Brain is produced by Hidden Brain Media our production team includes Bridget McCarthy Annie Murphy-Paul Kristen Wong Laura Corral Ryan Katz Autumn Barnes and Andrew Chadwick Tara Boyle is our executive producer I'm Hidden Brain's executive editor our unsung hero this week is Sereda McDonough Sereda is one of the listeners who's made a financial contribution to Hidden Brain Sereda thank you so much for your signal of support for our work it truly provides an emotional boost to all of us on the team if you'd like to join Sereda in supporting the show you can do so at support.hiddenbrain.org we're working on an episode and we're wondering if you have a story that we could feature on the show can you think of a time when an enemy became a friend what caused you to clash with the other person in the first place and what was the moment that changed your relationship if you have a personal story you are willing to share with the Hidden Brain audience record a voice memo and email it to us at ideas.hiddenbrain.org use the subject line enemies that email again is ideas.hiddenbrain.org I'm Shankar Paitanthan see you soon

Frequently asked questions

How long is this episode of Hidden Brain?

This episode is 52 minutes long.

When was this Hidden Brain episode published?

This episode was published on March 7, 2022.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this Hidden Brain episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.