Preparedness and Research

All right, welcome to season five of Inside Mike and New Head. This is the Canadian podcast. It's all about the integration of leading an incredible, awesome life, individual responsibility, individual emergency preparedness, and how do we navigate life's incredible, wild disruptions that we see among us. Sit back, grab a beverage, and enjoy the episodes.

All right, my friends. Welcome back to Auto Studios Inside Mike and New Head. This week, we are going to talk about research, how and why people are putting out all what sometimes seems to be like a bunch of ridiculous malarkey about preparedness and other elements of disaster and emergency management. Sit back, grab a beverage, let's get at her.

So research is much maligned. And most of what we talk about is always that, you know, that mysterious scientific process, the scientific method, where somebody says, well, it's in a peer review journal, so be quiet, and it must be right. So today we're going to talk a bit about the research paradigm, how it's supposed to be done, the way different ways it's done. Just a few interesting things.

I only say this because I think it's a really important point right now because COP27 is coming to an end, eventually, thank the Lord in Sharmel, Shaken, Egypt. And a lot of the world is talking about all of these things we have to do and look at all the wonderful research we have about the effects of climate change and the disastrous future we're going to leave our children if we don't make some massive fundamental choices. Look at the IPCC reports and look at all the other things that are coming out telling us the catastrophic wave, which is probably already over top of us, but it's coming nonetheless. So you ask yourself, like, how do they do this research?

What do we know? So today I thought I would dive into it because number one, I'm in the middle of within a month, I'm going to be defending my own PhD dissertation on emergency preparedness. And so I'm knee deep in it. So I figured why not?

Let's take a break from my own studying and talk a bit about how science in general comes to the conclusions that are put in the public sphere and then politicians grab a hold of and start to make some fundamental decisions. You can translate what we're going to talk about today to the COVID pandemic and many other issues that people say, well, I trust the science. Well, what does that actually mean? And you might want to assume it doesn't mean quite different things to different people.

So that's the plan today. I just wanted to start off with thanks for all the wonderful comments about my coffee episode. It was mirrored to a LinkedIn post and it got a lot of great traction. A lot of people, wonderful comments about how the difference between instant coffee and people like me who are a bit of a pour over can really differentiate between the type of preparedness, information is out there and the type of preparedness solution you're looking for.

You're looking for this rapid, quick, easy buy and go. I've got a busy life where you're actually interested in being involved. So that was great. I appreciate your feedback.

Keep it coming. We're a little light on the stuff coming out on our social media feeds right now because as I said, I am knee deep in the final preparations of ending, hopefully, a five plus year pursuit of a PhD. So really probably until January 2023, it's going to be a bit slow. But here we are.

The world is what it is. We all got our own personal going forward. So how do we start this, right? So where science really needs to step in is the number one.

What's the problem? The proper definition of the problem. What is it that you are concerned about? What is it that bothers you?

How do you define the problem? Is the fundamental foundational issue that we have in any pursuit of any scientific question? It doesn't matter your field. What exactly is the problem that you are facing that you would like to inquire about?

How do you clearly and unequivocally define that problem? And I'll use throughout today my research that I'm doing just because it's near and dear to my heart and it's easy to explain. My problem was defined as despite decades of messaging from all three levels of government about emergency preparedness, Canadians remain unprepared for disasters. See that's how I frame the problem.

I've framed the problem in the fact that I know what's going on. The problem is well defined. We know based upon the seper, which is the survey of emergency preparedness and resilience that was done by StatsCan in 2014. We haven't done another one since, but that's the most recent detailed assessment of tens of thousands of Canadian households as to their level of emergency preparedness and a number of other smaller studies that have ends in the somewhere between 50 and 1,000 that have been done in the subsequent decade within Canada.

But collectively, we know you're defining a problem that needs to be a statement of fact, right? It's not I don't know what's going on. The problem is well defined. Here's the fact Canadians are not prepared for disasters based upon all of this evidence that we have, despite efforts of the government.

That's the problem. You're not asking a question at this point. You are not asking a question whatsoever. You are defining the problem in so consider it like a space.

This is the space in which you want to investigate. So now once you've got the problem defined, the next thing you have to do is understand what we know about that problem. So what do we know about the problem of Canadians not being prepared for disasters? And to answer that question, you look at what has been done in this space.

What has people written about? And this is where you start what academia and science looks at a hierarchy of documents. The first is your peer review literature review. So basically a literature review on a problem like I have to find earlier would be the activity of going into all of the known research and all of the known publications about emergency, individual emergency preparedness to figure out what the world knows, what questions have been asked, what were the conclusions of those questions, what were the highlights.

And so in peer reviewed journals, it simply means that someone did a scientific study, came to a set of conclusions. They've handed the body of work or their article about their scientific study and they've given it to somebody else who has the necessary credentials to look at the methodology. Your peer review is not necessarily going to be somebody of the exact same discipline, right? Because they're looking at the scientific method, which is the overarching process that all scientists, whether it's natural, those are the folks in the chemistry, physics, that type of field, to the social sciences, people like me that are in humanities, economics, et cetera.

We all follow the scientific method, right? And this is kind of what I'm explaining here today. So the peer review process looks at it to make sure that you followed the method, that it is tested, that it is defendable, that is repeatable, and that your conclusions are based upon the evidence which you've presented. So if you try to write a peer reviewed article or an article for peer review and you make a whole bunch of conclusions, but they're not backed up in any evidence that you present in charts, et cetera, it falls flat.

The number one place that everybody goes to is chapter three in a publication, which is your methodology. So your methodology paragraphs are the first thing that peer review will go at. They want to look to make sure what you did was defendable, was the appropriate methods and checks and balances you use. So the idea is peer review is supposed to have this significant review done on it.

Now, peer review has been best described as it's the best process of all the worst choices. There are so many holes in the peer review process, but by what we have right now in science, it's the best process, but it's bad. It's bad because so much of peer review is influenced by individuals wanting to get their own publications. It's rife with stories of where reviewers will say, hey, cite these three additional papers and I love what you're doing except they wrote the three papers because authors need to get published and need to get cited because there's this thing called influence factor or impact factor, sorry, that's in academia where it actually scores your publications and sightings on a ratio that you're able to publicly compare with your peers.

It's almost like an Olympic decathlon scoring mechanism. There's a lot of other problems and sometimes a peer review process. You can give your article to two different peer reviewers. Most of it is through blind anonymity and it comes back.

One loves it. A few minor changes. The other one throws it in the bin from the beginning saying it's a horrible piece, but peer review is meant to be a higher standard because it's gone supposedly a rigorous process to evaluate it. Then we look at gray literature and gray literature is what kind of like government reports would be your IPCC reports that come out of the United Nations about climate change, reports out of the World Bank.

So organizational reports that have not gone through the peer review literature, but they're held in high enough stature that these are really scientifically thinking people that wrote these type of things. Then the last thing that you check is what we call, colloquially referred to as open source media. You're going to look at what everybody says online. You're going to look at what the media has said.

So you're looking at a balance of those three, but not balanced equally as I said. What that does is it tells you what do we know now about the problem? What do we know about that problem we defined? So I did a literature view.

It literally took me two and a half months of full-time work to complete the literature view and individual emergency preparedness and household emergency preparedness at the household community level, because there's that much that has been written about this subject matter. So once you're done that and you know what the world has said and written about it, now you have to define your question. All scientific examinations that respect the scientific method follow a question. Define the question.

We always try to avoid the why because why is so subjective and so difficult. We also almost completely avoid, except a few nutcases out there, the future tense, because scientific evaluation predicts the future, not by looking at what might happen. We predict the future based upon evidence as to what did happen. So if I want to know what the future climate is going to look like in North America in the next hundred years, I can't look out.

What I have to do as a scientist is I have to look back at what has happened over a significant period of time. I have to look at the evidence we have about what actually happened to be a predictor of what likely is to occur in the future. So scientists that are asking about what's going to happen and just colloquially goes on about based upon their opinion and experience what determines what will happen in the future. They're not scientists.

They're just authors. Scientists don't go based upon opinion and experience. We look at evidence. Evidence determined what happened.

Right? I really don't care what your opinion about things. All right, you can call yourself an expert. I don't care what your experience was in the middle of something.

I really don't at this point. I'm looking at the evidence. What are the facts? What actually did happen?

Because the evidence will tell me what happened beyond any reasonable doubt. Right? Then I'm going to move to the question, tell me why this happened. This is where I'm interested in your opinion and I'm interested in your experience.

Right? I don't want you to tell me what happened. I already know that. Right?

Because if you, if I ask you what happened, it's going through such a bias lens that I have to throw at all that material. Right? But when I come down to sit and interview you about a question, I already know what happened. Right?

I've already studied the evidence. I know unequivocally without question and arguably exactly what happened. What I want from you is your opinion and experience about why you think that occurred. Because what I'm going to get is it's from a point of view.

It's not the truth. Right? Because if you think of an event, if you look at a major disaster that has occurred, say a hurricane Ian that recently went through Southeastern United States, significant damage, you asked people what went on there. They're going to tell you their experience through their point of view.

To them, it's the truth. But the evidence tells us exactly what did happen. Now you can say, yeah, their house is blown away. Okay.

Well, the house isn't there anymore. But what happened? Was it something else? Was it a bunch of factors?

That's what the evidence difference is. A lot of times we see this debate on facts. And we hear it all the time in the news media about the fight on COVID-19 and vaccine and masks work. Well, the facts are this, the facts are that.

Well, you know what? The facts are never ever in dispute. There is no debating what happened. It's never about whether you liked it, whether you agreed with it, whether it fell into what your little belief system is.

I don't care. Facts are not in dispute. This is why as a researcher asking any important question within a field of study, you're going to go after the evidence first. And the evidence clearly tells you what exactly unequivocally happened.

That is your statement of fact, your interviews with human beings and your interviews of secondary material like looking at open source media, that type of thing is going to start to inform you what were the public opinions? What were people thinking? Why did people say they did the things they did? Those are all great things to add on, but they're not facts.

Facts are the evidence that underline. And then once you have the facts, you're in a position now where you ask all those other questions. I went back and I found all the facts related to the emergency preparedness, what's going on in Canada. And my question was about the relationship between municipal emergency managers and the communities they serve.

In the province of Ontario, it is required under legislation that all 444 municipalities conduct emergency preparedness communications and warn the public. That's in the law. It's actually in the law and it's amplified in Regulation 384. So that is the evidence I have.

The evidence tells me that there's a legal framework requirement for a municipality in the province of Ontario to issue preparedness information. So then what I have to do before I talk to anybody is I'm going to go out and I'm going to ask myself and inquire what happened, what was done. And so now the next part of my research was I went out and I looked at two years of social media detail because I had defined emergency, my examination is on emergency preparedness information on social media. So I defined my spaces social media.

So I wasn't looking at emergency preparedness information elsewhere. So I looked at the social media, right, which I also included their online presence at a municipal website and I looked at two years of data. I looked at absolutely everything that a select group, a purposeful sample, a purposeful then random sample of municipalities that I chose. I looked at everything that they said and did online about emergency preparedness communication on social media, right?

So I built the evidence. I looked at one year 2019 pre pandemic and I looked at 2020 in the pandemic because I was very much interested in how the pandemic influenced emergency preparedness communications in Ontario. And so I spent a large amount of time examining a year worth of social media detail from a number of municipalities because what that did is that's the fact, right? I now know unequivocally without question exactly what the evidence tells me.

I know exactly the extent of the emergency preparedness communications on social media for this group of municipalities, right? I don't need to go ask the municipal emergency manager or the person responsible for emergency preparedness communication and municipality. What did you do? What type of emergency preparedness information did you send out?

What was your intent? What did you ask any of that because I already know the answer so that when I sat down and interviewed the municipal emergency managers, the questions were framed in, why did you choose this set of wording? Why did you choose this frequency? Why did you choose this social media platform versus the other social media platform?

Do you use a branded emergency management account? Or sorry, the question is why didn't you use a branded emergency management account because none of them in my study did. There are a few in Ontario that do have a branded emergency management social media account, but they didn't. I'm asking a whole bunch of questions about the evidence that I already know.

So it's not a question of me not understanding what happened. It's me not understanding and really interested as to the why did this phenomenon occur? Why did you conduct emergency preparedness communications in the way that you did? And then I can say to them, and this was one of the big questions that started a lot of great debate with them, was what this is what you did in 2019 pre-pandemic.

So this is we can call your steady state normal operations. I looked at 12 months, so I would see all of the seasonal weather related effects of all the different types of emergencies that can occur related to weather. We're in Canada. We get heat storms.

We get hurricanes. We get ice storms. We get massive winter dumps and everything that goes along with it. Nor Easter if you're on these goes.

So you look at a year and so I saw the type of information they put out and then I contrasted it to exactly what they did in 2020, which had the same weather impacts. The pandemic did not affect the weather. There are some not cases that think it does, but the pandemic did not affect the weather. The weather carried on as per normal expectations, but there was a significant change in the emergency preparedness communications in the fact that it for all intents and purposes disappeared from the scope and all the emergency management folks were doing is rinse and repeat public health messaging.

Now, I'm not going to ask them, did they do it? Because I know they did it. I know they almost completely abandoned their legal responsibility for communicating emergency preparedness information, but the question then is why that and why only that? Why did you make the conscious choice to suspend emergency preparedness communications for that of pandemic communications and public health communications?

These arguing the public health communications are not good, but you have a legal framework requirement, right? And then you know, you ask them simple question. Did you receive any communications from the commissioner emergency management within the solicitor general of Ontario to advise you that they are for a short period of time to spending your legal requirement to execute this? And they're like, well, no, that didn't happen.

Oh, okay. Well, then maybe why this? And so you understand the process of the scientific method requires a lot of very specific steps that you take. The first one is always going to be defining the problem for which you want to investigate, being able to really codify it in a sentence or two so that anybody not within your field can understand it.

Then you have to understand and the biggest part about this is the literature view and most graduate students hate this portion of the process because it is unforgiving, right? It is literally unforgiving. You never really truly know until you start wrapping around in a circle that you've completed the literature view and wrapping around is the process where you're into about your, you know, your seventh or your eight hundredth piece of literature that you're looking at and you start to hear the same themes and you start to see the same authors and you start to see that you've now followed the process and the information almost full circle, right? So literature view is the most important part because unless you know what we already as a species, as humanity as we already know about something, it's very difficult for you to jump into the middle.

And a lot of people, this is confused a lot of people about what's been going on in the pandemic and the quote unquote trust the science, right? You got to understand that science is just trustworthy enough to be reasonably reliable for a short period of time until we learn something new. So when something is never settled in science, right? So right now what we know about the effects of the COVID-19 infection on humanity is because it started, we know it started in September of 2019, but let's follow the government lines in the official pandemic in Canada in 20, in say March of 2020.

So we're coming up on year three. So we literally have three years of data to tell us what has happened in the, in, within the COVID-19 pandemic, right? So we can look at that. We can look at what, how it pro, you know, how it went along.

We can look at the best available medical data we have and what scientists have to do with that is they have to look at the data. We know the evidence, the facts, not the ones in public dispute, but the facts and then be able to use that to predict what is likely to occur in the future. And the difficulty we have with this is, and the challenge with it is, is that the vast majority of information from the last major pandemic of this scope, the Spanish flu is a freaking guess. And I'll challenge any researcher that wants to come on this podcast.

All we have is a freaking guess from what went on back then, right? We don't have accurate records. We don't have accurate testing. We don't have accurate numbers.

We don't even have an accurate timeline process. All of that is in dispute because the evidence is not in a position where we can say it's a fact. So basically we're doing our best to try to understand what we think happened in 20, sorry, in the 1917, 1918, and 1919, and then we're going to try to inform what might happen now. And because they're making that correlation, they're doing it based upon little to no evidence, and they're not facts.

And so we have a lot of difficulty. So basically we have three years of really, really strong evidence to think what the next 20 years will look like. And that's a very difficult position. So when somebody states out to you says, trust the science, always ask how we got here.

What is the evidence? So if you want to help yourself out and everything that you've heard today goes in a bit about how we do science and how science is supposed to be done from a research perspective, ask the methodological questions. When somebody says a paper did so and so, go into the paper, look at the paper itself. It's that trust but verify, right?

And once you're in, once you're needy, like I am and many others are in graduate study work, you understand that we always trust but verify, right? Okay, I'm going to trust the government to produce a piece of data that is accurate, right? But I'm actually going to verify it because I'm going to go in and I'm going to look at the methodological decisions that were made in the construct of the experiment, the scientific experiment to see whether it meets the necessary rigor and standard of a scientific experiment. It's like the mice joke that's out there but it's not, it's true that something like eight mice were the subjects for the variant, the omnichron variant booster.

I was trying to remember the exact name for it but they actually studied it on eight mice and then sent the results forward. Now most of us in the scientific community would say that is far less than an appropriate level. Other people would say we don't have the time and effort to go through a complete and full proper trial because lives are on the line. Well, that's a balance and that's a question for another bidet but in the scientific method we know that there are appropriate ways to do things and if things follow an appropriate methodological process and it is done in accordance with the standards of the scientific method then we can make correlations and conclusions based upon the results of our experiment.

But if you read papers, you understand that almost all of the authors at the end have a caveat that we found this with a study of 800 patients over a longitudinal period of two years. And based upon that small sample size, we believe it's very informative. We believe it's very important for future studies but we highly recommend that this study be repeated on a larger group of people for a longer period to confirm our results. That's what scientists are saying at the end but you never see that published.

That's not what, you know, news hounds and the news anchors want to find. They'll take a couple of quick snippets out of the discussion and conclusion paragraphs of a research article, etc. So if you do nothing else as a result of this pandemic, go on to open source, go to Google Scholar, Google the problem you want to find, find the latest piece of research. Now you're going to have to jump around a bit because a lot of it is behind a paywall, find something that's not behind a paywall and go look at their methodology, look at their decisions that they made, the sample size, and then the test and either the most of the time it's a mathematical statistical test whether it's a p-score regression analysis or something that they conducted on the data to come to their conclusions.

Google what all that means, the regression and stuff like that. Google will tell you you can watch a short YouTube video on it. If you understand that process, you are a far more informed and a far more intelligent human being and you're much better prepared to jump into your Twitter Wars on whatever the problem is. So I hope this was helpful.

I thought it was be really a, you know, a neat idea to research. I'm a research nerd. Some people out there are not going to find this the best and greatest thing since sliced bread. But the idea is this is what you have to go through in the process to get to a point to be able to say this is what I'm doing.

This is how we conduct research when we believe research needs to be conducted. So thank you very much for taking the time. I'm going to drop by my website at www.insidemycanoohead.ca. Hang around my social medias.

We're having a lot of fun right now. We're really not putting a massive amount of effort into it. I've got something else on the go. Wait until 2023.

It's going to be incredible. Take care.