Thanks for downloading and welcome to Take Aurelie, the podcast from Dream Queens Medical Centre, Nottingham. This episode is a live recording of the fifth session of Unblinding Research on how clinical guidelines are made. As ever, all information is correct at the time of recording. Any and all guidelines mentioned are correct for Nottingham University Hospitals NHS Trust. Other trust guidelines may vary. All views and opinions are the speaker's own. Right then, so welcome back to Unblinding Research. Apologies that last month's session can't go ahead, so that's now being held today. Thank you.
For anyone who wants to mention us on Twitter, the hashtag is unblindingresearch. And there's the new Twitter handle for Dream and for our lovely research team. Thank you very much for coming in with your help with this session. Once again, as ever, this is a safe zone. Please feel free to talk. And if you have a question, put your hand up to make a point. And as I said, it's nice and friendly. If you want to say anything, just put your hand up.
So, on previous sessions, if you won't remember way back when, in our very first session of Unblinding Research, we looked at an introduction to research, the research method. We used the example of uniforms with ED nurses, and part of that research method, we would have to do a literature review, and we'd have to think about what outcome we want, and then that outcome will guide us with our method. We then had a look at formulating a clinical question, a research question, and we thought about PICO. What does P stand for? Population, yeah.
I stands for intervention, brilliant, yep. C, control, yep. And O is our outcome, yeah, okay. First used in the 18th century when we were studying the effects of citrus fruits and how they can prevent scurvy, yep. Bonus question, do you know who invented the quality improvement project? So it was Florence Nightingale, she invented the quality improvement project, she also invented the pie chart. So after thinking about PICO, we had a session on ethics.
We talked about a tale of two cities. We looked at Nuremberg and Helsinki declarations. We had a talk about GCP and how to get ethical approval. And then in our last session, we looked at types of trial. And we had a go at designing a randomized trial using the medium of sweets. We are not sponsored by Maynard, sadly. But we had a talk about how you would go about doing a randomized trial, looking at comparing Maynard's wine gums to generic wine gums. So that leads on to our scenario today, how our clinical guidelines produced.
I think we all use clinical guidelines every day. I think a common one that immediately springs to mind are the nice CT head guidelines. How many CT heads do you request? Yeah, eight a day, I recommend, especially if you're in an IAU, if you're at the front of the department, you're going to be seeing a lot of patients, you're going to be requesting a lot of CT heads because you follow nice CT head guidelines. pick-up rate's probably very, very low. I won't be surprised if I get one positive finding for every 100 CT heads that I request. If you're a GP with different guidelines and a different patient group who comes to you, you've probably got a higher rate of pick-up because you do fewer CT heads.
So in this session we're going to understand levels of evidence and quality of evidence, in particular meta-analysis and systematic review, we're going to have a chat about why they're so good, we're going to understand the role and limitations of evidence in developing clinical guidelines, and there's advantages and disadvantages to using guidelines, and we're going to have a think about NICE and how they make guidelines. So first up, we're going to have a bit of group work. So in your groups, you've got some made up studies that I've put in front of you, all about nurses and sweets. What I'd like you to do is rank them. So there are seven studies that I'd like you to rank them from the lowest level of evidence, that's the least useful, up to the highest level of evidence, the more useful the evidence is. Because as with computer games, there are levels of evidence. So to inspire you as we're going along, I'm going to put in the Super Mario Brothers theme tune.
Okay, so if you want to have a discussion and just rank those studies from the lowest level of evidence to the highest level of evidence. Right then, so, levels of evidence. So if we look at our absolute lowest level, this is our PlayStation 1, right there at the bottom. Yeah, I still have mine too as well. Yeah, absolutely. I've not used it in, well, well over a decade, but I still have it somewhere.
So yeah, so the bottom level is the opinion piece. So we present an article written by the RCN president who argues that research nurses should be given free sweets as a sign of goodwill, which will undoubtedly cause their recruitment of studies to go up. So that's an opinion piece. There's somebody of good standing who's written a piece, but it's just their opinion. Opinion pieces may include some research in them. They may quote other studies, but should always be taken with a pinch of salt. And then after that is our case report. Yeah, so we report the case of a research nurse who after being given three wine gums a day, as well as their normal lunch, increased their recruitment of studies by 24%.
So case reports, how useful are they? Well, as we said, they're low down. They're interesting, as Leon said, because they can identify new trends. So HIV and AIDS, as we now know, first spotted back in 1983, was a case report, the Mobility and Mortality Weekly Report from the CDC, noticed that a group of homosexual men were suffering from a rare form of pneumonia. And from that, it was the first identification of potentially this new disease, which then went on to become HIV and AIDS. So they serve an educational purpose, they're not generalisable, and they may focus on the rare.
As we've all been, I'm sure, to conferences where they like to go, this rare case of this came up and it's useful, it's interesting, but actually how useful is it going to be? And so after that, we're on to our PlayStation 2. And so now we're into our case series. So this is our study into access to suites and research nurse retainment. Ten research nurses began working in our department in May last year. They each received a free bag of wine gums on arrival. A year later, six of the nurses remain in post.
So case series look at patients with a known exposure, so that could be a treatment, it could be a vaccination, it could be an education program, but you've done something and you've followed them up. There's no comparison, so they don't have a control, so that's why they're low down here. Following on from there is our case control. So this was an example of a case control study. You have a known outcome. So this could be a disease, this could be death, etc. And you look back and you want to see, well, what was different about that group and a similar group? So you're basically looking to see the relationship between a risk factor or an exposure and that outcome. Quicker and more useful for initial studies in rare diseases because we know the outcomes occurred. But with the retrospective, like all studies, we call bias already.
And above, just above them, but still within this PlayStation 2 realm, we have our cohort study. So we report our study into research nurse intake of suites. We compared a group of research nurse staff nurses. We count that research nurses become a retreat in the cohort suites and work for key nurses. So this is a cohort study, which you're looking at by cohort and you're following them through perspective. And then you can assess the effects of certain factors. Participants in the group can be matched with a subject in another group. Variables, of course, if you go that you haven't paid for the background, you can make sure that there's an investment with that group as well that you're looking at.
That then tries to limit variables as much as possible. They are easier to carry out than randomized control trials. There is no randomization though, they do take a lot of time. And they are susceptible to confounding factors. So they're all our sort of PlayStation 2 level. So PlayStation 3 level, this is where we're into our randomized control trials. We present our study into suites and research nurse recruitment to studies.
We compared Maynard wine gums to an identical tasting placebo. We randomly allocated 100 research nurses into two groups of 50. One group had Maynards every day, while the other group had the placebo. Neither the participants or study staff knew the allocation. The Maynard group recruited 12% more patients to study. So this is a randomized control trial. These randomly assigned participants to either the treatment or the placebo group. Blinding is usually involved. They're expensive, but they are considered the gold standard if you're doing a single clinical trial, but they can't prove causation.
And as we talked before, and we'll talk again in later sessions, you can have single, double, triple blinding even. And so finally, that leaves us with our systematic review. They are our PlayStation 4 when it comes to levels of evidence. We report our study to determine the optimum suites for research nurse productivity. We performed a literature review, critically appraised 200 randomised controlled studies of suites and research nurse productivity. We find that the Maynard's winegums are the superior suite.
And yet again, Waynard's still not funding us, but maybe eventually. Systematic reviews are an exhaustive review of the current literature. They take less time than a new study, and the results can be extrapolated more broadly than if you have another study. As I said, they are very time-consuming, and not all studies can be combined. So why are systematic reviews and meta-analyses so good? So these are our top level, and that's where you're going to see these two guys, and then you compare these terms as to use, systematic reviews and meta-analysis.
Systematic reviews, this involves a panel of experts who, you know, you can have doctors, nurses, various different backgrounds, all looking, and they are assessing many, many, many papers in a similar way that you guys have done. They're looking at the evidence. And from that, they try and find exhaustive summary of that current evidence so this is where we're at at the moment and you can have systematic reviews of cohort studies you can have systematic reviews of case control studies okay each time that is better than that particular type of systematic review of a particular type of study is better than that type of study in itself better analysis this is a this assumes that there's a common statistical truth between studies and you look at the quantitative data that there is between those studies and you apply statistical methods to find the common truth amongst those. So if you've got one paper that says 30%, one says 40%, one says 60%, you're looking at those and you're trying to find the common truth. So these are two very useful types of study that are used in particular when we're producing guidelines. So a major source of guidelines is this group, the Cochrane group, if you've all heard of the Cochrane group.
So the Cochrane group are a major source of guidelines, they employ 37,000 doctors and nurses, specialists in their field to come in and they look at papers and they come up with, they look at research and they come up with guidelines. The way they do that is to first of all think of a problem with something broad like PE, try and narrow down the scope, so what is our actual question, so diagnosis of PE in A&E for example, and then from there that's when you're starting to look at the evidence, you're balancing it up. You're looking for bias and trying to overcome that. You're looking at the limitations. You're rejecting some research because it's rubbish. You're accepting others because it's really good. And you have a bit of a debate. And you can write up your research and write up your recommendations. And you can then debate that. And you can start again if you need to be. This is a very dynamic process. And finally, they publish. And then once it is published, it's out there in the domain. And then after a few years, they will reevaluate it and look at it.
Going through that process, like Lucy said, you could at any point decide, hang on, we need to do more research and jump in and say, well, we need to do some randomized control trial, for example, because we don't actually know enough at this point. A major measurement that we use, particularly when we're looking at the guidelines and for a new treatment is the QALY. Have you all heard of the QALY? So that's a quality adjusted life year or QALY. Yeah, exactly. So this will be part of the research trials. One year of perfect health is one QALY.
Okay, and Leon asked how we measure perfect health. We'll have a look at that in the next slide. So one year of perfect health is one QALY. Death is zero. So a QALY is measured somewhere between zero, which is death, up to one, which is perfect health. If you have one year of perfect health, that's one QALY. One year of half perfect health, that's half a QALY. Two years of half perfect health is a QALY overall. So you see how it measures. So it's not about the length of time that you get. It's about the quality of that. And
Therefore, it's important because it works out as how much money we're spending for each quality that our treatment is giving. And this then becomes very, very important when we're thinking about preventative measures as well. So the model that we look at when we're measuring, yeah, you've heard of this one as well, yeah, absolutely, so it's the EQ5D model, okay, so there are different models used for looking at measuring quality of life, but the EQ5D is one, and it looks at those five domains that you were just talking about, so yeah, absolutely, so yeah, so number one is mobility, how mobile is the patient, number two, self-caring, sort of the personal hygiene, essential activities, so going out, doing their shopping, cleaning, et cetera.
Pain is the absence of pain. And then finally, anxiety and depression, whether they're free from that. And if you think about it, that gets quite a nice rounded view of the patient's overall well-being. And that can be measured through a questionnaire as well as an objective study being done by somebody in the trial to look at them. So if you tick all those boxes, you are therefore in perfect health. If you are in perfect health for a year, that is therefore your quali.
And that's important because if you look at this graph, patient A has spent 40 years and then died suddenly, but within those 40 years they've had perfect health, they've had 40 qualities. Now patient B has lived double the length of time. to be 80 but they spend that whole time in half perfect health so they've still got 40 qualities so it's not about our length of time it's our it's the quality of that yeah exactly it is a it is a bit slightly controversial but you know if you save a patient's life but they are spending you know if you save a patient's life and they are on a ventilator for five more years before they die you know it's a useful way of going well i've given you five years extra life but those five are on a ventilator you know that seems that's the useful thing with a quality And so here we can compare our patients with a treatment getting 79 QALYs, patient without the treatment getting 66 QALYs. So we can say that we have given our patient with treatment 14 extra QALYs, sorry 13 extra QALYs than our patient without the treatment. And that's useful because we can say well how much money did that cost and when it comes to writing up guidelines and recommending treatment that's important. So like Leon said
Guidelines are only really a starting point and whenever you read guidelines, it always says at the bottom, clinician suspicion can overrule at any time. And a guideline is only good if we properly use it. Okay, PE's guidelines, for example, have a minefield to follow. and you have to have an understanding of the evidence behind it, otherwise it becomes difficult. Fever pain, this is a new guideline that's been brought out. It used to be the Centaur criteria, it's now fever pain. This is looking, if you have a patient who has a sore throat, how likely is it that that patient has a bacterial infection that you then have to give antibiotics to. Basically, it gives you a score out of five. So if you think you've got a patient who's got five out of five, you might think, okay,
definitely got a streptococcal infection, I need to give you antibiotics. But actually, if you have five out of five and you read the literature, you only have a 65% chance still of having a bacterial infection. So four and a half patients out of every 10 you see, if they have a fever pain of five, four and a half of them still have a viral infection and don't need antibiotics. So as I said, guidelines are only as useful as if you understand them and have a good think about them. And like I said, they always put that bit to the bottom, get out of jail free. Your suspicion overrides this at any time.
So NICE, so this link will take you, the previous link will take you to the Cochrane page where they go through how they create guidelines. This link will take you to the NICE guidelines and how they produce them. Dr. Simmons, one of our consultants here, was recently part of the group that was looking at sepsis guidelines and we've podcast on that previously. They follow very much the same routine as the Cochrane group do, so they look at a problem, they narrow it down, what is our question, they look at the evidence, they reject, they accept, they may do more studies themselves, they then go out, they write up, they draft their recommendations, they'll go through a few iterations of that before they eventually publish, and again they'll always come out with publication with a review date. And that's important thing with guidelines is that review date, so we can have
departmental, we can have trust, we can have national, we can have international guidelines. There should always be a review date to look back and has practice changed, has evidence changed that actually challenges that and that is a crucial part of the process. So a bit like plain Tetris, you have to juggle these different sized bits of evidence and trying to fit it together to come out with an overall picture when you're producing guidelines. So just to give you an idea of what that's like, I've invented a condition called Maynard's disease. So this affects research nurses who Don't eat enough wine gums. I want you to imagine that you are a committee formed by the NUH Trust to create guidelines to protect research nurses from Maynard's. Maynard's is an acquired condition affecting research nurses who do not get enough wine gums. However, wine gums are expensive and they contain a lot of sugar. You want to create UK guidelines for the right number of wine gums needed to prevent Maynard's disease whilst not causing harm to your nurses. Look at the evidence below. Create your guidelines. Just have a look at them and have a think. Would you follow that piece of guidance? Would you not? And sort of inspire you. We've got the Tetris music. The quicker you get through, the quicker you'll finish.
Yeah, so like Lucy said, a systematic review of RCTs in Japan recommended five wine gums a day, which reduces Maynard's by 65% with 10% diabetes. People in Japan have a different diet. Is it necessarily a representative group? And so is that transferable? Exactly. So they may have a much healthier diet than we do over here in the West.
A research unit in Leicester gave their nurses 10 wine gums a day for a year. None of them have developed Maynard's disease or diabetes. And like Leon said, we don't know... anything else about these nurses they could have been all triathlon athletes all very fit and they've only looked at them for a year and we don't actually know how many nurses they've got and that's the same thing for these randomized control trials down here a single blind randomized control trial in the uk or a double blind randomized control trial again you know the uk is a very diverse area we don't know you know were these in a deprived area where there is more likely to be disease were they in the you know an affluent area where it's less likely to be have these diseases A research unit in London has looked back on all their nurses employed in the past 15 years. Their nurses who develop Maynard's 8 on average 2 winegums a day or fewer. No nurse who ate 4 or more winegums a day developed Maynard's. Their nurses who developed diabetes and obesity 8 on average 5 or more winegums a day. None of their nurses who ate 2 or fewer developed diabetes.
Yeah, exactly, Megan, they've gone to some effort, they've looked back over 15 years. Recall bias, again, like Lucy said, is that necessarily going to be as accurate as you would expect? You know, can you remember 10 years ago how many wine gums a day you were eating? And again, you know, it says in London, you know, was this an affluent area? Were these nurses who were otherwise fit? What were their age groups? Etc. Six-year-old research nurse in Scotland has never eaten a single wine gum, never developed Maynard. She does have diabetes, though.
I think you all projected this quite quickly as a case study, as a case report, sorry. So, I mean, there isn't much there as well done for her, but as policy, it's not useful, is it? And she still ended up having diabetes, and I think that shows that diabetes is still prevalent even if you don't eat wine gums. We then had a meta-analysis of RCTs in the USA, and Lucy, you pointed out that the USA is that transferable, you know, Is the wine gum recipe the same over in the USA as it is over here? Et cetera, et cetera. Are there other lifestyle factors over there? You know, USA is known to have higher obesity and diabetes than here, although we're catching them up. So that's all things to think about. Then we had two different studies, very similar, in France and in Germany. One which recommended two wine gums a day. The one in Germany recommends three wine gums a day.
I think, you know, Lucy, you were saying, you know, is it necessarily transferable? Are there other things going on, you know, was it in France, the South of France, the Mediterranean diet, you know, adding in wine gums is necessarily different to maybe a more urban-deprived area, etc. You know, they're both Western European, but there are still different factors within that. And it's important to remember that, you know, ethnic diversity, we're a very international society now, especially in the West of Europe. Different areas will have different ethnic groups. Were these studies all of one ethnic group, etc., so could there be other confounding factors that you need to think about? So it's always worth, you know, going beyond the headline.
And yeah, I think the headline thing is really important, like Lucy said, I think, as I think going back to that research nurse in Scotland, when I was at university, they liked in exams to throw up an ethical thing where they'd have an actor playing a relative and they would show a made-up Daily Mail article in front of you going, oh, look at this one drug that's being mentioned. I want that for my relative. And to discuss that, you know, that's not research. And we already talked in previous sessions about the idea of primary and secondary outcomes. What's the study actually powered for that outcome? And this is something you find with things like statins, as I've already mentioned. A study could be powered to look at cholesterol levels with statins and then find out that none of their participants got cancer. But that's a secondary outcome. It's not actually powered. You can't then go, statins prevent cancer, but then that's often the thing that the tabloids will grab, and then that becomes the headline.
So like Lucy said, there are websites there that sort of debunk some of the headlines, I think that's important. So finally, a multi-centred double blind trial, double blind randomised control trial in Europe, looking at two or more wine gums a day, offer no benefit of Maynard's for over to 35 and increases the risk of diabetes by 1%. So yeah, were we part of that? That's the thing, you know, were we included in that randomised control trial? It sounds grand, but you know, was it? Europe's obviously a very large continent. Were we included in that data? And that's important.
before we start recommending it. So I think you've got there an idea of what it is like to sort of try and come up with guidelines and an appreciationist is very difficult and this is why it takes a long time and why it constantly has to be reviewed because evidence changes and as it changes you have to be prepared to challenge and to change. Right then, so thank you very much, guys. Our next session next month is on qualitative and quantitative data. I know Lucy is a fan of qualitative data, and we'll be looking at the differences between those. For this session, we've covered a lot of information. More information is available at the blog site. Follow this link to takeorally.com. This podcast will be there. The slides will be there. We'll also have all the information that we talked about and the links that we've mentioned all in there. So, you know, please check that out. And obviously, if you take Orally, follow us on Twitter.
So yeah, just remember those things and ultimately I think if we use guidelines properly and we're smart about them and we have an appreciation of where they come from, we can ultimately be nice with our guidelines and do the right things for our patients. They are there to protect us, they are useful, but ultimately they're only as useful as the person who's following them and you don't want to end up like Mario here. Thank you very much everyone.