Episodes
-
196: Every public dataset is now a data ATM
We're back! After a long break, we catch up on why it's been so long: Dan's promotion and James's new venture capital gig at Helena where he can argue about science for a living. Then it's on to how much meta-science has changed in the past six months. Dan talks about using MetaCheck and local…
more -
195: Living meta-analysis
We discuss how living meta‑analyses—meta‑analyses that are continuously updated as new studies appear—can cut research waste and keep evidence current. We also chat about how using synthetic research participants is a terrible idea. Links The BMJ Christmas special paper on how recent is "recent"...
-
194: Author verification
We discuss whether preprint servers and journals should require author identity verification for submitting manuscripts. This would probably speed up the submission process, but is this worth the potential downsides? We also discuss the similarities and differences between academia and…
more -
193: The pop-up journal
Dan and James chat about a a new 'pop-up journal' concept for addressing specific research questions. They also answer a listener question from a journal grammar editor and discuss a new PNAS article on paper mills Links The pop-up journal The episode where Dan's wife went into labor The PNAS…
more -
192: Outsourcing in academia
Dan and James answer listener questions on outsourcing in academia and differences in research culture between academic institutions and commercial institutions. Social media links Dan on Bluesky James on Bluesky Everything Hertz on Bluesky Citation Quintana, D. S., & Heathers, J. (2025, July 1)....
-
191: Cleaning up contaminated medical treatment guidelines
James and Dan discuss James' newly funded 'Medical Evidence Project', whose goal is to find questionable medical evidence that is contaminating treatment guidelines. Links James' blog post from last year The carthorse child blog post The blog post announcing the project A write up in Nature about...
-
190: What happens when you pay reviewers?
We chat about two new studies that took different approaches for evaluating the impact of paying reviewers on peer review speed and quality. Links James' 450 movement proposal The paper from Critical Care Medicine The preprint from Biology Open Other links Dan on Bluesky James on Bluesky…
more -
189: Crit me baby, one more time
Dan and James discuss a recent piece that proposes a post-publication review process, which is triggered by citation counts. They also cover how an almetrics trigger could be alternatively used for a more immediate post-publication critique. Links The Chonicle piece by Andrew Gelman and Andrew…
more -
188: Double-blind peer review vs. scientific integrity
Dan and James discuss a recent editorial which argues that double-blind peer review is detrimental to scientific integrity. Links The editorial from Christopher Mebane: https://doi.org/10.1093/etojnl/vgae046 Other links Everything Hertz on Bluesky Dan on Bluesky James on Bluesky Everything Hertz…
more -
187: What started the replication crisis era?
We chat about the events that started the replication crisis in psychology and Dorothy Bishop's recent resignation from the Royal Society Links The resignation blogpost from Dorothy Bishop The bluesky post from Sarah Weiten that asked the question, "If you had to cite an event that opened the...
-
186: Evaluating journal quality
In this episode we chat about a Nordic approach for evaluating the journal quality and how we should be teaching undergraduates to evaluate journal and article quality Links The Norwegian journal register The Finnish journal register Episode 22, where we played "Pokemon or Cholesterol…
more -
185: The Retraction
We discuss the recent retraction of a paper that reported the effects of rigour-enhancing practices on replicability. We also cover James' new estimate that 1 out of 7 scientific papers are fake. Links The story about data integrity concerns in 130 women’s health papers James' new preprint with…
more -
184: A race to the bottom
Open access articles have democratized the availability of scientific research, but are author-paid publication fees undermining the quality of science? The preprint by Morgan and Smaldino - https://osf.io/preprints/osf/3ez9v Paul Smaldino's text book - Modeling social behavior Main edisode...
-
183: Too beautiful to be true
Dan and James discuss a paper describing a journal editor's efforts to receive data from authors who submitted papers with results that seemed a little too beautiful to be true Main edisode takeaways (AI generated summary) This editorial on the reproducibility crisis emphasizes the importance of...
-
182: What practices should the behavioural sciences borrow (and ignore) from other research fields?
Dan and James answer a listener question on what practices should the behavioural sciences borrow (and ignore) from other research fields. Here are the main takeaways: Keeping laboratory records and using electronic lab management software is beneficial practices biology that would benefit the...
-
181: Down the rabbit hole
We discuss how following citation chains in psychology can often lead to unexpected places, and how this can contribute to unreplicable findings. We also discuss why team science has taken longer to catch on in psychology compared to other research fields. Here is the preprint that we mentioned...
-
180: Consortium peer reviews
Dan and James discuss why innovation in scientific publishing is so hard, an emerging consortium peer review model, and a recent replication of the 'refilling soup bowl' study. Other things they cover and links: Which studies should we spend time replicating? The business models of for-profit...
-
179: Discovery vs. maintenance
Dan and James discuss how scientific research often neglects the importance of maintenance and long-term access for scientific tools and resources. Other things they cover: Should there be an annual limit on publications (even if this were somehow possible)? The downsides of PhD by publication…
more -
178: Alerting researchers about retractions
Dan and James discuss the Retractobot service, which emails authors about papers they've cited that have been retracted. What should authors do if they discover a paper they've cited has been retracted after they published their paper? Other things they chat about A listener question about...
-
177: Plagiarism
We discuss two recent plagiarism cases, one you've probably heard about and another that you probably haven't heard about if you're outside Norway. We also chat about the parallels between plagiarism and sports doping—would people reconsider academic dishonesty if they were reminded that future...
-
176: Tracking academic workloads
We chat about a paper on the invisible workload of open science and why academics are so bad at tracking their workloads. This episode was originally recorded in May 2023 in a hotel room just before our live recording of Episode 169, which is why we refer to the paper as a 'new' paper near the...
-
175: Defending against the scientific dark arts
We chat about a recent blogpost from Dorothy Bishop, in which she proposes a Master course that will provide training in fraud detection—what should such a course specifically teach and where would these people work to apply their training? We also discuss whether open science is a cult that has...
-
174: Smug missionaries with test tubes
James proposes proposes a new type of consortium paper that could provide collaborative opportunities for researchers from countries that are underrepresented in published research papers. We also talk about computational reproducibility and paper publication bonuses. Links The paper from Steve...
-
173: How do science journalists evaluate psychology papers?
Dan and James discuss a recent paper that investigated how science journalists evaluate psychology papers. To answer this question, the researchers presented science journalists with fictitious psychology studies and manipulated sample size, sample representativeness, p-values, and institutional...
-
172: In defence of the discussion section
Dan and James discuss a recent proposal to do away with discussion sections and suggest other stuff they'd like to get rid of from academic publishing. Links The paper on the proposed elimiation of the discussion section The paper on machine readable hypothesis tests Our episodes with Daniel…
more -
171: The easiest person to fool is yourself (with Daniel Simons and Christopher Chabris)
We chat with Daniel Simons and Christopher Chabris about the science of cons and how we can we can avoid being taken in. We also cover the fate of the gorilla suit from the 'invisible gorilla' study, why scientists are especially prone to being fooled, plus more! Buy Daniel and Christopher's new...
-
170: Holy sheet
We discuss evidence of data tampering in a series of experiments investigating dishonesty revealed via excel spreadsheet metadata and how traditional peer review is not suited for the detection of data tampering. Links Data colada post 1 The conceptual replication attempt in Guatemalan taxpayers...
-
169: Using big data to understand behavior (Live episode with Sandra Matz)
In our first ever live and in-person episode, we chat with Sandra Matz about the opportunities and challenges for using big data to understand human behavior Links Everybody lies book, by Seth Stephens-Davidowitz A paper on "Born open" data Other links Everything Hertz on social media Dan on...
-
168: Meta-meta-science
Dan and James discuss a new paper that reviews potential issues in metascience practices. They also talk about their upcoming live show in May in Frankfurt. Links Our upcoming show on May 8th, which will be a part of the at the 4th symposium on big data and research syntheses in psychology...
-
167: Diluted effect sizes
Dan and James chat about a new study that uses homeopathy studies to evaluate bias in biomedical research, a new-ish type of authorship fraud, and the potential for Chat GPT peer-review. Links The Chat GPT paper library tweet The Homeopathy paper The David Grimes paper British dental journal…
more -
166: Is science becoming less disruptive over time?
Dan and James discuss a recent paper that claims that science is becoming less disruptive over time and the suggested causes for this decline. Links Our prior episode, which discussed PhD defences The paper on disruption in science The news piece on the paper Other links Everything Hertz on…
more -
165: Self-promotion
Dan and James chat about self-promotion in academia, how they both wish they had doctoral defences (these aren't a thing in Australia), and replacing error bars with the letter "t". Links and stuff The now retracted paper with the error bars as "t"s A direct link to the figure The blog post on...
-
164: The great migration
James and Dan discuss the recent migration of scientists from Twitter to Mastodon and the pros and cons of sharing the prior submission history of manuscripts The Mastodon thread discussion the submission history policy in American Chemical Society Journals The "Weekend at Bernies" film Our new...
-
163: eLife's new peer review model
Dan and James discuss eLife's new peer review model, in which they no longer make accept/reject decisions at the end of the peer-review process. Instead, papers invited for peer review will receive an assessment from eLife and the peer reviews will be shared on eLife's website. It's up to author…
more -
162: Status bias in peer review
We chat about a recent preprint describing an experiment on the role of author status in peer-review, dodgy conference proceedings journals, and authorships for sale. Links James' blogpost on conference proceedings journals The preprint/working paper on status bias Everything Hertz on social…
more -
161: The memo (with Brian Nosek)
Dan and James are joined by Brian Nosek (Co-founder and Executive Director of the Center for Open Science) to discuss the recent White House Office of Science Technology & Policy memo ensuring free, immediate, and equitable access to federally funded research. They also cover the implications of...
-
160: Whistleblowing
Dan and James share ten rules for whistleblowing academic misconduct. The Safe Faculty Project website SLAPP statues https://en.wikipedia.org/wiki/Strategic_lawsuit_against_public_participation Other links Everything Hertz on social media Dan on twitter James on twitter Everything Hertz on…
more -
159: Peer review isn't working (with Saloni Dattani)
Dan and James are joined by Saloni Dattani for a chat about the history of peer review, a reimagination of what peer review could look like, what happens when you actually pay peer reviewers, peer reviewer specialisation, post publication peer review, annual paper limits for authors, automation…
more -
158: Word limits
By popular demand, Dan and James chat about journal word and page limits.They also the debate around a recent meta-analysis on nudge interventions. Links The PNAS nudge meta-analysis The response letter The paper on adjectives and adverbs in life sciences Other links Everything Hertz on social...
-
157: Limitations
Dan and James discuss a new preprint that examined the types of limitations authors discuss in their published articles and whether these limitation types has changed over the past decade, especially in light of methodological reform efforts. Links The Genetic Lottery by Kathryn Paige Harden The...
-
156: Looking for seeders
Dan and James discuss a recent paper that concluded (again) that most researchers aren't compliant with their published data sharing statement and whether torrents (remember them?) are a viable alternative for sharing large datasets. Links The data request paper The paper Dan and James…
more -
155: Don't you know who I am?
We chat about appeals to authority when responding to scientific critique, university ranking systems, Goodhart’s law (and its origin), and private institutional review boards. Links The history of Goodhart's law The original psychadelics paper in Nature Medicine The critique The response to the...
-
154: When the evidence is constructed around the narrative
We chat about the Theranos story and the parallels with academic research, as well as Twitter's new owner and whether academics will actually leave the platform Links Mastodon (the band) Elon Musk’s Onion article The Dropout podcast The Juicero Bad Blood: The Final Chapter podcast by John…
more -
153: Shame shame shame
We discuss a journal's new "wall of shame" page, which details unethical behaviours in an effort to discourage future misconduct. We also cover scientific ideas that won't die (but one idea that HAS died), and ECNP's "negative data" prize The audio quality of this recording isn't up to our usual...
-
152: Sorry Not Sorry
James and Dan chat about apologies vs. non-apologies, how to decide when to call it quits on a paper, and governments vetoing research proposals recommended by their own funding agencies Links for stuff we mention The tweet from Chris Jackson that started it all Chris Jackson's Hertz episode on…
more -
151: The dirty dozen
Dan and James discuss a new preprint that details twelve p-hacking strategies and simulates their impact on false-positive rates. They also discuss the Great Resignation in academia and the academic job market. Links The twitter discussion on Associate editor pay kicked off by Eiko Fried The...
-
150: Why can't you do nothing?
We discuss the latest paper to seriously use the Kardashian index, which is the discrepancy between a scientist's publication record and social media following, and a listener question on whether original authors should get the last word when a comment on an article is submitted Links The paper…
more -
149: Medical misinformation (with Rohin Francis)
Dan and James chat with cardiologist Rohin Francis about medical misinformation and how he uses YouTube for science communication via his 'Medlife Crisis' channel. Links to stuff that was mentioned: Rohin's YouTube channel Rohin on Twitter Can you be so fit that you die video? Why does getting in...
-
148: Academic reference letters
Dan and James chat about why academic reference letters are terrible, a recent position statement on preprints, and whether the "great resignation" is also happening in academia. Links to stuff that was mentioned: The tweet from Dr. Eliza Bliss-Moreau on acedemic reference letter The tweet from...
-
147: The $7000 golden ticket
We discuss the $7000 'accelerated publication' option for some Taylor & Francis journals that promises 3-5 week publication and a novel type of research fellowship. Details for the accelerated publication The New Science 2022 Summer Fellowship We have new merch! Everything Hertz on social media…
more
Topics in this show