Automating Hypothesis Generation with LLMs and Meta-Analysis

This paper presents a two-stage framework utilizing Large Language Models (LLMs) for generating statistically supported scientific hypotheses from a body of literature, effectively automating a process akin to meta-analysis. The first stage focuses on data extraction from scientific articles and tables using sophisticated prompting techniques to obtain relevant information, including numerical yields and experimental parameters. The second stage formulates hypothesis generation as an optimization problem, where the LLM iteratively proposes, evaluates, and refines predicates to identify subgroups with significant positive effect sizes, ensuring statistical rigor through measures like p-values and effect sizes. The framework is demonstrated in a case study within agricultural science, illustrating its potential to streamline scientific discovery, though it acknowledges limitations related to data accuracy and LLM bias.