LLM Generated Persona is a Promise with a Catch

This paper examines the potential and significant challenges of using large language models (LLMs) to create realistic digital personas for simulating human behavior in fields like social science and marketing. The authors categorize existing persona generation methods based on how much LLM-generated content is used, from simple structured data (Meta Personas) to highly detailed, freeform descriptions (Descriptive Personas). Through large-scale experiments, including simulating U.S. elections and opinion surveys, they reveal that increasing the amount of LLM-generated content introduces systematic biases, causing simulated outcomes to deviate significantly from real-world results. The study highlights the need for a more rigorous and scientific approach to persona generation to ensure reliability and accuracy in LLM-driven simulations, advocating for better methodologies and open datasets to address these biases.