Recovering Coherent Event Probabilities from LLM Embeddings

This paper details research on recovering coherent event probabilities from Large Language Model (LLM) embeddings, as current LLMs often produce probability judgments in text that violate the axioms of probability theory. The authors propose a novel unsupervised learning method using a variational autoencoder (VAE) approach to enforce axiomatic constraints, specifically the additive rule for complementary events, within the latent space of LLM embeddings. They demonstrate that this method extracts an interpretable latent space where a specific variable corresponds to event probability, and that probabilities recovered this way are more coherent and accurate than directly prompted LLM judgments. Experiments on dice-rolling scenarios with open-weight models like Gemma-2-9b-instruct support their findings, suggesting that LLM embeddings may encode more coherent probabilistic information than is expressed in text outputs.