Active Statistical Inference

Thiis paper introduces Active Statistical Inference, a novel approach for statistical inference that strategically utilizes a machine learning model to guide data collection under a labeling budget. By prioritizing the labeling of data points where the model is uncertain, this method aims to achieve more powerful inferences and smaller confidence intervals compared to traditional methods that collect data uniformly at random, even while using a black-box machine learning model and handling any data distribution. The authors present batch and sequential settings for this methodology and demonstrate its effectiveness on datasets from public opinion research, census analysis, and proteomics, showing significant sample budget savings over existing baselines including prediction-powered inference.