A decade's battle on dataset bias: are we there yet?

This academic paper explores dataset bias, revisiting a decade-old experiment by Torralba & Efros (2011) called "Name That Dataset" in the context of modern neural networks and large, diverse datasets. Surprisingly, the authors found that neural networks can still classify images by their source dataset with very high accuracy (e.g., 84.7% for a three-way classification), even with datasets presumably less biased. The study demonstrates that this capability is robust across various model architectures, sizes, training data volumes, and augmentation strategies, suggesting models learn generalizable patterns related to dataset identity rather than simply memorizing images. This research indicates that despite efforts to create less biased datasets, the problem of dataset bias persists and is readily detected by advanced AI systems, prompting further discussion on the representativeness of current pre-training datasets.