EPISODE · Feb 20, 2025 · 9 MIN
Is AI Benchmarking Broken? The Truth Behind "con@64" Revealed Brought to you by Avonetics.com
from Beaker Banter · host Beaker Banter
Discover the controversial "con@64" technique, where AI models are prompted 64 times to reach a consensus answer. Is this a legitimate way to reduce variance or a sneaky trick to inflate benchmark scores? Dive into the heated debate on whether this practice skews real-world performance comparisons and unfairly impacts perceptions of model capabilities. Learn why some accuse XAI engineers of overhyping AI and how differing "con" values could be misleading the industry. For advertising opportunities, visit Avonetics.com.
Embed this episode
NOW PLAYING
Is AI Benchmarking Broken? The Truth Behind "con@64" Revealed Brought to you by Avonetics.com
No transcript for this episode yet
Similar Episodes
No similar episodes found.