Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models

This research paper introduces Energy-Based Fine-Tuning (EBFT), a novel method for refining language models by matching feature statistics of generated text with ground-truth data. Traditional training relies on next-token prediction, which often causes models to drift or fail during long sequences because they lack global distributional calibration. By optimizing a feature-matching objective using a frozen feature network, EBFT provides dense semantic feedback at the sequence level without requiring a manual reward function. Experimental results across coding and translation show that EBFT outperforms standard supervised fine-tuning and equals reinforcement learning in accuracy. Furthermore, it achieves superior validation cross-entropy, proving that aligning feature moments effectively stabilizes model behavior and preserves high-quality language modeling.