Test-Time Alignment Strategies for Large Language Models

We explore the evolving field of Large Language Model (LLM) alignment, shifting from traditional, static training-time methods like RLHF to more dynamic test-time approaches. It introduces four distinct test-time alignment frameworks: Alignment as Reward-Guided Search (ARGS), Adaptive Best-of-N (ABoN), Controlled Decoding (CD), and Test-Time Alignment via Hypothesis Reweighting (HyRe). The analysis compares these methods across various axes, including their point of intervention (e.g., token-level, post-generation), required alignment signal (e.g., reward models vs. labeled examples), computational profile (training vs. inference costs), and their flexibility in handling multiple objectives or adapting to distribution shifts. Ultimately, the text argues that selecting the optimal test-time alignment method depends on the specific application's needs, resources, and tolerance for complexity, indicating a move towards a versatile toolkit for AI alignment rather than a single, universal solution.