Logits are All We Need to Adapt Closed Models

This academic paper proposes a novel method called Plugin for adapting closed-source Large Language Models (LLMs) to specific tasks without needing to access their internal weights or original training data. The key idea is to leverage token logits, which are the raw probability scores before the final token selection, viewing the adaptation as a label noise correction problem in a sequence setting. A separate, smaller model is trained on task-specific data to reweight these probabilities during inference, effectively steering the black-box LLM's output. The research includes theoretical justification for this approach and empirical results on various datasets and LLMs, demonstrating its effectiveness compared to prompting and training new models. The authors advocate for broader access to token logits in commercial LLMs to enable such efficient and privacy-preserving customization.