CoT-Self-Instruct: High-Quality Synthetic Prompt Generation

The research introduces CoT-Self-Instruct, a novel method for generating high-quality synthetic data to train Large Language Models (LLMs). This approach enhances data quality by first guiding LLMs through a Chain-of-Thought (CoT) reasoning process, enabling them to generate more complex and relevant prompts. Subsequently, the method employs automated filtering techniques, like Answer-Consistency for verifiable tasks and Rejecting Instruction Preferences (RIP) for non-verifiable ones, to ensure only the best data is used for training. Experiments demonstrate that LLMs trained with CoT-Self-Instruct data significantly outperform those trained on existing human-annotated or standard self-instruct datasets across both reasoning and non-reasoning benchmarks. The core innovation lies in leveraging LLMs' reasoning capabilities to create superior synthetic data, addressing the challenges of data scarcity and human annotation biases.