EP062: VOYAGER AI Masters Minecraft by Writing Code

The paper introduces VOYAGER, the first Large Language Model (LLM)-powered embodied lifelong learning agent designed to continuously explore, acquire diverse skills, and make new discoveries in the open-ended world of Minecraft without human intervention. By interacting with GPT-4 through blackbox queries, VOYAGER completely bypasses the need for explicit gradient-based training or model parameter fine-tuning.The agent operates using three core components:An automatic curriculum that proposes progressively challenging tasks based on the agent's current state and exploration progress to maximize open-ended exploration.A skill library that stores successful action programs (written as executable code) for future retrieval, allowing the agent to compose complex behaviors rapidly while alleviating catastrophic forgetting.An iterative prompting mechanism that continually refines the generated code by incorporating environment feedback, execution errors, and a self-verification module to check for task success.Empirically, VOYAGER demonstrates exceptional in-context lifelong learning capabilities. When evaluated against other LLM-based agents like ReAct, Reflexion, and AutoGPT, VOYAGER discovers 3.3 times more unique items, travels 2.3 times longer distances, and unlocks key tech tree milestones up to 15.3 times faster. Furthermore, VOYAGER exhibits strong zero-shot generalization, successfully utilizing its learned skill library to solve entirely novel tasks from scratch in newly instantiated Minecraft worlds.