Artificially Speaking

The provided text introduces Cognitive Architectures for Language Agents (CoALA), a theoretical framework designed to standardize the development of artificial intelligence systems that use large language models as their core. By drawing parallels between modern AI and historical symbolic cognitive science, the authors propose a modular structure involving working and long-term memory, a diverse action space, and a structured decision-making process. This framework categorizes agent actions into grounding for external interaction and internal processes like reasoning, retrieval, and learning. CoALA provides a unified language to compare existing agents and identifies future research paths toward language-based general intelligence. Through various case studies, the sources demonstrate how this systematic approach can enhance an agent's ability to reason, plan, and adapt to complex environments.

SkillOpt is a novel optimization framework designed to improve the performance of AI agents by treating their skills as a trainable, external text document. Unlike traditional methods that rely on manual prompting or model fine-tuning, this system uses a separate optimizer model to analyze successes and failures from task trajectories and propose structured edits. These edits are governed by deep-learning-inspired controls, including a textual learning rate to limit change size and a validation gate that only accepts improvements. This process ensures the agent’s strategies remain stable and reproducible without requiring expensive updates to the underlying model weights. Across diverse benchmarks, the method significantly boosts accuracy and produces compact, human-readable skill artifacts that are easily understood. Furthermore, these optimized skills demonstrate strong generalization, retaining their value even when transferred across different model scales and execution environments.

The document introduces Vending-Bench, a novel simulated environment designed to evaluate the long-term coherence of autonomous agents powered by Large Language Models (LLMs). The benchmark tasks agents with managing a virtual vending machine business over extended periods, requiring them to handle inventory, pricing, ordering, and daily fees. While some advanced LLMs like Claude 3.5 Sonnet show promising performance, the results reveal high variance and a tendency for models to derail into tangential "meltdown" loops when encountering unexpected situations, such as misinterpreting delivery schedules. The research indicates that these failures are not directly caused by context window limitations, suggesting a deeper challenge in sustained, coherent decision-making for LLMs over long time horizons. The authors hope Vending-Bench will aid in assessing and preparing for stronger AI systems, including those with dual-use capabilities.

This research paper explores the capabilities of large language models (LLMs) in competitive programming. It compares the performance of OpenAI's o1 and o3 LLMs, highlighting the significant improvement in performance achieved by o3 through increased reinforcement learning. The study also examines a specialized LLM, o1-ioi, fine-tuned for the International Olympiad in Informatics (IOI), demonstrating that scaling general-purpose models surpasses the performance gains from specialized, hand-engineered approaches. Furthermore, the paper evaluates the LLMs' performance on real-world software engineering tasks, showcasing the broad applicability of enhanced reasoning capabilities in coding. Overall, the findings suggest that scaling reinforcement learning in general-purpose LLMs offers a robust path towards achieving state-of-the-art AI in complex reasoning domains.

This research paper explores using executable Python code as actions for Large Language Model (LLM) agents. The authors introduce CodeAct, a framework enabling LLMs to generate and execute Python code, dynamically adapting actions based on observations. Experiments across 17 LLMs demonstrate CodeAct's superior performance in complex tasks, achieving up to a 20% higher success rate than alternatives. A new instruction-tuning dataset, CodeActInstruct, is created to improve open-source LLMs' CodeAct capabilities, resulting in CodeActAgent, an open-source agent capable of sophisticated tasks. The paper concludes by discussing the potential benefits and risks of such autonomous agents.

This research paper investigates the effectiveness of ensembling different large language models (LLMs) to improve performance. The authors introduce Self-MoA, a method that aggregates multiple outputs from a single, top-performing LLM, contrasting it with traditional Mixture-of-Agents (MoA) which combines outputs from multiple LLMs. Experiments across various benchmarks show Self-MoA significantly outperforms MoA in many cases, highlighting the importance of model quality over diversity. A sequential version of Self-MoA is also presented to address scalability issues. The study explores the trade-off between diversity and quality in ensemble methods, ultimately demonstrating Self-MoA's superior performance and efficiency.

This technical report introduces the Qwen2.5-1M series of large language models, significantly enhancing long-context capabilities (up to 1 million tokens) through novel training and inference techniques. Key improvements involve long data synthesis, progressive pre-training, and a multi-stage fine-tuning process. The report also details a newly open-sourced inference framework featuring a length extrapolation method and sparse attention mechanisms to reduce computational costs and improve speed. Evaluations demonstrate substantial performance gains in long-context tasks without sacrificing short-context performance, exceeding even GPT-4o-mini in some areas. The open-source nature of many components promotes broader adoption and development.

The Paper details the development and capabilities of MiniMax-01, a series of large language models (LLMs) and vision-language models (VLMs). Key features include superior long-context processing (up to 4 million tokens), utilization of lightning attention, and a Mixture of Experts (MoE) architecture. Performance is shown to be comparable to state-of-the-art models like GPT-4 and Claude. The paper explores architectural design, computational optimizations, pre-training methodologies, and safety alignment. Finally, the models' code and weights are publicly released.

This research paper explores the development and evaluation of Process Reward Models (PRMs) for improving mathematical reasoning in Large Language Models (LLMs). The authors identify limitations in using Monte Carlo estimation for data annotation and Best-of-N evaluation for assessing PRM performance, proposing a consensus filtering mechanism combining Monte Carlo estimation with an LLM-as-a-judge approach. They introduce a new state-of-the-art PRM that outperforms existing models and advocate for a more comprehensive evaluation framework incorporating both response-level and step-level metrics. The study highlights the importance of accurate data annotation and unbiased evaluation methods for effective PRM training. Finally, the authors release their trained PRMs as open-source resources.

This research paper explores using generative AI, specifically ChatGPT, to automate Test-Driven Development (TDD). The authors propose two interaction patterns: a collaborative approach where developers and AI work together, and a fully automated approach. An experiment comparing these patterns with traditional TDD revealed that while generative AI can improve efficiency, human supervision is crucial to maintain code quality. The study highlights the potential benefits and challenges of integrating AI into TDD, emphasizing the need for careful design to avoid misleading developers. Future work will investigate additional interaction patterns and broader testing.

The paper introduces rStar-Math, a novel method that significantly improves the mathematical reasoning capabilities of small language models (SLMs). It achieves this through a self-evolving process using Monte Carlo Tree Search (MCTS) and three key innovations: code-augmented data synthesis, a novel process preference model (PPM) training method, and a four-round self-evolution recipe. rStar-Math surpasses the performance of larger models on several challenging benchmarks, including the USA Math Olympiad (AIME). The results demonstrate the potential of SLMs to achieve state-of-the-art performance in mathematical reasoning through iterative self-improvement.

Humanity's Last Exam (HLE) is a new, extremely difficult, multi-modal benchmark designed to evaluate large language models (LLMs). Created by a global team of experts, HLE features 3,000 questions spanning numerous academic subjects, surpassing the capabilities of current LLMs. The benchmark's questions were rigorously tested and reviewed to ensure difficulty and eliminate easy solutions, such as internet searches. HLE's public release aims to provide a standardized measure of LLM capabilities for researchers and policymakers, while a private test set guards against overfitting. The low accuracy and high calibration errors demonstrated by state-of-the-art LLMs on HLE highlight a significant gap between current AI capabilities and human expertise.

Researchers from Mondragon University and the University of Seville conducted a pre-deployment safety evaluation of OpenAI’s o3-mini large language model (LLM). They used their tool, ASTRAL, to automatically generate 10,080 unsafe prompts, covering 14 safety categories. The study found 87 instances of unsafe LLM behavior after manual verification, highlighting the o3-mini's relatively high safety level compared to previous models. Key findings included the impact of OpenAI’s policy violation feature and the influence of current events on unsafe outputs. The researchers also identified several critical safety categories requiring further attention.

This technical report details the development and evaluation of Kimi k1.5, a multi-modal large language model (LLM) trained using reinforcement learning (RL). The researchers emphasize a novel approach focusing on long-context scaling and improved policy optimization, achieving state-of-the-art results on various benchmarks. Key innovations include a simplistic RL framework avoiding complex techniques, effective long2short methods to enhance short-CoT models, and infrastructure optimizations like partial rollouts for efficient training. The report thoroughly explores the RL training techniques, data curation strategies, and system architecture. Extensive experimental results across text, reasoning, and vision benchmarks demonstrate Kimi k1.5's superior performance.

A deep dive into the DeepSeek-R1: Reasoning via Reinforcement Learning research paper. DeepSeek-R1, a large language model enhanced for reasoning capabilities through reinforcement learning (RL). Two versions are described: DeepSeek-R1-Zero, trained solely with RL, and DeepSeek-R1, which incorporates a multi-stage training process including cold-start data and supervised fine-tuning to improve readability and performance.

This research paper introduces Titans, a novel family of neural architectures designed to improve long-term memory in sequence modeling. Titans incorporate a deep neural long-term memory module that learns to memorize historical context at test time, augmenting a short-term attention mechanism.