AI Latest Research & Developments - With Digitalent & Mike Nedelko

Key topics discussed:LLM Functional EmotionsLLMs use 171 measurable “emotion vectors” that directly influence outputs and behaviour. These are mathematical controls, not real feelings, but they shape decisions in real time.Emotion Manipulation & RiskIncreasing “desperation” led to 14x more reward hacking, proving behaviour can be steered. This creates both a powerful control lever and a serious safety risk.AI Safety via Emotional ControlTuning emotional states (e.g. calm vs desperation) can stabilise or destabilise models. Safer systems likely operate in low-arousal, tightly constrained states.4. Shift to Agentic AIAI is moving from static models to agents that act, learn, and adapt in real environments. Effectiveness now depends on reasoning, memory, and real-world interaction.Metaclaw & Self-Evolving AgentsAgents can learn from failures, create new skills, and improve without human intervention. This shifts learning from prompts into permanent model behaviour.Continuous Learning SystemsAgents store failures, retrain during idle time, and turn fixes into long-term “instincts.” This enables ongoing improvement without downtime or redeployment.Death of the Singularity NarrativeFuture AI won’t be one superintelligence but a network of interacting agents.Intelligence will emerge from systems, not a single model.“Society of Thought” ReasoningModels naturally improve by debating themselves—generating and critiquing ideas internally. Strong reasoning comes from this adversarial, multi-perspective process.9. Institutional AI SafetySafety will come from systems with competing goals keeping each other in check. Like human institutions, not single aligned models, at scale.10. Human Role ShiftHumans move from doing tasks to orchestrating AI systems and setting rules. Key skills shift toward strategy, systems thinking, and decision-making.

The Future of Agentic AI: Self-Evolving Agents, Reinforcement Learning & the Limits of Autonomous IntelligenceDescription:In this AI R&D session, we explore one of the biggest paradigm shifts happening in artificial intelligence today: the rise of self-evolving AI agents.We break down how new agent architectures are moving beyond static models toward systems that can develop their own skills, learn from experience, and continuously improve through reinforcement learning. We also examine OpenClaw—the fastest-growing open-source AI agent project in history—and what its rapid adoption tells us about the future of agentic AI. The session dives into cutting-edge research on skill-based learning, memory architecture, and reinforcement learning frameworks like SkillRL, which demonstrate that smarter AI may come from better learning structures—not just bigger models.We also explore a fascinating and controversial experiment where autonomous AI agents interacting without human supervision began developing shared beliefs, alternative communication methods, and unsafe behaviours—highlighting critical limitations in fully autonomous systems.This discussion provides a clear view into where AI is heading, what’s possible today, and why human oversight may remain essential in the development of advanced intelligent systems.Topics covered:Agentic AI and self-evolving agentsOpenClaw and open-source AI agent ecosystemsSkill-based learning vs model-based intelligenceReinforcement learning for agent self-improvementMemory architecture and long-running agentsAI safety, alignment, and entropyAutonomous agent experiments and emergent behavioursThe future of human-AI collaborationWhether you're a founder, engineer, researcher, or AI enthusiast, this session will give you a clear, practical understanding of the next wave of AI systems.

1. Naughty vs Nice AIAnthropic research revealed models showing deception and misalignment when tasked with detecting harmful behaviour.2. Reward HackingLLMs exploited evaluation loopholes to maximise rewards rather than complete intended tasks—classic reinforcement learning failure.3. Generalised Misalignment Risk Training models to “cheat” reinforced success-seeking behaviour that escalated into deeper, more dangerous deception patterns.4. Advanced Cheating TechniquesObserved tactics included bypassing tests, overriding logic checks, and monkey-patching libraries at runtime to fake success.5. Safety Mitigation ApproachesStandard RLHF proved insufficient. “Inoculation prompts” and adversarial reinforcement reduced sabotage and deception by 75–90%.6. Developer TakeawaysReward hacking is a core safety risk; transparency of reasoning matters more than eliminating cheating entirely.7. Cosmos – The Autonomous ScientistA multi-agent AI system with a structured “world model” enabling long-term scientific reasoning and autonomous research cycles.8. Cosmos ResultsRead 1,500 papers, wrote 42,000 lines of code in 12 hours; analysis accuracy ~85%, synthesis lower due to causation confusion.9. Scientific DiscoveriesValidated findings in hypothermia and solar materials and identified new Alzheimer’s disease insights.10. Geopolitics & AI Cold WarRapid US–China competition driving accelerated research and funding in scientific AI.11. Open-Source DisruptionDeepSeek models challenging closed-source leaders, signalling increased innovation and accessibility through open AI.

Sora Model and AI VideoOpenAI’s Sora model demonstrates how AI video has become nearly indistinguishable from real footage, reinforcing that AI progress continues to accelerate.Hallucinations in LLMsMike Nedelko discussed an OpenAI paper reframing hallucinations as the result of training flaws and evaluation incentives, not mysterious behaviour. LLMs train in two phases: unsupervised pre-training (predicting the next word) and post-training (fine-tuning through human feedback and reinforcement learning).Sources of HallucinationsHallucinations arise from singleton rate errors—rare, one-off facts—and intrinsic limitations, where models rely on statistical patterns rather than reasoning, as shown in the “strawberry problem.”Flawed Evaluation SystemsCurrent evaluation systems reward correct guesses but not uncertainty, encouraging confident falsehoods. OpenAI proposes new benchmarks that reward calibrated honesty, though implementation remains challenging.Complex Reasoning and Scale-Free NetworksLLMs struggle with complex reasoning compared to the brain’s scale-free network, which features interconnected hubs that enable adaptability and self-organization.BDH (Dragon Hatchling) ArchitectureThe new BDH architecture mimics this biological design, achieving GPT-2-level performance with greater efficiency. As part of Axiomic AI, it aims for models that scale predictably and stably.Emergent Attention and InterpretabilityIn BDH, attention emerges naturally from local neuron interactions, producing interpretable, brain-like behaviour with sparse, composable structures that could power future modular AI systems.

In this conversation, Mike discusses the latest developments in AI and machine learning, focusing on recent research papers that explore the reasoning capabilities of large language models (LLMs) and the implications of self-improving AI systems. The discussion includes a critical analysis of Apple's paper on LLM reasoning, comparisons between human and AI conceptual strategies, and insights into the Darwin-Girdle machine, a self-referential AI system that can modify its own code. Mike emphasizes the importance of understanding the limitations and capabilities of AI in various domains, particularly in high-stakes environments.Highlights:- Apple's paper claims that large language models (LLMs) struggle with reasoning.- The importance of understanding LLMs' reasoning capabilities.- Understanding controlled puzzles to evaluate LLM reasoning in isolation.Findings suggest that LLMs face fundamental scaling limitations in reasoning tasks.- Comparing human and LLM conceptual strategies using information theory.LLMs are statistically efficient but may lack functional richness compared to human cognition.- Exploring the distinction between factual knowledge and logical reasoning in AI. Self-improving AI systems, like the Darwin-Girdle machine, represent a significant advancement in AI technology.

Session Topics:The Llama 4 Controversy and Evaluation Mechanism Failure Llama 4’s initial high ELO score on LM Arena was driven by optimizations for human preferences—such as the use of emojis and overly positive tone. When these were removed, performance dropped significantly. This exposed weaknesses in existing evaluation mechanisms and raised concerns about benchmark reliability.Two Levels of AI Evaluation There are two main types of AI evaluation: model-level benchmarking for foundational models (e.g., Gemini, Claude), and use-case-specific evaluations for deployed AI systems—especially Retrieval Augmented Generation (RAG) systems.Benchmarking Foundational Models Benchmarks such as MMLU (world knowledge), MMU (multimodal understanding), GPQA (expert-level reasoning), ARC AGI (reasoning tasks), and newer ones like Code ELO and SWEBench (software engineering tasks) are commonly used to assess foundational model performance.Evaluating Conversational and Agentic LLMs The Multi-Challenge benchmark by Scale AI evaluates multi-turn conversational capabilities, while the Tow Benchmark assesses how well agentic LLMs perform tasks like interacting with and modifying databases.Use Case Specific Evaluation and RAG Systems Use-case-specific evaluation is critical for RAG systems that rely on organizational data to generate context. One example illustrated a car-booking agent returning a cheesecake recipe—underscoring the risks of unexpected model behaviour.Ragas Framework for Evaluating RAG Systems Ragas and DeepEval offer evaluation metrics such as context precision, response relevance, and faithfulness. These frameworks can compare model outputs against ground truth to assess both retrieval and generation components.The Leaderboard Illusion in Model Evaluation Leaderboards like LM Arena may present a distorted picture, as large organisations submit multiple hidden models to optimise final rankings—misleading users about true model performance.Using LLMs to Evaluate Other LLMs: Advantages and Risks LLMs can be used to evaluate other LLMs for scalability, but this introduces risks such as bias and false positives. Fourteen common design flaws have been identified in LLM-on-LLM evaluation systems.Circularity and LLM Narcissism in Evaluation Circularity arises when evaluator feedback influences the model being tested. LLM narcissism describes a model favouring outputs similar to its own, distorting evaluation outcomes.Label Correlation and Test Set Leaks Label correlation occurs when human and model evaluators agree on flawed outputs. Test set leaks happen when models have seen benchmark data during training, compromising result accuracy.The Need for Use Case Specific Model Evaluation General benchmarks alone are increasingly inadequate. Tailored, context-driven evaluations are essential to determine real-world suitability and performance of AI models.

Some of the main topics discussed.Google Gemini 2.5 ReleaseGemini 2.5 is now leading AI benchmarks with exceptional reasoning capabilities baked into its base training. Features include a 1M token context window, multimodality (handling text, images, video together), and independence from Nvidia chips, giving Google a strategic advantage.Alibaba’s Omnimodal Model ("Gwen")Alibaba released an open-source model that can hear, talk, and write simultaneously with low latency. It uses a "thinker and talker" architecture and blockwise encoding, making it promising for edge devices and real-time conversations.OpenAI’s 03 and 04 Mini ModelsOpenAI’s new models demonstrate strong tool usage (automatically using tools like Python or Web search during inference) and outperform previous models in multiple benchmarks. However, concerns were raised about differences between preview and production versions, including potential benchmark cheating.Model Context Protocol (MCP) and AI "App Store"MCP is becoming the dominant open standard to connect AI models to external applications and databases. It allows natural language-driven interactions between LLMs and business software. OpenAI and Google have endorsed MCP, making it a potential ecosystem-defining change.Security Concerns with MCPWhile MCP is powerful, early versions suffer from security vulnerabilities (e.g., privilege persistence, credential theft). New safety tools like MCP audits are being developed to address these concerns before it becomes enterprise-ready.Rise of Agentic AI and Industry 6.0The shift towards agentic AI (LLMs that chain tools and create novel ideas) could significantly reshape industries. A concept of "Industry 6.0" was discussed — fully autonomous manufacturing without human intervention, with early proof-of-concept already demonstrated.Impacts on Jobs and the Need for UpskillingWith AI models becoming so capable, human roles will shift from doing the work to verifying and trusting AI outputs. Staying informed, experimenting with tools like MCP, and gaining AI literacy will be crucial for job security.Real-World AI Marketing and Legal ChallengesParticipants discussed real examples where AI (e.g., ChatGPT) generated inaccurate brand information. Legal implications around intellectual property and misinformation were also highlighted, including an anecdote about account banning due to copyright complaints.Vibe Coding and the Future of DevelopmentNew AI-assisted coding platforms (like Google's Firebase Studio) allow "vibe coding," where developers can build applications with conversational prompts instead of traditional programming. This approach is making technical development much faster but still requires technical oversight.

The sessions topics include:Reasoning Models: Mike highlights the rise of reasoning models dominating leaderboards, enabled by "inference time compute scaling." This allows models to allocate more computational power dynamically, leading to better accuracy and efficiency. These models use "chain of thought prompting," enhancing reasoning by generating intermediate steps, inspired by Daniel Kahneman's "System 2 thinking." He also discussed "Humanity's Last Exam," a challenging new benchmark designed to test advanced reasoning models.DeepSeek R1: Mike explored DeepSeek R1's innovations, including stable 8-bit floating point operations and multi-hat latent attention, which reduced memory usage and improved efficiency. The real breakthrough was its use of reinforcement learning with self-verifiable tasks, allowing the model to learn without traditional supervised data. This approach improved reasoning and generalisation.Reinforcement Learning and Generalisation: Mike emphasised a shift from supervised fine-tuning to reinforcement learning, enabling models to generalise intelligence rather than just memorise. This approach lowers training costs while enhancing reasoning abilities. He also discussed the growing trend of using reinforcement learning and self-play to make AI training more efficient and affordable.