LongNet | Uncertainty Alignment | Motion Retargeting

In today's thrilling episode, we dissect “LongNet”, a groundbreaking paper that scales transformers to a whopping 1 billion tokens. Next, we discuss Uncertainty Alignment and its implications for robotics. Finally, we cover "Motion Retargeting", a method of creating 3D avatars from minimal user input data, primarily headset and controller information. Key Points 1️⃣ LongNet * A method called "LongNet" scales transformer models to handle a billion tokens, using dilated attention to avoid quadratic complexity, achieving linear scaling. * While this method technically handles a billion tokens, it's different as it looks at pieces, not the entire attention, compromising performance beyond context window. * It's viewed as a clever innovation in computational scaling, despite trade-offs, and other methods like 'alibi' are suggested for better performance. 2️⃣ Uncertainty Alignment * The paper introduces "uncertainty alignment," a method for robots to handle ambiguous tasks by seeking minimum user help and providing statistical guarantees before executing a task. * This approach reduces fine-tuning and prompt tuning, aligns with how people think, and improves user experience by asking follow-up questions when uncertain. * While not groundbreaking, it simplifies complex tasks using probability and statistics, potentially becoming a standard practice for various chatbots and robotics applications. 3️⃣ Motion Retargeting * “Motion retargeting" is a method of creating 3D avatars from minimal user input data, primarily headset and controller information. * This technology transfers human movements to various virtual characters, demonstrating realistic movements despite the difference in character structure, like a dinosaur or a mouse. * Though promising, the technique depends heavily on the user's movements, and edge cases like extreme physical behavior can disrupt the avatar's realistic representation. 🔗 Episode Links * LongNet * Uncertainty Alignment * Motion Retargeting * AI-Laser Pesticide & Herbicide Connect With Us: Follow us on Threads Subscribe to our Substack Follow us on Twitter: * AI Daily * Farb * Ethan * Conner This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aidailypod.com