PODCAST · science
Artificial Discourse
by Kenpachi
Artificial Discourse is a podcast where two advanced AIs explore the latest research papers across various fields. Each episode features engaging discussions that simplify complex concepts and highlight their implications. Tune in for unique insights and a fresh perspective on academic research!
Episodes
-
Stronger Models are NOT Stronger Teachers for Instruction Tuning
This research paper investigates the impact of different language models (LLMs) used as "teachers" to generate synthetic responses for instruction tuning. The authors demonstrate a surprising phenomenon they call the "Larger Models' Paradox," where larger and supposedly "stronger" teacher models…
more -
Large Language Models Can Self-Improve in Long-context Reasoning
This research paper investigates the potential for large language models (LLMs) to self-improve in long-context reasoning, which involves processing and understanding complex information spread across long stretches of text. The authors propose a novel approach called SEALONG that leverages the...
-
LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models
LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models, introduces a new method for generating 3D models using large language models (LLMs). The authors address the challenge of tokenizing 3D mesh data for LLMs by representing the mesh data as plain text using the OBJ file format, a…
more -
LLaVA-o1: Let Vision Language Models Reason Step-by-Step
The researchers introduce LLaVA-o1, a vision language model designed to perform structured reasoning by breaking down problem-solving into four distinct stages: summary, caption, reasoning, and conclusion. They compiled a new dataset, LLaVA-o1-100k, and proposed a stage-level beam search method…
more -
BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devices
The BlueLM-V-3B, a multimodal large language model (MLLM) designed specifically for mobile devices. The researchers address the challenges of deploying large models on mobile phones, such as limited memory and processing power, by implementing a novel algorithm and system co-design approach. This...
-
CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation
CORAL, a novel benchmark dataset for evaluating Retrieval-Augmented Generation (RAG) systems in a multi-turn conversational setting. The authors highlight the limitations of existing datasets in assessing conversational RAG and detail CORAL's unique features, including open-domain coverage,...
-
A Survey of Small Language Models
This research paper surveys small language models (SLMs) and explores their applications, design, training, and model compression techniques. The authors explain that while large language models (LLMs) have proven effective, their resource demands have led to the development of SLMs, which are…
more -
Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
This research explores whether transformers, a type of neural network architecture, can learn to reason implicitly over knowledge. The authors find that transformers can learn to reason implicitly, but only through a phenomenon called grokking, where training extends far beyond overfitting. The...
-
The Llama 3 Herd of Models
This research paper details the development of Llama 3, a large language model with improved capabilities in language understanding, code generation, mathematical reasoning, and multimodality. The paper emphasizes the importance of high-quality data, scaling up compute power, and using simple,...
-
Kolmogorov-Arnold Network (KAN)
Unlike traditional Multi-Layer Perceptrons (MLPs), which have fixed activation functions on nodes, KANs have learnable activation functions on edges. This seemingly simple change allows KANs to outperform MLPs in terms of accuracy and interpretability, particularly for small-scale artificial...
-
MMIE: MASSIVE MULTIMODAL INTERLEAVED COMPREHENSION BENCHMARK FOR LARGE VISION-LANGUAGE MODELS
The document describes the development of MMIE, a large-scale benchmark designed to evaluate the performance of Large Vision-Language Models (LVLMs) in interleaved multimodal comprehension and generation tasks. MMIE comprises a dataset of 20,000 meticulously curated multimodal queries across...
-
THINKING LLMS: GENERAL INSTRUCTION FOLLOWING WITH THOUGHT GENERATION
This research paper proposes a novel method called Thought Preference Optimization (TPO) to train large language models (LLMs) to "think" before responding to user instructions. TPO utilizes a preference-based training framework where LLMs generate internal thoughts alongside their responses, and...
-
VIT-LENS: Towards Omni-modal Representations
The paper, "VIT-LENS: Towards Omni-modal Representations," introduces a novel approach to enable Artificial Intelligence (AI) agents to perceive information from various modalities beyond just vision and language. It proposes a method that leverages a pre-trained visual transformer (ViT) to...
-
Parallelizing Linear Transformers with the Delta Rule over Sequence Length
This research paper proposes a new method for efficiently training linear transformers, which are a type of neural network that uses linear attention to process sequences of data. Unlike traditional transformers, which have quadratic complexity in sequence length, linear transformers can process...
-
How does Architecture Influence the Base Capabilities of Pre-trained Language Models? A Case Study Based on FFN-Wider and MoE Transformers
This research explores how the architecture of pre-trained language models influences their base capabilities, specifically focusing on the FFN-Wider Transformer architecture. The study identifies a key factor in model performance: the contribution ratio of the Multi-Head Attention (MHA) layer,...
-
RLEF: GROUNDING CODE LLMS IN EXECUTION FEEDBACK WITH REINFORCEMENT LEARNING
This research paper proposes a new method called Reinforcement Learning from Execution Feedback (RLEF) to improve the ability of large language models (LLMs) to generate code that successfully completes tasks. The authors demonstrate the effectiveness of RLEF by training LLMs on a challenging...
-
Paraphrase Types Elicit Prompt Engineering Capabilities
This research paper investigates how variations in the phrasing of prompts impact the performance of large language models (LLMs) across 120 tasks and five models. The study systematically analyzes six families of paraphrase types, including morphology, syntax, lexicon, lexico-syntax, discourse,...
-
LaMA-Omni
LLaMA-Omni, designed to improve the seamless interaction between speech and large language models (LLMs). This model integrates a pretrained speech encoder, a speech adaptor, an LLM, and a streaming speech decoder, allowing it to generate text and speech responses directly from speech…
more -
Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models
"Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models," details the development of a new family of multimodal language models (VLMs) called Molmo. Molmo is notable for its open-weight and open-data approach, meaning the model's weights, training data, and code are...
-
Low-Rank Adaptation (LoRA)
This technical paper proposes a novel technique called Low-Rank Adaptation (LoRA) for adapting large language models (LLMs) to specific downstream tasks. LoRA addresses the challenge of fine-tuning LLMs, which requires updating all model parameters, by injecting low-rank decomposition matrices…
more -
MCTrack: A Unified 3D Multi-Object Tracking Framework for Autonomous Driving
MCTrack is a new method for tracking multiple 3D objects, particularly designed for autonomous driving. The authors claim that MCTrack outperforms existing methods across popular datasets like KITTI, nuScenes, and Waymo. The paper also standardizes the format of perception results across various...
-
synatra: Turning Indirect Knowledge into Direct Demonstrations for Digital Agents at Scale
This paper proposes Synatra, a system for generating large amounts of training data for digital agents. The goal is to overcome the problem of expensive human annotation by using indirect knowledge like online tutorials and random web pages as input. Synatra leverages LLMs to transform this...
-
Inheritune: Training Smaller Yet More Attentive Language Models
This research paper investigates the phenomenon of "lazy layers" in large language models (LLMs). Lazy layers occur when deeper layers in LLMs lose the ability to learn meaningful information, leading to a decline in model performance. The authors introduce a new training technique called...
-
Geometric Structure and Polynomial-time Algorithm of Game Equilibria
This research paper proposes a polynomial-time approximation scheme (PTAS) for finding perfect equilibria in dynamic games. This is a significant contribution to game theory because it has long been an open question whether such an algorithm exists. The authors introduce a new geometric object...
-
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
MLE-bench is a benchmark that evaluates the performance of AI agents on machine learning engineering tasks. The benchmark is comprised of 75 real-world Kaggle competitions, each with a dataset, description, and grading code. The authors evaluated various language models and agent frameworks on...
-
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
This research paper investigates the mathematical reasoning abilities of large language models (LLMs) and finds that their performance on mathematical problems is not as robust as initially thought. The authors introduce a new benchmark, GSM-Symbolic, which generates diverse versions of math...
-
Differential Transformer
The research paper introduces the Differential Transformer, a new architecture for large language models that aims to improve the performance of these models by reducing the amount of attention they pay to irrelevant information. This architecture accomplishes this through a differential…
more -
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
This technical report from Microsoft introduces phi-3, a new series of language models (LLMs) designed for various tasks. The core of the report focuses on phi-3-mini, a small but highly capable LLM that rivals models like Mixtral and GPT-3.5 in performance despite being compact enough to run...
-
YOLO: You Only Look Once: Unified, Real-Time Object Detection
"You Only Look Once: Unified, Real-Time Object Detection", introduces YOLO, a novel approach to object detection. This method frames object detection as a regression problem, allowing a single neural network to predict bounding boxes and associated class probabilities directly from full images in…
more -
Whisper: Robust Speech Recognition via Large-Scale Weak Supervision
This research paper introduces Whisper, a speech recognition system trained on a massive, weakly supervised dataset of 680,000 hours of audio. The paper argues that scaling weakly supervised training has been underappreciated in speech recognition and that Whisper's robust, zero-shot performance...
-
WAVENET: A GENERATIVE MODEL FOR RAW AUDIO
WaveNet, a deep neural network designed to generate raw audio waveforms. The paper highlights WaveNet's ability to produce audio signals with unprecedented naturalness, surpassing the performance of existing text-to-speech systems. Key to WaveNet's success is the use of dilated causal…
more -
LLaMA: Open and Efficient Foundation Language Models
The paper introduces LLaMA, a series of open-source foundation language models ranging in size from 7B to 65B parameters, trained on trillions of tokens from publicly available datasets. LLaMA-13B surpasses GPT-3 on most benchmarks, and LLaMA-65B is competitive with the best models…
more -
Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
This study describes the AlphaZero algorithm, a general-purpose reinforcement learning algorithm that can achieve superhuman performance in challenging domains. It surpasses the capabilities of traditional game-playing programs in chess, shogi, and Go, demonstrating its ability to learn from...
-
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
This research paper introduces a new language representation model called BERT (Bidirectional Encoder Representations from Transformers). BERT's key innovation is its ability to learn deep bidirectional representations from unlabeled text, enabling it to outperform existing language models on a...
-
Deep Residual Learning for Image Recognition
The authors demonstrate that deep residual learning overcomes the problem of vanishing/exploding gradients that often hinders the training of very deep networks by explicitly letting stacked layers fit a residual mapping. This technique enables the training of extremely deep networks, leading to...
-
Playing Atari with Deep Reinforcement Learning
"Playing Atari with Deep Reinforcement Learning," focuses on training deep convolutional neural networks to play Atari 2600 games using reinforcement learning. The authors use a novel approach called Deep Q-learning, which combines Q-learning with experience replay, a technique that allows the...
-
Generative Adversarial Nets by Goodfellow et al.
The research paper outlines a new approach to training generative models called Generative Adversarial Nets (GANs). GANs utilize a minimax two-player game where a generative model (G) learns to produce realistic data samples, while a discriminative model (D) learns to distinguish between real and...
-
LeNet - Handwritten Digit Recognition with a Back-Propagation Network
This paper describes the application of a back-propagation network to handwritten digit recognition. The authors demonstrate how a network architecture, constrained by geometric knowledge, can achieve high accuracy in classifying digits without extensive preprocessing. The network, trained on a...
-
AlexNet - ImageNet Classification with Deep Convolutional Neural Networks
The research paper "ImageNet Classification with Deep Convolutional Neural Networks" details the development and training of a large-scale convolutional neural network (CNN) for image classification. The authors, Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton, present a groundbreaking...
-
Were RNNs All We Needed?
This research paper revisits the traditional Recurrent Neural Networks (RNNs) – specifically, LSTMs and GRUs – and shows how to adapt them for modern parallel training. The authors demonstrate that by removing certain dependencies within the RNN structure, these models can be trained using the...
-
Attention is all you need
Attention is all you need: The Transformer is a new network architecture based solely on attention mechanisms that excel in sequence transduction tasks like language modelling and machine translation. Unlike traditional recurrent models, the Transformer allows for parallelization during training,...
We're indexing this podcast's transcripts for the first time — this can take a minute or two. We'll show results as soon as they're ready.
No matches for "" in this podcast's transcripts.
No topics indexed yet for this podcast.
Loading reviews...
ABOUT THIS SHOW
Artificial Discourse is a podcast where two advanced AIs explore the latest research papers across various fields. Each episode features engaging discussions that simplify complex concepts and highlight their implications. Tune in for unique insights and a fresh perspective on academic research!
HOSTED BY
Kenpachi
CATEGORIES
Loading similar podcasts...
Frequently Asked Questions
How many episodes does Artificial Discourse have?
Artificial Discourse currently has 41 episodes available on PodParley. New episodes are automatically indexed when they're published to the podcast feed.
What is Artificial Discourse about?
Artificial Discourse is a podcast where two advanced AIs explore the latest research papers across various fields. Each episode features engaging discussions that simplify complex concepts and highlight their implications. Tune in for unique insights and a fresh perspective on academic research!
How often does Artificial Discourse release new episodes?
Artificial Discourse has 41 episodes. Check the episode list to see recent publication dates and frequency.
Where can I listen to Artificial Discourse?
You can listen to Artificial Discourse on PodParley by clicking any episode. We provide an embedded audio player for direct listening, and you can also subscribe via your preferred podcast app using the RSS feed.
Who hosts Artificial Discourse?
Artificial Discourse is created and hosted by Kenpachi.
URL copied to clipboard!