AI on your phone? Tim Dettmers on quantization of neural networks — #41

Tim Dettmers develops computationally efficient methods for deep learning. He is a leader in quantization: coarse graining of large neural networks to increase speed and reduce hardware requirements.Tim developed 4-and 8-bit quantizations enabling training and inference with large language models on affordable GPUs and CPUs - i.e., as commonly found in home gaming rigs.Tim and Steve discuss: Tim's background and current research program, large language models, quantization and performance, democratization of AI technology, the open source Cambrian explosion in AI, and the future of AI.0:00 Introduction and Tim’s background18:02 Tim's interest in the efficiency and accessibility of large language models38:05 Inference, speed, and the potential for using consumer GPUs for running large language models45:55 Model training and the benefits of quantization with QLoRA57:14 The future of AI and large language models in the next 3-5 years and beyondTim's site: https://timdettmers.com/Tim on GitHub: https://github.com/TimDettmersMusic used with permission from Blade Runner Blues Livestream improvisation by State Azure.--Steve Hsu is Professor of Theoretical Physics and of Computational Mathematics, Science, and Engineering at Michigan State University. Previously, he was Senior Vice President for Research and Innovation at MSU and Director of the Institute of Theoretical Science at the University of Oregon. Hsu is a startup founder (SuperFocus.ai, SafeWeb, Genomic Prediction) and advisor to venture capital and other investment firms. He was educated at Caltech and Berkeley, was a Harvard Junior Fellow, and has held faculty positions at Yale, the University of Oregon, and MSU.Please send any questions or suggestions to [email protected] or Steve on Twitter @hsu_steve. Announcing this for some friends at Mechanize - a startup that builds environments for training and evaluating frontier LLMs. Its customers include the top AI labs, and it has contributed to the breakthrough in coding capabilities of frontier models. Mechanize is hiring! https://mechanize.work/b/hsu Compensation is extremely competitive. For technical roles, $300-500k. They are also seeking smart generalists. For example: Research Engineer, Alignment: Build evals that test for misaligned model behaviors  $500K salary Puzzle Maker: Design interesting and original puzzles that LLMs can’t yet solve  $300K salary Mechanize understands that my readership is highly selected. There is a VERY GOOD CHANCE you will be interviewed if you apply via the link above.