Skip to content
Accelerating LLMs with TornadoVM: From GPU Kernels to Model Inference episode artwork

May 18, 2025 1 hr 11 min

Accelerating LLMs with TornadoVM: From GPU Kernels to Model Inference

from airhacks.fm podcast with adam bien Hosted by adam-bien.com

An airhacks.fm conversation with Juan Fumero (@snatverk) about: tornadovm as a Java parallel framework for accelerating data parallelization on GPUs and other hardware, first GPU experiences with ELSA Winner and Voodoo cards, explanation of TornadoVM as a plugin to existing JDKs that uses Graal as a library, TornadoVM's programming model with @parallel and @reduce annotations for parallelizable code, introduction of kernel API for lower-level GPU programming, TornadoVM's ability to dynamically reconfigure and select the best hardware for workloads, implementation of LLM inference acceleration with TornadoVM, challenges in accelerating Llama models on GPUs, introduction of tensor types in TornadoVM to support FP8 and FP16 operations, shared buffer capabilities for GPU memory management, comparison of Java Vector API performance versus GPU acceleration, discussion of model quantization as a potential use case for TornadoVM, exploration of Deep Java Library (DJL) and its ND array implementation, potential standardization of tensor types in Java, integration possibilities with Project Babylon and its Code Reflection capabilities, TornadoVM's execution plans and task graphs for defining accelerated workloads, ability to run on multiple GPUs with different backends simultaneously, potential enterprise applications for LLMs in Java including model distillation for domain-specific models, discussion of Foreign Function & Memory API integration in TornadoVM, performance comparison between different GPU backends like OpenCL and CUDA, collaboration with Intel Level Zero oneAPI and integrated graphics support, future plans for RISC-V support in TornadoVM Juan Fumero on twitter: @snatverk

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Accelerating LLMs with TornadoVM: From GPU Kernels to Model Inference

0:00 1:11:04

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of airhacks.fm podcast with adam bien?

This episode is 1 hour and 11 minutes long.

When was this airhacks.fm podcast with adam bien episode published?

This episode was published on May 18, 2025.

Can I download this airhacks.fm podcast with adam bien episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.