Skip to content
From SIMD to CUDA with TornadoVM episode artwork

Aug 16, 2025 45 min

From SIMD to CUDA with TornadoVM

from airhacks.fm podcast with adam bien Hosted by adam-bien.com

An airhacks.fm conversation with Michalis Papadimitriou (@mikepapadim) about: GPU acceleration for LLMs in Java using tornadovm, evolution from CPU-bound SIMD optimizations to GPU memory management, Alfonso's original Java port of llama.cpp using SIMD and Panama Vector API achieving 10 tokens per second, TornadoVM's initial hybrid approach combining CPU vector operations with GPU matrix multiplications, memory-bound nature of LLM inference versus compute-bound traditional workloads, introduction of persist and consume API to keep data on GPU between operations, reduction of host-GPU data transfers for improved performance, comparison with native CUDA implementations and optimization strategies, JIT compilation of kernels versus static optimization in frameworks like tensorrt, using LLMs like Claude to optimize GPU kernels, building MCP servers for automated kernel optimization, European Space Agency using TornadoVM in production for simulations, upcoming Metal backend support for Apple Silicon within 6-7 months, planned support for additional models including Mistral and gemma, potential for distributed inference across multiple GPUs, comparison with python and C++ implementations achieving near-native performance, modular architecture supporting OpenCL PTX and future hardware accelerators, challenges of new GPU hardware vendors like tenstorrent focusing on software ecosystem, planned quarkus and langchain4j integration demonstrations Michalis Papadimitriou on twitter: @mikepapadim

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

From SIMD to CUDA with TornadoVM

0:00 45:08

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of airhacks.fm podcast with adam bien?

This episode is 45 minutes long.

When was this airhacks.fm podcast with adam bien episode published?

This episode was published on August 16, 2025.

Can I download this airhacks.fm podcast with adam bien episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.