Skip to content
AI Insights – EP.2: Unlocking Cost-Effective AI with Small Language Models episode artwork

Feb 26, 2026 22 min

AI Insights – EP.2: Unlocking Cost-Effective AI with Small Language Models

from Cisco Podcast Network Hosted by Cisco

In the latest episode of the Cisco AI Insights Podcast, hosts Rafael Herrera and Sónia Marques welcome Cisco AI operations engineer James Tidd for a discussion on the world of small language models (SLMs) and the evolution of efficient AI inference. Together, they unravel the complexities behind “Fast Inference from Transformers via Speculative Decoding,” a groundbreaking paper from Google that explores how smaller draft models can speed up large language model predictions while maintaining accuracy. James shares his hands-on experience experimenting with the technique, leveraging knowledge distillation and speculative execution. The trio also discusses the potential of this approach to optimize AI, reduce power consumption and costs, and help businesses of all sizes get more out of existing hardware. A special thank you to Google’s AI team for developing this month's paper. If you are interested in reading the paper yourself, please visit this link: https://research.google/blog/looking-back-at-speculative-decoding/.

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

AI Insights – EP.2: Unlocking Cost-Effective AI with Small Language Models

0:00 22:27

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Cisco Podcast Network?

This episode is 22 minutes long.

When was this Cisco Podcast Network episode published?

This episode was published on February 26, 2026.

Can I download this Cisco Podcast Network episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.