Spark DataFrame Documentation - Programmers

What this episode covers

Spark is a library for structured data processing. Unlike the basic Spark RDD API, the interfaces provided by Spark SQL provide Spark with more information about the structure of both the data and the computation being performed. Internally, Spark SQL uses this extra information to perform extra optimizations. There are several ways to interact with Spark SQL including SQL and the Dataset API. When computing a result the same execution engine is used, independent of which API/language you are using to express the computation. This unification means that developers can easily switch back and forth between different APIs based on which provides the most natural way to express a given transformation.

Share this episode

Similar Episodes

Plant Manager with Tony Wong, 49th Parallel Coffee Roasters

Jun 21, 2026 ·43m

Programming in English

Jun 14, 2026 ·31m

The Rabbit Was Always There

May 25, 2026 ·43m

Senior Human Resources Manager with Olga Cenic, Voortman Bakery | HR Mini-series

May 16, 2026 ·49m

VP of People and Culture with Steve MacIntyre, Fine Choice Foods | HR Mini-series

May 9, 2026 ·47m

Talent Aquisition Manager with Nicolas Pretorius, High Liner Foods | HR Mini-series

Apr 25, 2026 ·45m

Similar Podcasts

Khanyisa Keke TV Khanyisa Keke On Khanyisa Keke TV, developers can learn and improve their Android for Kotlin Development skills. On this podcast, programmers can learn Android for Kotlin coding from scratch, improve their existing programming skills, get tips, be kept up to date with all the latest happenings and get access to free resources. Powered by Firstory Hosting Command Line Heroes Red Hat Hear the epic true tales of how developers, programmers, hackers, geeks, and open source rebels are revolutionizing the technology landscape. Command Line Heroes is an award-winning podcast hosted by Saron Yitbarek and produced by Red Hat. Get root access to show notes, transcripts, and other associated content at https://redhat.com/commandlineheroes Android Developers Backstage Android Developers Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google.Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs Startup Data Science Edderic Ugaddan, Apurva Naik, Alex Au Startup Data Science is the podcast where you learn startup-ready data science with programming basics. We discuss how to bootstrap data science techniques and understand their underlying mechanics by discussing open-source learning materials. Startup Data Science helps forward-thinking entrepreneurs, novice programmers, and seasoned software engineers to use Data Science to make a bigger impact.

Frequently Asked Questions

How long is this episode of Programmers?

This episode is 23 minutes long.

When was this Programmers episode published?

This episode was published on April 29, 2020.

What is this episode about?

Spark is a library for structured data processing. Unlike the basic Spark RDD API, the interfaces provided by Spark SQL provide Spark with more information about the structure of both the data and the computation being performed. Internally, Spark...

Can I download this Programmers episode?

Yes, you can download this episode by clicking the download button on the episode player, or subscribe to the podcast in your preferred podcast app for automatic downloads.