Spark DataFrame Documentation

from Programmers · host Software Engineering

Spark is a library for structured data processing. Unlike the basic Spark RDD API, the interfaces provided by Spark SQL provide Spark with more information about the structure of both the data and the computation being performed. Internally, Spark SQL uses this extra information to perform extra optimizations. There are several ways to interact with Spark SQL including SQL and the Dataset API. When computing a result the same execution engine is used, independent of which API/language you are using to express the computation. This unification means that developers can easily switch back and forth between different APIs based on which provides the most natural way to express a given transformation.

Episode metadata supplied by the publisher feed · Published Apr 29, 2020

Embed this episode

Attribution link and audio player

Ready to play

Spark DataFrame Documentation

0:00 23:56

1×

No transcript for this episode yet

We transcribe on demand. Request one and we'll notify you when it's ready — usually under 10 minutes.

Share this episode

Similar Episodes

No similar episodes found.

Similar Podcasts

Command Line Heroes Red Hat Hear the epic true tales of how developers, programmers, hackers, geeks, and open source rebels are revolutionizing the technology landscape. Command Line Heroes is an award-winning podcast hosted by Saron Yitbarek and produced by Red Hat. Get root access to show notes, transcripts, and other associated content at https://redhat.com/commandlineheroes Android Developers Backstage Android Developers Android Backstage, a podcast by and for Android developers. Hosted by developers from the Android engineering team, this show covers topics of interest to Android programmers, with in-depth discussions and interviews with engineers on the Android team at Google.Subscribe to Android Developers YouTube → https://goo.gle/AndroidDevs DevDiscuss DevDiscuss is the first original podcast from DEV, a global community of software developers of all backgrounds and experience levels. The show covers burning topics that impact the daily lives of programmers and beyond, hosted by Forem Co-Founder, Ben Halpern, as well as a rotating cast of Forem developers. Two's Complement Ben Rady and Matt Godbolt If you come to a fork in the road, take it! Two’s Complement is a programming podcast, hosted by Matt Godbolt and Ben Rady; two programmers who both grew up wanting to make video games. One of them did, one of them didn’t, but now they both work together despite coming from very different backgrounds.

Frequently Asked Questions

How long is this episode of Programmers?

This episode is 23 minutes long.

When was this Programmers episode published?

This episode was published on April 29, 2020.

Can I download this Programmers episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.

URL copied to clipboard!