Skip to content
Bring Geospatial Analytics Across Disparate Datasets Into Your Toolkit With The Unfolded Platform episode artwork

Jun 27, 2022 1 hr 7 min

Bring Geospatial Analytics Across Disparate Datasets Into Your Toolkit With The Unfolded Platform

from Data Engineering Podcast Hosted by Tobias Macey

Summary The proliferation of sensors and GPS devices has dramatically increased the number of applications for spatial data, and the need for scalable geospatial analytics. In order to reduce the friction involved in aggregating disparate data sets that share geographic similarities the Unfolded team built a platform that supports working across raster, vector, and tabular data in a single system. In this episode Isaac Brodsky explains how the Unfolded platform is architected, their experience joining the team at Foursquare, and how you can start using it for analyzing your spatial data today. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don’t forget to thank them for their continued support of this show! Atlan is the metadata hub for your data ecosystem. Instead of locking your metadata into a new silo, unleash its transformative potential with Atlan’s active metadata capabilities. Push information about data freshness and quality to your business intelligence, automatically scale up and down your warehouse based on usage patterns, and let the bots answer those questions in Slack so that the humans can focus on delivering real value. Go to dataengineeringpodcast.com/atlan today to learn more about how Atlan’s active metadata platform is helping pioneering data teams like Postman, Plaid, WeWork & Unilever achieve extraordinary things with metadata and escape the chaos. Modern data teams are dealing with a lot of complexity in their data pipelines and analytical code. Monitoring data quality, tracing incidents, and testing changes can be daunting and often takes hours to days or even weeks. By the time errors have made their way into production, it’s often too late and damage is done. Datafold built automated regression testing to help data and analytics engineers deal with data quality in their pull requests. Datafold shows how a change in SQL code affects your data, both on a statistical level and down to individual rows and values before it gets merged to production. No more shipping and praying, you can now know exactly what will change in your database! Datafold integrates with all major data warehouses as well as frameworks such as Airflow & dbt and seamlessly plugs into CI workflows. Visit dataengineeringpodcast.com/datafold today to book a demo with Datafold. Unstruk is the DataOps platform for your unstructured data. The options for ingesting, organizing, and curating unstructured files are complex, expensive, and bespoke. Unstruk Data is changing that equation with their platform approach to manage your unstructured assets. Built to handle all of your real-world data, from videos and images, to 3d point clouds and geospatial records, to industry specific file formats, Unstruk streamlines your workflow by converting human hours into machine minutes, and automatically alerting you to insights found in your dark data. Unstruk handles data versioning, lineage tracking, duplicate detection, consistency validation, as well as enrichment through sources including machine learning models, 3rd party data, and web APIs. Go to dataengineeringpodcast.com/unstruk today to transform your messy collection of unstructured data files into actionable assets that power your business. Your host is Tobias Macey and today I’m interviewing Isaac Brodsky about Foursquare’s Unfolded platform for working with spatial data Interview Introduction How did you get involved in the area of data management? Can you describe what the Unfolded platform is and the story behind it? What are some of the core challenges of working with spatial data? What are some of the sources that organizations rely on for collecting or generating those data sets? What are the capabilities that the Unfolded platform offers for spatial analytics? What use cases are you primarily focused on supporting? What (if any) are the datasets or analyses that you are consciously not investing in supporting? Can you describe how the Unfolded platform is implemented? How have the design and goals shifted or evolved since you started working on Unfolded? What are the new constraints or opportunities that are available after the merger with Foursquare? Can you describe a typical workflow for someone using Unfolded to manage their spatial information and build an analysis on top of it? What are some of the data modeling considerations that are necessary when populating a custom data set with Unfolded? What are some of the techniques that you needed to build to allow for loading large data sets into a users’s browser while maintaining sufficient performance? What are the most interesting, innovative, or unexpected ways that you have seen Unfolded used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on Unfolded? When is Unfolded the wrong choice? What do you have planned for the future of Unfolded? Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast helps you go from idea to production with machine learning. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story. To help other people find the show please leave a review on Apple Podcasts and tell your friends and co-workers Links Unfolded Platform H3 Hexagonal Map Tiles Library Carto Mapbox Open Street Map Raster Files Hex Tiles The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Sponsored By: Unstruk: ![Unstruck Data](https://files.fireside.fm/file/fireside-uploads/images/c/c6161a3f-a67b-48ef-b087-52f1f1573292/J3_WeYmj.png) Unstruk Data offers an API-driven solution to simplify the process of transforming unstructured data files into actionable intelligence about real-world assets without writing a line of code – putting insights generated from this data at enterprise teams’ fingertips. The company was founded in 2021 by Kirk Marple after his tenure as CTO of Kespry. Kirk possesses extensive industry knowledge including over 25 years of experience building and architecting scalable SaaS platforms and applications, prior successful startup exits, and deep unstructured and perception data experience. Unstruk investors include 8VC, Preface Ventures, Valia Ventures, Shell Ventures and Stage Venture Partners. Go to [dataengineeringpodcast.com/unstruk](https://www.dataengineeringpodcast.com/unstruk) today to transform your messy collection of unstructured data files into actionable assets that power your business! Support Data Engineering Podcast

Episode details come from the podcast's RSS feed.

Embed this episode

Ready to play

Bring Geospatial Analytics Across Disparate Datasets Into Your Toolkit With The Unfolded Platform

0:00 1:07:01

Chapters

  1. 0:13 Introduction and Sponsor Messages
  2. 1:28 Interview with Isaac Brodsky: Introduction and Background
  3. 3:18 The Unfolded Platform: Origins and Development
  4. 5:30 Challenges in Working with Geospatial Data
  5. 11:04 Geospatial Data Tools and Ecosystem
  6. 20:08 Capabilities and Use Cases of the Unfolded Platform
  7. 25:30 Technical Implementation and Architecture
  8. 28:15 Impact of Foursquare Acquisition
  9. 30:43 Workflow for Building Geospatial Analytics
  10. 37:55 Data Modeling and Preparation for Geospatial Analysis
  11. 48:01 Customer Education and Conceptual Challenges
  12. 51:11 Interesting and Unexpected Use Cases
  13. 54:23 Lessons Learned from Building the Platform
  14. 57:20 When Unfolded is Not the Right Choice
  15. 59:54 Future Plans and Focus Areas
  16. 1:03:04 Closing Remarks and Contact Information

No transcript yet.

Requests help us decide what to transcribe next.

Frequently asked questions

How long is this episode of Data Engineering Podcast?

This episode is 1 hour and 7 minutes long.

When was this Data Engineering Podcast episode published?

This episode was published on June 27, 2022.

Is there a transcript available for this episode?

Yes, a full transcript is available for this episode. You can read the complete transcript on the episode page.

Can I download this Data Engineering Podcast episode?

Yes. Use the download control on the episode player to save the publisher-provided media file.