Discover this podcast and so much more

Podcasts are free to enjoy without a subscription. We also offer ebooks, audiobooks, and so much more for just $11.99/month.

Build Your Python Data Processing Your Way And Run It Anywhere With Fugue

Build Your Python Data Processing Your Way And Run It Anywhere With Fugue

FromData Engineering Podcast


Build Your Python Data Processing Your Way And Run It Anywhere With Fugue

FromData Engineering Podcast

ratings:
Length:
61 minutes
Released:
Feb 21, 2022
Format:
Podcast episode

Description

Python has grown to be one of the top languages used for all aspects of data, from collection and cleaning, to analysis and machine learning. Along with that growth has come an explosion of tools and engines that help power these workflows, which introduces a great deal of complexity when scaling from single machines and exploratory development to massively parallel distributed computation. In answer to that challenge the Fugue project offers an interface to automatically translate across Pandas, Spark, and Dask execution environments without having to modify your logic. In this episode core contributor Kevin Kho explains how the slight differences in the underlying engines can lead to big problems, how Fugue works to hide those differences from the developer, and how you can start using it in your own work today.
Released:
Feb 21, 2022
Format:
Podcast episode

Titles in the series (100)

Weekly deep dives on data management with the engineers and entrepreneurs who are shaping the industry