45 min listen
Build Your Python Data Processing Your Way And Run It Anywhere With Fugue
Build Your Python Data Processing Your Way And Run It Anywhere With Fugue
ratings:
Length:
61 minutes
Released:
Feb 21, 2022
Format:
Podcast episode
Description
Python has grown to be one of the top languages used for all aspects of data, from collection and cleaning, to analysis and machine learning. Along with that growth has come an explosion of tools and engines that help power these workflows, which introduces a great deal of complexity when scaling from single machines and exploratory development to massively parallel distributed computation. In answer to that challenge the Fugue project offers an interface to automatically translate across Pandas, Spark, and Dask execution environments without having to modify your logic. In this episode core contributor Kevin Kho explains how the slight differences in the underlying engines can lead to big problems, how Fugue works to hide those differences from the developer, and how you can start using it in your own work today.
Released:
Feb 21, 2022
Format:
Podcast episode
Titles in the series (100)
Pachyderm with Daniel Whitenack - Episode 1 by Data Engineering Podcast