Discover this podcast and so much more

Podcasts are free to enjoy without a subscription. We also offer ebooks, audiobooks, and so much more for just $11.99/month.

Add Version Control To Your Data Lake With LakeFS

Add Version Control To Your Data Lake With LakeFS

FromData Engineering Podcast


Add Version Control To Your Data Lake With LakeFS

FromData Engineering Podcast

ratings:
Length:
50 minutes
Released:
Nov 3, 2020
Format:
Podcast episode

Description

Data lakes are gaining popularity due to their flexibility and reduced cost of storage. Along with the benefits there are some additional complexities to consider, including how to safely integrate new data sources or test out changes to existing pipelines. In order to address these challenges the team at Treeverse created LakeFS to introduce version control capabilities to your storage layer. In this episode Einat Orr and Oz Katz explain how they implemented branching and merging capabilities for object storage, best practices for how to use versioning primitives to introduce changes to your data lake, how LakeFS is architected, and how you can start using it for your own data platform.
Released:
Nov 3, 2020
Format:
Podcast episode

Titles in the series (100)

Weekly deep dives on data management with the engineers and entrepreneurs who are shaping the industry