42 min listen
Add Version Control To Your Data Lake With LakeFS
Add Version Control To Your Data Lake With LakeFS
ratings:
Length:
50 minutes
Released:
Nov 3, 2020
Format:
Podcast episode
Description
Data lakes are gaining popularity due to their flexibility and reduced cost of storage. Along with the benefits there are some additional complexities to consider, including how to safely integrate new data sources or test out changes to existing pipelines. In order to address these challenges the team at Treeverse created LakeFS to introduce version control capabilities to your storage layer. In this episode Einat Orr and Oz Katz explain how they implemented branching and merging capabilities for object storage, best practices for how to use versioning primitives to introduce changes to your data lake, how LakeFS is architected, and how you can start using it for your own data platform.
Released:
Nov 3, 2020
Format:
Podcast episode
Titles in the series (100)
Rebuilding Yelp's Data Pipeline with Justin Cunningham - Episode 5 by Data Engineering Podcast