4 min listen
Unlocking The Power of Data Lineage In Your Platform with OpenLineage
Unlocking The Power of Data Lineage In Your Platform with OpenLineage
ratings:
Length:
58 minutes
Released:
May 18, 2021
Format:
Podcast episode
Description
Data lineage is the common thread that ties together all of your data pipelines, workflows, and systems. In order to get a holistic understanding of your data quality, where errors are occurring, or how a report was constructed you need to track the lineage of the data from beginning to end. The complicating factor is that every framework, platform, and product has its own concepts of how to store, represent, and expose that information. In order to eliminate the wasted effort of building custom integrations every time you want to combine lineage information across systems Julien Le Dem introduced the OpenLineage specification. In this episode he explains his motivations for starting the effort, the far-reaching benefits that it can provide to the industry, and how you can start integrating it into your data platform today. This is an excellent conversation about how competing companies can still find mutual benefit in co-operating on open standards.
Released:
May 18, 2021
Format:
Podcast episode
Titles in the series (100)
Introducing The Show - Episode 0 by Data Engineering Podcast