33 min listen
Low Code And High Quality Data Engineering For The Whole Organization With Prophecy
Low Code And High Quality Data Engineering For The Whole Organization With Prophecy
ratings:
Length:
73 minutes
Released:
Jul 16, 2021
Format:
Podcast episode
Description
There is a wealth of tools and systems available for processing data, but the user experience of integrating them and building workflows is still lacking. This is particularly important in large and complex organizations where domain knowledge and context is paramount and there may not be access to engineers for codifying that expertise. Raj Bains founded Prophecy to address this need by creating a UI first platform for building and executing data engineering workflows that orchestrates Airflow and Spark. Rather than locking your business logic into a proprietary storage layer and only exposing it through a drag-and-drop editor Prophecy synchronizes all of your jobs with source control, allowing an easy bi-directional interaction between code first and no-code experiences. In this episode he shares his motivations for creating Prophecy, how he is leveraging the magic of compilers to translate between UI and code oriented representations of logic, and the organizational benefits of having a cohesive experience designed to bring business users and domain experts into the same platform as data engineers and analysts.
Released:
Jul 16, 2021
Format:
Podcast episode
Titles in the series (100)
MarketStore: Managing Timeseries Financial Data with Hitoshi Harada and Christopher Ryan - Episode 24: Fast and Scalable Financial Timeseries Dataframes with MarketStore (Interview) by Data Engineering Podcast