Summary
Every business has customers, and a critical element of success is understanding who they are and how they are using the companies products or services. The challenge is that most companies have a multitude of systems that contain fragments of the customer's interactions and stitching that together is complex and time consuming. Segment created the Unify product to reduce the burden of building a comprehensive view of customers and synchronizing it to all of the systems that need it. In this episode Kevin Niparko and Hanhan Wang share the details of how it is implemented and how you can use it to build and maintain rich customer profiles.
Announcements
Hello and welcome to the Data Engineering Podcast, the show about modern data management
RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their extensive library of integrations enable you to automatically send data to hundreds of downstream tools. Sign up free at dataengineeringpodcast.com/rudderstack (https://www.dataengineeringpodcast.com/rudderstack)
Your host is Tobias Macey and today I'm interviewing Kevin Niparko and Hanhan Wang about Segment's new Unify product for building and syncing comprehensive customer profiles across your data systems
Interview
Introduction
How did you get involved in the area of data management?
Can you describe what Segment Unify is and the story behind it?
What are the net-new capabilities that it brings to the Segment product suite?
What are some of the categories of attributes that need to be managed in a prototypical customer profile?
What are the different use cases that are enabled/simplified by the availability of a comprehensive customer profile?
What is the potential impact of more detailed customer profiles on LTV?
@kevin: do you have jeff's all hands talk track where he threw out specific numbers with our reference customers?
How do you manage permissions/auditability of updating or amending profile data?
Can you describe how the Unify product is implemented?
What are the technical challenges that you had to address while developing/launching this product?
Two months before launch, our public beta was completely oversubscribed and we had a huge list of over 100 customers who wanted PS. The best problem to have....but a lot of "sweat smiles emojis going around"
Onboarding requires requires a one-time backfill of all historical events in time. And these 100+ customers in the queue required XYZ rows of Profiles data to be backfilled. Expected ETA: 4 months (TBC), which would be a day for day slip to GA.
Team built an enhanced backfill system in 4w....and now we can onboard customers in max X days.
What is the workflow for a team who is adopting the Unify product?
What are the other Segment products that need to be in use to take advantage of Unify?
H2: Today folks still need Connections to create the identity-resolved Profiles based on their customer events. However, that's also something we're re-thinking with our warehouse-centric strategy. Let us know if you have opinions there!
What are some of the most complex edge cases to address in identity resolution?
How does reverse ETL factor into the enrichment process for profile data?
What are some of the issues that you have to account for in synchronizing profiles across platforms/products?
How do you mititgate the impact of "regression to the mean" for systems that don't support all of the attributes that you want to maintain in a profile record?
What are some of the data modeling considerations that you have had to account for to support e.g. historical changes (e.g. slowly changing dimensions)?
What are the most interesting, innovative, or unexpected ways that you have seen Seg