Revenues drop unexpectedly, and management pulls aside the data science team into a room. The team is given its marching orders: “your job,” they’re told, “is to find out what the hell is going on with our purchase orders.”
That’s a very open-ended question, of course, because revenues and signups could drop for any number of reasons. Prices may have increased. A new user interface might be confusing potential customers. Seasonality effects might have to be considered. The source of the problem could be, well, anything.
That’s often the position data scientists find themselves in: rather than having a clear A/B test to analyze, they frequently are in the business of combing through user funnels to ensure that each stage is working as expected.
It takes a very detail-oriented and business-savvy team to pull off an investigation with that broad a scope, but that’s exactly what Medium has: a group of product-minded data scientists dedicated to investigating anomalies and identifying growth opportunities hidden in heaps of user data. They were kind enough to chat with me and talk about how Medium does data science for this episode of the Towards Data Science podcast.