Fighting Churn with Data: An Interview with Zuora Chief Data Scientist Carl Gold

We sat down with Carl Gold, Chief Data Scientist at subscription software company Zuora, to learn more about his new book, Fighting Churn with Data.

The ability to attract and retain customers is fundamental to the success of any company, but it’s easier said than done. In fact, businesses spend millions of dollars every year trying to fight customer churn and crack the code to keep customers happy and coming back for more.

As a former VP of Customer Operations at Gainsight, a company quite literally dedicated to customer success, this problem was near and dear to my heart. So when I read Carl Gold’s Fighting Churn with Data, the world’s first book dedicated to exploring the intersection of data science and churn, I couldn’t put it down.

Carl is the Chief Data Scientist at Zuora, creator of the leading subscription management platform, and has spent a great deal of time thinking about how to solve this problem. Not only does he lead the team responsible for applying data science and machine learning to help fight customer churn for Zoura’s customers, but he literally invented the Subscription Economy Index, the company’s landmark index tracking the ascent of the Subscription Economy.

When it comes to optimizing customer engagement, Carl says, the details are in the data.

We sat down with Carl to discuss his journey to Zuora, the inspiration behind his new book, and how companies can leverage data and statistics to fight churn at scale.

You have a really interesting background — you studied neural networks and computer science for your PhD at the California Institute of Technology (Caltech), built machine learning models for Wall Street, and now serve as Chief Data Scientist of Zuora. What encouraged you to make the move from academia to industry, specifically working for companies in the financial space?

Well, it’s complicated. Back in the day, I pursued a PhD in science and it was really in physical science. My PhD was really more biophysics than machine learning, but I was in an interdisciplinary brain and AI program. So I did plenty of machine learning back before it was cool and data science wasn’t a thing yet. So I got into data science the old fashioned way, which was by being a scientist and then needing a job.

Back at the time, I went into finance because that was one of the few industries that hired scientific PhD graduates as quantitative analysts, or quants. This was before data science was a distinct profession. And this was before the financial crash when finance actually had a lot more appeal. 

The early 2000s were the heyday of finance because the economy was booming, and people seemed to think that what they were doing on Wall Street was actually good for the economy.

Now all that changed in the crash of 2008 and I had a front row seat. I was actually at a subsidiary of Morgan Stanley as the economy was melting down. During that time, data science started becoming a thing for businesses to invest in and people actually began to use machine learning outside of academia. Like I said, I researched machine learning models and published papers on machine learning back before it was cool, so it wasn’t too hard for me to become a data scientist. In fact, I don’t see data science and quantitative finance as that different, because they’re both using mathematical models for industrial problems. It’s all on a spectrum.

As the Chief Data Scientist of Zuora, you’re in charge of leading the team responsible for developing the Subscription Economy Index and driving the data science behind Zuora’s Subscriber Insights. Why do you think the subscription model has become so popular these last few years compared to say, ten or twenty years ago?

I can’t call myself a tech industry analyst, but I think most people would agree that the shift to the cloud has enabled new business models. With the cloud, you can do continuous delivery of software, so it just makes more sense for the software to be on subscription than to be selling perpetual licenses. And at the same time, the cloud technology which enables this continuous update of software also enables the mass management of subscriptions, because by and large, subscriptions are not that simple. The reason Zuora exists is because for any significant subscription product, the subscriptions become pretty complicated once you have multiple tiers and different options, not to mention start offering discounts. Suddenly, it gets very hard to keep everything straight.

Cloud technology has enabled everyone to master that process of keeping your data up to date. In fact, I think the biggest advantage of subscriptions is not recurring revenue, but recurring data. 

It’s that you can continuously maintain the data about your customer, which allows you to optimize your business models. The shift to cloud and cheap computing made clear that a subscription economy is one of the most efficient and optimal ways to deliver services.

One of the biggest challenges for the Subscription Economy is customer churn. To address this topic, you just published a new book, Fighting Churn with Data, about how to use data science techniques to reduce churn in subscription services. What inspired you to write this book?

When I first made my switch from finance to data science in 2014, I actually worked at a startup that was looking at churn problems. So, then I came to Zuora a few years later and kept working on churn problems for our customers. By the time I set out to write the book, I had a pipeline process developed over several years to predict churn.

Ultimately, I was inspired to write the book for two reasons. One, I wanted to write a book for my past self — in other words, data scientists tackling these problems for the first time at their own companies. I wanted to share best practices and lessons learned from my experiences so they wouldn’t make the same mistakes I made. Two, no one else had written a book on churn yet so I thought, “I better get on it!” because if someone else wrote a similar book a few years later, I’d be kicking myself over not beating them to it,

In the book, you mention that most companies use customer acquisition as a signal of growth, but that “low customer churn” is just as significant of an indicator of success, if not more so. Can you elaborate?

Churn is a direct drag on your growth. Growth equals acquisition minus churn. So it’s natural for companies to focus on acquisition at the early stage, when you don’t have any customers, but as soon as you get past that to the extent that you can reduce churn, low churn directly translates into growth. And to the extent that you don’t reduce your churn, it simply cancels out your growth. 

In short, churn is the enemy of any business trying to grow.

At the end of the day, low customer churn is always about value. I often say the techniques in my book are just making intuition rigorous, because the most basic intuition about churn, is that anyone who’s not getting a good value on the product is not going to hang around. You don’t need a PhD or background in statistics to figure that out.

Now, it does help to use some math to answer the question: how much use of your product/service is enough to predict low churn? The most important thing when predicting churn is to determine whether or not customers are getting value from the product. If you really dig deep into the numbers, you can clearly see that people who use a product more often churn less frequently.

How can data analysts and data scientists best measure the value of their product or service for customers?

Well, the best practice that I teach in the book is actually an analytic approach. You can measure what customers get simply by how much they use, for instance, just how many times do they produce a document using your product or text a friend about your service (i.e., a referral)? The simplest thing you can do is just cohort your customers by how much they use the product. You’ll always see your bottom cohort churn the most when you calculate the cohort churn rate. It’s a kind of cohort analysis, and it’s always very clear that the bottom cohorts churn the most, the top cohorts churn the least.

The interesting part is what happens to the curve in between these two cohorts. Usually, there’s a plateau in retention where you can clearly see how much use of a certain feature is enough, when the churn rate stops declining, which is very tangible and easy for people to see. I put several examples of it in chapter one of Fighting Churn with Data. It’s the main method I talk about, just because it’s so easy to both explain to people and understand what level of churn is healthy and what is considered high.

How does data quality and reliability factor into the analytics behind forecasting churn?

Data quality is the foundation of analytic forecasting. In short, if you don’t have sufficient data quality, you can’t do anything with data. That being said, you don’t need perfect data to do the science. 

My usual rule of thumb is if your data is more than 90 percent good, then there’s nothing to worry about when it comes to data quality. In my opinion, if only 10 percent of your data is unusable, your data is clean.

You can have less than clean data and still have it be usable, but definitely, the importance of high data quality can’t be overstated. In the book, I actually go over data quality assurance testing techniques at length. I wanted the book to be useful for people to show that you can’t just expect some of these techniques to work automatically. You really have to monitor the data at every stage because events and other data points are usually collected through unreliable systems. And the more complex your systems get, the more likely you are to have problems merging them.

If you could have dinner with any scientist or data leader, dead or alive, who would it be and why?

I have two answers. First, for a living scientist I’d love to meet Nouriel Roubini, the NYU economics professor who warned us about the 2006 recession caused by the credit and housing market bubble (that I lived through on Wall Street). I became a big fan of his during the financial crisis because he kind of coined the Black Swan theory as it relates to economics, showing us the limits of all statistical thinking based on the past. Whatever you can do with your statistics or machine learning, it’ll never predict Black Swan events, as he calls them.

Second, for someone who is no long with us, I’d say William Playfair. He’s basically credited with inventing data visualization. He came up with histograms, bar charts, and many other things that we consider the foundation of data visualization back in the 1700s. He was also a British spy and was somehow involved in the French Revolution. He sounds like a really interesting character. You read about some of the other mathematicians and scientists who were child prodigies, doing differential equations when they were 12 and I think, “Hmm, would I want to have a beer with that guy? I don’t know.” But Playfair seems like someone you would definitely want to hang out with and hear his stories.

Learn more about Carl’s book, Fighting Churn with Data, or his work at Zuora on the Subscription Economy Index.

Book a time to speak with us using the form below.

Our promise: we will show you the product.