MassMutual’s Gareth Ross: The hardest work of analytics “sits around the edges”

On MassMutual’s data strategy:

Gareth Ross: [I]f a decision is getting made in the company, whether that’s a customer acquisition decision, whether it’s a choice around which agent we hire or don’t hire, how we match an agent to a customer, what that particular customer’s risk profile looks like, … or if we should investigate a fraud claim in 15 or 20 years, our view is that all those decisions need to get made by, augmented by, or supported in some way by the best possible data, and therefore the best possible analytics and models to do that. 

On organizing the analytics function:

[A]s we’ve metastasized deeper into the company,… what we’ve ended up doing is creating a series of domains or sort of depth of expertise, where we put data scientists in groups focused on an area for a three to four-year time horizon or cycle…. And then we rotate them around, so that by the end, what you end up having, in theory—and I think, in practice, this is beginning to show up—you’ve got data scientists who kind of understand the full depth of the company through both a data perspective and a business problem perspective.  That requires decentralization. 

In hindsight, my guess is that that proves to be a decent approach, which is very targeted and focused in the beginning; decentralize once you get to scale.

On learning from early mistakes:

In the early days, we had a belief system that said the hardest work is the building of the model.  And that is the place where we need to invest the most, get it right.  If you ask me or any of the data scientists in our world today, I think without being dismissive of the work, the actual characterization of a model is some of the easiest work.  And the hardest work that sits around the edges of that is the adoption and the cultural norms that go with that so that people want to use the model. 

A similar one is, and it’s of the same type, but data engineering and the movement of data through ecosystems and the availability of data.  We massively underinvested that early on, for the same reason, believing you just get to the model and you’re good.  While, actually, for this to work, you’ve got to have the engineering to move the data around to allow those models to be fresh and alive. 

On explainable models:

[If you think of a model that] kicks back after millions and millions of computations for any individual a continuous variable between 1-100 that’s predictive of your mortality.  If you’re using that in an underwriting process, it is insufficient to tell someone that their score is 84, and this is why we’re giving you this particular price for your product.  So, it is critically important that we be able to sort of then reverse engineer that model and provide insight to those individuals….  So actually we have a whole program within data science, that where it’s appropriate, we kind of reverse engineer its models and actually run them again for explanatory power. 

On the increasing reliance on automation:

This field is amazing, it can do a lot.  But what we have to do is have the seamless symbiosis between humans that can apply judgment and models that can apply science.  And knowing where to draw that tradeoff is not a one-time game.