Where intuition ends and predictive analytics begins

A client once asked me a question that was almost philosophical: “Where does intuition end and data science start?” In this post, I share my answer, and why expert knowledge is still the best starting point for any predictive model.
It’s a great question. It shows a desire to understand how far expert judgment can take a company, and when data should take over. That transition matters a lot for small and medium-sized companies starting their first data projects.
What experts do
To answer, I used my wife as an example. She’s a designer, and a very good one. When we met, she was still in college with almost no real-world design experience. Today she has more than a decade of experience designing for clients that include the National Football League, Monster and a few celebrities. When I ask how she approaches a website, she says:
“I get closely involved with each client, understanding their needs. Then I use that information to match their vision with the latest design trends. I don’t have a crystal ball to tell me exactly what users will like, but strong design sense and an eye for white space are the best starting place we have.”
I love that explanation. To me, it says that experienced professionals are the best at setting a course when there’s no other information to go on. We see the same thing in AI work.
Why domain knowledge matters
A careful understanding of the business gives us the material for strong predictive models. It’s also why AI work is hard to copy from one client to the next: every business needs its own careful thought. Here are three places where we rely on domain knowledge all the time.
Feature engineering. Data scientists combine or break apart pieces of data to create new, more useful inputs. A messy timestamp might become “weekday” or “weekend.” That’s a simple example, and often a powerful one, but domain experts are essential for finding the inputs that really matter. It’s a big part of how people win data science competitions on Kaggle: the leaders spend a lot of time understanding the subject.
Simple models. Experts often build simple, effective rules. A recommender that shows everyone the five most popular products is just a count of what’s popular. It’s hardly data science, but it can perform well. Add some simple customer segments and keep the data fresh, and it can pass for personalization until something better comes along. I wrote about its limits in how popularity can kill personalization.
Knowing the customer. Some experts just know which parts of a product matter most to customers at each stage of their relationship with it. That knowledge shapes the best models, and it’s one reason we put so much weight on clear communication.
The handoff
So where does intuition end and data science start? I think there’s a handoff, and it depends on a lot of clear communication. It’s part of why I see AI as part of the product more than as a technology, which I explain in why AI teams should report to product leaders. There’s a lot of math and computing involved, but it’s all in service of the product. The more the people building the models understand the product, the better their models get.
That handoff is how we start every engagement: we sit down with the people who know the business before anyone builds anything. You can read the steps in how we work with clients.