← Blog

Automated machine learning won’t replace your data team

Bennett Data Science Automated machine Learning Tech Tuesdays

Automated machine learning tools promise that you can upload data and get a working predictive model back. This post explains why that rarely works without an experienced person, where these tools do help, and how to use them to save time and money.

The promise

Building predictive models is hard to do well. It generally takes years of education and practical experience to build a proper foundation.

Big firms like Google and Amazon have built platforms for automated machine learning, which does what it sounds like. You upload some data, and the tool learns to make the predictions you want. Say you want to predict churn. You give it data on customer attributes, actions and past cancellations, and the tool gives you a predictor that tells you whether a given customer is likely to come back.

I’d love a tool that made it that easy. With my years of experience in analytics, I’d know exactly how to use it and what data to feed it. I’d get a lot of value from it.

Most people wouldn’t. That isn’t clear from the marketing, even in the fine print.

Why it doesn’t work on its own

No tool can get around the oldest rule in data work: garbage in, garbage out. And garbage isn’t as black and white as you might think.

Data scientists think of data as having predictive power. That power isn’t on or off. It sits on a range. Judging how much predictive power your data has is essential, and someone without experience usually isn’t in a position to judge it.

Bias is another big problem that’s hard for a newcomer to spot. Models have been found to favor or penalize particular groups of people, often because of what was or wasn’t in the data used to train them.

I’m not saying automated machine learning has no use. I’m saying it isn’t a tool for everyone, and it may never be.

How to use automated machine learning well

Think of automated machine learning as an advanced tool for skilled data scientists, a big time saver for the right company in the right conditions. It won’t fix the difficulty of hiring and managing a data science team, and you can’t buy a license and let your team of five talented data scientists go.

Here’s a better recipe:

  1. Have a mid-level or senior data scientist collect, understand and clean the relevant data.
  2. Use automated machine learning to test how well that data predicts the outcome you care about, such as churn.
  3. Let the experienced data scientist read the results and improve the data and the model from there.

The same holds for today’s AI coding and analysis assistants. They make experienced people much faster. They don’t replace the judgment about which data to trust and which problem to solve. That’s why I’d start without machine learning until the data and the goal are clear.

At Bennett, we pick the tool for the task, and we put a named person on your team in front of every consequential decision the system makes. You can see how in how Bennett ONE works.

Let's talk about what you want to improve

Talk to an engineer