← Blog

How much value should AI deliver? Set expectations

Bennett Data Science Increase User Engagement A.I. Machine Learning

Business leaders often ask how much lift to expect from AI. This post shares a rule of thumb I heard years ago that still holds, an example from a client, and how to set expectations before an AI project starts.

A rule of thumb from a panel

Several years ago, I sat in the audience at a conference listening to data scientists from Netflix, Amazon and Spotify talk about the gains they usually see from their work. One panelist said something that has stuck with me ever since:

Major product changes can lift user engagement by 300%. Using AI to personalize that same product can lift engagement by 30%.

That’s a tenfold difference between what product changes and personalization can achieve. Treat the figures as one practitioner’s rule of thumb. The reason behind them is easy to see.

Imagine taking a good shopping site and making it bad on purpose by removing every product image. Who would shop there, with only names, descriptions, reviews and prices? Nobody. Engagement would collapse. No amount of personalization could win back as much as simply putting the images back.

That’s an extreme example, but it shows how much a good customer experience matters, and it puts what AI can do for engagement in perspective.

AI should support the product

It’s usually unrealistic to expect huge gains from personalizing content alone. You can expect steady, incremental gains. If you want a big jump in engagement, and hopefully revenue, pair smart changes to the product with personal messaging and get the best of both.

In e-commerce, AI’s job is to support the product.

We’ve seen this many times with our clients. One client started offering its products by subscription, so shoppers didn’t have to log in and reorder every month. That simple change produced big revenue gains. Data science helped with the timing and the personal presentation of the offer, but the driving factor was the option to subscribe.

Great products need great customer experiences. AI can help, but it won’t rescue a poor product or a bad interface. Too many companies hope it will, and I think that hope is where a lot of the hype around AI comes from.

How to set expectations before you start

Agree on the metric you want to improve, measure where it stands today, and decide what a realistic gain would be worth before anyone builds anything. That’s the job of the Discovery stage in how we work with clients. If you want to understand why projects fall short of expectations, read why AI projects fail to pay off.

If you’d like to talk through how much lift AI could bring to your business, the first call with one of our senior engineers is free and takes 30 minutes.

Let's talk about what you want to improve

Talk to an engineer