Designing Your SaaS Feature Set In The Age Of AI

A few weeks ago I took my car in for service. I had two questions about what it might need, the kind of thing you ask a service advisor for two minutes and you're done.

I called. First thing I got was an AI voice agent. It almost sounded like a person. Almost. The pauses ran a beat too long. Some answers had nothing to do with what I asked. It mispronounced things nobody in that job would.

I escaped that loop into option two, an overseas call center. Nice people, no way to reach an advisor either, and no answers to my questions.

You know how I finally got my answers? I drove to the dealership and stood in front of a human being. How convenient.

Every tier of that system existed to keep me away from the one person who could help. The AI wasn't the failure. It was doing exactly what it was designed to do, which was absorb calls. Nobody designed a path to an actual answer.

That's not one bad dealership. That's the default pattern right now, and it's what happens when you start with "we should use AI" instead of "what does the customer need."

AI is a tool, not magic

Here's what got lost.

Think about it. Why do we build software? To automate a bottleneck, save time, make a miserable job less miserable. That was the job before AI and it's still the job now. AI changed what's possible, not what you're trying to accomplish.

Which brings me to your actual advantage. Your customers think AI is magic. You know it's a probability engine predicting the next token. Astonishing at some jobs, structurally bad at others.

That's not cynicism, that's literacy. Every tool has a shape, and knowing where that shape fits is your job.

The evidence is already in

This isn't theoretical anymore.

In May, Sinch surveyed 2,527 senior decision makers across ten countries. Of the enterprises running live AI customer communications agents, 74% had already rolled one back or shut it down.

Here's the counterintuitive part. Among teams with the most mature guardrails, the rollback rate was higher, at 81%. The best-monitored teams rolled back more, since they could see the failures everybody else just blindly shipped and ignored.

On the demand side, Gartner found half of US consumers prefer to buy from brands that don't use generative AI in customer-facing work. They aren't neutral about it. They prefer to avoid it.

Two years ago "AI-powered" was an innovation signal. It's now a liability signal for a real slice of the market.

Four ways AI features hurt

Four distinct failure modes, and the fix is different for each.

One. Nondeterminism where reproducibility is required. I build fintech software, which means building for people in that world like accountants, who need the same input to produce the same output every time. Thinking Machines Lab ran the same prompt a thousand times at temperature zero and got 80 different completions. That's a liability no firm would touch.

Two. The trust tax. Half your market may prefer you didn't. It's invisible in your analytics, since the people it drives off never tell you. They just don't convert. They bail.

Three. The liability surface. Apple suspended its AI news summaries after they generated false headlines. Meta pulled an Instagram image capability days after launch. Neither company lacks resources. Both walked it back publicly.

Four. The bloat cost. Maintenance, docs, onboarding, the inference bill. Forever, for a feature that may be working against you.

None of those is "the model wasn't smart enough." Every one is a fit problem, and creating the right fit is something you need to own, always.

Deflection is not resolution

One distinction from my car service story is worth making explicit. Every AI deployment optimizes for one of two things.

Deflection is how many contacts you kept away from a human. A cost metric, and it goes up when people give up.

Resolution is how many people got what they came for. An outcome metric, and it doesn't care which tier delivered it.

Those look similar on a dashboard for about a quarter. Then they diverge hard, since a deflected customer isn't a solved customer. They're a customer who's annoyed and still has the problem.

My dealership had a great deflection rate that day. Their resolution rate was zero.

If you're going to use AI this way, every path needs an exit that leads to a real solution for your customer. If you can't do that, you're adding liability to your product experience.

Just yesterday I was shopping for a trash can on Amazon and figured I'd give their agent a shot, so I asked a few questions. Not only did I get different answers to repeat questions, I was able to talk the agent into different answers. Not ideal, but definitely hilarious. Try it yourself to see how bad it is.

The four places AI can live

Here's a framework that makes this easier. Any feature has four options, and most of the founders I coach are only looking at one of them.

Option one: runtime AI

The model runs when your customer clicks. It's helpful when variance is acceptable or valuable: drafting, summarizing, extracting from messy documents, classifying. Messy input, a human reviews the output, and slightly different each time is fine or even better.

Option two: the extension layer

This is the one people miss, and it's the "if you can't beat them, join them" move.

Sometimes AI genuinely is the best fit, but every customer wants it tuned to their own process, and it isn't core to your value proposition yet. Building that into your product experience early means taking on liability early in your go-to-market.

So don't build it. Expose it. Give them a clean API or an MCP server and show them how to wire it up with the AI tools they already have. They experiment on their terms, you don't have to maintain it, and their workflows run through your product instead of around it.

Then watch. If customers converge on the same pattern, it standardized itself and you have a pre-validated feature to absorb into the platform. If they don't converge, you dodged a feature that was never going to work as a fixed product capability anyway.

Option three: build-time AI

The model never touches your customer. You use AI to write the deterministic software that solves their problem, and ship plain, testable code. Same input, same output, forever. Nobody has to trust a model at runtime, since there isn't one. This is the old school way, but leveraging AI to build it speeds the process up considerably, so that's still a big win.

Option four: no AI

A form, a query, a rule. Correct far more often than the current discourse suggests, and completely legitimate. As they say, garbage in, garbage out. Know when to avoid that trap.

The five-question test

Here's a five-question test for every feature on your roadmap, to know which category is correct for each.

One. Does the same input have to produce the same output? If yes, it's custom predictable code, which can be written by AI, but AI never makes the decisions there.

Two. Does somebody sign their name to this, or carry the consequence if it's incorrect? If yes, AI drafts, a human owns it, and the system never decides alone.

Three. Would your customer be annoyed to learn AI did this? If yes, either don't, or be upfront, since finding out later is worse. This is where you experiment with the balance of human versus AI. The results are the ultimate determining factor for where you wind up.

Four. Is variance actually useful here? If yes, you're looking at genuine runtime AI fit. Build it and observe. This is a category where you can add new value for your customer.

Five. Is this core to your value, and does every customer want it the same way? If either answer is no, expose it instead of embedding it, and absorb it later if it standardizes.

Use AI to build it, not to be it

Option two, extending with AI, is underutilized. Option three, building custom code with AI, is severely underrated and often overlooked.

Extending with AI makes pretty much anything possible. Use it to test and experiment. Think of it as a way to try out ideas you or your customers come up with that you aren't quite sure about yet.

Building with AI is still where most of your value is today. You can build anything that works well and fast, so build a better mousetrap and do it faster than ever. This use case, while revolutionary, is often overlooked because of the whole point of this article. Everyone is falling into the trap of over-utilizing AI where it either isn't a fit or is actively making the experience worse. Looking your way, dealership and Amazon.

Custom software used to be too expensive to build for narrow problems, so everybody bought general-purpose tools and bent their process around them. Now the economics work.

The winning move in a lot of markets isn't AI-powered software. It's AI-built software that behaves exactly the same way every single time.

Your playbook to follow

Write the objective at the top of your roadmap. Which bottleneck, and how much time saved. Every feature gets measured against that, not against how heavily you're using AI to solve it.
Run every feature through the five-question test before it gets built. That tells you when to leverage AI and when to leave it out.
Default everything else to build-time AI. If AI can write the deterministic version, that's almost always the better product.
Anything customers want customized that isn't core to you, expose through an API or MCP server instead of building it. Absorb it later if it standardizes.
Where you do ship runtime AI, build a fail-safe into the experience. Make sure your customer can get off the ride if they feel themselves getting sick of it.
Stop leading with "AI-powered." That's how the food gets made, and for the most part nobody cares. What they care about is how well you solve their problem. Faster, better, cheaper. Lead with the outcome they want.

Sources

Where to go next

Step one is where people get stuck, since finding the top problem in your market is harder than it sounds. That's what my free Problem Finder does. Use it to find real problems in your market worth solving first: tools.bootstrappersparadise.com/problem-finder

If you want my help improving your product roadmap with strategies like these, I take a small number of founders for private 1:1 coaching: bootstrappersparadise.com/#coaching

Nobody at that dealership set out to make my day worse. They installed a semi-capable tool where it couldn't do the real job, and never designed what happens when a customer gets stuck in a doom loop. Don't do that to your customers.

They think it's magic. You know it's a tool. Use it like one.