Where not to use AI (and why nobody tells you)

We are a company that builds AI systems. This article therefore runs against our short-term interest. But a client who concludes a year from now that somebody sold them nonsense is worse than a deal we never took.

So: where AI does not belong.

Anywhere ordinary code will do

This is by far the most common case, and the most embarrassing.

We once found a language model deciding whether an order qualified for free shipping. The rule was: over a hundred euros, yes. One condition in code. Instead there was an API call that cost money, took two seconds, occasionally returned the wrong answer, and could not be properly tested.

If the rule can be written, write the rule. It is cheaper, faster, and it does not get things wrong. Reach for a model only where you cannot write the rule.

Exact arithmetic

A language model does not calculate. It predicts what the result should look like, which is a very different thing.

With small numbers it usually lands, because it saw similar examples in training. With your numbers — fourteen line items, three VAT rates, a discount and rounding — it usually lands too. That "usually" is precisely the problem.

Prices, tax, payroll, stock, accounting. All plain arithmetic in code. A model may at most read the numbers off a document — something else has to add them up.

Decisions with legal effect

Rejecting a warranty claim. Judging whether a customer breached the terms. Assessing an application. Anything ending in a decision that harms somebody who can then contest it.

The problem is not only that the model might be wrong. The problem is that you cannot explain why. When a regulator or a customer asks on what basis you decided, "the model rated it that way" is not an answer. And for certain kinds of decisions GDPR requires you to provide human intervention — an automated decision with legal effect is not merely a technical question.

A model can prepare the material here. A person who puts their name to it has to decide.

When you have no data

"We'd like to predict which customers will churn." Good. How many historical churn cases do you have? "Well… about thirty."

Thirty examples is not a dataset. It is an anecdote. Whatever you build will learn noise and look convincing right up until you deploy it.

This does not apply to language models, which arrive pre-trained — there you need no data of your own. It applies to anything meant to predict something specific to your business.

Sensitive data with no thought-out framework

Not a prohibition, a warning. When you send health records, payroll or client personal data to an external API, you need to know where it goes, how long it is kept, whether anyone trains on it, and whether you have a lawful basis.

Those answers exist and are usually fine — providers offer modes where your data is not trained on. But somebody has to verify it beforehand, not after being asked.

When nobody checks the output

This is a summary of everything above.

An AI feature nobody looks at is a time bomb. Not because the model is bad, but because you have no way to learn that something broke. Language model failures are quiet — the output looks equally good whether it is right or wrong. That is what separates them from ordinary software, which fails loudly.

If "who looks at this?" does not have a name as an answer, the project is not finished.

This is not scepticism

None of this means AI does not work. It means it works on certain tasks, and forcing it onto the others gives you an expensive, unreliable way to do something that could have been done better.

The best AI projects we have delivered had one thing in common: we threw a large part of the original brief out, because it could be solved the ordinary way.