When should a SaaS use AI? And when it absolutely should not

Three of my products are, at bottom, decision engines: Atamatax maps a tax situation to forms, Horvera decides whether a flight entitles you to compensation, Wabiro turns answers into a score. None of the three uses a language model to make the decision. That is not distrust of AI; it is the consequence of a question I ask before every feature.

The question is not “can we” but “what happens when it is wrong”

A language model can do almost anything plausibly. The useful question is the cost of the error. If a wrong output is caught by a person, fixed in one click, or has no consequence — AI belongs. If a wrong output sends a mistaken claim to an airline, declares a fund non-PFIC when it is one, or bills the wrong amount — it does not, whatever its average accuracy.

That question brings a second one: can you measure? Without about fifty real examples with the known correct answer, you will never know whether the system works, or whether it regressed after a model change. An AI project without an evaluation set is a project where nobody knows if it works.

Three cases where AI belongs

Structured extraction. Turning a PDF, an email, a statement or an invoice into typed fields is the most profitable case I know: weeks of hand-written rules replaced by one call with an enforced output schema, a confidence per field and human review when confidence is low. The result then feeds the rest of the system like any validated data.

Classification and routing. Sorting requests, tickets, documents or replies into the right queue. The model proposes; a deterministic rule decides the sensitive cases; a person takes over what the model marks as uncertain. Reading an airline’s reply and classifying it — accepts, refuses, asks for documents — is exactly that kind of task: AI prepares, the rule decides the next step.

Structured generation. Producing a letter, a summary or a report from already-validated data, with a template, constraints and JSON output — not free text. Horvera’s claim letter is generated from the facts of the case; what matters is that the facts come from the engine, not from the model.

Three cases where it does not

The decision that must be justifiable. Eligibility, tax computation, amounts, compliance: the answer has to be reproducible, traceable, and identical tomorrow for the same facts. Horvera’s engine produces every answer with its rule ids and sources; Atamatax’s shows its assumptions and confidence level. A probabilistic model cannot honour that contract. It can prepare the facts; it does not deliver the verdict.

What a query already does. “How many customers paid this month” is a SQL query. Adding a model to answer it adds cost, latency and a source of error, and brings nothing. A surprising share of “AI” requests are requests for a better database.

What cannot be evaluated. Without real examples, without a way to know whether the output is right, you do not build. You collect first — sometimes with a manual version of the process — and you come back when there is something to measure.

When not to use RAG

Retrieval-augmented generation — chunking documents, embedding them, retrieving the relevant passages and letting the model answer with citations — has become the default answer to “our data in a chatbot”. It is useful when the corpus is large, changes often, and the question is open-ended. It is unnecessary in three frequent cases.

• When the data is structured. If the information lives in tables, a query is more precise, cheaper and verifiable. A good schema makes RAG unnecessary more often than people think.
• When the corpus fits in the context. A current model reads hundreds of pages in one call; for a modest, stable corpus, passing the documents directly beats a retrieval infrastructure.
• When the documents contradict each other. Without version governance, RAG retrieves the right answer and the old one, and the model picks. The problem is editorial before it is technical.

When RAG is justified, it lives in Postgres with pgvector, as hybrid search, with verifiable citations — not in one more service.

The output contract: what AI must honour to enter an engine

What makes a deterministic engine reliable is its output contract: a fixed structure, a confidence, named assumptions, a trace. When an AI step slots into that engine, it honours the same contract. Every call is a typed function — validated input, versioned prompt, output schema validated with Zod, model swappable by configuration, fallback if the provider is down. The rest of the application never sees free text.

Evaluation ships with the feature: a set of examples, a score, a run on every change. Every human correction in production joins the set. Observability covers every call — input, output, model, tokens, cost, prompt version — and budgets are explicit, per user and per day, with planned degradation instead of a surprise invoice.

Deciding in twenty minutes

• What happens today when this task is done badly? If the answer is “we lose a customer” or “we break a rule”, the decision stays deterministic.
• Do fifty real examples with the correct answer exist? If not, collect first.
• Is the answer already a query? Then it is a query.
• Can a person review before the effect? If yes, AI can prepare; if not, it cannot decide.
• Which data leaves, where, under which terms? Settled before the first line of code.

In one line

AI replaces repetitive human work on text and documents when the error is recoverable and measurable; it does not replace a rule that must be justifiable or a query that already exists. The deterministic surrounds the probabilistic. That is how I integrate AI into a production product.