How to integrate AI into your product

Start with a job, not a model

The failure pattern is always the same: a team decides to "use an LLM" and then hunts for somewhere to put it. That's building a solution and looking for a problem, and it reliably produces features nobody keeps. Reverse it. Look at where your users waste time, get stuck, abandon a flow, or stare at a blank field, and ask whether a language model genuinely helps there.

Good candidates share a recognizable shape:

• Unstructured input - free text, documents, messy data a rule can't parse.
• Tolerant of near-misses - a good-enough answer is genuinely useful, and the user can catch mistakes.
• Tedious for humans - the task is real work that people currently do by hand and dislike.

Summarizing long threads, drafting a first version, extracting fields from messy text, answering questions over your own docs - these fit the shape. Bad candidates need exact answers, hard guarantees, or already have a perfectly good deterministic solution.

Where LLMs earn their keep

• Drafting and rewriting. Generating a starting point a human then edits is a natural fit - the bar is "better than blank," not "perfect." Reply drafts, first-pass content, summaries.
• Extraction and classification. Turning free text into structured data your app can use - pulling fields off an invoice, tagging tickets, routing messages.
• Question answering over your data. Grounded retrieval against your own content, so users get answers from your knowledge base instead of hunting through it.
• Natural-language interfaces. Letting users ask for what they want in plain language instead of clicking through menus and filters.

Where they don't

Resist AI for anything that must be exactly right every time, anything where a confident wrong answer is dangerous or expensive, or anything a simple rule already handles well. A regex, a lookup table, or a PostgreSQL query is cheaper, faster, more reliable, and infinitely more debuggable than a model for deterministic work. Calculating a total, validating an email, enforcing a permission - none of these want an LLM. Using one there adds cost, latency, and a brand-new failure mode for zero gain. The most senior thing you can do in an AI project is say "this part doesn't need AI."

Choosing an approach: prompting, RAG, or fine-tuning

Once you've found a real job, pick the lightest technique that does it. Reach for more complexity only when the simpler tier demonstrably falls short.

• Prompting a capable model. Start here, always. A well-written prompt against a strong general model solves a surprising share of tasks with zero training and no data pipeline. Cheapest to build, easiest to change.
• Retrieval-augmented generation (RAG). When the model needs to know your facts - your docs, your product data - retrieve the relevant content and put it in the prompt. This is how you ground answers without retraining anything, and it's the right default for "answer questions over our data." Vector search via pgvector (see PostgreSQL for startups) keeps this in the database you already run.
• Fine-tuning. Only when you need a consistent style, format, or narrow behavior that prompting can't reliably produce, and you have quality examples to train on. It's the most expensive and least flexible option - most products never need it.

The mistake is starting at tier three. Begin with a prompt, add retrieval when the model needs your facts, and reserve fine-tuning for the rare case that truly demands it.

A worked example: turning a real job into a feature

Abstractions only get you so far, so make it concrete. Suppose a support team spends hours each day reading long customer email threads and manually filling a ticket with the customer's issue, urgency, and product area. That's the right shape for AI - unstructured input, tedious for humans, tolerant of a near-miss because an agent reviews every ticket anyway.

Here's how the tiers we described map onto it. Start with prompting: send the thread to a capable model and ask it to return the issue summary, an urgency level from a fixed set, and the product area, as structured JSON. If it needs to know your specific product taxonomy, add retrieval - pull the relevant catalog entries into the prompt so it classifies against your real categories rather than guessing. You'd almost certainly never fine-tune here; prompting plus retrieval solves it.

Then you design for it being wrong: the model proposes the ticket fields, the agent sees them pre-filled and approves or edits in one click, and you log both the suggestion and the final human-corrected version. Those logged corrections become your eval set and, if you ever needed it, your fine-tuning data. The feature saves real time from day one, degrades gracefully when it's uncertain, and gets measurably better as you learn - which is exactly what "shipped" looks like, versus a demo that impresses once and helps no one.

Design for the model being wrong

The single biggest difference between a demo and a shipped AI feature is how it handles being wrong. Models are probabilistic - they will occasionally produce a fluent, confident, completely incorrect answer. A demo hides this; a real product plans for it:

• Keep a human in the loop where stakes are high - the AI proposes, the person approves. This turns "the AI made a mistake" into "the AI saved someone time and they caught the edge case."
• Ground responses in your data via retrieval so answers cite real sources instead of inventing them. Show the source; let users verify.
• Constrain the output to structured formats (JSON schemas, enums) you can validate before acting on them. Reject or retry anything that doesn't parse.
• Fail gracefully. A clear "I'm not sure - here's what I found" beats a confident hallucination every single time. Design the not-sure state deliberately.

Cost, latency, and privacy

Three practical realities decide whether an AI feature survives contact with production:

• Cost. Every call costs money and it scales with usage. Cache aggressively, use smaller/cheaper models for simpler steps, and don't send more context than the task needs.
• Latency. LLM calls are slow compared to a database query. Stream responses so the user sees progress, do the work in the background where you can, and never block the whole UI on a model call.
• Privacy. Be deliberate about what user data leaves your system and where it goes. Know your provider's data-retention terms, and for sensitive domains consider what must stay in-house.

Evaluate before you trust

You cannot improve what you don't measure, and "it looked good in a few tries" is not measurement. Before an AI feature ships - and continuously after - build a small eval set: a collection of real inputs paired with what a good output looks like. Run it whenever you change a prompt, swap a model, or adjust retrieval, and you'll catch the silent regressions that plague AI features, where a tweak that helps one case quietly breaks five others.

Evals don't need to be elaborate. Even twenty to fifty representative examples, scored by a simple rule or a second model acting as a judge, will tell you more than any amount of manual spot-checking. The teams that keep their AI features healthy over time are the ones who treat prompts and models like code that can regress, and guard them with tests accordingly. This is the same "measure, don't guess" discipline that separates a shipped SaaS MVP from a science project.

Ship small, measure, expand

Don't launch an "AI everything" release. Pick one high-value job, ship it behind a flag to a slice of users, and watch two things: do people use it, and do they keep using it. Instrument acceptance - how often users take the AI's suggestion versus editing or discarding it. That single number tells you whether the feature is real value or a novelty that spikes on launch day and flatlines a week later.

This is the same discipline we apply to any launch, as in our SaaS MVP guide: ship the smallest real thing, measure, then expand from evidence. Build the plumbing to swap models and tune prompts without a redeploy, because both will change constantly. A thin, well-typed TypeScript service around the model call - with logging, retries, structured output validation, and a small eval suite of real examples - pays for itself the first time you need to change providers or a prompt regresses silently. Treat prompts and model choice as configuration, not hardcoded constants.

If you want to add AI that users actually keep, starting from a real job rather than the hype, let's talk. Originally published on the Doktouri Agency blog. We build web, mobile, SaaS, and AI products - let's talk.