The short version
A practical guide to integrating LLMs and agents into real products.
Building an impressive AI demo takes an afternoon. Building an AI feature people rely on every day — and complain about when it is switched off — takes considerably longer, and most of the extra work has nothing to do with the model.
The gap between the two is trust. A demo only has to work once, in front of a friendly audience, on an example you chose. A feature has to behave predictably on the messy input your users bring, admit when it does not know, and never quietly invent something that costs someone money.
Choose a first use case you can be wrong about
The instinct is to point AI at the most valuable, most visible decision in the business. That is usually the worst place to start, because the cost of a confident mistake is highest exactly there.
Strong first candidates share a shape: the work is repetitive, a human already reviews the output, and being 90% right saves real time. Drafting a reply that a person edits. Extracting fields from an invoice into a form someone confirms. Summarising a long thread. Classifying an enquiry so it reaches the right team. In every case a wrong answer is an annoyance, not an incident — and you learn how the model behaves on your data before you give it authority.
Ground the model in your own data
A general model knows a great deal about the world and nothing about your business. Asked about your refund window or your part numbers, it will produce something plausible and wrong. That is not a bug you can prompt your way out of; it is a missing input.
Two techniques do most of the work:
- Retrieval — fetch the relevant documents, policies or records first, and instruct the model to answer only from what it was given. Show the source alongside the answer so the user can check it in one click.
- Tools — let the model call your systems for facts that must be live: stock levels, order status, pricing. A model that can look up an order does not need to guess at one.
Retrieval quality, not model choice, is what usually separates a useful assistant from a frustrating one. If the right paragraph is never retrieved, no amount of prompt engineering recovers it.
Decide what it is not allowed to do
Guardrails are product decisions, not a safety afterthought. Before launch, write down the answers:
- What must it never state without a source?
- Where does it stop and hand over to a human — refunds, legal or medical questions, anything above a value threshold?
- What data must never be sent to the model, and how is that enforced in code rather than in a prompt?
- What happens when it is unsure — and does the interface have a way to say so?
Enforce the important ones outside the model. A rule that only exists in a system prompt is a request; a rule in your application code is a constraint.
Evaluate before you launch, and keep evaluating
Teams that ship dependable AI features have one habit in common: a test set. Collect 50 to 200 real examples with the answer you would accept, and run them on every prompt change, model upgrade or retrieval tweak. Without it, “the new prompt seems better” is a feeling, and you will regress something you cannot name.
After launch, the evaluation continues in production. Log the input, the retrieved context and the output. Give users an obvious way to flag a bad answer, and read those flags weekly — they are the cheapest product research you will ever get.
Users forgive an AI feature that says “I’m not sure, here’s the policy page”. They do not forgive one that was confidently wrong about their money.
Design the interface for uncertainty
Most trust is won or lost in the UI, not the model.
- Show the working. Cite the document, link the record, quote the line the answer came from.
- Make it editable. Present output as a draft the user can change, not a verdict they must accept.
- Be honest about what it is. Label AI-generated content. Users who know what they are looking at check it appropriately.
- Fail visibly. If retrieval found nothing, say so and offer the search box. Silence reads as an answer.
- Stream, and stay responsive. Perceived speed matters; a spinner with no output feels broken long before it times out.
Cost, latency and the boring engineering
Token cost is easy to model and easy to underestimate once a feature is popular. Measure cost per successful task rather than per request — a cheaper model that needs two attempts and a human correction is not cheaper.
Cache aggressively where inputs repeat. Route simple requests to a smaller, faster model and reserve the larger one for genuinely hard cases. Set timeouts and a fallback path, because the model API will have a bad afternoon at some point and the rest of your product should not go down with it.
Handle data properly from day one
Decide explicitly what leaves your infrastructure, and tell your users. Redact identifiers you do not need to send. Check your provider's retention and training terms, and keep the agreement on file. If you work with client data under a contract, confirm that sending it to a third-party model is permitted before you build on the assumption that it is — retrofitting that decision is expensive.
Start narrow, then earn scope
The pattern that works is unglamorous: one workflow, one clearly-defined output, a human in the loop, a test set, and a metric you agreed in advance. Ship it, watch it for a few weeks, and let measured results decide whether it earns more autonomy.
That is how we approach AI and automation work, whether it is agents that carry out multi-step tasks or automation that removes repetitive process work. If you have a candidate workflow in mind, describe it to us and we will tell you honestly whether AI is the right tool for it.
