Learnings From Building Agentic AI

There is plenty written about how agents should work. This is about what happened when we put them into systems that were already running, with real users and real permissions, and what we would do differently.

Some of it confirmed what we expected. A fair amount did not.

The model was never the bottleneck

We expected to spend our time on prompts and model choice. We spent it on authorization, retries, logging and evaluation.

Swapping models turned out to be close to a configuration change. Working out what an agent should be allowed to do, and proving afterwards what it did, was the actual project. If you are budgeting an agent build as an AI project, you will get the shape of it wrong. It is an integration and access control project with a model attached.

Narrow tools beat flexible ones, every time

Our first instinct was to expose a flexible query interface and let the model work it out. It did work it out, in ways nobody anticipated, including filter combinations that were technically valid and operationally meaningless.

Replacing that with ten specific verbs made behaviour predictable overnight. GetOverdueInvoices(accountNumber) gives you a system you can reason about. QueryInvoices(filter) gives you a system you can only observe.

The related lesson, which cost us a week: the description on a tool is the interface, not documentation. Two tools with overlapping descriptions produce a model that picks the wrong one and looks, from the outside, like a model quality problem. It almost never is. We now review tool descriptions the way we review API contracts.

More detail on the mechanics is in building production AI agents in .NET.

Permissions are the thing that will stop your project

This is the one we now raise on the first call, because it has derailed more timelines than any technical issue.

An agent must run as the person asking, not as itself. That sounds obvious and it is genuinely hard in estates where the underlying systems were never designed for it. Retrieval is where it bites hardest: an index built without permission trimming will happily surface a document the asker was never allowed to see, and it will do so fluently and with a citation.

We have seen this discovered late twice. Both times it added weeks. Now we check, before anything else, whether every source system can answer “what can this specific user see?” at query time. If the answer is no, that is the first piece of work, and pretending otherwise just moves the pain later.

Evaluation is what lets you change anything

Early on we shipped prompt changes and hoped. That works until the second or third change, when you can no longer tell whether you improved things or quietly broke a case that used to work.

A set of fifty real inputs with expected outcomes, run on every change, converted the whole thing from guesswork into engineering. The highest value assertion by a distance is simply did it pick the right tool, which is also the cheapest to write.

What we would do differently: build that set in week one, from real user questions, rather than in month three after the first regression scared everyone.

People trusted it more than we expected, which is its own problem

We assumed users would be sceptical. Mostly they were not. Once an agent answered well a few times, people stopped checking, including on the answers that deserved checking.

That changed how we design output. Every answer carries its sources. Uncertainty is stated rather than smoothed over. Anything that writes to a record or sends a message goes through a human until there is evidence it is safe. Not because the model is unreliable, but because confident wrong answers are far more dangerous than obviously wrong ones.

Scope small, then earn more

The agents that worked did one job. Answer questions from this document set. Draft this specific reply. Reconcile these two systems.

The ones that struggled were assistants in general, which meant nobody could say what they were for, and no evaluation set could be written because there was no defined right answer. If you cannot describe the job in one sentence, it is not ready to build. That is also the argument in choosing your first AI agent use case.

Things we got wrong

Worth stating plainly, because the write-ups that only contain successes are not much use to anyone.

We under-estimated logging. “Why did it say that?” arrives months later and the only acceptable answer is a full trace of the run.
We let conversation history grow. Context filled with turns nobody needed, cost went up, quality went down. Cap it and summarise.
We did not attribute token cost per feature early enough. By the time finance asked, the answer took a week to assemble. It should have taken a minute.
We built for a two-way sync when nobody needed one. The same over-engineering instinct we warn clients about, applied to ourselves.

Where this leaves us

Agentic AI is real and it is useful, and most of the work is unglamorous. The teams getting value are not the ones with the best prompts. They are the ones whose integrations, permissions and logging were in reasonable shape before they started, because an agent inherits every weakness of the estate it sits in.

That is the honest summary. It is also why a decade of .NET, SharePoint and Dynamics work turned out to matter more in this era, not less.

We build agentic systems in .NET on Azure OpenAI, Semantic Kernel and Copilot Studio, wired into the permissions you already have. See our AI agents and copilots service, or get in touch.