How to Add AI to an Existing .NET Application Without Rebuilding It

The most common AI request we get is not “build us an AI product”. It is “we have a .NET application that works, it holds years of business logic, and we want AI inside it without starting again.”

That is entirely reasonable, and it is usually a few weeks of work rather than a rebuild. The mistake teams make is jumping straight to the hardest version.

Three levels, in the order you should attempt them

Level 1: a model call behind a service

The simplest useful thing. You add a service that calls a model and returns text, and you wire it into one screen.

public interface ISummaryService
{
    Task<string> SummariseAsync(string content, CancellationToken ct = default);
}

Drafting a reply, summarising a long record, classifying an incoming item, extracting structured fields from a document. Interface, implementation, dependency injection. Nothing about your architecture changes.

Do this first, always. It proves the plumbing, the auth, the cost model and the latency in your real environment, and it delivers something users notice within a fortnight.

Level 2: grounding it in your own data

Level 1 has an obvious ceiling: the model knows nothing about your business, so it will answer confidently and wrongly.

Retrieval-augmented generation fixes that. You index your own content, retrieve the relevant passages at question time, and instruct the model to answer only from those passages, with citations. In .NET this usually means Azure AI Search alongside Azure OpenAI, and the pipeline is: chunk, embed, index, retrieve, generate.

The engineering here is mostly not AI work. It is data work: what to index, how to chunk it, and how to filter retrieval by the user’s permissions so nobody sees a document they could not already open. Teams underestimate this and then blame the model.

Level 3: an agent that does things

Only now does an agent make sense. You give the model a set of tools, which in a .NET application are just your existing service methods, and let it decide which to call.

The tools are functions you already have. GetOverdueInvoices, CreateSupportTicket, UpdateCustomerRecord. Your business logic, validation and authorisation stay exactly where they are, which is the whole point: the agent calls your code, it does not replace it.

Do not start here. An agent built before you have grounding and evaluation in place is how projects end up in the majority of AI pilots that never reach production.

Semantic Kernel or the SDK directly

For Level 1, call the Azure OpenAI SDK directly. Adding a framework to make one call is overhead you do not need.

From Level 2 onward, Microsoft’s Semantic Kernel starts earning its place: it handles function calling, conversation state, planning and telemetry in idiomatic C#, and it keeps you from writing an orchestration layer that you will then have to maintain. Its function-calling model maps cleanly onto your existing services, which is exactly what you want when the goal is to expose what you already have rather than build something parallel.

Where the code belongs

Put it in its own layer, not scattered through your controllers.

• An abstraction you own. Never let a vendor SDK type appear in your domain code. You will change models, and possibly providers, more often than you expect.
• Configuration, not constants. Model name, temperature, prompts and token limits belong in configuration so they can change without a deployment.
• Prompts in version control, reviewed like code. A prompt change alters behaviour as surely as a code change.
• Async and cancellable throughout. Model calls are slow by the standards of the rest of your app, and users will navigate away.

This structure also means you can swap the implementation for a fake in tests, which you will want, because calling a real model in your test suite is slow and expensive.

The guardrails that decide whether it survives

Cost visibility from day one. Log tokens per request and attribute them to a feature. AI spend is the easiest thing in a modern architecture to lose track of, and the conversation with finance is much better if you can answer it precisely.

A human in the loop wherever it writes. Reading and drafting can be automatic. Writing to a record, sending a message or committing a transaction should be reviewed until you have evidence it is safe.

Evaluation before rollout. Assemble fifty real inputs with expected outputs and run them on every prompt or model change. Without this you cannot tell an improvement from a regression, and you will eventually ship one thinking it was the other.

Timeouts, retries and a graceful failure. The model service will be slow or unavailable sometimes. The feature should degrade quietly rather than break the page.

Log the inputs and outputs. When someone asks in six months why the system said something, you need an answer.

A realistic first project

Pick one screen where a person currently reads something long and writes something short. That is the highest-value, lowest-risk starting point in almost every business application, and it maps to Level 1.

Ship that, measure the time it saves, then decide whether Level 2 is worth the data work. Most teams find it is. Rather fewer need Level 3, and the ones who do get there far more safely for having built the first two.

---

We add AI to existing .NET systems, and modernise the ones that need it first. See our AI agents and copilots and Azure OpenAI solutions services, or get in touch to talk through your application.