Building AI Agents With Semantic Kernel: A Practical 101 Guide

AI agents are evolving beyond traditional chatbots, taking on complex problem-solving tasks that demand deep contextual understanding and intelligent reasoning. Developers today aim to build systems that can not only respond intelligently but also act autonomously — combining domain knowledge, business logic, and specialized tools to create decision-making agents tailored to specific problems. Achieving this requires a powerful orchestrator capable of coordinating models, tools, and workflows seamlessly. 

Microsoft’s Semantic Kernel (SK) provides exactly that: a lightweight framework that bridges large language models (LLMs) with your own code, data, and APIs. In this article, we'll build a simple AI agent and explore the key components that make it work.

What Is Semantic Kernel?

Semantic Kernel (SK) is Microsoft’s open-source framework for building AI agents that combine an LLM with your own code and data. It was created to make 'do-things' agents practical and simple: you register your agent's capabilities as plugins (tools), SK exposes JSON-based inputs and outputs for those tools, and the LLM decides which tool to call based on the conversational context. SK handles orchestration — executing the requested tool, passing results back to the model, and maintaining structured communication — so you can focus on your business logic rather than low-level prompt wiring.

Three Pillars of an AI Agent

At the heart of every Semantic Kernel Agent are three components — the Kernel, the LLM, and Plugins (Tools).

1. Kernel: The Orchestrator

The kernel is the central coordinator. It manages interactions between the model and your tools, handles context, and ensures a smooth flow between reasoning and execution.

2. LLM (Reasoning Model): The Brain

The large language model provides reasoning capabilities. It interprets user intent, determines what data or tools it needs, and instructs the kernel to execute them.

3. Plugins: The Tools

Plugins are regular functions in your code that perform actions, for example, fetching data or computing values. By exposing them to the kernel, your agent can interact with real systems rather than staying purely conversational. Recent LLMs are trained well for tool calls and can pick the right tool based on the current context that they're aware of.

Now, let's build a simple AI agent equipped with tools to perform actions, an LLM to understand the user intent and make decisions, and a powerful kernel to help the agent achieve its goal. In this article, you will find code snippets using libraries specific to Azure, but you can find their equivalents based on your LLM provider in the official Semantic Kernel docs.

Step 0: Defining Your Agent

Before writing any code, let's start by clearly defining what our agent is and how it should behave. This definition, called the Agent's persona, becomes the system prompt later in our code, which guides the model's tone, style, and reasoning boundaries. In our example, we will build an agent that can greet the user with an upbeat tone and customize it based on the user's location.

Step 1: Building the Kernel

Every Semantic Kernel project starts with a kernel, which acts as the orchestrator connecting your LLM and tools. Creating it is simple — you register your model and the endpoint where the model is hosted, and the credential to use for authN/Z. If you are using Azure for production use, it's best to use Azure RBAC with Managed Identity rather than storing API keys. This setup securely connects your agent to the model without embedding any secrets in your code.

using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using Azure.Identity;   // for RBAC

// Build the kernel and register the model (e.g., Azure OpenAI GPT-4)
var builder = Kernel.CreateBuilder();
builder.AddAzureOpenAIChatCompletion(
    deploymentName: "<YOUR_DEPLOYMENT_NAME>",  // name of your LLM deployment (e.g., gpt-4.1, gpt-4o)
    endpoint: "<YOUR_AZURE_OPENAI_ENDPOINT>",
    credential: new DefaultAzureCredential());  // uses managed identity or local credentials

var kernel = builder.Build();

Console.WriteLine("Kernel is ready (RBAC enabled)!");

Step 2: Registering Plugins (Tools)

Plugins extend what the agent can do. Each plugin is a regular C# class whose methods can be invoked by the kernel when the model requests them. You expose a method by adding [KernelFunction] and [Description], which tells the model what the tool does. We'll define two simple plugins and a small helper client for our example. Each [KernelFunction] method becomes a callable tool, and its [Description] tells the model what it does, enabling reasoning-driven orchestration. You can also register prompts as tools, essentially creating modularity in prompts. This comes in handy when there is a use case for reusing certain prompts, e.g., summarization prompts.

using Microsoft.SemanticKernel;
using System.ComponentModel;

// Simple client to resolve a time zone from a city name
public class LocationClient
{
    public string GetTimeZone(string city)
    {
        return city.ToLower() switch
        {
            "seattle" => "Pacific Standard Time",
            "new york" => "Eastern Standard Time",
            "london" => "GMT Standard Time",
            "tokyo" => "Tokyo Standard Time",
            _ => "UTC"
        };
    }
}

// Plugin 1: Get user location and time zone
public class LocationPlugin
{
    private readonly LocationClient _client = new();

    [KernelFunction, Description("Get the user's current city.")]
    public string GetUserLocation() => "Seattle";

    [KernelFunction, Description("Get the time zone for a given city.")]
    public string GetTimeZone(string city) => _client.GetTimeZone(city);
}

// Plugin 2: Get time of day for a given time zone
public class TimePlugin
{
    [KernelFunction, Description("Get the current time of day (morning, afternoon, evening, or night) for a given time zone.")]
    public string GetTimeOfDay(string timeZone)
    {
        try
        {
            var tz = TimeZoneInfo.FindSystemTimeZoneById(timeZone);
            var localTime = TimeZoneInfo.ConvertTime(DateTime.UtcNow, tz);
            var hour = localTime.Hour;

            return hour switch
            {
                >= 5 and < 12 => "morning",
                >= 12 and < 17 => "afternoon",
                >= 17 and < 21 => "evening",
                _ => "night"
            };
        }
        catch
        {
            return "unknown time of day";
        }
    }
}

// Register plugins
var locationPlugin = new LocationPlugin();
var timePlugin = new TimePlugin();

kernel.Plugins.AddFromObject(locationPlugin, "Location");
kernel.Plugins.AddFromObject(timePlugin, "Time");

Console.WriteLine("Plugins registered: Location, Time");

Step 3: Creating an Agent

With the kernel and plugins in place, let's create our agent using SK's built-in agent templates, which abstract away a lot of the semantics behind the kernel invocation.

using Microsoft.SemanticKernel.Agents;

// Define the agent's persona
string systemPrompt = """
You generate a short, upbeat Message of the Day.
Personalize it using the user’s city and the local time of day there.
Keep responses concise and warm.
""";

// Create the chat-completion agent
var agent = new ChatCompletionAgent(
    kernel,
    instructions: systemPrompt,
    name: "MotdAssistant");

Console.WriteLine($"Agent '{agent.Name}' created.");
    

Step 4: Invoking the Agent

Invoking the agent is simple — just send a message. But when you invoke the agent, you can specify how tool calls are handled using execution settings. By setting FunctionCallBehavior.Auto, you allow the kernel to automatically execute any tool calls that the model decides are needed. In this setup, the model determines which function to call and when, while the kernel performs the actual call and provides the results back to the model.

using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.ChatCompletion;

// Define execution settings to automatically handle tool calls
var executionSettings = new ChatRequestSettings
{
    FunctionCallBehavior = FunctionCallBehavior.Auto
};

// Invoke the agent with a simple greeting
var reply = await agent.InvokeAsync("Hello", executionSettings);

Console.WriteLine($"\nAgent: {reply}");
    

Behind that single call, a structured conversation unfolds:

From one simple InvokeAsync("Hello"), you get a complete reasoning-and-execution loop that feels natural to the user. In the table above, I have provided the 'Flow' only to illustrate the data flow, not the actual contract used for that interaction.

Observability: Logging, Metrics, and Tracing

Production systems require more than clever prompts - they need visibility. Observability is the ability to understand the internal state of a system through logs, metrics, and traces. Semantic Kernel supports OpenTelemetry, providing data on function invocation times, token usage, latency, and model completion metrics, giving developers full visibility into cost and performance.

Adding a Logging Filter

You can add filters that run before and after function calls. For example, a simple logging filter can record when each plugin method starts and finishes:

using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Filters;

public sealed class LoggingFilter(ILogger<LoggingFilter> logger) : IFunctionInvocationFilter
{
    public async Task OnFunctionInvocationAsync(
        FunctionInvocationContext context,
        Func<FunctionInvocationContext, Task> next)
    {
        logger.LogInformation("Invoking {Plugin}.{Function}", context.Function.PluginName, context.Function.Name);
        await next(context);
        logger.LogInformation("Invoked {Plugin}.{Function}", context.Function.PluginName, context.Function.Name);
    }
}

// Register the filter
builder.Services.AddSingleton<IFunctionInvocationFilter, LoggingFilter>();

This makes it easy to trace function usage and latency in real time. You can also use these filters for a variety of purposes, e.g., to get consent from the user before the agent can perform any sensitive action, checking for any prompt violations, etc.

Advanced Concepts

Memory and Persistent Context

Memory lets agents retain context between sessions - for example, remembering a user's name or preferences. Semantic Kernel supports external memory stores like Redis or vector databases to persist and retrieve this context. SK now supports out-of-the-box connectors for your vector databases as well, which makes RAG and other semantic search capabilities available to the agent, making it even more powerful.

Making Agents Production-Ready

1. Clear System Prompts

The system prompt is your agent's behavioral contract. A clear, scoped prompt reduces unnecessary back-and-forth reasoning and lowers token usage.

2. Token Awareness

Track token counts for prompts and completions to manage cost and prevent truncation.

3. Security

Validate user inputs, restrict tool capabilities, and store credentials securely.

4. Scalability

• Cache frequent tool results (for example, static location or configuration lookups)
• Cache prompt templates or embeddings for reuse
• Use asynchronous operations for model and tool execution to maximize throughput
• Keep agents stateless or store shared context externally for horizontal scaling
• Use observability data to identify high-frequency tool calls that benefit most from caching

By combining concise prompts, token observability, strong security, and smart caching, you can turn a simple proof-of-concept agent into a scalable, efficient, and reliable production service.

Takeaways

• Semantic Kernel simplifies building AI agents by connecting LLMs with your own code and data.
• The Kernel orchestrates the flow between reasoning and execution, while plugins expose real capabilities as callable tools.
• A well-defined system prompt is the foundation for agent consistency and reasoning clarity.
• Clear system prompts and concise design reduce unnecessary back-and-forth between the kernel and the LLM, improving cost and latency.
• Observability features like otel support and filters make it easier to monitor tool usage and model behavior.
• Advanced concepts like connectors, text-search for RAG, and memory enable multi-step reasoning and persistent context.

In short, Semantic Kernel turns large language models into reliable, observable, and extensible agents that can reason, act, and scale.