Microsoft Tech for Modern Developers

Mastering Azure AI Foundry: A Hands-on Guide to Building Intelligent Applications

· 6 min read · Tech Staff

Microsoft Foundry (recently rebranded from Azure AI Foundry) is Azure's unified platform for building production-ready AI applications and agents. If you've been navigating the maze of Azure AI services wondering how to tie everything together, Foundry is Microsoft's answer—a single endpoint, consistent APIs, and tools that work across multiple model providers.

Understanding Microsoft Foundry: What You Actually Get

Microsoft Foundry isn't a single service—it's a platform that unifies agents, models, and tools under one management umbrella. Think of it as your AI operations center where you can build, deploy, monitor, and scale intelligent applications without wrestling with dozens of disparate Azure resources.

The platform offers two distinct experiences: Foundry (classic) and Foundry (new). Unless you're maintaining legacy hub-based projects, you'll want the new experience—it's where all the modern agent capabilities, multi-agent orchestration, and enhanced tooling live.

Foundry Projects vs Hub-Based Projects

Here's where things get confusing if you've worked with Azure AI before. Microsoft Foundry supports two project types, and choosing the wrong one will frustrate you quickly:

Foundry Projects (what you want):

  • Managed under a Microsoft Foundry resource
  • General availability (GA) support for agents and the Foundry API
  • Project-level file isolation and conversation storage
  • Direct access to Azure OpenAI, DeepSeek, xAI, and marketplace models
  • Cleaner resource management—no need to create separate hubs

Hub-Based Projects (legacy):

  • Hosted by a Microsoft Foundry hub resource
  • Only preview support for agents
  • Required if you need Prompt Flow or models deployed on managed compute
  • More complex resource hierarchy

The simple decision tree: Building agents or working with modern models? Use a Foundry project. Need Prompt Flow or HuggingFace models on Azure compute? You're stuck with hub-based for now.

Getting Started: First Application in 10 Minutes

Let's build a working intelligent application. We'll use C# since this is MSCloud, but the pattern applies across Python, TypeScript, and Java.

Prerequisites

You need:

  • An Azure subscription with contributor access
  • .NET 8.0 or later
  • Azure CLI (az login to authenticate)
  • Visual Studio or VS Code

Step 1: Create a Foundry Project

The fastest way is through the portal:

  1. Navigate to ai.azure.com
  2. Toggle to Foundry (new) if you see the classic banner
  3. Select a model from the catalog (I recommend gpt-4o-mini for development)
  4. Click Use this model and create a new project
  5. Deploy the model—this automatically creates your Foundry project

Copy your Foundry project endpoint from the Overview page. It looks like:

https://your-resource.services.ai.azure.com/api/projects/your-project

Step 2: Install the SDK

Create a new console app and add the preview packages:

dotnet new console -n FoundryQuickstart
cd FoundryQuickstart
dotnet add package Azure.AI.Projects --prerelease
dotnet add package Azure.Identity

The --prerelease flag matters—you need Azure.AI.Projects 2.x for the new Foundry API.

Step 3: Chat with a Model

Replace the contents of Program.cs:

using Azure.AI.Projects;
using Azure.AI.Projects.OpenAI;
using Azure.Identity;
using OpenAI.Responses;

// Set these from your environment or directly
string projectEndpoint = Environment.GetEnvironmentVariable("PROJECT_ENDPOINT")
    ?? "https://your-resource.services.ai.azure.com/api/projects/your-project";
string modelName = "gpt-4o-mini";

// Authenticate using Azure CLI credentials
var credential = new AzureCliCredential();
var projectClient = new AIProjectClient(new Uri(projectEndpoint), credential);

// Get an OpenAI-compatible client from the project
var responsesClient = projectClient.OpenAI
    .GetProjectResponsesClientForModel(modelName);

// Send a simple query
var response = await responsesClient.CreateResponseAsync(
    "Explain dependency injection in .NET in one sentence.");

Console.WriteLine(response.Value.GetOutputText());

Run it:

dotnet run

You'll see a model-generated response. This confirms your authentication, endpoint, and model deployment are working.

What Just Happened?

The AIProjectClient is your entry point to everything Foundry. Notice we didn't specify:

  • OpenAI API keys
  • Model-specific endpoints
  • Provider-specific configuration

The Foundry SDK abstracts all of that. You can swap gpt-4o-mini for gpt-4.1-mini or even a different provider's model without changing your code.

Building an Agent: The Real Power

Agents are where Foundry shines. An agent encapsulates instructions, tools, and behavior—you define it once, then interact with it repeatedly without resending instructions every turn.

Creating an Agent with Instructions

using Azure.AI.Projects;
using Azure.AI.Projects.Models;
using Azure.Identity;

string projectEndpoint = "https://your-resource.services.ai.azure.com/api/projects/your-project";
string modelName = "gpt-4o-mini";

var credential = new AzureCliCredential();
var projectClient = new AIProjectClient(new Uri(projectEndpoint), credential);

// Define the agent
var agentDefinition = new PromptAgentDefinition(modelName)
{
    Instructions = "You are a C# code reviewer. Focus on performance, " +
                   "security, and modern .NET best practices. Be direct."
};

// Create a versioned agent
var agent = projectClient.Agents.CreateAgentVersion(
    "code-reviewer",
    options: new(agentDefinition));

Console.WriteLine($"Agent created: {agent.Id} (v{agent.Version})");

Agents are versioned automatically. This means you can update instructions, compare behavior across versions, and roll back if needed.

Multi-Turn Conversations with Agents

Conversations maintain state across multiple interactions:

using Azure.AI.Projects.OpenAI;
using OpenAI.Responses;

var openAIClient = projectClient.GetOpenAIClient();

// Create a conversation (think of it as a thread)
var conversation = openAIClient.Conversations.Create();

var responsesClient = projectClient.OpenAI.GetProjectResponsesClientForAgent(
    defaultAgent: "code-reviewer",
    defaultConversationId: conversation.Id);

// First turn
var response1 = responsesClient.CreateResponse(
    "Review this: public static string GetName() => UserName;");
Console.WriteLine("Turn 1: " + response1.Value.GetOutputText());

// Follow-up question - agent remembers context
var response2 = responsesClient.CreateResponse(
    "What would you change in that code?");
Console.WriteLine("Turn 2: " + response2.Value.GetOutputText());

// Clean up when done
openAIClient.Conversations.Delete(conversation.Id);

The conversation maintains full context. The agent knows "that code" refers to the snippet from Turn 1.

Key Concepts That Trip People Up

1. Authentication: Azure CLI vs Managed Identity

In development, AzureCliCredential is fine—it uses your az login session. In production, use DefaultAzureCredential:

// Production pattern
var credential = new DefaultAzureCredential();

DefaultAzureCredential tries multiple auth methods in order:

  • Environment variables
  • Managed identity (if running in Azure)
  • Visual Studio credentials
  • Azure CLI
  • Azure PowerShell

Assign your app's managed identity the Azure AI Developer role on the Foundry resource.

2. Model Names vs Deployment Names

With Foundry projects, you reference models directly by their canonical names (gpt-4o-mini), not deployment names. The Foundry API routes to the correct deployment automatically.

This is different from Azure OpenAI's direct API where you specify a deployment name.

3. Conversation Lifetime Management

Conversations aren't automatically cleaned up. In production, implement a retention policy:

// Store conversation ID in your database with creation timestamp
var conversation = openAIClient.Conversations.Create();
await SaveConversationToDb(conversation.Id, userId, DateTime.UtcNow);

// Background job to clean up old conversations
async Task CleanupOldConversations()
{
    var oldConversations = await GetConversationsOlderThan(TimeSpan.FromDays(30));
    foreach (var convId in oldConversations)
    {
        try
        {
            openAIClient.Conversations.Delete(convId);
        }
        catch (Exception ex)
        {
            // Log but don't fail the cleanup job
            _logger.LogWarning(ex, "Failed to delete conversation {ConvId}", convId);
        }
    }
}

4. Error Handling Patterns

Foundry API calls can fail for multiple reasons. Handle them explicitly:

try
{
    var response = await responsesClient.CreateResponseAsync(userInput);
    return response.Value.GetOutputText();
}
catch (Azure.RequestFailedException ex) when (ex.Status == 429)
{
    // Rate limit exceeded - implement backoff
    await Task.Delay(TimeSpan.FromSeconds(5));
    // Retry logic here
}
catch (Azure.RequestFailedException ex) when (ex.Status >= 500)
{
    // Server error - log and fail gracefully
    _logger.LogError(ex, "Foundry service error");
    return "I'm having trouble connecting. Please try again.";
}
catch (Exception ex)
{
    // Unexpected error
    _logger.LogError(ex, "Unexpected error in AI response");
    throw;
}

Advanced Patterns: Tool Integration

Agents become powerful when they can call your code. Foundry supports function calling natively:

// Define a tool the agent can call
var weatherTool = new FunctionToolDefinition(
    name: "get_weather",
    description: "Gets current weather for a location",
    parameters: BinaryData.FromObjectAsJson(new
    {
        type = "object",
        properties = new
        {
            location = new { type = "string", description = "City name" },
            unit = new { type = "string", @enum = new[] { "celsius", "fahrenheit" } }
        },
        required = new[] { "location" }
    }));

var agentWithTool = new PromptAgentDefinition(modelName)
{
    Instructions = "You are a helpful weather assistant.",
    Tools = { weatherTool }
};

// When agent calls the tool, you'll receive the function name and parameters
// Implement a handler to execute your code and return results

Tool integration is what separates toy demos from production AI applications. Your agent can query databases, call APIs, or execute business logic without the model needing to know implementation details.

Monitoring and Observability

Foundry (new) includes built-in observability. In the portal under Operate, you'll see:

  • Request traces and latencies
  • Token usage per conversation
  • Error rates and types
  • Model performance metrics

For programmatic access, enable Application Insights in your Foundry resource. Then use the Azure Monitor SDK:

// Configure telemetry
var telemetryConfig = TelemetryConfiguration.CreateDefault();
telemetryConfig.ConnectionString = 
    "InstrumentationKey=your-appinsights-key";
var telemetryClient = new TelemetryClient(telemetryConfig);

// Track custom metrics
telemetryClient.TrackEvent("AgentInteraction", new Dictionary<string, string>
{
    ["AgentName"] = "code-reviewer",
    ["ConversationId"] = conversation.Id,
    ["TokensUsed"] = response.Value.Usage?.TotalTokens.ToString() ?? "0"
});

Cost Management Strategies

Foundry pricing is based on the services you consume—mainly model API calls and token usage. A few patterns to keep costs reasonable:

1. Use model tiers strategically:

// Use smaller models for simple tasks
string quickModel = "gpt-4o-mini";  // Cheaper, faster
string powerModel = "gpt-4.1";      // More expensive, more capable

// Route based on complexity
var model = IsComplexQuery(userInput) ? powerModel : quickModel;
var responsesClient = projectClient.OpenAI.GetProjectResponsesClientForModel(model);

2. Limit conversation context:

// Don't keep infinite history
var recentMessages = await GetRecentMessages(conversation.Id, limit: 10);

3. Set token budgets:

var response = await responsesClient.CreateResponseAsync(
    userInput,
    new ResponseCreateOptions
    {
        MaxTokens = 500  // Cap response length
    });

Deployment Considerations

Local Development

  • Use AzureCliCredential for quick iteration
  • Set PROJECT_ENDPOINT in your .env file or User Secrets
  • Keep conversation cleanup manual during debugging

Production

  • Use DefaultAzureCredential with managed identity
  • Store PROJECT_ENDPOINT in Azure Key Vault or App Configuration
  • Implement conversation retention policies
  • Enable Application Insights for monitoring
  • Use Azure Front Door or API Management for rate limiting

Example Production Configuration

public class FoundrySettings
{
    public string ProjectEndpoint { get; set; }
    public string DefaultModel { get; set; }
    public int MaxConversationAgeDays { get; set; } = 30;
    public int MaxTokensPerResponse { get; set; } = 1000;
}

// In Startup.cs or Program.cs
builder.Services.Configure<FoundrySettings>(
    builder.Configuration.GetSection("Foundry"));

builder.Services.AddSingleton<AIProjectClient>(sp =>
{
    var settings = sp.GetRequiredService<IOptions<FoundrySettings>>().Value;
    var credential = new DefaultAzureCredential();
    return new AIProjectClient(new Uri(settings.ProjectEndpoint), credential);
});

When NOT to Use Microsoft Foundry

Foundry isn't always the right choice:

  • Pure Azure OpenAI usage: If you only need OpenAI models and nothing else, the Azure OpenAI SDK is simpler
  • On-premises requirements: Foundry is cloud-only; use Azure AI containers if you need on-prem
  • Custom model hosting: If you're running HuggingFace models on your own compute, hub-based projects might fit better
  • Non-Azure environments: Foundry ties you to Azure infrastructure

What You Should Remember

Microsoft Foundry is Microsoft's bet on making enterprise AI accessible to working developers. The key insights:

  1. Choose Foundry projects over hub-based unless you have specific legacy requirements
  2. Agents maintain state—use them for conversational experiences, not one-shot queries
  3. Authentication matters—use managed identity in production, not API keys
  4. Monitor token usage—implement budgets and routing to control costs
  5. Conversations don't auto-cleanup—build retention policies into your app

Start with the simple model query pattern, graduate to agents when you need stateful conversations, and add tools when you need to integrate with your systems. The SDK is consistent across languages, so the patterns you learn in C# apply equally to Python or TypeScript.

The platform is evolving fast—Microsoft is actively adding features like multi-agent orchestration and expanded tool catalogs. Stay current by checking the Microsoft Foundry documentation for updates.

Now go build something intelligent.