What it does
Azure AI Foundry is a unified development platform and management layer for building, evaluating, and deploying AI applications and agents. It is the workspace and governance layer on top of the underlying model and compute services — not itself a billable inference service.
Core capabilities:
Hubs and Projects — organises AI workloads into Hubs (shared infrastructure, networking, and governance) and Projects (team- or workload-scoped workspaces within a Hub).
Model Catalogue — a curated marketplace of foundation models: Azure OpenAI models, Meta Llama, Mistral, Phi, Cohere, and others. Models can be deployed directly from the catalogue.
Prompt Flow — a visual authoring and testing environment for chaining LLM calls, tools, and logic into repeatable pipelines.
Evaluations — built-in and custom evaluation frameworks for measuring AI application quality (groundedness, coherence, relevance, safety) against test datasets.
Tracing and observability — end-to-end tracing of agent and prompt flow executions using OpenTelemetry.
AI Agent Service — a managed runtime for deploying and operating AI agents with tool use, memory, and multi-agent orchestration.
Azure AI Search and storage integration — first-class connectors to index and retrieve grounding data for RAG workloads.
Responsible AI tooling — content safety evaluation, red-teaming tools, and model monitoring dashboards.
When you'd use it
You are building a production AI application and need a structured workspace with access control, deployment management, and evaluation pipelines — not just raw API access.
You want to compare models side-by-side in the playground before committing to one for a workload.
You need to evaluate AI application quality systematically (e.g. hallucination rate, answer relevance) rather than ad-hoc testing.
You are deploying AI agents that need tool use, persistent memory, or multi-step orchestration.
Your organisation needs governance: audit logs of model calls, centrally managed credentials, and policy-enforced content safety.
Sample use case
A software company is building a customer-facing product documentation assistant. The team uses Azure AI Foundry to manage the project: they deploy GPT-4o and an embedding model through the Hub, build and test a RAG prompt flow in the Foundry portal, run automated evaluations measuring groundedness and relevance against a test set of 200 questions, and deploy the approved flow as a managed endpoint. Monitoring dashboards track latency, token consumption, and safety metric trends in production.
Example reference architecture
.png)
Pricing & How Costs Work
Azure AI Foundry itself has no direct service charge. The Foundry portal, Hub management plane, project workspaces, evaluation framework, and prompt flow authoring are all free to use. Costs come entirely from the underlying Azure resources that Foundry provisions and orchestrates.
Where the costs actually come from
Cost source | Azure service billed | Billing model |
|---|---|---|
Model inference (chat, embeddings, image gen) | Azure OpenAI Service | Per token (PAYG) or PTUs |
Open-source model inference (Llama, Mistral, Phi via serverless) | Azure AI Model Inference / Marketplace | Per token; rate varies by model and publisher |
Evaluation runs | Azure OpenAI Service (judge model calls) + Azure ML compute | Tokens consumed by the evaluator model + compute hours for the evaluation job |
Prompt Flow execution (managed compute) | Azure ML managed compute | Per compute hour (instance type × duration) |
AI Agent Service (managed runtime) | Azure AI Agent Service | Per message/session (consumption-based; pricing as per current Azure rates) |
Storage (datasets, model outputs, logs) | Azure Blob Storage / ADLS Gen2 | Per GB stored + egress |
Vector index / search | Azure AI Search | Per search unit (SU) per hour |
Monitoring and tracing | Azure Monitor / Application Insights | Per GB of logs and traces ingested |
Private networking | Azure Virtual Network / Private Endpoints | Per private endpoint per hour + data processed |
Key Vault (secret management) | Azure Key Vault | Per 10,000 operations |
Model Catalogue deployment modes and their cost implications
When deploying a model from the Foundry Model Catalogue, there are three deployment options with different cost models:
Deployment type | How it works | Cost model |
|---|---|---|
Azure OpenAI managed | Microsoft-managed infrastructure, dedicated to the model | Pay-per-token (PAYG) or PTU |
Serverless API (pay-per-token) | Model hosted by Microsoft or publisher; no infrastructure to manage | Pay-per-token; rate set by model publisher; billed through Azure Marketplace |
Managed compute | Model runs on an Azure ML managed online endpoint you provision | Pay-per-hour for the underlying VM SKU; you control the instance type and count |
Key mechanics:
Serverless API deployments for third-party models (e.g. Llama, Mistral) appear as Azure Marketplace charges in Cost Management, not as standard Azure resource charges — they are shown under a separate "Marketplace" section in the Cost Management portal.
Managed compute deployments are billed as Azure ML compute: you pay for the VM as long as the endpoint is active, regardless of request volume. This is an always-on cost.
Evaluation jobs run as Azure ML experiment runs — they spin up compute on demand and consume judge model tokens (typically GPT-4o or GPT-4o mini). A single evaluation run against a 200-row dataset can consume significant tokens if the evaluator generates long reasoning per row.
Azure Cost Data — Meters & Meter Subcategories
Because Foundry is a management plane, its costs are spread across multiple Azure services and service families in Cost Management. There is no single Microsoft.MachineLearningServices/foundry resource type — costs appear under the services Foundry provisions.
eg: Azure OpenAI inference (via Foundry)
Appears under Cognitive Services — identical meters to a standalone Azure OpenAI resource. The ResourceId will be the Azure OpenAI account attached to the Hub.