AI Content Safety

What it does

Detects harmful, inappropriate, or policy-violating content in text and images before it reaches users or is stored:

• Classifies content across four harm categories: hate, violence, sexual, self-harm.

• Returns a severity score (0–7) per category, allowing configurable thresholds.

• Prompt Shields — detects prompt injection and jailbreak attempts in LLM inputs.

• Groundedness detection — flags when an LLM response is not grounded in the provided context (hallucination detection).

When you'd use it

• Any public-facing AI feature where user-generated text or images could contain harmful content.

• As a pre- or post-processing guardrail around Azure OpenAI calls.

• Platforms with user-generated content (forums, social features, marketplaces) that need automated moderation.

Sample use case

An online learning platform allows students to submit written assignments and discussion posts. Azure AI Content Safety screens every submission before it is published. Posts above the configured severity threshold are held for human review; the student is notified without the post going live, preventing exposure of harmful content to other students.

Example reference architecture

Pricing & How Costs Work

Azure AI Content Safety (now marketed as Content Safety in Foundry Control Plane) is priced per 1,000 text records for text features and per 1,000 images for image features. All text-based features share the same per-record rate — they are not individually priced.

What is a text record? A text record is up to 1,000 characters (measured by Unicode code points). If a text input exceeds 1,000 characters it is split into multiple records, each billed separately:

• 500-character input → 1 record

• 1,500-character input → 2 records

• 7,500-character input → 8 records

This means longer text costs more — unlike a pure per-call model, cost scales with input length.

Key pricing mechanics:

• Free tier: 5,000 text records/month + 5,000 images/month at no cost.

• Standard tier: All text features (moderation, Prompt Shields, Protected material detection, Groundedness detection) are charged at the same $0.38/1,000 records rate — there is no premium rate for advanced features.

• Image billing: Each image submitted = one billable unit, regardless of image dimensions or content.

• Disconnected container (commitment tier): Available for large-scale on-premises deployments at a fixed monthly commitment price.

• There is no per-feature surcharge — enabling Groundedness detection costs the same per record as basic harm category detection.

What Drives Costs

Costs are driven by two dimensions: volume of submissions and length of text submitted.

Cost optimisation levers:

• Truncate or summarise long inputs before submission where the full length is not needed for accurate moderation.

• Only call Groundedness detection for use cases where hallucination detection is genuinely required — not as a default on all completions.

• Limit Prompt Shields to user-controlled inputs; skip for internally generated prompts.

• Cache moderation results for identical or near-identical content using a content hash.

Azure Cost Data — Meters & Meter Subcategories

When you query Cost Management or export billing data, Azure AI Content Safety appears under the Cognitive Services service family. The meters you will encounter are:

Meter subcategory notes:

• All text features share a single Text Records meter — there is no separate meter per feature (no distinct meter for Prompt Shields vs harm categories vs Groundedness detection). To see a breakdown by feature you need to instrument your application or use Azure Monitor diagnostics logs.

• The Quantity in billing exports represents text records consumed (not API calls). A single API call submitting 3,000 characters generates a quantity of 3 on this meter.

• The meter subcategory reflects the pricing tier (Standard S). The free tier (F) usage appears as zero-cost rows with the Free F subcategory.

• The ResourceId in billing data maps to the specific Cognitive Services account resource, making it possible to allocate costs per environment (dev/prod) if you use separate resources.

• Because all features roll into one meter, cost spikes that indicate a specific feature being overused (e.g. Groundedness detection being accidentally called on every request) are not directly visible in Cost Management — application-level logging is the only way to attribute record consumption by feature.

Common Developer Mistakes That Drive Up Costs

• Not accounting for text length in cost estimates. The billing unit is a text record of up to 1,000 characters, not an API call. A single API call submitting a 5,000-character document costs 5 records — the same as 5 separate short-message submissions. Teams that estimate cost based on API call count without considering average input length significantly underestimate their spend on document or long-form content moderation workloads.

• Running Groundedness Detection on every LLM response at scale. Groundedness detection is priced at the same per-record rate as basic harm detection, but LLM responses passed for grounding checks tend to be long (multiple paragraphs of response + source context), generating many records per call. Teams that enable it as a default post-processing step on all completions drive up record consumption without necessarily needing hallucination detection for every response.

• Enabling all harm categories when only a subset is needed. The API allows selective category enabling. Calling with all four harm categories (hate, violence, sexual, self-harm) when only one is relevant to the use case does not cost extra per call, but it invites future scope creep where unnecessary categories are retained out of habit.

• Not implementing caching for identical or near-identical content. Content moderation on user-generated content platforms often sees the same or very similar strings submitted repeatedly (copy-paste posts, spam campaigns). Without a cache keyed on content hash, every submission generates a billable API call.

• Applying Prompt Shields to all API calls including internal/trusted ones. Prompt Shield is billed separately and is designed to protect against untrusted external input. Applying it to prompts generated entirely by the application's own code (not user-controlled) wastes calls.

What Could Make Your Bill Go Up or Down Next Month

Could go UP:

• A public-facing AI feature is launched, significantly increasing the volume of user-submitted text or images being screened.

• Groundedness Detection is enabled as a default for all Azure OpenAI responses in a high-volume pipeline.

• A spam or abuse campaign on the platform causes a spike in user-generated content submissions, each requiring moderation.

• Image moderation is added to cover a new user profile picture upload feature, introducing a new call type at scale.

• Long-form content (articles, documents, support transcripts) is submitted for moderation, multiplying records per submission due to the 1,000-character record boundary.

Could go DOWN:

• Response caching is implemented for identical content strings, reducing repeat API calls.

• Groundedness Detection is scoped to only the highest-risk response categories rather than all LLM outputs.

• Prompt Shields are limited to user-controlled input paths only, removing calls from internal pipeline steps.

• A seasonal reduction in platform user activity (e.g. post-holiday) reduces submission volume.

Most Common Optimisation Techniques