Azure OpenAI Service

With OpenAI direct, application code commonly sends a model value in the body. With Azure OpenAI, the important routing decision is usually in the URL: the client calls a specific Azure resource endpoint and a specific deployment name. That deployment is an Azure object that maps to a model version and capacity configuration.

This matters operationally. A deployment name such as customer-support-gpt4o can stay stable while the platform team changes the underlying model version through a controlled rollout. It also means quota, PTUs, content policies, and monitoring can be reasoned about at the deployment boundary instead of being hidden inside arbitrary model strings sent by each application.

Azure OpenAI requests include an api-version query parameter. The version controls the REST shape, feature availability, response fields, preview behavior, and sometimes compatibility with model capabilities. Treat it like any other external API version, not as incidental boilerplate.

Pinning an api-version makes behavior reproducible. Upgrading it should be reviewed with tests for streaming, tool calling, structured outputs, content-filter fields, and error handling because a newer API version can expose new capabilities or slightly different response metadata.

API keys are simple: the client sends a resource key in the request header. They are useful for quick prototypes and some server-side integrations, but they are bearer secrets that must be stored, rotated, and kept away from browsers and mobile apps.

Microsoft Entra ID is the enterprise pattern. A workload identity or managed identity obtains a token for Azure AI services and Azure RBAC decides whether that identity can call the Azure OpenAI resource. This gives you central revocation, least privilege, auditability, and no long-lived secret in application configuration.

An Azure OpenAI resource lives in an Azure region, and the chosen deployment type affects where requests are processed and what capacity pool is used. For strict residency, engineers must choose the right region and deployment option, then verify the current Azure data-residency documentation for that SKU and geography.

Private networking is also a product decision. Private Link can expose the resource through a private endpoint inside a VNet, private DNS can resolve the service name to that private address, and public network access can be disabled. That combination lets a backend service call the model without traversing the public internet.

Azure OpenAI includes content filters that evaluate prompts and completions for safety categories such as hate, sexual content, violence, and self-harm, with additional classifiers depending on capability and policy. A request can be allowed, annotated, or blocked depending on severity and configuration.

The response can include fields such as prompt_filter_results and content_filter_results. Production code should log and interpret these annotations because they explain why an answer was truncated, filtered, or refused. Do not treat a filtered response as a generic model failure.

Azure OpenAI enterprise commitments state that prompts, completions, embeddings, and fine-tuning data are not used to train foundation models and are not used to improve Microsoft or third-party products. That is a major difference from consumer-style assumptions and is often why enterprises choose Azure.

At the same time, requests are still processed by the service for inference, content filtering, logging, abuse monitoring, and operational security according to Azure terms and approved configurations. A careful design separates model-training privacy from runtime data handling, retention, monitoring, and access control.

A deployment name is not just a cosmetic alias. It is the stable contract your application calls, and it can represent a particular model version, regional capacity choice, content-filter setup, and quota allocation. This makes it a natural unit for blue-green rollouts and workload isolation.

For portability, use logical names in your app such as chat_reasoning_primary and map them to either an OpenAI model or an Azure deployment in configuration. That prevents product code from depending on provider-specific URL shapes.

Basic chat completion calls may look similar across versions, but advanced features such as tool calling, structured outputs, streaming fields, and filter metadata can depend on the chosen api-version. A silent version change can create confusing failures where one environment supports a feature and another does not.

Treat api-version as pinned infrastructure configuration. Include it in logs and regression tests, and roll it forward deliberately the same way you would upgrade a database driver or cloud SDK.

API keys minimize setup but create secret-management work: storage, rotation, leak detection, environment separation, and blast-radius control. They also do not express which workload is calling unless you wrap them with additional gateway logic.

Managed identities and Entra ID shift that burden to the cloud identity plane. They work especially well for Azure-hosted compute because the service can obtain tokens without a client secret. The tradeoff is that local development and cross-cloud workloads need a clear credential strategy.

Pay-as-you-go deployments consume quota and share regional service capacity. They are flexible and often the right starting point, but they can hit tokens-per-minute, requests-per-minute, or transient capacity limits as traffic grows.

Provisioned throughput units reserve model-serving capacity for a deployment. PTUs can improve predictable throughput and latency for steady production load, but they require capacity planning and cost commitment. They do not remove the need to control prompt size, concurrency, retries, and fallback behavior.

A filtered response should not be collapsed into a generic error message. Filter annotations can tell you which category and severity were involved, whether the prompt or completion triggered the decision, and whether the user experience should ask for safer wording, escalate, or show a policy explanation.

Applications should track filter rates by feature and deployment. Sudden changes can mean a prompt regression, a new abuse pattern, a model behavior change, or a data-quality problem in retrieved context.

The fact that prompts are not used to train foundation models does not mean every prompt is safe to send. Data can still be subject to runtime processing, logging choices, abuse monitoring, customer support access controls, and retention policies.

Good designs minimize sensitive input, redact unnecessary PII, use private networking when appropriate, restrict RBAC, and document which data categories each deployment may process. Enterprise privacy is a platform capability, not a substitute for application data governance.

Log deployment name, api-version, authentication mode, region, latency, token counts, status codes, retry count, and content-filter annotations. Deployment-level observability lets platform teams find noisy tenants, quota hot spots, unsafe prompt patterns, and version-specific regressions.

Retry only transient failures with exponential backoff and jitter. Do not blindly retry 400, 401, 403, or blocked content-filter outcomes. For 429 and temporary capacity pressure, use queueing, request shedding, smaller prompts, cached answers, a secondary deployment, or a lower-cost fallback model.

Prefer managed identities for server workloads, store API keys only in a secure secret store when keys are unavoidable, disable public network access when private connectivity is required, and never expose Azure OpenAI credentials to a browser. Pair RBAC with application-level authorization so users cannot call a model just because the backend can.

Plan for content-filter events, abuse-monitoring review, user appeals, audit trails, model access approvals, and policy updates. Responsible AI is not only a launch checklist; it is an operating process that explains why a request was allowed, blocked, escalated, or rate limited.

This Python example uses DefaultAzureCredential so the same code can use a managed identity in Azure or a developer login locally. Notice that the URL contains the Azure endpoint, deployment name, and api-version, while the request body contains messages and generation settings rather than a model field.

A small adapter can map one logical model name to provider-specific configuration. Product code asks for policy_chat, while configuration decides whether that means an OpenAI model or an Azure deployment.