Tutorial · 2026

Connect Azure OpenAI to SAP BTP: Generative AI Orchestrator Setup

Connect Azure OpenAI to SAP BTP: provision a deployment via Azure CLI, register it in SAP AI Launchpad's Generative AI Orchestrator, and invoke it over REST.

advanced 11 min read Updated Nov 8, 2026

30-second summary

  • The path: Azure OpenAI deployment -> register in SAP AI Launchpad Generative AI Orchestrator -> call over REST via AI Core.
  • Keys work, Entra ID / managed identity is the production answer - we show both, with the actual error payloads you will hit.
  • You need the SAP AI Core 'extended' plan for orchestration; the default plan will not register external models.
  • Four failure points account for most setup time: RBAC, region quota, service key scope, model restriction config.
Contents

Bringing Azure OpenAI models into an SAP Business Technology Platform (BTP) landscape is the most common “first AI project” for SAP architects in 2026 — whether you are extending Joule, building a custom RAG assistant on SAP Core AI services, or wiring GPT-4o into a CAP application. There are two fundamentally different ways to do it, and confusing them is the single biggest source of failed setups:

  1. SAP-managed path (Generative AI hub / Orchestrator in SAP AI Launchpad): Azure OpenAI models such as gpt-4o, gpt-4o-mini and the gpt-4.1 family are offered through the azure-openai executable in SAP AI Core. SAP hosts and manages the Azure capacity on tenant-isolated hyperscaler infrastructure; your application authenticates with a BTP service key, and your own Azure API keys are never involved.
  2. Bring-your-own (BYO) path: You consume your Azure OpenAI resource — your subscription, your quota, your region — directly from a BTP application via REST, or behind a proxy. This is the only way to bill inference against your own enterprise agreement.

This tutorial sets up both, then runs a real orchestration call (prompt templating plus PII masking) and covers the errors you will actually hit.

Architecture Overview

SAP BTP subaccount (Cloud Foundry / Kyma)
│
│  service key (clientid/clientsecret) ──► OAuth2: $AUTH_URL/oauth/token ──► Bearer token
│
├── Path A: SAP AI Core, "extended" plan (Generative AI hub)
│     scenarios: foundation-models ─ executable "azure-openai" (SAP-managed Azure capacity)
│     scenarios: orchestration     ─ executable "orchestration" (templating, grounding,
│                                    masking, filtering, translation, fallbacks)
│     $AI_API_URL/v2/lm/...                 runtime: $ORCH_DEPLOYMENT_URL/v2/completion
│     SAP AI Launchpad UI  ── configures, restricts and tests both scenarios
│
└── Path B: BYO Azure OpenAI resource (your subscription)
      Azure OpenAI endpoint: https://<resource>.openai.azure.com
      auth: api-key header ── or ── Entra ID bearer token + RBAC role
      POST /openai/deployments/<deploymentName>/chat/completions?api-version=2024-10-21
      egress: direct internet from the BTP runtime (public endpoint)

Key distinction: in Path A the hub’s Azure OpenAI models are remote, SAP-managed endpoints. The hub does not expose a form where you paste your own *.openai.azure.com URL and key into a Launchpad “register endpoint” screen; that registration concept exists for the orchestration service instance (configuration + deployment of the orchestration executable) and for model restrictions, not for customer-owned Azure keys. If your requirement is “traffic must run on our own Azure subscription and quota”, use Path B from your BTP app (or a proxy pattern — see the archived SAP-samples reference below).

Prerequisites

SAP BTP side

  • A BTP subaccount with admin access, plus role assignments for the subaccount and space. For SAP AI Launchpad you need a role collection containing at least one of mloperations_editor or scenario_deployment_editor to create configurations and deployments, and orchestration_executor, genai_experimenter or genai_manager to build and run orchestration workflows.
  • SAP AI Core service instance on the extended plan. The Generative AI hub (and therefore the foundation-models and orchestration scenarios) is not available in the Free or Standard plans — if the scenarios are missing, check your tenant service plan.
  • SAP AI Launchpad subscribed from the Service Marketplace (subaccount → Service Marketplace → SAP AI Launchpad → Create).
  • A service key for AI Core exposing clientid, clientsecret, url (auth), identityzone, and serviceurls.AI_API_URL.

Azure side (Path B)

  • An Azure subscription with permissions to create Cognitive Services resources (e.g. Cognitive Services Contributor on the resource group).
  • Azure CLI 2.x logged in (az login), curl and jq.
  • Quota headroom for your model in the chosen region — Azure OpenAI quota is allocated per subscription, per region, per model, in Tokens-per-Minute (TPM) (default for GPT-4o in East US was 240,000 TPM at the time of writing; verify in your own subscription).

Step 1 — Create the Azure OpenAI Resource and Deployment (Azure CLI)

Create the resource (kind OpenAI, the only SKU tier is S0):

RG=OAIResourceGroup
LOC=eastus
RES=saplab-oai-01

az group create --name $RG --location $LOC

az cognitiveservices account create \
  --name $RES \
  --resource-group $RG \
  --kind OpenAI \
  --sku s0 \
  --location $LOC \
  --tags {}

Deploy a model. The deployment name — not the model name — is what appears in every API call:

az cognitiveservices account deployment create \
  --name $RES \
  --resource-group $RG \
  --deployment-name gpt-4o-prod \
  --model-name gpt-4o \
  --model-version "2024-11-20" \
  --model-format OpenAI \
  --sku-name Standard \
  --sku-capacity 1

--sku-name accepts Standard, GlobalStandard, GlobalBatch, and ProvisionedManaged (PTU). --sku-capacity is the number of allocation units; for Standard deployments each unit maps to fixed RPM/TPM slices (for example, older chat models: 6 RPM + 1,000 TPM per unit), so capacity 1 is a pilot number only — for production set the deployment’s TPM instead (increments of 1,000) and request quota increases via the Azure quota form before scaling.

Grab the connection material:

az cognitiveservices account show \
  --name $RES --resource-group $RG \
  --query properties.endpoint -o tsv

az cognitiveservices account keys list \
  --name $RES --resource-group $RG --query key1 -o tsv

Step 2 — Grant Access: API Key vs Entra ID / Managed Identity

Two authentication modes for Path B:

API key Entra ID (user/managed identity + RBAC)
Secret in code Yes (api-key header) No — short-lived tokens
Fine-grained scope Whole resource Role per identity
Rotation burden Manual Automatic

For keyless access, note that BTP apps cannot natively use an Azure managed identity — the usual pattern is an Entra ID confidential client (service principal) or a managed identity on an Azure-hosted middle tier. Assign the data-plane role — Cognitive Services OpenAI User (role ID 5e0bd9bd-7b93-4f28-af87-19fc36ad61bd) permits inference calls but not reading keys or managing deployments; Cognitive Services OpenAI Contributor (a001fd3d-188f-4b5d-821b-7da978bf7442) additionally manages deployments; neither creates resources:

az role assignment create \
  --role "Cognitive Services OpenAI User" \
  --assignee <client-or-identity-object-id> \
  --scope "/subscriptions/<sub-id>/resourceGroups/$RG/providers/Microsoft.CognitiveServices/accounts/$RES"

A token is then requested for the data-plane scope https://cognitiveservices.azure.com/.default and sent as Authorization: Bearer <token> instead of api-key. Note the trap: the management-plane roles above govern the resource; calling chat/completions requires the data-plane role assignment you just created.

In Path A there is nothing to grant on Azure: the hub authenticates you with the OAuth2 token from your BTP service key, and SAP’s managed Azure credentials never surface.

Step 3 — Wire Up SAP AI Core and the Orchestration Instance

Create the service key and fetch a token (tokens expire after ~2 hours, expires_in: 7199):

TOKEN=$(curl -s --request POST "$AUTH_URL/oauth/token" \
  --header "Content-Type: application/x-www-form-urlencoded" \
  --data "grant_type=client_credentials" \
  --data "client_id=$CLIENT_ID" \
  --data "client_secret=$CLIENT_SECRET" | jq -r .access_token)

Every AI Core call carries AI-Resource-Group (start with default). You likely already have a running orchestration deployment in default; create one only per new resource group, or when you want model restriction. Register it exactly as the orchestrator expects — scenario orchestration, executable orchestration:

curl -s -X POST "$AI_API_URL/v2/lm/configurations" \
  -H "Authorization: Bearer $TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{"name":"orchestration-config","executableId":"orchestration","scenarioId":"orchestration"}'
# → {"id":"f7ac7f77-...","message":"Configuration created"}

curl -s -X POST "$AI_API_URL/v2/lm/deployments" \
  -H "Authorization: Bearer $TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{"configurationId":"f7ac7f77-..."}'

curl -s "$AI_API_URL/v2/lm/deployments/$DEPLOYMENT_ID" \
  -H "Authorization: Bearer $TOKEN" -H "AI-Resource-Group: default" | jq .

Poll until "status":"RUNNING" and export deploymentUrl as ORCH_DEPLOYMENT_URL.

Step 4 — SAP AI Launchpad: Register Models, Restrict, and Test

In Launchpad (Workspaces app → select your AI Core connection → pick the resource group):

  1. ML Ops app → Scenarios: confirm foundation-models and orchestration exist. Missing? You are on the wrong service plan.
  2. ML Ops app → Configurations → Create: for direct model access, choose scenario foundation-models, executable azure-openai, enter the model name (e.g. gpt-4o) and version (latest for auto-upgrade, or pin 2024-11-20 for manual control; deprecated versions stop working on their retirement date — track SAP Note 3437766). Then Create Deployment.
  3. Model restriction: on the orchestration configuration, add a modelFilterList with modelFilterListType allow or deny, e.g. [{"modelName":"gpt-4o","modelVersion":"latest"}] — this enforces which LLMs your developers may target through your shared orchestration deployment.
  4. Generative AI Hub → Orchestration → Create: the visual workflow builder (roles orchestration_executor / genai_experimenter) shows the pipeline — grounding → templating → input translation → data masking → input filtering → model configuration → output filtering → output translation. Templating and Model Configuration are mandatory and cannot be hidden. Test prompts directly in the prompt-experimentation chat, then save the orchestration config (it becomes referenceable via config_ref in REST calls).

BYO note: to run on your own Azure resource, call Path B directly from your CAP/Kyma app — a Cloud Foundry destination or Kyma service instance holding the endpoint + key (or Entra client). If you are setting that destination up for the first time, our BTP destinations + Cloud Connector guide covers the ProxyType, truststore, and principal-auth details. The classic reference implementation of proxying your Azure endpoint through AI Core is the (now archived) SAP-samples/azure-openai-aicore-cap-api repository, which registered a CAP “serving” scenario taking endpoint and api-key parameters.

Step 5 — Invoke the Orchestrator over REST

A production-shaped minimal call: templating with placeholders, GPT-4o params, and PII masking. Note max_completion_tokens — max_tokens is deprecated in the current Azure OpenAI spec, and orchestration’s drop_unsupported_params (default true) reports dropped keys in the X-Orchestration-Dropped-Model-Params response header.

curl -s -X POST "$ORCH_DEPLOYMENT_URL/v2/completion" \
  -H "Authorization: Bearer $TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "modules": {
        "prompt_templating": {
          "prompt": {
            "template": [
              {"role": "system", "content": "You are an SAP HR assistant. Answer concisely."},
              {"role": "user", "content": "Summarise this candidate profile: {{ ?candidate_resume }}"}
            ]
          },
          "model": {
            "name": "gpt-4o",
            "params": {"max_completion_tokens": 500, "temperature": 0.2}
          }
        },
        "masking": {
          "masking_providers": [
            {"type": "sap_data_privacy_integration",
             "method": "anonymization",
             "entities": ["profile-email", "profile-phone"],
             "allowlist": ["SAP"]}
          ]
        }
      }
    },
    "placeholder_values": {"candidate_resume": "Jane Roe, jane.roe@example.com, +49 170 5550101, 8 yrs ABAP..."}
  }'

The response contains intermediate_results per module and final_result.choices[0].message.content. Masked tokens are re-substituted after output filtering. A modules array instead of object turns the call into a fallback chain — orchestration uses the first configuration that succeeds, ideal for gpt-4o → gpt-4o-mini degradation.

For Path B, the BYO call from your BTP app looks like:

curl "https://$RES.openai.azure.com/openai/deployments/gpt-4o-prod/chat/completions?api-version=2024-10-21" \
  -H "api-key: $AZURE_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello SAP"}],"max_completion_tokens":50}'

Troubleshooting Common Errors

HTTP Where Likely cause Fix
401 UNAUTHORIZED Hub /v2/... Expired/malformed bearer token; wrong $AUTH_URL (must be the identityzone URL from the service key) Re-request token with grant_type=client_credentials; check key JSON
401 Azure endpoint Entra token without the data-plane RBAC role, or wrong scope Assign Cognitive Services OpenAI User; scope https://cognitiveservices.azure.com/.default
403 FORBIDDEN Hub Missing AI-Resource-Group for that group, Launchpad role missing, or plan lacks the scenario Grant mloperations_editor/scenario_deployment_editor; use extended plan
404 NOT_FOUND Hub Deployment still PENDING, deleted (TTL), or wrong deploymentUrl segment; model name not in registry Poll deployment status; re-pin the config; check /v2/lm/scenarios/foundation-models/models
404 Azure endpoint Deployment name used where model name was typed (or vice versa) — the URL takes the deployment name, the body’s model takes it too az cognitiveservices account deployment list
400 Hub v2/completion Missing placeholder value; prompt_templating absent (required); denied by model restriction Provide all placeholder_values; check modelFilterList
429 QUOTA_EXCEEDED Hub SAP-side rate limits per SAP Note 3437766 Add orchestration fallbacks; request capacity via your SAP account team
429 Azure endpoint Deployment TPM/RPM exhausted, or transient system-capacity throttling with no quota change (x-ratelimit-limit-tokens below configured TPM) Honour retry-after-ms; exponential backoff with jitter; rebalance TPM across deployments; distinguish capacity 429s from quota 429s before filing increase requests

Security Notes

  • Keyless by default on Path A. Your Azure key (if any) stays in your BTP destination store or Entra app — never in Launchpad. On the hub, all auth is BTP OAuth2; rotate service keys yourself (x.509 certificate credentials are supported: "credential-type": "x509" in the key JSON).
  • Data-residency caveat: in SAP’s model-availability matrix, Yes* means the model may be served cross-region for load balancing/failover — pin regions and check the matrix before promising EU-only processing.
  • Private networking: Path B to the public Azure OpenAI FQDN needs ordinary internet egress from the CF/Kyma runtime — SAP Cloud Connector is not required (it tunnels to private networks behind it; it does not proxy arbitrary outbound REST). If you disable public access on the Azure resource and use a private endpoint, the caller must sit on peered networking (VNet/internet gateway sidecar or an Azure-hosted proxy) with the resource’s Network setting set to Selected networks or Disabled + private endpoint.
  • PII: run the masking module (with SAP Data Privacy Integration) ahead of the LLM, keep allowlist small, and combine with input/output filtering; never ground on repositories containing unmasked personal data.

FAQ

Do I need an Azure subscription to use GPT-4o from SAP BTP?

No. The Generative AI hub’s azure-openai executable runs on SAP-managed Azure capacity and is billed through your SAP AI Core subscription. You need your own Azure subscription only for the BYO path: your own quota, region, or private data boundary.

What is the difference between the foundation-models and orchestration scenarios?

foundation-models gives raw model access: you create a configuration (executable azure-openai + model name/version) and a deployment, then call its OpenAI-compatible URL. orchestration is SAP’s pipeline service (templating, grounding, masking, filtering, translation, fallback chains) reached through one orchestration deployment that can route to any permitted model. Most production apps should standardise on orchestration.

Why can’t I see the Generative AI hub in my Launchpad?

The hub requires the extended SAP AI Core service plan; it is unavailable in Free and Standard. Update the service instance to extended (data and models are kept), then re-check the foundation-models scenario. Also confirm your resource group — the hub is scoped per resource group, and orchestration deployments must exist in each group you use.

Can I point the orchestrator at my own Azure OpenAI deployment instead of SAP’s?

Not directly — as of this writing there is no first-party Launchpad form to register a customer-owned Azure endpoint with its key. The supported options are: call your Azure resource from the BTP application (Path B above), or deploy a proxy/serving layer (the archived SAP-samples/azure-openai-aicore-cap-api pattern) that exposes your endpoint behind AI Core. Watch for “bring-your-own” hub capabilities in the SAP Road Map, which has signaled BYO-model investments.

References

Next Steps