Query route-optimized serving endpoints

This article describes how to fetch the appropriate authentication credentials and URL so you can query your route-optimized model serving or feature serving endpoint.

Requirements

• A model serving endpoint or feature serving endpoint that has route optimization enabled. See Route optimization on serving endpoints.
• Querying route-optimized endpoints only support using OAuth tokens. Personal access tokens are not supported.

Quick start: end-to-end query recipe

The following recipe combines every step needed to query a route-optimized endpoint from an external client into a single runnable flow. Use this section if you want to verify a working setup quickly. See the following sections for more details on each step.

# 1. Set the variables for your environment.
export DATABRICKS_HOST="https://<your-workspace>.cloud.databricks.com"
export ENDPOINT_NAME="<your-endpoint>"
export WORKSPACE_ID="<workspace-id>"

# 2. Create an account-level service principal and an OAuth secret for it.
SP_ID=$(databricks account service-principals create \
  --json '{"displayName":"my-app","active":true}' --output json | jq -r '.id')
SECRET_JSON=$(databricks account service-principal-secrets create "$SP_ID" --output json)
export CLIENT_ID=$(databricks account service-principals get "$SP_ID" --output json | jq -r '.applicationId')
export CLIENT_SECRET=$(echo "$SECRET_JSON" | jq -r '.secret')

# 3. Assign the service principal to the workspace and grant CAN_QUERY on the endpoint.
databricks account workspace-assignment update "$WORKSPACE_ID" "$SP_ID" \
  --json '{"permissions":["USER"]}'
ENDPOINT_ID=$(databricks serving-endpoints get "$ENDPOINT_NAME" --output json | jq -r '.id')
databricks permissions update serving-endpoints "$ENDPOINT_ID" \
  --json "{\"access_control_list\":[{\"service_principal_name\":\"$CLIENT_ID\",\"permission_level\":\"CAN_QUERY\"}]}"

# 4. Mint an endpoint-scoped OAuth token. `authorization_details` is required for
# route-optimized endpoints -- a plain `scope=all-apis` token is rejected with
# 401 "Missing authorization details" when used against the route-optimized URL.
TOKEN=$(curl -sS -X POST -u "$CLIENT_ID:$CLIENT_SECRET" \
  --data-urlencode "grant_type=client_credentials" \
  --data-urlencode "scope=all-apis" \
  --data-urlencode "authorization_details=[{\"type\":\"workspace_permission\",\"object_type\":\"serving-endpoints\",\"object_path\":\"/serving-endpoints/$ENDPOINT_ID\",\"actions\":[\"query_inference_endpoint\"]}]" \
  "$DATABRICKS_HOST/oidc/v1/token" | jq -r '.access_token')

# 5. Invoke the endpoint at its route-optimized URL.
RO_URL=$(databricks serving-endpoints get "$ENDPOINT_NAME" --output json | jq -r '.endpoint_url')
curl -sS -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"inputs":[[0.12,0.34]]}' "https://$RO_URL"

Fetch the route-optimized URL

When you create a route-optimized endpoint, the following route-optimized URL is created for the endpoint.

https://<unique-id>.<shard>.serving.azuredatabricks.net/<workspace-id>/serving-endpoints/<endpoint-name>/invocations

You can get this URL from the following:

Serving UI

REST API

Use the GET /api/2.0/serving-endpoints/{name} API call. The URL is present in the response object of the endpoint as endpoint_url. This field is only populated if the endpoint is route-optimized.

GET /api/2.0/serving-endpoints/my-endpoint
{
  "name": "my-endpoint"
}

Databricks SDK

Use the Serving Endpoints API get call. The URL is present in the response object of the endpoint as endpoint_url. This field is only populated if the endpoint is route-optimized.

from databricks.sdk import WorkspaceClient

workspace = WorkspaceClient()

workspace.serving_endpoints.get("my-endpoint")

Fetch an OAuth token and query the endpoint

To query your route-optimized endpoint you must use an OAuth token. Databricks recommends using service principals in your production applications to fetch OAuth tokens programmatically. The following sections describes recommended guidance on how to fetch an OAuth token for test and production scenarios.

Fetch an OAuth token using the Serving UI

The following steps show how to fetch a token in the Serving UI. These steps are recommended for development and testing your endpoint.

For production use, like using your route-optimized endpoint in an application, your token is fetched using a service principal. See Fetch an OAuth token programmatically for recommended guidance for fetching your OAuth token for production use cases.

From the Serving UI of your workspace:

1. On the Serving endpoints page, select your route-optimized endpoint to see endpoint details.
2. On the endpoint details page, select the Use button.
3. Select the Fetch Token tab.
4. Select Fetch OAuth Token button. This token is valid for 1 hour. Fetch a new token if your current token expires.

After you fetch the OAuth token, query your endpoint using your endpoint URL and OAuth token.

REST API

The following is a REST API example:

URL="<endpoint-url>"
OAUTH_TOKEN="<token>"

curl -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OAUTH_TOKEN" \
  --data "@data.json" \
  "$URL"

Python

The following is a Python example:

import requests
import json

url = "<url>"
oauth_token = "<token>"

data = {
    "dataframe_split": {
        "columns": ["feature_1", "feature_2"],
        "data": [
            [0.12, 0.34],
            [0.56, 0.78],
            [0.90, 0.11]
        ]
    }
}

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {oauth_token}"
}

response = requests.post(url, headers=headers, json=data)

# Print the response
print("Status Code:", response.status_code)
print("Response Body:", response.text)

Fetch an OAuth token programmatically

For production scenarios, Databricks recommends setting up service principals to embed within your application to programmatically fetch OAuth tokens. These fetched tokens are used to query route-optimized endpoints.

Follow the steps in Authorize service principal access to Azure Databricks with OAuth through step 2 to create your service principal, assign permissions and create an OAuth secret for your service principal. After your service principal is created, you must give the service principal at least Query permission on the endpoint. See Manage permissions on a model serving endpoint.

The Databricks Python SDK provides an API to directly query a route-optimized endpoint.

The next example requires the following to query a route-optimized endpoint using the Databricks SDK:

• Serving endpoint name (the SDK fetches the correct endpoint URL based on this name)
• Service principal client ID
• Service principal secret
• Workspace hostname

from databricks.sdk import WorkspaceClient
import databricks.sdk.core as client

endpoint_name = "<Serving-Endpoint-Name>" ## Insert the endpoint name here

# Initialize Databricks SDK
c = client.Config(
    host="<Workspace-Host>", ## For example, my-workspace.cloud.databricks.com
    client_id="<Client-Id>", ## Service principal ID
    client_secret="<Secret>"   ## Service principal secret
)
w = WorkspaceClient(
    config = c
)

response = w.serving_endpoints_data_plane.query(endpoint_name, dataframe_records = ....)

Fetch an OAuth token manually

For scenarios where the Databricks SDK or the Serving UI can not be used to fetch your OAuth token, you can manually fetch an OAuth token. The guidance in this section mainly applies to scenarios where users have a customized client that they want to use for querying the endpoint in production.

When you fetch an OAuth token manually, you must specify authorization_details in the request.

• Construct the <token-endpoint-URL> by replacing https://<databricks-instance> with the workspace URL of your Databricks deployment in https://<databricks-instance>/oidc/v1/token. For example, https://my-workspace.0.azuredatabricks.net/oidc/v1/token

• Replace <client-id> with the service principal's client ID, which is also known as an application ID.
• Replace <client-secret> with the service principal's OAuth secret that you created.

• Replace <endpoint-id> with the endpoint ID of the route-optimized endpoint. This is the alpha-numeric ID of the endpoint that you can find in the hostName of the endpoint URL. For example, if the serving endpoint is https://abcdefg.0.serving.azuredatabricks.net/9999999/serving-endpoints/test, the endpoint ID is abcdefg.

• Replace <action> with the action permission given to the service principal. The action can be query_inference_endpoint or manage_inference_endpoint.

REST API

The following is a REST API example:

export CLIENT_ID=<client-id>
export CLIENT_SECRET=<client-secret>
export ENDPOINT_ID=<endpoint-id>
export ACTION=<action>  # for example, 'query_inference_endpoint'

curl --request POST \
--url <token-endpoint-URL> \
--user "$CLIENT_ID:$CLIENT_SECRET" \
--data 'grant_type=client_credentials&scope=all-apis'
--data-urlencode 'authorization_details=[{"type":"workspace_permission","object_type":"serving-endpoints","object_path":"'"/serving-endpoints/$ENDPOINT_ID"'","actions": ["'"$ACTION"'"]}]'

Python

The following is a Python example:

import os
import requests

# Set your environment variables or replace them directly here
CLIENT_ID = os.getenv("CLIENT_ID")
CLIENT_SECRET = os.getenv("CLIENT_SECRET")
ENDPOINT_ID = os.getenv("ENDPOINT_ID")
ACTION = "query_inference_endpoint" # Can also be `manage_inference_endpoint`

# Token endpoint URL
TOKEN_URL = "<token-endpoint-URL>"

# Build the payload, note the creation of authorization_details
payload = { 'grant_type': 'client_credentials', 'scope': 'all-apis', 'authorization_details': f'''[{{"type":"workspace_permission","object_type":"serving-endpoints","object_path":"/serving-endpoints/{ENDPOINT_ID}","actions":["{ACTION}"]}}]''' }

# Make the POST request with basic auth
response = requests.post( TOKEN_URL, auth=(CLIENT_ID, CLIENT_SECRET), data=payload )

# Check the response
if response.ok:
  token_response = response.json()
  access_token = token_response.get("access_token")
  if access_token:
    print(f"Access Token: {access_token}")
  else:
    print("access_token not found in response.")
else: print(f"Failed to fetch token: {response.status_code} {response.text}")

After you fetch the OAuth token, query your endpoint using your endpoint URL and OAuth token.

REST API

The following is a REST API example:

URL="<endpoint-url>"
OAUTH_TOKEN="<token>"

curl -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OAUTH_TOKEN" \
  --data "@data.json" \
  "$URL"

Python

The following is a Python example:

import requests
import json

url = "<url>"
oauth_token = "<token>"

data = {
    "dataframe_split": {
        "columns": ["feature_1", "feature_2"],
        "data": [
            [0.12, 0.34],
            [0.56, 0.78],
            [0.90, 0.11]
        ]
    }
}

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {oauth_token}"
}

response = requests.post(url, headers=headers, json=data)

# Print the response
print("Status Code:", response.status_code)
print("Response Body:", response.text)

Calling from an AI agent or external application

AI coding assistants and external applications that query route-optimized endpoints cannot use a developer's personal access token or workspace OAuth tokens. They must use a service principal with the OAuth M2M flow and include authorization_details in the token request. The flow is:

1. Create a service principal at the account level and an OAuth secret for it. See Authorize service principal access to Azure Databricks with OAuth.
2. Assign the service principal to the workspace and grant it CAN_QUERY on the endpoint.
3. From the application, mint an endpoint-scoped token by calling POST <workspace-host>/oidc/v1/token with the service principal credentials and authorization_details referencing the endpoint ID. See Fetch an OAuth token manually.
4. Invoke the route-optimized URL with the resulting token.

The Quick start section above contains a single copy-pasteable script that covers every step.

Troubleshooting