OpenAI Compatibility

Xinference's inference endpoints accept OpenAI-style request bodies and return OpenAI-style responses. Requests are proxied to your running deployment's Xinference server, so payloads and responses match the OpenAI schema. Because the inference endpoints accept a Bearer API key, the official OpenAI SDKs and compatible frameworks work by pointing the base URL at Xinference and using your API key.

Endpoints

POST /v1/chat/completions
POST /v1/embeddings

The base URL is https://api.xinference.co. Every request must include a model field, which the platform uses to route to your matching running deployment.

Authentication

Send an API key as a Bearer token. API keys (prefix xi-sk-) are created under the organization's API keys and are scoped to specific deployments — see API Keys. The session cookie also works for browser-based calls.

curl https://api.xinference.co/v1/chat/completions \
  -H "Authorization: Bearer xi-sk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Using OpenAI Client Libraries

Point the client's base URL at Xinference and pass your API key. The model must be one of the API key's allowed deployments.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.xinference.co/v1",
    api_key="xi-sk-...",
)

resp = client.chat.completions.create(
    model="qwen2.5-instruct",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

Frameworks that read OPENAI_API_KEY / OPENAI_BASE_URL (e.g. LangChain, LlamaIndex) work the same way — set the base URL to https://api.xinference.co/v1 and the API key to your xi-sk- key.

Supported OpenAI Parameters

The proxy validates model, messages, input, and stream, and forwards any additional OpenAI parameters to the underlying Xinference server. Support for a given parameter ultimately depends on the deployed model:

Known Differences

• API keys are created in the organization's API key settings and are scoped to specific deployments (a browser session cookie also works).
• The model field selects your running deployment and is reflected in responses as the Xinference model name, not an OpenAI model ID.
• Which OpenAI features and parameters work depends on the deployed model and the underlying Xinference server, since the platform forwards requests to it.