Inference API

Once a deployment is running, Xinference exposes OpenAI-compatible inference endpoints. The platform proxies your request to that deployment's Xinference server and returns its response unchanged, so OpenAI-style request and response bodies work as-is.

Base URL

https://api.xinference.co

Inference endpoints are served under the /v1 prefix.

Authentication

The inference endpoints accept either of two credentials:

• API key (recommended for programmatic access) — send Authorization: Bearer xi-sk-…. API keys are created under the organization's API keys (see API Keys) and are scoped to specific deployments.
• Session cookie — the same browser session used by the dashboard.

The rest of the platform (management endpoints) uses the session cookie only; the Bearer API key is accepted on the inference endpoints.

Supported Endpoints

How It Works

Every request must include a model field. The platform uses it to locate your matching running deployment and forwards the request to that deployment's supervisor endpoint. When authenticating with an API key, the model must resolve to one of the key's allowed deployments; if a model name matches more than one allowed deployment, use the deployment id instead. For chat completions, setting "stream": true returns a streamed response.

curl https://api.xinference.co/v1/chat/completions \
  -H "Authorization: Bearer xi-sk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Response Format

Responses are returned exactly as produced by the underlying Xinference server, following the OpenAI response schema (including usage fields):

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1718000000,
  "model": "qwen2.5-instruct",
  "choices": [ ... ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 47,
    "total_tokens": 59
  }
}

Errors

If no running deployment matches the requested model, the proxy responds 404:

{ "detail": "Model is not running." }

Any error produced by the underlying Xinference server is passed back to the caller.

Next Steps

• Chat Completions →
• Embeddings →
• OpenAI Compatibility →