Xinference Docs
Get started

Self-Hosting Xinference

Xinference can be deployed in your own AWS account, giving you control over data residency, network configuration, and billing.

What self-hosting covers

Self-hosting runs the Xinference control plane — the backend, frontend, and database — on infrastructure you own. GPU workers are still provisioned as EC2 instances in your AWS account, so an AWS account remains a prerequisite. Running inference on hardware you already operate is a different deployment model and is not covered by these pages.

Architecture

A self-hosted Xinference deployment consists of:

┌─────────────────────────────────────────┐
│  Your Infrastructure                     │
│                                          │
│  ┌──────────┐    ┌────────────────────┐ │
│  │ Frontend │◄──►│  Backend (FastAPI)  │ │
│  │ (React)  │    │  + PostgreSQL DB    │ │
│  └──────────┘    └────────┬───────────┘ │
│                            │             │
│                   ┌────────▼───────────┐ │
│                   │  AWS EC2 Clusters  │ │
│                   │  (GPU Workers)     │ │
│                   └────────────────────┘ │
└─────────────────────────────────────────┘

Components:

Component Description
Backend Python FastAPI application — handles auth, deployments, billing, and proxies inference
Frontend React SPA — dashboard UI
Database PostgreSQL (or SQLite for development)
AWS EC2 instances for GPU inference workers

Prerequisites

  • AWS account with permissions to launch EC2 instances, create IAM roles, and manage security groups
  • Docker (optional, recommended for production)
  • Python 3.11+ and uv for local development
  • Node.js 18+ for frontend builds

Deployment Options

The quickest way to run the full stack locally or on a single server. See Docker Setup →.

Render / PaaS

The backend can be deployed as a web service on Render, Railway, Fly.io, or similar platforms. The frontend can be deployed to Vercel, Netlify, or as a static site on S3/CloudFront.

Kubernetes

For production-scale deployments, the backend is stateless (except for the database) and can run as a Kubernetes Deployment with horizontal pod autoscaling.

Quick Start

Access required

The xinference-saas repository is private. The clone below fails unless your account has already been granted access, so request access before you start.

git clone https://github.com/TeohCapital/xinference-saas
cd xinference-saas
cp backend/.env.example backend/.env
# Edit .env with your configuration
docker compose up

Next Steps