A self-hosted, OpenAI-compatible AI gateway for private RAG, natural-language data access, and tool-calling agents — run it in your own environment across 37+ model providers.
Quick Start • Demos • Features • Tutorial • Docs
ORBIT is a self-hosted AI gateway that lets you chat with private data through one OpenAI-compatible endpoint.
Put it in front of local or cloud models, connect files, SQL, NoSQL, vector stores, APIs, and tools, then operate everything from an admin panel with keys, quotas, prompts, metrics, and audit logs.
Reach for ORBIT when you need to:
- 💬 Query SQL, NoSQL, vector stores, REST/GraphQL APIs, Elasticsearch, and files in plain English
- 🔀 Route one API contract across local (Ollama, llama.cpp, vLLM) and cloud models
- 🛠️ Give models scoped access to MCP tools and multi-step agent workflows
- 🔒 Ship private RAG with auth, quotas, moderation, metrics, and admin controls
- 📨 Drive inference asynchronously over a message queue for decoupled, batch-style workloads
- 🧪 Prototype locally, then keep the same architecture in production
Comparisons: ORBIT vs. Open WebUI · ORBIT vs. LiteLLM
multimodal.mp4
Upload private files, ask questions in chat, and keep the whole workflow behind your own gateway.
Install the latest stable release, then start ORBIT locally.
curl -LO https://github.com/schmitech/orbit/releases/download/v2.9.2/orbit-2.9.2.tar.gz
tar -xzf orbit-2.9.2.tar.gz && cd orbit-2.9.2
./install/setup.sh # add --wizard for interactive setup
./bin/orbit.sh startOpen the admin panel:
| URL | Login |
|---|---|
| http://localhost:3000/admin | admin / admin123 |
Verify the gateway:
curl -X POST http://localhost:3000/v1/chat \
-H 'Content-Type: application/json' \
-H 'X-API-Key: default-key' \
-H 'X-Session-ID: local-test' \
-d '{"messages": [{"role": "user", "content": "Summarize ORBIT in one sentence."}], "stream": false}'Install orbitchat:
npm install -g orbitchat@latestOpen a browser chat client:
ORBIT_ADAPTER_KEYS='{"simple-chat":"default-key"}' npx orbitchat --openWhat you should see: the server responds through the OpenAI-compatible /v1/chat endpoint, the admin panel shows live health and adapters, and OrbitChat opens a local chat UI.
For a Docker-based local demo with Ollama, use the development compose stack:
git clone https://github.com/schmitech/orbit.git
cd orbit/docker
docker compose up -dThe first Docker run can take a few minutes while the local model is downloaded.
Warning
Docker from main is for local evaluation and development. For production installs, use the latest stable release tarball from GitHub Releases.
More setup paths: Docker Guide · Tutorial · Windows native install
If ORBIT saves you setup time, a GitHub star helps other developers find it.
Pick the path closest to what you want to build.
| Demo | What it shows | Run it |
|---|---|---|
| Chat with private files | Upload PDFs, spreadsheets, and images, then query them in one thread. | Tutorial |
| Ask SQL questions in English | Generate and run safe parameterized queries over a sample HR database. | Tutorial |
| Operate the gateway | Create personas and API keys, inspect adapters, watch metrics, and audit changes from /admin. |
First chat |
Chat with private files
multimodal.mp4
Analysts stop hunting across files: upload PDFs, spreadsheets, and images, then query them together in one thread with context cached across the conversation.
Ask SQL questions in English
db-query.mp4
No SQL and no ticket queue: ask in plain English, and ORBIT generates the query, runs it against your database, and charts the result in chat.
Operate the gateway from the admin panel
orbit-admin.mp4
Monitor health, latency, tokens, sessions, adapters, and logs from one dashboard behind API keys, quotas, and rate limits.
Every request flows through the ORBIT gateway, gets authenticated and routed by an adapter, and lands on the right provider — REST, OpenAI-compatible, MCP, A2A, or message queue in; inference, RAG, structured data, tools, or generation out. |
| Capability | What you get |
|---|---|
| OpenAI-compatible gateway | One /v1/chat interface across local, self-hosted, and cloud providers. |
| Model routing (37+ providers) | Local: Ollama, llama.cpp, vLLM, TensorRT-LLM, Transformers, LM Studio, BitNet. Cloud: OpenAI, Anthropic, Gemini, Bedrock, Vertex, Azure, Groq, Mistral, DeepSeek, xAI, and many more. |
| Natural-language data access | Plain English → SQL, MongoDB, Elasticsearch DSL, REST, GraphQL across Postgres, MySQL, Oracle, SQL Server, DuckDB, Athena, Supabase, Cassandra, Redis, and composite multi-source answers. |
| Vector RAG | Chroma, Qdrant, Pinecone, Milvus, Weaviate, Marqo, pgvector, FAISS, DuckDB, Redis, Elasticsearch. |
| File & multimodal RAG | PDFs, DOCX, spreadsheets, CSVs, markdown, images, and audio — with cached file context across conversations. |
| Pluggable file storage | Store uploads and generated media on local disk (default) or in the cloud — AWS S3, MinIO / SeaweedFS (S3-compatible), Azure Blob, or Google Cloud Storage — selected with one config key. |
| File encryption at rest | AES-256-GCM encryption for uploaded files, opt-in per adapter — keep classified or regulated documents encrypted on disk or in the cloud without touching adapters that don't need it. |
| Cloud secrets management | Resolve API keys, DB passwords, and connection strings from AWS Secrets Manager, Azure Key Vault, or GCP Secret Manager instead of (or alongside) .env — selected with one config key. Guide. |
| Web search | Provider-agnostic real-time context via DuckDuckGo (free), Brave, SearXNG, Serper, Tavily, Google PSE, Perplexity — decoupled from synthesis so any LLM can answer. |
| MCP tool agents | Connect MCP servers (filesystem, GitHub, Slack, Postgres, Jira, Notion, and more) over stdio/SSE with bounded, server-side agent loops. |
| A2A peer protocol | Google Agent-to-Agent support — discovery via /.well-known/agent.json and task delegation over JSON-RPC. Guide. |
| Message-queue (async) surface | Broker-native RabbitMQ ingestion — publish requests to a queue, ORBIT consumes them through the same pipeline and replies on a results queue. Decoupled, at-least-once, batch/async processing instead of synchronous HTTP. Guide. |
| Media generation | Image, video, audio (TTS), PDF, Word, Excel, PowerPoint, CSV, and markdown generation adapters in the same chat flow. |
| Voice (STT/TTS) | OpenAI, Whisper, Google, Gemini, ElevenLabs, Coqui, and more. |
| Production controls | API keys, RBAC, SSO via Entra ID & Auth0 (OIDC), per-key routing, rate limits, token quotas, moderation, circuit breakers, fallback routing, metrics, audit logs, and hot adapter reloads. |
| Config-first | Define adapters, providers, datasources, prompts, and guardrails in YAML — no application code. |
📚 Deep dive: Docs index · Adapter guide · Configuration
Same gateway, different jobs. ORBIT is useful anywhere a team needs private data access, controlled model routing, and operational visibility behind one API.
| Business outcome | Who feels the pain | What it replaces |
|---|---|---|
| Answers from scattered documents | Insurance, legal, finance, research | Hours spent manually cross-referencing PDFs, spreadsheets, and scans |
| Self-service data access | Retail, operations, finance, SaaS | BI ticket backlogs and waiting on the data team to write SQL |
| Faster incident response | SRE, DevOps, security ops | Bottlenecks where only a few specialists can write query DSL |
| Automated internal workflows | IT ops, support, back-office | Manual glue work across databases, Slack, files, and tickets |
| Conversational tool use, zero orchestration code | Product, support, SaaS builders | Bolt-on agent frameworks (LangChain, AutoGen) just to get tool calling |
| AI on regulated data | Healthcare, government, legal, banking | Compliance blocks on sending sensitive data to cloud LLMs |
| Governed AI rollout | Any regulated or enterprise org | Shadow AI with no audit trail, cost control, or visibility |
| In-flow content generation | Marketing, e-learning, communications | Slow, fragmented media-production tooling |
Cut incident response time by searching logs in plain English
es-logs.mp4
Investigations no longer stall on DSL expertise: ask operational questions naturally, and ORBIT compiles the Elasticsearch Query DSL and returns the answer.
Automate multi-step workflows across your internal tools
mcp-tool-demo.mp4
Replace manual glue work: give models scoped, server-side access to MCP tools — filesystem, Slack, Postgres, GitHub, Jira — with bounded agent loops.
Let the model call tools on its own, like ChatGPT or Claude
Beyond explicit, client-requested tool skills, any conversational adapter can
opt into opportunistic MCP tool calling: the model decides,
turn by turn, whether an external tool is needed and calls it inline — no
skill field, no adapter swap, same thread throughout. Multi-step
chains, self-correction from tool errors, and mixing several tools in one turn
all work out of the box, powered by providers' native function calling with
no LangChain, AutoGen, or CrewAI dependency — just a small,
bounded, fully-owned server-side loop. See the
opportunistic mode guide.
Keep regulated data in-house by running AI fully offline
sensitive-data.mp4
Meet data-residency and compliance rules: run local llama.cpp/Ollama models so sensitive PII never leaves your environment.
Generate on-brand media inside the same chat flow
puppy.mp4
Produce content without leaving the conversation: generate images and video as cross-adapter skills that carry conversation context.
More capabilities
svg-rendering.mp4
Render dynamic LLM-generated SVGs inline.
second-opinion.mp4
Switch inference models mid-conversation without breaking chat history.
business-analytics-demo.mp4
Sub-conversation threading and document caching for faster retrieval.
| Client | Description |
|---|---|
| ORBIT Chat | React web chat client. Run it: ORBIT_ADAPTER_KEYS='{"simple-chat":"default-key"}' npx orbitchat |
| Node.js SDK | Node library for integrating ORBIT into apps. |
| Python client | CLI and API examples for key management and chat. |
| Topic | Start here |
|---|---|
| Getting started | Tutorial · First chat |
| Configuration | Configuration guide · Adapter config |
| Adapters & RAG | Adapters overview · File adapter |
| NL data access | SQL retriever architecture · Intent SQL RAG |
| MCP agents | MCP agent guide |
| Production ops | Rate limiting · Fault tolerance |
| Auth & SSO | Authentication guide — built-in users, Entra ID & Auth0 |
Contributions are welcome — new retrievers, adapters, and provider integrations; better examples and deployment guides; tests, bug fixes, and docs. Start with CONTRIBUTING.md, open an issue, or send a PR. Roadmap and active work live in GitHub Issues.
Maintained by Remsy Schmilinsky.
ORBIT is licensed under the Apache 2.0 License. See LICENSE for details.