Koaiu
Sign inGet started

Self-hosted multi-provider AI gateway

One API key.Every AI provider.

Koaiu pools Anthropic API keys, Claude Code OAuth, AWS Bedrock, GCP Vertex, OpenAI keys, and Codex OAuth behind one base URL — with model routing, automatic failover, prompt-cache optimization, and per-tenant metering on every request.

Get started View API reference

MIT licensed · Postgres + Valkey · Anthropic and OpenAI dialects, ingress and egress

gateway.example.com — first request
curl https://gateway.example.com/v1/messages \
  -H "x-api-key: $KOAIU_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-3-5-sonnet",
    "max_tokens": 256,
    "messages": [{"role": "user", "content": "Hello, Koaiu!"}]
  }'
base_urlhttps://gateway.example.com
  • 09upstream account kinds pooled
  • 04funding arbitration modes
  • 01unified Bearer key per caller

01Upstream surfaces

Every provider surface. One pool, one scheduler.

Credentials are sealed with AES-256-GCM at rest; the scheduler is surface-agnostic and per-surface adapters translate auth, URL, and body on the hot path.

  1. First-party API

    Anthropic API

    anthropic_console_api_key

    Console sk-ant-… keys against api.anthropic.com. Static, operator-rotated.

    MessagesToken CountPrompt Caching
  2. OAuth

    Claude Code OAuth

    claude_code_oauth

    Subscription-backed OAuth with prompt-cache mimicry for the cheapest cache buckets.

    MessagesMimicryOAuth Refresh
  3. First-party API

    OpenAI API

    open_ai_api_key

    Static sk-… keys against Responses, Chat Completions, and Embeddings routes.

    ChatResponsesEmbeddingsImages
  4. OAuth

    Codex OAuth

    open_ai_codex_oauth

    ChatGPT OAuth (PKCE) on the Codex backend; short-lived tokens auto-refreshed.

    ResponsesChatPKCE Auto-refresh
  5. OAuth

    Grok (xAI)

    grok_oauth_and_key

    OAuth subscriptions and static API keys for Responses, Images, and Video generation.

    ResponsesChatImagesLive Sync
  6. Enterprise Cloud

    Google Gemini & Vertex

    gemini_api_and_vertex

    AI Studio keys and Google Cloud Vertex AI service accounts with Responses translation.

    ResponsesMessagesVertex SA JSON
  7. Enterprise Cloud

    AWS Bedrock

    anthropic_bedrock

    SigV4-signed requests to bedrock-runtime, region pinned on the credentials.

    MessagesSigV4 AuthCross-Region
  8. Enterprise Cloud

    Azure OpenAI

    openai_azure

    Enterprise Azure OpenAI resources with custom deployment mapping.

    ChatResponsesDeployment Mapping
  9. Compatible

    Preset & Compatible

    openai_compatible

    DeepSeek, Kimi Moonshot, Zhipu GLM, Ollama Cloud, and custom HTTPS gateways.

    ChatMessagesResponsesModel Discovery

02Architecture Philosophy

Three-Way Separation.Zero Credential Leaks.

Koaiu strictly decouples Authorization, Funding, and Execution. Callers authenticate against a Project, get metered against an authorized Billing Account, and execute through an optimal upstream pool without ever knowing the underlying credentials.

01授权域 · 租户治理

Authorization Domain

Multi-tenant workload boundaries, Bearer keys, capability grants, and concurrency limits.

Unified Bearer Keyx-api-key / Bearer

One key unlocks all authorized model families

Project IsolationProjectRequestPolicy

Workload quotas, IP allowlists, member roles

Capability MatrixProjectCapabilities

Chat, Messages, Responses, Embeddings, Images

Zero provider secrets or upstream IDs exposed to tenant callers.
02资金域 · 商业化结算

Funding & Monetization

4-way funding arbitration across Subscriptions, Wallets, Daily/Weekly Caps, and Price Books.

Funding ArbitrationFundingDecision

Subscription-first, Balance-first, or strict mode

Immutable PlansPlanVersion

Versioned subscription allowances and rate limits

Wallet & Budget CapsWalletLedger

Hard stops, soft alerts, and reservation locks

Funding decisions resolve in-memory with sub-millisecond overhead.
03执行域 · 上游容量与调度

Execution & Gateway Router

Multi-provider account pooling, priority targets, proxy health, and prompt caching.

9-Provider Capacity PoolUpstreamAccountPool

Claude OAuth, Codex, Bedrock, Vertex, GLM, etc.

Smart Failover & CooldownAccountScheduler

Automatic 429/5xx backoff and health probing

Cache & Pipeline OptimizationProxyEngine

H2 ping, TLS profiles, prompt cache reuse

Credentials AES-256-GCM sealed; proxy egress isolated per upstream account.
1. Bearer Auth (Project)
2. Funding Decision (Quota / Wallet)
3. Scheduled Pool (AES-Sealed Egress)
4. Realtime Streaming & Settle

03Why a gateway

The pipeline does the work. The interface proves it.

Route & failover

Project capabilities authorize provider families; Account Scheduling Profiles choose healthy upstream capacity and fail over without changing the key or funding decision.

Meter & bill

Every request settles against a subscription window or Billing Account wallet — token-class precision, cache reads and creations split, spend caps and concurrency gates enforced pre-flight.

Observe & audit

Live account status over SSE, request captures with diffs, Prometheus metrics, and an append-only audit event for every admin mutation.

04How it works

From cold deploy to routed traffic in three steps.

No SDK forks, no sidecar agents. The gateway is one binary in front of Postgres and Valkey; everything below happens in its admin surface.

  1. Pool capacity

    Onboard upstream accounts of any of the nine supported kinds. Credentials are sealed with AES-256-GCM; cooldowns, quota probes, and load tracking keep the pool honest.

    9 account kinds · priority + proxy per account

  2. Define Projects

    Bind members, payer Billing Accounts, capabilities, and Account Scheduling policy to Projects, then issue one key that can use every permitted provider family.

    reveal-once · multi-provider · audit-logged

  3. Route traffic

    Point any Anthropic or OpenAI SDK at the gateway. Routing, failover, cache optimization, and metering happen per request — no client changes beyond the base URL.

    base_url = https://gateway.example.com

05Plans & Monetization

Your operator sets the pricing.

Koaiu publishes no price list. The operator defines immutable plan versions, Project price books, and wallet controls. Exact terms and live usage appear in your portal after sign-in.

subscription

Plan-backed funding with versioned capability entitlements.

Quota windows
5h / 24h / 7d
Capabilities
plan entitlements
Selection
automatic priority

Pricing — set by your operator

wallet

Prepaid USD funding with explicit payer authorization.

Wallet
Billing Account
Spend caps
daily / weekly / monthly
Rates
Project price book

Pricing — set by your operator

fallback

Eligible subscriptions first, then an optional wallet.

Primary
ranked subscriptions
Fallback
payer-approved wallet
Decision
one settled source

Pricing — set by your operator

Sign in to view billing

Start routing

Route your first request in minutes.

Postgres, Valkey, and the Koaiu binary. Auth, billing, scheduling, observability, and the audit log ship in the box — no SaaS lock-in to undo later.

Get started View API reference