AI gateway
Every model behind one OpenAI-compatible endpoint.
Point your SDK at the gateway and keep the client you already have. An alias decides which model serves the call, a 429 moves it to the next provider, and the response tells you which one answered.
Start free trial
14 days, no card
ai pipeline
your app
Penstoke gateway
- OpenAI
- Anthropic
- Gemini
- Groq
- OpenRouter
The first one that answers wins. A 429, a 5xx or a timeout moves the same request to the next provider in the order, and the response says which one served it.
A request leaves your app once, and is accounted for by the time it returns.
The gateway resolves the alias, picks the provider, forwards the call with your key, and writes the model, the tokens, the latency and the cost to the log before the response closes.
See also the storage gateway and the push gateway, which sit behind the same key and the same audit trail, or what a plan includes.
What the AI gateway does
- One endpoint, your SDK
- The gateway speaks the OpenAI API, so an existing client changes its base URL and nothing else. Streaming passes through untouched, token by token.
- Aliases, not model ids
- Your app asks for “fast” or “writing”. The alias names a primary model and an ordered list of fallbacks, so a model can be swapped without redeploying anything.
- Failover on 429 and 5xx
- A rate limit, an outage or a timeout moves the same request down the order instead of failing it. The response says which provider actually served it.
- Keys scoped per endpoint
- Each endpoint has its own key, its own allowed models and its own monthly budget, so one app can never spend another's.
- Budgets that stop spend
- A cap is checked before the request leaves the building, not reconciled on an invoice at the end of the month.
- Every call accounted for
- Model, provider, tokens, latency and cost are recorded per request, per endpoint and per day — including which fallback was used and why.
Change one line
import OpenAI from "openai";
const ai = new OpenAI({
baseURL: `${process.env.PENSTOKE_URL}/v1`,
apiKey: process.env.PENSTOKE_KEY,
});
// "fast" is an alias you define: a primary model, then the fallbacks, in order.
const reply = await ai.chat.completions.create({
model: "fast",
messages: [{ role: "user", content: question }],
stream: true,
});Keys travel as Authorization: Bearer. The same host also serves /storage/v1 and /push/v1.
AI gateway questions
- What is an AI gateway?
- An AI gateway is a single endpoint your applications call instead of calling model providers directly. It holds the provider credentials, decides which model serves each request, retries elsewhere when a provider fails, and records what every call cost. Penstoke's AI gateway is OpenAI-compatible, so any SDK that can talk to OpenAI can talk to it.
- Which AI providers does Penstoke support?
- OpenAI, Anthropic, Google Gemini, OpenRouter, Groq, Ollama, and any OpenAI-compatible API — which includes self-hosted and private deployments. Through OpenRouter a single alias also reaches GPT, Claude, Gemini, Llama, Mistral, DeepSeek, Qwen, Kimi and Grok on one account.
- Does it support streaming responses?
- Yes. Server-sent events pass straight through the gateway unbuffered, so the first token reaches your app as soon as the provider emits it. Failover happens before the stream opens, so a fallback never arrives mid-sentence.
- Does Penstoke charge per token or mark up model calls?
- No. You bring your own provider accounts and keys, and those providers bill you at their own rates. The subscription caps what you keep configured — providers, endpoints, keys, budgets — not the volume of traffic. Nothing passing through the gateway is metered or marked up.
- How is this different from calling OpenAI directly?
- Calling a provider directly means one account's rate limits and one provider's outages are your application's. Behind the gateway, a 429 becomes a retry on another provider, keys live on the server instead of in every app, spend is capped before it happens, and every call is logged in one place with its cost.
- Can I use my own domain for the AI endpoint?
- Yes. Every account is served at <slug>.penstoke.com from signup, and a domain you own can be pointed at the gateway as well. Both serve the API, so your SDK snippets can read https://ai.yourcompany.com/v1.