Which key pays for gpt-4o? You probably can't answer that

A script on your laptop asks for gpt-4o. The request succeeds. Now answer a simple question: which account just paid for it?

For most setups the honest answer is that you'd have to go and look. There's an OpenAI key in a shell profile, an Anthropic key in a .env file, an Ollama running on a port you set months ago, and at work a Bedrock role that only resolves on the VPN. Every tool that wants a model wants its own copy of all of it - its own credential, its own base URL, its own idea of what to do when a request times out. Add an MCP server or two and the sprawl doubles, because now there are commands, arguments and environment variables to keep straight as well.

That sprawl works. What it can't do is answer a question. You can't tell which endpoints are live, which models are actually reachable, or which key pays when a tool asks for gpt-4o. The information exists, spread across a dozen files written at different times, and checking means opening each one.

Agent Gateway is a Kunobi extension that puts all of it in one place and then answers those questions on screen. It runs the agentgateway data plane in-process: however many proxies you like, each on its own loopback port, each speaking the OpenAI wire protocol. You declare your providers and your MCP servers once, front them with a local endpoint, and get client authentication, rate limits, retries and weighted model routing on top. Secrets go into the OS keychain and never appear in the config file or the synced stores.

That's the whole control plane - sixteen records across seven types in one table. But the table is the least interesting view of it, because it can't show the shape those records take at request time:

Clients hold a virtual key and talk to a loopback port. The provider and target secrets are read from the keychain on the way out, so the client side of that boundary never holds them.

Provider order is a billing decision

A provider record is a name, a kind, an optional list of models it serves, and a secret that lives in the keychain. There are eight kinds: openai, anthropic, compatible, copilot, gemini, bedrock, vertex and azure. Most are what they sound like. compatible covers everything else - it's how Ollama, vLLM, Groq and OpenRouter get in, and it's the only kind that requires a base URL, because there's no default host to fall back on.

Then you point a proxy at several of them, in an order. That order is load-bearing, and it's the part worth stopping on.

The gateway builds one flat routing table, provider by provider and model by model, in exactly the order you configured them. An incoming model name resolves to the first entry that matches - not the most specific one. Reordering two providers therefore changes which account pays for gpt-4o, with no edit to either provider record, and nothing in either record to suggest it happened.

Three rules stack on top of that walk:

• An empty model list means wildcard, not "no models". A provider carrying no allow-list catches any model name a client sends. Those catch-all routes are appended after every explicitly named model, so a name another provider lists by hand still wins it.
• A provider's name, not its id, is its routing key upstream. Two providers on the same proxy sharing a name is a resolution failure that stops the proxy from starting.
• The matcher takes four forms and only four: a bare *, a trailing prefix*, a leading *suffix, and everything else compared literally. So foo*bar isn't a glob the router recognizes. It's compared literally, which makes it a real model name a client can send.

No single record can tell you the outcome of all that. So the Connect tab rebuilds the gateway's own routing table, ranks it the way the gateway ranks it, walks it with the same first-match rule, and labels every result.

Shadowed is the label that matters, and it has three causes the tab tells apart. An earlier provider listing the same literal is the ordinary case - Azure — East US above. An earlier pattern can also swallow a later literal, because first-match doesn't prefer the more specific entry. And a virtual model intercepts a provider literal of the same name no matter who declared it, since the router matches virtual models before any provider route.

The other six labels: Model for a literal routed to a named provider, Pattern for an entry like Qwen/* that matches model names but isn't itself a name you can send, Pattern (shadowed) for one an earlier route swallows whole, Catch-all for a provider carrying no allow-list, Alias for a virtual model, and Alias (unvalidated) where one of that alias's targets is served by nobody.

Below it, a curl command and an OpenAI Python SDK snippet, generated against the live endpoint with a model name the proxy will really accept. They're labeled templates because the bearer is left as a placeholder: the key value stays in the keychain and there's no read path to it.

The alias that fails twenty percent of the time

A virtual model maps a client-facing name onto one or more provider models, scoped to a single LLM proxy. One target makes it a plain alias, useful when you want chat-prod in your code instead of a model string you'll want to change next quarter. Several model:weight entries make it a per-request weighted split, which is how you A/B test two models behind one name.

Here's the failure mode nobody plans for. The backend picks a weighted target first, then resolves that concrete model. So one unserved target in gpt-4o:80, o3-mini:20 produces a 404 on roughly twenty percent of requests and success on the rest.

That's the kind of failure that runs for weeks. It doesn't look like an outage, it looks like flakiness. Every target has to be checked, and that check lives on the proxy's Connect tab rather than on the virtual model - chat-next is the unvalidated alias in the model surface above, with o3-mini named as the leg nothing serves.

An alias can also take a name a provider already serves. Virtual models are matched ahead of every provider route, so the alias wins the name outright and requests never reach the provider that declared it. The model surface labels that name as intercepted, and says which provider it displaced.

A rate limit that isn't one

A virtual key is the credential your clients present to the proxy. It's deliberately not the provider key, so a local script never holds a credential that can spend money directly against OpenAI. On save the value goes to the keychain, and there's no read path back out.

That split is also how the AI-facing half of the extension avoids handling raw secrets. An agent can create a provider or a virtual key, but the tools that take raw secret values aren't exposed to it. It creates the record with the secret marked pending, and the amber "Needs secret" badge - on agent-fleet in the inventory shot - stays until a human supplies the value.

A rate limit counts either requests or tokens over an interval in seconds. It's a fixed-capacity refill bucket, not a continuous rate, which is why the interface only says "per minute" when the window is genuinely sixty seconds, and spells out the real numbers otherwise.

And it carries a footgun the extension warns about rather than quietly repairing: a zero amount or a zero interval does not mean unlimited-by-policy. The policy fails to build and the proxy runs with no limit at all. Nothing but that warning tells you the limit you wrote isn't in force.

Why won't it start

Connect opens with Readiness, because the most valuable thing this screen can tell you is why the proxy won't start.

Blockers stop it dead: a reference to a record someone deleted, a provider or key still waiting on its secret, two referenced providers sharing a name. Warnings let it start but you should know about them - the config drifted since it started and needs a restart to apply, there's no gateway authentication at all, that rate limit with the zero interval, a port set to 0.

Every one of those is silent everywhere else in the app. A field dump can't report that provider 3f2a… was deleted a week ago; it can only show you the id. Readiness names the affected record and what to do about it.

Below it, Endpoint composes the base URL from the port the proxy is actually listening on and states the path (POST /v1/chat/completions). Authentication resolves the virtual keys by name and says plainly that the key is sent as an Authorization: Bearer header, and that its value lives only in the keychain. Routing resolves the UUIDs: providers by name and kind, in configured order, so the precedence rule above is visible on screen instead of implied. Reliability collapses the retry and timeout fields into one line - "3 total attempts (2 retries), 200 ms backoff, 30 s timeout" - since retries only mean anything above one attempt.

Cloud providers are the one place that resolution stops short, and it starts at the add dialog.

The Kind field drives the rest of the form, and the cloud kinds take no API key at all. Bedrock signs requests with SigV4 against the machine's ambient AWS credentials or an access key you supply, Vertex uses Application Default Credentials or a pasted service-account JSON, Azure uses DefaultAzureCredential or a tenant and client id pair. All three sit behind the same record shape, and the form shows only the fields the chosen kind needs.

Once saved, though, those fields have no read-only view of their own. To see which region a Bedrock provider points at, you open the edit form.

MCP: the same job, for tools

The other half of the extension does all of this for MCP servers. An MCP target is one upstream server, either stdio with a command, arguments and environment variables, or remote over streamable HTTP with bearer, custom-header, query-parameter or environment auth. Plain SSE still works, since the older servers are still out there. An MCP proxy fronts any number of targets behind one endpoint, with the same virtual keys and rate limits as its LLM counterpart.

You never call an MCP endpoint by hand, so its Connect tab answers a different question: what will a client see, and how do I point one at it?

Tools shows the naming scheme. When two targets both export search, one of the names has to change, so tools carry their target's name as a prefix - github_read_file above. It shows the scheme and not the names themselves, because exact names need a live session: a client has to connect and call tools/list, and the upstream can change them at any time.

Which is what makes tool access worth setting. Fronting several MCP servers behind one endpoint means every tool they export becomes callable by anything holding the key. The Tool Access tab starts from a default - expose everything, or hide everything - and you list the exceptions, scoped to the target that exports them, with a deny always beating an allow. The names come from the servers themselves: the tab opens a one-shot MCP session per target, calls tools/list, and renders the reply as a checklist. A target that fails to answer is reported, and Save stays disabled while any target is unanswered, so a policy written from a partial picture can't be saved as though it were complete. Editing needs the proxy stopped, since a running data plane rejects a rule change.

Sessions spells out what the session options do. A stateful proxy persists sessions and drops idle ones after a TTL; a stateless one opens a new upstream session per request, doesn't support GET or DELETE, and respawns stdio targets every time. Below it the failure mode: fail closed means one target that won't start fails the session, fail open means it's skipped and the rest still serve.

Client configuration generates the actual config block for Claude Code, Claude Desktop, Cursor, VS Code or Codex, because each expects a different shape in a different file. Where a client has a CLI that can register itself, the tab leads with that - claude mcp add, code --add-mcp, codex mcp add, each complete with this proxy's URL - and demotes the file to "Or edit …" underneath.

Two carry warnings worth having before you debug them yourself. Claude Desktop speaks stdio only, so its block bridges through mcp-remote, and adding a url field makes Desktop discard the whole section silently. Claude Code's .mcp.json has no documented field for a header, so on a key-protected proxy the JSON block alone would 401 - the tab says so and points at the command form, the only one that can carry the key.

An MCP proxy can also skip client files entirely. Publish over Kunobi MCP, a checkbox on the record and on by default, exposes that proxy's tools through Kunobi's own MCP hub, filtered by the tool-access rules above. It's available only while the proxy has no virtual key, because the hub reaches the proxy on your behalf and has no key to present. Attaching a key later turns publication off, so the setting can't go stale against the record it describes.

The graph

The Flow view draws the same records as nodes and the references between them as edges: a proxy routes to a provider, proxies to an MCP target, requires a virtual key, enforces a rate limit, serves a virtual model. Above is gateway-prod alone - three providers, one virtual key, one rate limit, two virtual models.

The edges answer what the table can't. A proxy with no provider edge routes nothing, and that's visible without opening the record and resolving its ids. Precedence isn't drawn, so the ordering question still belongs to Connect's Routing list.

What it doesn't do yet

Three gaps, all pointing the same way. Cloud auth internals are form-only, as above. The MCP Connect tab's Tools section shows the naming scheme rather than the live tool names, because discovery runs in the Tool Access editor and Connect doesn't open a session of its own. And the model surface is computed entirely from configuration - there's no request log in the extension yet, so gpt-4o resolving to your production OpenAI provider describes the config and says nothing about how many requests actually took that route.

The keys were never the problem

Every setup has keys, and every setup has too many of them. The problem is that a pile of .env files can't be asked a question. Not which endpoint is live, not which tools a client can reach, and not which account pays for gpt-4o.

Order decides the bill. Empty means wildcard. A zero rate limit means no limit. A weighted alias with one dead leg fails a fifth of the time and looks like flakiness. None of that is visible until something reads the whole configuration the way the router will - which is the actual work here. The proxies are the easy half.

Agent Gateway is version 1.1.0. It ships with Kunobi and installs itself on first run, and every view in this post is free tier.

👉 Download Kunobi and find out which key has been paying.