How Gate works

Every request to Gate runs through the same set of checks before it reaches the upstream provider, and the same set of records after the upstream responds. This page walks through what happens, from your client’s POST to the response in your terminal.

At a glance

client → Gate → upstream provider → Gate → client

Gate sits inline between your application and the model provider. You send a normal Anthropic Messages (/v1/messages) or OpenAI Chat Completions (/v1/chat/completions) request to Gate’s base URL, and Gate forwards it on, adding routing, security, caching, cost accounting, and an audit trail on the way through.

Which upstream Gate talks to depends on your billing mode:

• Paying through Gate. Gate forwards through one of the provider accounts your org has enabled, using its credentials, and bills the request against your prepaid balance. Today paying through Gate bills two upstreams: Amazon Bedrock and OpenRouter.
• Use your own keys. You point Gate at your own upstream and supply your own credentials. Gate works with any upstream that speaks the Anthropic Messages or OpenAI Chat Completions API, including Anthropic, OpenAI, Google Vertex AI, Google AI Studio, OpenRouter, and any other OpenAI-compatible endpoint. You pay that provider directly. Gate is a policy, optimization, and logging layer in front of your own account.

Note: Google Vertex AI is fully routable but is your-own-keys-only today. Gate does not yet bill Vertex usage against a prepaid balance.

Between client and upstream, Gate:

1. Authenticates your key and decides which upstream serves the request.
2. Rate-limits by key and org, and enforces any usage caps you have configured.
3. Screens the prompt for injection, jailbreak, PII, PHI, and credentials.
4. Caches the request (when applicable) and adds compression and prompt-cache hints so you get the most value per token.
5. Forwards to the upstream, retrying transient errors.
6. Screens the response for sensitive data on the way back.
7. Records the request (token counts, cost, security verdict, upstream) in your request log and audit trail, then settles the bill.

Each step is a feature you control. Here is the user-visible knob for each.

1. Authentication and routing

Two things decide where a request goes:

• The headers on the request (see Authentication).
• The billing mode, which Gate infers from the request shape. There is no per-key flag to set.

Every request must carry a Gate API key (it starts with sk-gw-), most commonly as Authorization: Bearer $GATE_API_KEY. The key binds the request to your org so it can be rate-limited, billed, and audited.

Your own keys: you choose the upstream

Send the X-Gate-Upstream-Url header and Gate treats the request as your own keys. It forwards your bytes to that URL, appending your request’s full path (including the /v1 your client adds) to the value you supply — so give the provider’s base without that path, for example https://openrouter.ai/api rather than https://openrouter.ai/api/v1. Network protections keep a supplied URL from being used to reach internal systems. It never reaches into your org’s provider accounts, and the prepaid balance check is skipped. In using your own keys you also pass your own upstream credential (for example Authorization: Bearer <your-upstream-key>) alongside your Gate key.

curl -sS -X POST "https://gateway.constellationgate.ai/v1/messages" \
 -H "X-Gate-Api-Key: $GATE_API_KEY" \
 -H "Authorization: Bearer $ANTHROPIC_API_KEY" \
 -H "X-Gate-Upstream-Url: https://api.anthropic.com" \
 -H "Content-Type: application/json" \
 -d '{"model":"claude-sonnet-4-5","max_tokens":256,"messages":[{"role":"user","content":"Hello"}]}'

To save typing the header on every call, point your SDK’s base URL at Gate and pass the upstream once via your client config. Most agent tools (Claude Code, Codex) inject it for you. See the manual setup guides for per-tool setup.

Paying through Gate: Gate chooses the provider account

Omit the upstream header and Gate routes the request as paying through Gate. It picks an enabled provider account on your org by this precedence:

1. An explicit provider hint, if you send one.
2. The preferred provider you have pinned for that model on the Models page.
3. The cheapest enabled provider that serves the model.

To pin a provider explicitly, send the X-Gate-Provider header, or set "provider": "<name>" in the request body. The value can be a provider type (for example bedrock) or the name of a specific account.

curl -sS -X POST "https://gateway.constellationgate.ai/v1/messages" \
 -H "Authorization: Bearer $GATE_API_KEY" \
 -H "X-Gate-Provider: bedrock" \
 -H "Content-Type: application/json" \
 -d '{"model":"claude-sonnet-4-5","max_tokens":256,"messages":[{"role":"user","content":"Hello"}]}'

Note: X-Gate-Provider applies to paying through Gate only. It selects among your enabled provider accounts. X-Gate-Upstream-Url applies to your own keys only. It sets the upstream you supply yourself. Sending the upstream header always puts the request in using your own keys, so the provider hint is ignored there.

When several providers serve the same model, Gate routes to the most cost-effective eligible one automatically. You can always override that with a hint or a pinned preference.

If the request cannot be served (an unknown model, no enabled provider, a provider that does not serve the requested model, or a missing required header), Gate returns a 400, 402, or 403 with a short JSON explanation, often including a “did you mean” suggestion. It never makes a partial upstream call.

2. Rate limiting and usage caps

• Per-key rate limit. A sliding window, configurable per key in the dashboard.
• Per-org rate limit. A single window shared across every key on the org, set in the org’s entitlements.
• Usage limits. Cost, token, or request ceilings over a time window. Optional, and attached per key or per org.

Exceeding a rate limit returns 429 with X-RateLimit-* headers (limit, remaining, reset, and the scope that tripped). Exceeding a usage limit returns 429 with the limit’s id, type, threshold, current usage, and time window. Both block the request and record the breach to your audit trail. See Limits and retention for the configurable values and the exact 429 body.

3. Security screening

Inbound prompts and outbound responses are scanned for:

• Prompt injection and jailbreaks. Detection works in stages: a fast pattern-based pre-filter, an ensemble of purpose-built classifiers, and a large-model adjudicator for the highest-confidence calls. Every request is scanned. There are no trust shortcuts that skip the scan.
• Sensitive data. PII, PHI, credentials, API keys, and similar.
• Spend ceiling. An optional per-key cost guardrail enforced as a security criterion.

Each criterion has its own configurable sensitivity (High, Medium, or Low) and action: log only, flag, or block. You choose how detections are handled under Guardrails. When part of the detection stack is briefly unavailable, it fails closed rather than letting a request slip through unscreened. See Security and the Prompt-injection defense page.

4. Caching and compression

• Response cache. An identical request returns a cached response when your org enables it. Hits cost nothing (no upstream call, no tokens) and the response carries X-Gate-Cache: HIT.
• Prompt-cache injection. For Anthropic-family requests (served directly or via Bedrock), Gate automatically inserts cache_control breakpoints so the upstream charges a fraction of the input rate for the cached prefix on later turns. If you already set markers yourself, Gate leaves them alone.
• Compression. Text-bearing fields are losslessly rewritten with safe transforms before forwarding, and tool schemas are never altered. The model sees the same meaning, the request carries fewer tokens.

Typical impact: meaningful savings on token spend with no client changes.

Turning compression off, and true passthrough

These are two different things, and the difference matters if you are measuring Gate against a baseline.

Turning compression off for your org means Gate stops rewriting your prompts to save tokens. It does not mean Gate forwards your bytes untouched. A few request-shape features are keyed on the provider and wire rather than on that toggle, and they keep running: deferred tool loading on capable Anthropic models, prompt-cache marker placement, and the normalization of a per-request nonce some clients stamp into their system prompt (left as-is, it would defeat the upstream prompt cache on every turn for exactly the orgs that opted out).

If you want a genuinely untreated request, send the header X-Gate-Passthrough: 1. Gate then forwards your body byte for byte: no compression, no nonce normalization, no tool deferral, no cache-marker injection or relocation, and no answers from Gate’s own response cache. In Claude Code, set ANTHROPIC_CUSTOM_HEADERS="X-Gate-Passthrough: 1".

What passthrough does not turn off is anything that exists for correctness or safety, so a few things still run and any of them can change your bytes.

Security screening always runs, on every request, and no header turns it off. It can rewrite a body, for example by stripping invisible attack characters or redacting a detected secret.

Routing also still runs, because a gateway that does not name the model your chosen provider serves cannot reach that provider at all. On a managed plan that can mean a model alias is rewritten in the body. If you are bringing your own key and your own upstream (X-Gate-Upstream-Url), routing makes no changes, so that is the shape to use when you need a byte-for-byte baseline.

A small set of repairs stay on too. If your request would be rejected by the provider for a fixable reason, Gate still fixes it rather than letting the turn fail: an OpenAI-style system message is moved to where the Anthropic API expects it, and a stale tool reference left behind by an earlier turn is dropped instead of failing that conversation forever. On the Anthropic Messages API these fire only on a request that would not have worked as sent.

The system-message move also applies when you send the OpenAI Chat Completions format to a provider that does not need it, and there are two of those. To Bedrock it costs you nothing — Bedrock is in the group below that never reports verbatim anyway — though it does mean your system prompt reaches the model as slightly different text whenever it is more than one plain string. To a managed Anthropic account the move is not free: your system message is taken out of messages and written where the Anthropic API expects it, which an OpenAI-shaped endpoint does not read, so the system prompt can be dropped without an error. Gate reports that request as modified. If you are sending OpenAI-shaped bodies, use the bring-your-own-upstream shape.

And two destinations can never be byte-for-byte at all. Amazon Bedrock and Google Vertex AI do not accept the Anthropic or OpenAI request shape directly, so Gate rebuilds the payload into their own format before sending it. Requests routed there always report modified, whatever Gate itself did or did not do, rather than implying a fidelity that is not there.

So passthrough reports what actually happened rather than what you asked for. A forwarded request answers verbatim when the bytes upstream were exactly yours and modified when something still changed them, and it comes back on the response as X-Gate-Passthrough: verbatim or X-Gate-Passthrough: modified. Read it per request rather than treating the request header as a guarantee. Some requests answer neither, and that is itself the honest answer: anything stopped before it reached a provider (blocked by screening, rate-limited, or over a usage or balance limit), and anything carrying no prompt for the question to be about.

The verdict is usually also recorded against the request, retrievable through the API, though the dashboard does not surface it as a column today. Treat that as a convenience rather than the record: a request whose response was blocked, for example, keeps no such field. If you are running a comparison, capture the response header as you go rather than planning to reconstruct it later.

5. Forwarding

Gate forwards with retries on transient upstream errors (5xx and 429). Streaming responses (SSE, chunked JSON) are scanned in-flight, so you do not trade latency for a post-hoc check. Upstream 429s are relayed back to you verbatim, so your client’s own backoff logic stays in charge.

If a provider is unhealthy, Gate returns 503 early instead of leaving your client to time out, and skips that provider rather than hammering it. These early rejections are recorded as uncharged, so you are never billed for a request that never reached a provider.

6. Response screening

The same security checks that ran on the input run on the output. A response that leaks a credential, leaks PII, or matches an output policy can be blocked or redacted, depending on your configuration. Streaming responses are scanned incrementally as tokens arrive.

7. Recording and billing

Every request creates a row in your request log with:

• request id, key, org, and user (when present)
• model, upstream, and billing mode (either payment method)
• token counts (input, output, cached) and computed cost
• a cost-calculation status that explains how the cost was derived (for example calculated, cache_hit, byok, or free_request)
• security verdict (per criterion)
• latency and HTTP status

It also writes an immutable entry to your audit trail (see audit trail). You can review any request from the dashboard or via the Messages view.

How the charge is computed

For paying through Gate, Gate prices each request against the provider catalog and verifies accuracy. When a provider reports the cost of a request in its response, Gate uses that figure. Otherwise it prices the reported token usage against the catalog rate for the routed provider. All arithmetic runs in integer micro-USD (1 USD = 1,000,000 µUSD), so charges never drift through floating-point rounding across millions of small requests. Your balance and transaction ledger are kept to the micro-USD, and the dashboard’s cent figures are a rounded display of the same underlying amount.

Before forwarding a request paid through Gate, Gate places a short-lived reservation for a conservative upper bound of the request’s cost, including a safety margin. This keeps a burst of concurrent requests from collectively overdrawing a low balance. The reservation is only a hold. It is released when the request settles, and you are always charged the real cost, never the estimate. If your balance cannot cover the worst-case bound, Gate returns 402 (insufficient_balance) before contacting the upstream rather than serving a request it cannot bill. A request whose price cannot be determined up front is refused with 503 (pricing_unavailable) rather than served at an unknown cost.

Requests on your own keys record cost_usd = 0 with a byok status. You pay your provider directly, so there is no gateway charge to compute.

The dashboard’s cost-verification view independently re-checks what Gate billed against what each provider charged, so you can confirm accuracy.

Conversations

When a client sends a multi-turn conversation, Gate identifies the session automatically by hashing the conversation prefix. The dashboard groups requests by session under Conversations, so you see whole exchanges instead of isolated calls. There is nothing to wire up on the client side.

Where Gate runs

Gate is a hosted, cloud service. You integrate by pointing your client at Gate’s base URL. There is no customer self-hosted or on-premises build of Gate itself today. The “self-hosted” option you will see for your own keys refers to the upstream: you can point Gate at a model server you run yourself (for example a self-hosted, OpenAI-compatible endpoint) and Gate will proxy to it like any other upstream.

What this looks like in practice

• A clean call. client → security passes → cache miss → forward → upstream returns → response scanned → record written and settled. Only a few milliseconds of Gate overhead on top of upstream latency.
• A cache hit. client → security passes → cache hit → response served from Gate with X-Gate-Cache: HIT. No upstream call, no token cost.
• A blocked call. client → security verdict is block → Gate returns the configured error → no upstream call, no token cost. The block lands in your audit trail with the reason.
• An unaffordable call paid through Gate. client → balance cannot cover the worst case → Gate returns 402 before any upstream call → nothing is billed.

Next steps

• Authentication: keys, headers, and your own keys passthrough.
• Security: guardrails, sensitivity, and detection actions.
• Limits and retention: rate limits, usage caps, request size, and retention.