The one-sentence definition
An AI gateway receives the request your app would have sent to a model provider, decides where it should go, forwards it, and records the result. Your code talks to one address; the gateway talks to the providers.
What it is not
It is not a model, and it does not make answers better on its own. It also does not replace your SDK: most gateways accept the request format you already send, so adopting one usually means changing a base URL, not rewriting code.
The four jobs of the gateway
Routing chooses the provider and model for each request and falls back when one is unavailable. Metering turns usage into a cost. Limits cap spend and request rate per key.
Recording writes down what happened, so the numbers can be audited later instead of trusted blindly.
Why teams end up needing one
With a single provider and a single key, you do not. The need appears when keys multiply across services, when a provider has an outage during a launch, or when you resell access and need to know exactly who spent what.
Pertanyaan yang sering diajukan
- Does a gateway add latency?
A little, and it should be measurable. Pecutin publishes its own load measurements on the home page, taken against a simulated upstream that holds each request for several seconds, so the gateway's own overhead is visible.
- Do I have to rewrite my code?
Usually not. You change the base URL in the SDK you already use and swap in the gateway key.