AI Gateway
An AI gateway is one server that every application in a company sends its model calls through. It holds the model-provider credentials, routes each call to the chosen model and applies rate limits, spend tracking and logging in one place. Teams run one so they can switch providers and see usage without changing every app.
Why It Matters
Teams run an AI gateway so every application talks to one place rather than to each model provider separately. One server then holds the provider credentials, reports what the company is spending and lets a team change providers without touching every app. LiteLLM is the common open-source example.
An AI gateway and an MCP gateway are two different endpoints that are easy to confuse, because both concentrate traffic in one place. An AI gateway carries a company's model calls out to model providers. An MCP gateway carries an agent's tool calls in to the company's own apps. Cakewalk builds an MCP Gateway and does not build an AI gateway.
Concentration is what makes either one worth attacking. In July 2026 Darktrace disclosed a break-in at a customer whose AI gateway, a LiteLLM deployment, held standing access to Amazon Bedrock and checked nothing about who was calling, so one exposed login was worth everything behind it. The risk belongs to the shape rather than to any one product, which is why what a gateway is built to hold decides how bad a breach gets.