Platform · AI Gateway
One governed endpoint
to every model.
Point every SDK, agent, and AI tool at a single gateway URL. Identity, policy, data protection, routing, metering, and audit are enforced on every call — in the request path, before a prompt ever leaves your company. Nothing gets rewritten; everything gets governed.
Why a Gateway
AI traffic became a first-class kind of network traffic. Almost nobody controls it yet.
When every team calls model providers directly, the company ends up with dozens of parallel arrangements: separate API keys, separate billing, separate ideas of what is allowed, and no shared record of what was sent where. Industry research through 2026 keeps finding the same pattern — the overwhelming majority of organizations report unsanctioned AI use, roughly half of employees reach AI through personal accounts at least some of the time, and nearly all AI-related breaches trace back to missing access controls rather than exotic attacks.
The fix is architectural, not procedural. A policy document cannot inspect a prompt; a gateway can. Placing one governed endpoint between your people and every model turns AI from hundreds of individual relationships into one piece of infrastructure — the same move companies made for internet egress, email, and identity years ago.
It is also a finance decision. Enterprise model spending is growing fast and unevenly — most of the bill typically comes from a small set of high-volume tasks running on models far stronger than the task needs. A gateway is the only place where visibility, budgets, and routing can be applied to every application at once, without asking each team to change code.
How It Works
What happens to a single request.
01 · Identify
The caller is authenticated
Every request carries a short-lived token issued through your SSO. The gateway knows the person or agent behind the call, their team, and their role — no anonymous traffic, no shared keys.
02 · Check
Policy is evaluated in the path
Allow/deny rules run on the live request: is this role permitted this model, this tool, this data destination? Enforcement happens before egress — not in a wiki, not after the fact.
03 · Protect
Sensitive data is handled
Detection runs on the prompt: personal data, credentials, and configured patterns are masked or blocked. Workloads flagged private divert to a local model that never leaves your network.
04 · Route
The right model is chosen
Routing rules weigh cost, speed, privacy, and quality. Simple, high-volume work lands on low-cost models; complex reasoning earns a premium model; sensitive data stays local.
05 · Execute
Provider keys stay vaulted
The gateway completes the call using your vaulted provider credentials. If a provider degrades, fallback chains keep the workload alive. Raw keys never reach a laptop or a repo.
06 · Record
Everything is attributed
Model, tokens, latency, and cost are logged against the user, team, tool, and agent. The call becomes a line in the audit trail and a line in the budget — automatically.
Policy
Every call carries its policy with it.
Drop-in compatibility
OpenAI- and Anthropic-compatible APIs for chat, completions, and embeddings, with streaming. Existing SDKs, IDE assistants, and agent frameworks work by changing one base URL.
Every major provider, plus your own
Anthropic, OpenAI, Google, Mistral, xAI and more — alongside fine-tuned, proprietary, and local models — under one ruleset, one meter, one audit trail.
Built to disappear
The gateway is designed to add minimal overhead to the request path; the work it does — policy, masking, routing — replaces logic your teams would otherwise rebuild in every app.
Where It Lands First
One endpoint, very different jobs per department.
Engineering
IDE assistants, CI agents, and internal apps point at the gateway URL. Personal provider keys disappear from laptops and repos; usage becomes visible per developer and per pipeline.
Support
High-volume triage and drafting run on low-cost models with customer identifiers masked before egress. Quality-critical escalations route up a tier automatically.
Finance
Ledger and payroll work is pinned to the local model by policy — the gateway refuses external routes for those data classes, and the audit trail shows it.
Sales
Reps draft with CRM context through scoped connectors. Access follows the role — a departing rep loses gateway access and every downstream connection in one revocation.
IT & Security
Shadow keys and personal accounts are migrated onto the governed path rather than banned. IT gets one control point; employees keep tools that work.
Regulation is arriving on a schedule
The gateway is where deployer duties become practical.
The EU AI Act phases in between 2025 and 2028: general-purpose model rules and AI-literacy duties are already in force, the EU AI Office's enforcement powers begin in August 2026, and — following the 2026 simplification package — obligations for high-risk uses such as employment decisions apply from December 2027. Companies that deploy AI (rather than build models) are expected to ensure human oversight, monitoring, and records of how systems are used. GDPR continues to apply alongside all of it.
Frameworks like ISO/IEC 42001 and the NIST AI RMF ask for the same substance in voluntary form: an inventory of AI in use, controls over access and data, and evidence that both work. A gateway makes those answers structural — every model in use is visible because every call passes through one place, and the logs regulators or auditors ask about already exist.
Rollout Guide
Observe first. Enforce second. Optimize third.
Point one team's traffic at the gateway in observe mode. Within days you know which tools, models, and prompts are actually in use, and what they cost. No rules yet — just truth.
Turn on the guardrails that prevent incidents: PII masking, budget caps that stop a looping agent, vaulted keys replacing personal ones. Nothing about daily work changes.
Apply role-based allow/deny rules learned from phase 1: which teams reach which models, tools, and data classes. Offboarding and audit are now one revocation and one query.
Add routing rules and local models where the data demands it. Spend falls where work was over-served; quality holds because routing is checked against real outcomes.
Evaluating Gateways
Ten questions to ask any AI gateway — including ours.
If a vendor can't answer these concretely, the gateway is a proxy, not a control plane.
Is policy evaluated on every request, or configured per application?
Do raw provider keys ever leave the vault — and who can read them?
Can sensitive fields be masked or blocked before egress, per data class?
Is there a local-model route for data that must not leave the network?
Are budgets enforced in dollars — not just tokens — per team and per agent?
What happens when a provider has an outage — is failover automatic?
Does one revocation remove a person's access to every model and tool?
Are agents identified separately from humans, each with an owner?
Can the audit trail answer "who sent what, where, at what cost" in one query?
Can the same policy plane run in our own cloud when compliance requires it?
Questions We Hear
AI Gateway — frequently asked.
Do we have to rewrite our applications?
No. The gateway exposes OpenAI- and Anthropic-compatible APIs, so existing SDKs, agent frameworks, and IDE tools work by changing the base URL and swapping personal keys for scoped tokens. Migration is a configuration change, done team by team.
What happens if the gateway itself goes down?
The gateway is deployed redundantly, and provider-side failures are handled with fallback chains and retries. Because it is the path to every model, we treat its availability the way you treat your identity provider's — as tier-zero infrastructure. Architecture details are shared under NDA on a strategy call.
Can employees just go around it?
Technically they can open a personal account — which is exactly today's problem. The gateway makes the sanctioned path the easy path: better models, shared agents, and no personal bill. Shadow-AI discovery then surfaces what remains outside, so IT can migrate it rather than chase it. Research consistently shows sanctioned alternatives reduce unsanctioned use far more effectively than bans.
Whose provider accounts does it use?
Yours, ours, or both. Bring your own Anthropic, OpenAI, or Google agreements and vault the keys, use Leapforce-managed capacity to start quickly, or mix — existing enterprise agreements and committed-use discounts keep working.
How does PII detection actually work?
A combination of pattern rules (identifiers with known shapes) and classifier-based detection for names, addresses, and free-text data, tuned per data class you define. Detected fields can be masked with placeholders, blocked, or allowed-with-log depending on policy.
Doesn't masking make answers worse?
Placeholders preserve the structure the model needs ("write an update for [CUSTOMER]") while removing the values it doesn't. For workloads where the values themselves matter, policy routes the task to a local model instead of masking — the choice is per data class, not global.
How much latency does it add?
The gateway is designed to add minimal overhead relative to model inference time, which dominates every call. Policy and masking run in-path; heavy analysis runs on the record after the fact. We benchmark it with you on your workloads rather than quoting a universal number.
Which models are supported?
Every major commercial provider — Anthropic, OpenAI, Google, Mistral, xAI and more — plus fine-tuned and proprietary models you bring, and local open-weight models for private workloads. New models become available by configuration, not by project.
Can the gateway run inside our own cloud?
Yes. Start on Leapforce SaaS; when data control or residency requires it, the same gateway and policy plane deploy into your cloud account. Policies, roles, and audit history carry over — see the Deployment page.
Is this an MCP gateway?
It includes one. Tool and MCP-server access is governed through the approved registry — version-pinned and allowlisted per team — but the gateway also covers model traffic, budgets, masking, and audit, which a bare MCP gateway does not. See Connectors.