ngrok AI Gateway

Overview

ngrok AI Gateway is a hosted LLM gateway for developers who want one API surface across public providers, custom endpoints, and self-hosted models. The official site positions it around a simple integration: change the SDK base URL to https://gateway.ngrok.ai, use an ngrok AI access key, and route model calls through the gateway.

The product is worth publishing because ngrok is a trusted developer infrastructure brand and the AI gateway category is becoming important for production AI agent systems. As teams add OpenAI, Anthropic, Gemini, local models, and private endpoints, the operational problem shifts from "can we call a model?" to "can we route, observe, retry, control, and pay for model calls predictably?"

ngrok AI Gateway is not a general chatbot. It is infrastructure for developers building AI products, agent backends, internal tools, and applications that need model routing without creating a separate gateway service from scratch. It also connects naturally to AI data governance, because access control, provider keys, and visibility are central to production AI operations.

Key Features

• One base URL for model calls - Use standard SDKs with a gateway URL instead of wiring every provider separately.
• Public and self-hosted model support - Route to OpenAI, Anthropic, custom endpoints, and models you run yourself.
• Fallback routing - Define alternatives so a request can move to another model or provider when the first path fails.
• Automatic retries - Reduce application-level error handling by retrying failed calls at the gateway layer.
• Usage observability - Track tokens, latency, errors, costs, providers, and model behavior across routed calls.
• Programmable configuration - Configure through APIs, Terraform, CLI, coding agents, or custom tooling.

How to Get Started

The fastest test is to take a small existing OpenAI-compatible script and change the base URL to the ngrok gateway endpoint. Then create an AI Gateway access key, choose a model route, and send a few non-sensitive requests while watching the dashboard.

For production evaluation, test three cases: normal routing, provider failure or slowness, and usage attribution across environments. Teams should also decide whether to use ngrok-managed provider keys or bring their own provider keys. The choice affects billing, security review, and how provider accounts are managed.

Pricing & Plans

ngrok AI Gateway pricing has two layers. The AI Gateway site lists a flat routing fee of $0.05 per million tokens plus inference cost. The ngrok documentation also says AI Gateway requires the Pay-as-you-go plan, while free and Hobbyist plans cannot use AI Gateway.

Teams should verify the current pricing page before deployment, especially if they plan to route high-volume traffic or combine AI Gateway with broader ngrok traffic features.

Best For

• Developers building production AI applications with more than one model provider
• Teams moving from prototype scripts to reliable AI infrastructure
• Companies routing to both hosted and self-hosted LLMs
• Platform teams centralizing AI observability and provider-key management
• Agent builders who need retries, fallback, and cost visibility

FAQ