Overview
Inferock Bench is a local diagnostic proxy for teams that want an independent receipt for LLM API calls. Instead of relying only on a provider dashboard or monthly invoice, developers route selected OpenAI, Anthropic, Gemini Developer API, or pinned OpenRouter traffic through a localhost process that records usage, timing, failure signals, and pricing evidence.
The tool is especially relevant for engineering teams with growing AI API spend, agent workloads, or production systems where one failed response can still consume tokens. It sits near the same operational problem space as AI data governance, but with a narrower focus: preserving per-call billing and reliability evidence before it disappears into aggregate invoices.
Inferock Bench is not a hosted spend-management dashboard. The open-source package runs locally, stores event records on the user's machine, and produces receipts from captured traffic. Hosted Inferock is mentioned as a separate product for teams that need managed routing, support, invoice reconciliation, or provider leverage.
Key Features
Local LLM proxy - Runs on localhost and forwards measured traffic to supported providers, giving teams a local evidence layer without changing the core application flow.
Per-call receipts - Records provider-reported usage, request and response metadata, timing, retry evidence, status, and detector signals for calls the proxy actually observes.
Billing-integrity checks - Looks for signals such as billed-empty output, truncation, token recount mismatches, duplicate request IDs, refusal states, cache-discount exposure, and provider-fault retries.
Provider coverage labels - Marks surfaces as watched-clean, signal, or not-openable, so teams can distinguish what was checked from what remains coverage debt.
Local key boundary - Provider keys are saved locally and attached only to provider requests; receipts remain local unless the user shares them.
SDK-friendly setup - Existing apps usually need only a base URL and API key setting change, and the README includes examples for Claude Code, OpenAI SDK, Gemini, OpenRouter, and CI usage.
How to Get Started
Inferock Bench is a technical tool, so the first setup should be small and controlled. Install Node.js 22 or newer, run npx inferock-bench, save a low-limit provider API key in the local dashboard or CLI, reveal the local bench key, and point a development app or SDK at the localhost endpoint.
The cleanest first test is not production traffic. Use a small script, an internal evaluation task, or a low-cost agent prompt to confirm that the proxy sees the call and can generate a compact receipt. Teams already using Claude Code or other agent CLIs can test the same pattern by swapping the base URL and local bench key for measured API traffic.
After that, decide which workflows are worth routing through the proxy. Good candidates include expensive batch jobs, agent evaluation runs, invoice spot checks, reliability experiments, and regression tests where token usage or response failure modes need a durable trail.
Pricing & Plans
Inferock Bench is published as an open-source local package. The repository uses an FSL-1.1-ALv2 license badge and the Product Hunt listing presents the launch as free.
That does not make provider usage free. Any real traffic routed through the proxy still uses the user's OpenAI, Anthropic, Gemini, or OpenRouter account, and the README warns users to start with low-limit keys and small tests. Hosted Inferock is a separate commercial product for production teams that want managed key custody, routing, recovery, invoice reconciliation, or support.
| Option | Price | Notes |
|---|---|---|
| Inferock Bench local package | Free | Local diagnostic proxy and receipt generator |
| Provider API usage | Varies | Paid to the upstream provider when real calls are made |
| Hosted Inferock | Separate product | For managed production workflows and support |
Best For
- Engineering teams with meaningful OpenAI, Anthropic, Gemini, or OpenRouter spend.
- Developers building agents who need to preserve response, retry, and token evidence.
- Finance or platform teams auditing AI API invoices with technical support.
- Reliability engineers comparing AI failures across providers and endpoints.
- Teams already following AI agent infrastructure and needing better cost visibility.
FAQ
What is Inferock Bench?
Inferock Bench is a local LLM cost-tracking proxy that captures per-call usage, timing, failure, retry, and billing-integrity evidence for supported AI API traffic.
Is Inferock Bench free?
The local package is free to run, but any real calls sent to upstream providers still use the user's provider account and may incur provider charges.
Which providers does it support?
The README names OpenAI, Anthropic, Gemini Developer API, and pinned OpenRouter endpoints as measured provider planes at launch.
Does Inferock Bench send my provider keys to Inferock?
Official materials state that provider keys are saved locally and attached only to provider requests. Receipts stay local unless the user shares them.
Can it audit my whole monthly AI bill?
Only for traffic it actually observes. It cannot audit calls that bypass the proxy or explain an invoice without matching captured evidence.
Is it a provider ranking tool?
No. The project frames receipts as diagnostic evidence for observed calls, not as a global provider ranking or proof that a failure pattern is widespread.
Who should avoid it?
Non-technical users who only want a simple spend dashboard may find it too low-level. Teams should also avoid treating any receipt as a legal conclusion without reviewing the underlying assumptions.
