Trace every request
Inspect provider, model, latency, time to first token, tokens, estimated cost, errors, and request lifecycle.
Latency · TTFT · Tokens · CostLLM Gateway & Observability
See what happens
behind every LLM
response
Route requests across providers, inspect latency and cost, trace failures, replay prompts, and compare models from one operational console.
Built with FastAPI, Next.js, Redis Streams, PostgreSQL, OpenTelemetry, and Prometheus.
Operational clarity
One console for the full request lifecycle
Recover from failures
See retries, fallback routing, circuit-breaker events, cancellations, and normalized provider errors.
Retry · Fallback · Circuit breakerReplay safely
Re-run a request with another provider or model while preserving the original trace.
Original trace · New executionCompare providers
Run the same prompt across multiple models and compare latency, tokens, cost, status, and output.
Latency · Cost · Output · StatusTechnical stack
Built as real LLM infrastructure
InferLens combines a provider-agnostic gateway, asynchronous ingestion, privacy-safe observability, and resilient worker processing.
Core stack
Next.jsTypeScriptFastAPIPostgreSQLRedis StreamsOpenTelemetry
Observability & providers
PrometheusDockerOpenAIGemini
✓ Real SSE streaming✓ Retry and fallback routing✓ Circuit breaker✓ W3C Trace Context✓ PII redaction✓ Dead-letter recovery✓ Concurrent comparisons
Start exploring
Observe your first LLM request
Open the InferLens console to explore streaming, traces, replay, comparisons, latency, tokens, cost, and provider behavior.
Open the console