What is Mesh API?
Mesh API is an AI API and LLM gateway that routes requests to over 300 large language models through a single API key. It covers models from OpenAI, Anthropic, Google, Meta, Mistral, DeepSeek, Alibaba, and others. The API is OpenAI-compatible, so existing code works with two configuration changes. Beyond basic routing, the platform handles agent memory, retrieval-augmented generation, smart model selection, and automatic failover. It supports audio, image generation, video generation, embeddings, and batch processing.
Features & Benefits
- Universal API Access: Connect to 300+ AI models through one AI API.
- Smart Routing: Automatically direct requests to the best-fit model based on task complexity, cost, and latency. Simple completions route to lightweight models. Complex reasoning routes to capable ones.
- Automatic Failover: Detect provider outages and reroute requests to a backup model within 100ms. Circuit breaking removes failing providers from the routing pool until they recover.
- Persistent Agent Memory: Store episodic and long-term context across sessions without configuring a separate vector store.
- Plug-in RAG: Accept PDF, DOCX, or CSV files and return grounded, cited answers through a standard API call. Chunking, embedding, and indexing happen inside the platform.
- Prompt Templates: Save, version, and switch system prompts via a secure templating system using a single parameter change.
- Multi-Model Comparison: Fan out a single request to multiple models concurrently and receive a structured evaluation covering accuracy, completeness, and clarity. Raw outputs are available without synthesis.
- Audio Input and Output: Send audio input and receive audio output through the standard chat completions endpoint. Audio payloads are base64 encoded in the request body.
- Speech to Text: Transcribe audio files via a dedicated endpoint. Accept file uploads, public URLs, or cloud storage URLs. Real-time transcription streams partial and final transcripts as audio is captured.
- Text to Speech: Convert text to audio through automatic provider selection based on the model passed. WebSocket streaming sends text chunks as they arrive and returns audio chunk by chunk.
- Transcription and Translation: Transcribe audio and translate the result to English in a single API call.
- Image Generation: Generate images through the same unified endpoint without a separate provider integration.
- Video Generation: Access video generation models through the same endpoint.
- Embeddings: Produce vector embeddings for downstream search and retrieval use cases.
- Batch API: Submit batch inference requests through the platform.
- Unified Billing: Fund one account balance and consume any model from any provider. Supports multiple currencies including INR and UPI.
- Rate Limiting: Apply spend caps and request limits per model or integration to prevent runaway costs.
- Audit Logging: Capture every request, token count, latency value, and cost in real time. Filter by model, status, or time range and export for compliance or debugging.
- Security: Enforce key scoping per account, apply a zero-storage policy on prompt and completion content, and store upstream provider keys in hardware-isolated GCP Secret Manager.
What can Mesh API do?
- route AI requests to multiple LLM providers through one API
- switch between LLMs without changing code
- add persistent memory to AI agents
- run retrieval-augmented generation from uploaded documents
- compare outputs from multiple LLMs in one request
- automatically failover to a backup model during an outage
- log token usage and cost per API request
- limit AI API spend with rate caps
- generate images through a unified LLM endpoint
- transcribe audio through a single API call
- generate embeddings for search and retrieval
Real-World Applications
A team building AI-powered products may use the LLM gateway to consolidate access across multiple providers. Instead of managing separate API keys and SDKs for OpenAI, Anthropic, and Google, they route everything through one endpoint. Smart routing sends low-complexity tasks to cheaper models and reserves expensive reasoning models for tasks that need them.
Production deployments that need high availability benefit from the automatic failover built into the AI API. If a provider goes down, traffic shifts to a backup model in under 100ms. Circuit breaking prevents degraded providers from receiving traffic until they recover, which keeps response rates stable.
Document-heavy workflows can use the plug-in RAG feature to feed the AI API PDFs, spreadsheets, or Word documents. The platform handles chunking, indexing, and retrieval internally and returns cited answers. This applies to use cases like contract review, internal knowledge bases, or customer-facing Q&A.
Teams running multi-agent systems may rely on the persistent memory layer built into the LLM gateway. Agents retain conversation history, prior decisions, and session context without custom vector store configuration, making it practical to build agents that operate across long or recurring task sequences.
