Vercel AI Gateway Now Supports Routing Rules
Vercel has added routing rules to its AI Gateway, a feature that lets developers redirect AI model requests at the infrastructure layer without touching application code. Announced on the Vercel blog, the update effectively turns the gateway into a firewall-style traffic controller for large language models.
What Happened
According to Vercel, the AI Gateway now supports two types of routing rules: Rewrite and Retry. A Rewrite rule serves a request intended for one model by routing it to another model entirely. For example, if OpenAI's GPT-4 experiences an outage, a team can instantly reroute all GPT-4 calls to Anthropic's Claude 3.5 Sonnet without deploying a single line of code. The Retry rule, meanwhile, automatically reattempts a failed request against a fallback model if the primary model returns an error or timeout.
The key architectural shift is that these rules live at the gateway level, not inside application logic. Previously, handling a model outage required editing a configuration file, merging a pull request, and redeploying. Now, a single rule update on the Vercel dashboard propagates instantly across all services using that gateway.
Why It Matters
Model availability has become a critical reliability concern for AI-powered applications. OpenAI, Anthropic, and Google have all suffered major outages in the past 18 months. A three-hour GPT-4 outage in early 2025 cost some enterprises an estimated $2 million in lost revenue per hour, based on internal estimates shared during industry conferences.
For developers, the implication is straightforward: you can now build for model diversity without adding complexity to your codebase. Most integration today uses a single provider's API. Routing rules let teams keep a primary model while maintaining a fallback chain. If GPT-4 returns a 5xx error, the AI Gateway can automatically forward the same prompt to Claude 3.5 Haiku or Gemini 1.5 Pro, returning a response to the user with no visible delay.
What It Means for Developers and Businesses
For development teams, routing rules reduce operational burden. AI engineering teams currently spend significant time monitoring model health endpoints and writing custom circuit-breaker logic. The Vercel approach treats model routing as a configuration problem, not a code problem.
From a security perspective, routing rules act as an access control layer. Teams can create rules that block calls to unauthorized models or enforce cost limits per endpoint. A developer cannot accidentally call an expensive model like GPT-4o if a rule restricts that endpoint to Claude 3 Haiku for low-priority tasks.
Technical Details and Usage
Vercel demonstrated the feature using a simple API call to the gateway's routing endpoint. The syntax follows a pattern familiar to anyone who has worked with reverse proxies or API gateways before:
- Rewrite rule: Match on model ID and rewrite to a different model ID. Vercel claims the rewrite adds under 5ms of overhead.
- Retry rule: Specify a primary model and up to three fallback models. The gateway tries fallbacks in order until one returns a valid response.
- Rate limit integration: Rules can include rate limits per model, preventing an overwhelmed fallback from becoming another bottleneck.
Competitive Landscape
Vercel's move puts it in direct competition with dedicated AI infrastructure providers like Portkey, Helicone, and LangChain's LangSmith. Portkey already offers model fallback and routing, but Vercel's advantage is its deep integration with the Vercel ecosystem and developers who already use its Edge Functions and Serverless computing platform. According to Vercel's product team, routing rules work transparently with any provider that supports OpenAI-compatible API formats, which covers most major LLM APIs today.
Pricing and Availability
Routing rules are available today on the AI Gateway Pro plan, which starts at $100 per month for 10 million tokens processed. Vercel has not yet announced a free tier for routing-specific features, though the basic AI Gateway remains free for the first 1 million tokens per month.
The Bigger Picture
The announcement reflects a broader industry trend: AI infrastructure is becoming more like web infrastructure. Developers no longer treat models as monolithic services but as interchangeable components that can be swapped, load-balanced, and circuit-broken just like database connections or microservice endpoints.
For businesses building production AI systems, the lesson is clear. Model lock-in is a liability. Routing rules eliminate the need to embed provider-specific logic into your application layer. When OpenAI retires GPT-4 on a Friday afternoon, you don't scramble to write code — you update a rule and move on.
Related: Claude Sonnet 5 Debuts on AWS Bedrock: Anthropic's Smartest Mid-Tier Model Arrives
Source: Vercel Blog. This article was produced with AI assistance and reviewed for accuracy. Editorial standards.
