Changelog

2024-09-26

Persistent logs

• Logs: AI Gateway now has logs that persist, giving you the flexibility to store them for your preferred duration.

Logpush

• Logs: Securely export logs to an external storage location using Logpush.

Pricing

• Pricing: Added pricing for storing logs persistently.

Evaluations

• Configurations: Use AI Gateway’s Evaluations to make informed decisions on how to optimize your AI application.

2024-09-10

Custom costs

• Configuration: AI Gateway now allows you to set custom costs at the request level custom costs to requests, accurately reflect your unique pricing, overriding the default or public model costs.

2024-08-02

Mistral AI

• Providers: Added Mistral AI as a new provider.

2024-07-23

Google AI Studio

• Providers: Added Google AI Studio as a new provider.

2024-07-10

Custom metadata

AI Gateway now supports adding custom metadata to requests, improving tracking and analysis of incoming requests.

2024-07-09

Logs

Logs are now available for the last 24 hours.

2024-06-24

Custom cache key headers

AI Gateway now supports custom cache key headers.

2024-06-18

Access an AI Gateway through a Worker

Workers AI now natively supports AI Gateway.

2024-05-22

AI Gateway is now GA

AI Gateway is moving from beta to GA.

2024-05-16

• Providers: Added Cohere and Groq as new providers.

2024-05-09

• Added new endpoints to the REST API.

2024-03-26

• LLM Side Channel vulnerability fixed
• Providers: Added Anthropic, Google Vertex, Perplexity as providers.

2023-10-26

• Real-time Logs: Logs are now real-time, showing logs for the last hour. If you have a need for persistent logs, please let the team know on Discord. We are building out a persistent logs feature for those who want to store their logs for longer.
• Providers: Azure OpenAI is now supported as a provider!
• Docs: Added Azure OpenAI example.
• Bug Fixes: Errors with costs and tokens should be fixed.

2023-10-09

• Logs: Logs will now be limited to the last 24h. If you have a use case that requires more logging, please reach out to the team on Discord.
• Dashboard: Logs now refresh automatically.
• Docs: Fixed Workers AI example in docs and dash.
• Caching: Embedding requests are now cacheable. Rate limit will not apply for cached requests.
• Bug Fixes: Identical requests to different providers are not wrongly served from cache anymore. Streaming now works as expected, including for the Universal endpoint.
• Known Issues: There's currently a bug with costs that we are investigating.