Supported Providers

InferShrink currently supports OpenAI and Anthropic. We route requests intelligently within each provider's ecosystem.

OpenAI Integration

Wrap the standard client. All chat.completions.create arguments are supported, including tools/functions and JSON mode.

import openai
from infershrink import optimize

client = optimize(openai.Client())
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

Routing Tiers

• Tier 1 (Simple): gpt-4o-mini
• Tier 2 (Standard): gpt-4o
• Tier 3 (Complex): gpt-4.5-preview

Anthropic Integration

InferShrink automatically maps complexity to the Opus/Sonnet/Haiku family.

import anthropic
from infershrink import optimize

client = optimize(anthropic.Anthropic())
response = client.messages.create(
    model="claude-opus-4-6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)

Routing Tiers

• Tier 1 (Simple): claude-sonnet
• Tier 2 (Standard): claude-opus-4-5
• Tier 3 (Complex): claude-opus-4-6

Coming Soon

We are actively working on support for:

• Google Vertex AI (Gemini)
• Mistral AI
• Local LLMs (via Ollama)