I Used the OpenAI SDK-and Claude Answered. Here’s Why.

The 10-second answer

Three independent choices are hiding in that snippet:

• The SDK decides how your application constructs requests and reads responses.
• The endpoint and API contract decide where the request goes and what shape crosses the network.
• The model identifier and routing rules decide what ultimately runs.

The package name is not the model name. In the example above:

OpenAI Python SDK
    ↓ speaks OpenAI-style HTTP
Anthropic compatibility endpoint
    ↓ maps the request
Claude Sonnet
    ↓ result is mapped back
OpenAI-shaped response
    ↓ parsed by the SDK
response.choices[0].message.content

Anthropic officially provides this compatibility layer for testing and comparing Claude with existing OpenAI integrations. It is not the recommended production path for most Claude-first applications, for reasons we will get to shortly. But first, let us separate the layers developers accidentally compress into the word “AI.”

The four layers hiding behind one API call

The useful mental model is not “SDK → model.” It is:

┌───────────────────────────────────────────────┐
│ 1. Application + SDK                          │
│   Builds a request and parses a response       │
├───────────────────────────────────────────────┤
│ 2. API endpoint / gateway                      │
│   Auth, routing, policy, protocol mapping      │
├───────────────────────────────────────────────┤
│ 3. Serving layer                               │
│   Scheduling, batching, streaming, metrics     │
├───────────────────────────────────────────────┤
│ 4. Inference runtime + model                   │
│   Prompt → tokens → generation → tokens        │
└───────────────────────────────────────────────┘

These layers may live in one program, several containers, or multiple companies' infrastructure. The boundaries are conceptual, but the responsibilities are different.

1. The SDK is a client, not a model

An SDK usually gives you: authentication and default headers; request and response types; serialization and validation; retries, timeouts, and error classes; streaming helpers; a nicer interface than raw HTTP.

Without an SDK, the same job can be done with an HTTP client:

import os
import httpx

response = httpx.post(
    "https://api.anthropic.com/v1/messages",
    headers={
        "x-api-key": os.environ["ANTHROPIC_API_KEY"],
        "anthropic-version": "2023-06-01",
        "content-type": "application/json",
    },
    json={
        "model": "claude-sonnet-4-6",
        "max_tokens": 100,
        "messages": [{"role": "user", "content": "Hello"}],
    },
)
data = response.json()
print(data["content"][0]["text"])

The native Anthropic response uses content[]. The OpenAI-style response uses choices[]. Those shapes are API contracts. Neither is the natural output format of a neural network.

2. The endpoint is more than a URL

Consider these clients:

# OpenAI
OpenAI(
    api_key=os.environ["OPENAI_API_KEY"],
    base_url="https://api.openai.com/v1",
)

# Anthropic's OpenAI compatibility layer
OpenAI(
    api_key=os.environ["ANTHROPIC_API_KEY"],
    base_url="https://api.anthropic.com/v1/",
)

# A local Ollama server
OpenAI(
    api_key="ollama",   # required by the client; ignored locally
    base_url="http://localhost:11434/v1/",
)

The calling style barely changes, but the request crosses three completely different trust, billing, latency, and data boundaries. The base_url can point to:

• the model provider itself;
• a multi-provider gateway such as OpenRouter or LiteLLM;
• a cloud proxy;
• an OpenAI-compatible server such as vLLM;
• a local runtime such as Ollama;
• your own internal policy and routing service.

This is why “we use the OpenAI SDK” tells an architect almost nothing about where prompts go. The next questions should be:

• What is the base_url?
• Who controls that endpoint?
• Which model identifier is sent?
• Can the gateway rewrite or reroute it?
• Which protocol features survive the translation?

3. The serving layer creates the API-shaped response

At the lowest useful level, a language model works with numbers. The prompt is formatted and tokenized. The model produces logits. A generation strategy selects new token IDs. Those IDs are decoded into text.

For example, Hugging Face Transformers documents that generate() returns token sequences-or a richer internal ModelOutput when requested:

generated_ids = model.generate(**model_inputs, max_new_tokens=50)
text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

There is no universal neural-network law requiring the result to contain:

{
  "choices": [],
  "finish_reason": "stop",
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 5
  }
}

That public JSON shape is assembled by software around the generation runtime. A serving system may also produce internal text, token IDs, finish metadata, timing information, cache statistics, and scheduler state. The important distinction is not “the engine only returns text.” The important distinction is:

Provider JSON is a network contract created by the serving/API layer-not an intrinsic property of the model weights.

Projects such as vLLM make this visible: the inference machinery returns internal request outputs, while the OpenAI-compatible server exposes endpoints with OpenAI-style schemas.

4. The model is selected downstream

The model field is a request to the service, not a Python import. The endpoint decides how to interpret it.

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4.6",
    messages=[{"role": "user", "content": "Hello"}],
)

A gateway might:

• map that identifier to Anthropic;
• choose one of several upstream regions;
• fail over to another provider hosting the same open model;
• reject the identifier;
• apply an alias configured by your organization.

So even the model string is not always the complete deployment identity. In production, log the resolved provider, model version, request ID, and routing decision whenever the platform exposes them.

“OpenAI-compatible” does not mean “identical”

This is where a convenient prototype becomes a quiet production bug. Two services can support /v1/chat/completions and still disagree on:

• accepted parameters;
• tool-call guarantees;
• multimodal content;
• structured output enforcement;
• streaming event details;
• token accounting;
• error payloads;
• provider-specific capabilities.

Anthropic documents several concrete limitations in its OpenAI compatibility layer:

• strict for function calling is ignored;
• response_format, logprobs, and several other fields are ignored;
• prompt caching is not supported through this surface;
• system and developer messages are hoisted and combined;
• the full Claude feature set requires the native Claude API.

Some unsupported fields are silently ignored. That last behavior is more dangerous than a clean error. Your code can compile, your request can return 200, and your assumption can still be false.

Ollama uses equally careful wording: it supports parts of the OpenAI API. vLLM documents its own list of supported and additional parameters. Compatibility is a spectrum, not a boolean.

The migration that “only changes base_url”

Suppose this prototype works:

client = OpenAI(
    api_key=os.environ["GATEWAY_API_KEY"],
    base_url=os.environ["LLM_BASE_URL"],
)

Changing an environment variable may be enough for basic chat completion. Then production adds:

• parallel tool calls;
• strict JSON schemas;
• images or PDFs;
• prompt caching;
• token-level streaming;
• provider-specific safety controls;
• detailed usage accounting.

Now the lowest common denominator begins to cost you. The abstraction was not free. You deferred the translation work to a gateway-and accepted its fidelity limits.

Which approach should you choose?

For a serious multi-provider application, I prefer a small internal interface whose implementation uses native provider SDKs. For example:

from typing import Protocol

class TextModel(Protocol):
    def generate(self, prompt: str) -> str: ...

class ClaudeModel:
    def generate(self, prompt: str) -> str:
        # Native Anthropic SDK implementation
        ...

class OpenAIModel:
    def generate(self, prompt: str) -> str:
        # Native OpenAI SDK implementation
        ...

This is more work than changing base_url, but it makes the lossy parts visible. Your domain code depends on your contract, while provider-specific capabilities remain available inside each adapter.

If you choose a universal gateway instead, create contract tests for every feature you rely on:

• ✅ plain text
• ✅ streaming
• ✅ tool call arguments
• ✅ strict structured output
• ✅ image input
• ✅ stop reasons
• ✅ usage accounting
• ✅ retryable error classification

Do not test only whether the first “Hello” request succeeds.

The mental model to keep

When someone says: “We use the OpenAI SDK.” Translate it to: “This code uses a client designed around an OpenAI API shape.” It does not automatically mean:

• OpenAI hosts the endpoint;
• GPT generated the answer;
• every OpenAI feature is supported;
• prompts never pass through a gateway;
• switching providers is lossless.

Remember the chain:

SDK → API contract → endpoint/router → serving runtime → model

The SDK speaks a protocol. The endpoint receives and may route the request. The serving stack runs-or delegates-the generation work. The model generates token probabilities. Once those responsibilities are separate in your head, “OpenAI SDK calling Claude” stops looking like magic. It becomes what it always was: one client speaking a compatible network contract to an endpoint that knows how to reach Claude.

What broke first when you switched LLM providers: streaming, tool calls, structured output, or usage accounting? Share the failure mode in the comments. Those edge cases are where “compatible” gets interesting.

References

• Anthropic: OpenAI SDK compatibility
• Hugging Face Transformers: Generation
• vLLM: OpenAI-Compatible Server
• Ollama: OpenAI compatibility

Disclosure: This article is based on the author's original technical material and subject-matter knowledge. AI was used to help restructure the article, edit the English, and create the cover image. The technical claims and cited sources were reviewed before publication.