Can I use the OpenAI Python SDK with a different provider?

Yes. Pass `base_url` and `api_key` to `OpenAI()` and the SDK sends all requests to that endpoint. Any provider that implements the OpenAI API format, including inference.eu, works with the same methods and response types.

How do I change the base URL in the OpenAI Node.js SDK?

Pass `baseURL` when you create the client, for example `new OpenAI({ baseURL: process.env.INFERENCE_EU_BASE_URL, apiKey: process.env.INFERENCE_EU_API_KEY })`. The same option works in Node.js, Deno, Bun and edge runtimes, and in TypeScript projects.

Does OPENAI_BASE_URL work instead of passing base_url?

Yes. Both SDKs read `OPENAI_BASE_URL` and `OPENAI_API_KEY` when no explicit values are given. This is useful for tools you cannot modify. In your own code, explicit arguments make it obvious which provider each client uses.

Which inference.eu model should I use with the OpenAI SDK?

Start with `gpt-oss-120b` for general tasks, `qwen3-coder-480b-a35b-instruct` for code and tool calling, and `glm-5.3` or `kimi-k3` for harder reasoning. Call `client.models.list()` to see every model id available to your key.

Does streaming work with inference.eu?

Yes. Pass `stream=True` in Python or `stream: true` in JavaScript and iterate over the chunks as usual. Add `stream_options={"include_usage": True}` if you need token counts, which arrive in the final chunk.

Does function calling work through the OpenAI SDK?

Yes, for models that support tool calling. Pass `tools` with JSON Schema definitions and read `tool_calls` from the response as you would with OpenAI. Qwen3 Coder 480B, GLM-5.3 and Kimi K3 handle tool calls well.

Can I use structured outputs with inference.eu?

You can pass `response_format` with a JSON schema on Chat Completions. Support for strict schema enforcement depends on the model, so validate the parsed result in your code, for example with Pydantic, and retry if a field is missing.

Why do I get a 404 Not Found error?

Either the base URL is wrong or the model id is unknown. The base URL must end in `/v1` with nothing after it, because the SDK adds `/chat/completions`. Then compare the model id with `client.models.list()`.

Does the OpenAI SDK send data to OpenAI if I change the base URL?

No. The SDK sends requests only to the base URL you configure and has no built-in telemetry. Make sure every client in your codebase uses the inference.eu base URL, including those created by libraries that read `OPENAI_BASE_URL`.

Do I need a separate SDK for inference.eu?

No. The official `openai` package for Python or JavaScript is all you need. If you prefer the Anthropic format, the official Anthropic SDK also works with inference.eu, so you can use whichever client your code already depends on.

SDKs · OpenAI SDK (Python and JavaScript)

Use the OpenAI SDK with open-source models hosted in the EU

The official OpenAI SDKs for Python and JavaScript work with any OpenAI-compatible API. Set the base URL to inference.eu and your existing code calls open-weight models processed in European data centres, with the same methods, types, streaming and tool calling you already use.

Contact sales Setup

01API format: OpenAI
02EU data centres
03Covered by one DPA

Set it up in a few minutes

1
Create an API key in your inference.eu account and store it as INFERENCE_EU_API_KEY.
2
Install the SDK with pip install openai or npm install openai.
3
Create the client with base_url (Python) or baseURL (JavaScript) set to the inference.eu base URL and api_key set from the environment.
4
Replace the model name with an inference.eu model id such as gpt-oss-120b.
5
Run a short chat completion to confirm the connection, then enable streaming or tools as needed.

python

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inference.eu/v1",
    api_key=os.environ["INFERENCE_EU_API_KEY"],
)

response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "Summarise GDPR Article 28 in two sentences."}],
)
print(response.choices[0].message.content)

# JavaScript / TypeScript:
# const client = new OpenAI({ baseURL: "https://api.inference.eu/v1", apiKey: process.env.INFERENCE_EU_API_KEY });

Why use the OpenAI SDK with inference.eu

The openai packages are the most widely used client libraries for language model APIs, and many other tools build on them. They handle retries, timeouts, streaming, typed responses and errors for you. Because inference.eu speaks the OpenAI API format, you keep all of that and only change where requests go.

This is often the quickest way to move an existing application to open-weight models in the EU: two constructor arguments and a model name change, with no new dependency and no change to the rest of your code.

How the configuration works

In Python, pass base_url and api_key to OpenAI(...) or AsyncOpenAI(...). In JavaScript and TypeScript, pass baseURL and apiKey to new OpenAI({...}). The base URL ends in /v1; the SDK appends paths such as /chat/completions itself.

Both SDKs also read OPENAI_BASE_URL and OPENAI_API_KEY from the environment when you do not pass them explicitly. That is convenient for tools you cannot edit, but explicit arguments with INFERENCE_EU_API_KEY are clearer in your own code and avoid mixing up keys when a project talks to more than one provider.

Which models to choose

Use client.models.list() to see the model ids available to your key. For general chat, extraction and summarisation, gpt-oss-120b is a fast, capable default. For code generation and agents that call tools, qwen3-coder-480b-a35b-instruct is a strong choice.

For complex reasoning, long documents or multi-step agent workflows, try glm-5.3 or kimi-k3. Because only the model string changes, you can compare models on your own prompts by running the same request against several ids and checking quality, latency and token usage in the responses.

Troubleshooting

A 401 AuthenticationError means the key is missing or wrong; check that you read INFERENCE_EU_API_KEY and not an old OPENAI_API_KEY. A 404 NotFoundError usually means the base URL is missing /v1 or has an extra path, or the model id does not exist.

Use Chat Completions, client.chat.completions.create, as the most widely supported path. For streaming usage, pass stream_options={"include_usage": True} to receive token counts in the last chunk. If tool calls come back as plain text, check the model supports tool calling and that tools follows the JSON Schema format.

Keeping data in the EU

With the base URL set to inference.eu, every request your application sends through the SDK is processed in EU data centres. Prompts and completions are never used for training. We sign a data processing agreement under Article 28 GDPR, publish our subprocessor list and bill in euros.

The SDK itself only talks to the base URL you configure; it sends no telemetry to OpenAI. Check the rest of your stack: logging, tracing and evaluation services may store prompts too. If you set OPENAI_BASE_URL globally, make sure no other code path still creates a client with OpenAI's default URL.

OpenAI SDK (Python and JavaScript): common questions

The questions buyers, developers and legal teams ask us most. Something missing? Ask us directly.

Ask a question

All integrations

Move your AI workloads to EU infrastructure

Tell us which models you use and roughly how much traffic you have. We reply within one working day with pricing and next steps.

Reply within one working day
Pricing for your volume
DPA ready to sign