Migrating from OpenAI to the inference.eu API: a practical guide

Point the client at the inference.eu base URL, use your inference.eu key, and change the model name. In the OpenAI Python and JavaScript SDKs that means passing base_url (or baseURL) and api_key when you create the client, or setting the OPENAI_BASE_URL and OPENAI_API_KEY environment variables, which the SDKs read automatically.

Everything else stays: the same SDK version, the same client.chat.completions.create and client.responses.create calls, the same streaming loop and the same error classes. Frameworks such as LangChain, LlamaIndex, the Vercel AI SDK and the OpenAI Agents SDK take the same two settings. Keep the model id in configuration, so the migration is a deployment setting you can roll back.

Map each use to a model rather than looking for one model that replaces everything. For general chat and agent work, a large open model such as gpt-oss-120b is a common starting point. For high-volume classification or extraction, a smaller and faster model often does the job. For embeddings, qwen3-embedding-8b covers multilingual text, and whisper-large-v3-turbo replaces OpenAI's transcription models.

Embeddings are the one place where you cannot mix: vectors from OpenAI models are not comparable with vectors from open models, so re-embed your collection with the new model before you switch search over. Run the batch API for that, then swap the index in one step.

Build a small evaluation set from real traffic: 50 to 200 prompts with the answers you consider good. Run it against the old and the new setup and compare quality, length and format, not just whether the request succeeds.

Check the features you rely on. Tool calling should produce the same function names and valid arguments. Structured outputs should pass your JSON schema. Streaming should end with a final chunk and usage. Vision prompts need a vision-capable model. Prompts tuned heavily for one model sometimes need small edits, such as a clearer instruction about output format or language, to get the best from another.

Move traffic gradually. Send a small share of requests to inference.eu, compare error rates and user feedback, then increase. Because the request format is the same, a feature flag that chooses the base URL and model is usually enough.

Some OpenAI-only features have no direct equivalent: OpenAI's own model names, its hosted assistants and vector stores, and settings tied to specific OpenAI models. Fields a model does not support are ignored or rejected with a clear 400 error, so watch your logs during the first days. Check rate limits for your plan before moving peak traffic, and ask us to raise them in good time.