Use the Vercel AI SDK with open-source models hosted in the EU

The AI SDK is an open-source TypeScript library from Vercel. It gives you one API for text generation, streaming, tool calls, structured output and agents across many model providers, plus UI hooks such as useChat for React, Svelte and Vue. It runs in Node.js, serverless functions and edge runtimes.

Providers are separate packages. The @ai-sdk/openai-compatible package works with any API that implements the OpenAI format, which is the recommended way to connect inference.eu. You keep the same AI SDK calls and swap only the provider instance.

createOpenAICompatible returns a provider function. name labels it in errors and telemetry, baseURL is the inference.eu base URL ending in /v1, and apiKey is sent as a bearer token. includeUsage: true asks for token counts in streamed responses so that usage is filled in after streamText finishes.

Call the provider with a model id to get a model, for example inferenceEu("qwen3-coder-480b-a35b-instruct"). Keep the provider in a server-only module: the key must never reach the browser. Route handlers, server actions and API routes then import it and stream results to the client through useChat or your own UI.

For chat features, summaries and extraction, gpt-oss-120b is a fast default that keeps streaming responsive in the UI. For agents built with tools and stopWhen, choose a model with dependable tool calling such as qwen3-coder-480b-a35b-instruct, glm-5.3 or kimi-k3.

glm-5.3 and kimi-k3 also suit long documents and multi-step reasoning. If you use generateObject with a Zod schema, set supportsStructuredOutputs: true on the provider only after testing that the chosen model returns valid JSON reliably; otherwise the SDK falls back to instructing the model and validates the result.

A 401 error means INFERENCE_EU_API_KEY is not set in the runtime that executes the call. Serverless and edge deployments need the variable configured in the hosting environment, not only in .env.local. A 404 usually means baseURL has an extra path.

"Model not found" means the id passed to the provider differs from the inference.eu id. If usage is empty after streaming, add includeUsage: true. If tool calls are ignored, check the model supports tools. Do not use the @ai-sdk/openai package for this: it targets OpenAI's Responses API by default, while the compatible provider uses Chat Completions.

With the provider pointed at inference.eu, every prompt and completion your app sends is processed in EU data centres and never used for training. We sign a data processing agreement under Article 28 GDPR, publish our subprocessor list and bill in euros.

The AI SDK itself only calls the provider you configure. Your hosting region matters too: run route handlers and functions in an EU region so that prompts are not handled by servers elsewhere before they reach us. If you enable telemetry with OpenTelemetry, check where your tracing backend stores prompt and response attributes.