How do I add an OpenAI-compatible model to Dify?

Install the OpenAI-API-compatible plugin under Settings > Model Provider, click Add Model, and enter the model type, model name, API key and API endpoint URL. Set the completion mode, context size and capabilities, then save.

What should I enter as API endpoint URL for inference.eu?

For LLM models, enter the inference.eu base URL ending in `/v1`. Dify appends `/chat/completions` itself, so do not add any further path. The plugin documentation notes that some non-LLM model types expect the URL without the version suffix.

Which inference.eu model is best for Dify agents?

Kimi K3 and GLM-5.3 are strong choices for agents and workflows that call tools; enable Tool Call when you add them. gpt-oss-120b is a fast choice for chatbots and classification nodes.

Why does my Dify agent not call tools?

Dify only uses native function calling if the model is configured with Function calling set to Tool Call. Edit the model in Model Provider settings, enable it, and choose a model with dependable tool support such as Kimi K3 or GLM-5.3.

Why are answers cut off in Dify?

The upper bound for max tokens, or the max tokens set in the app, is too low. Raise both to suit the model and your use case. Check the model context size as well, since Dify uses it to decide how much history and knowledge to send.

Can I use inference.eu in Dify workflows?

Yes. Once added, inference.eu models appear in every LLM, question classifier, parameter extractor and agent node. Each node can use its own model, so you can mix fast and stronger models in one workflow.

Do I have to add each inference.eu model separately?

Yes. The OpenAI-API-compatible plugin adds models one at a time, each with its own name, context size and capabilities. Add the models your teams need, for example gpt-oss-120b, GLM-5.3 and Qwen3 Coder 480B.

Does this work in self-hosted Dify?

Yes. Install the plugin from the marketplace in your self-hosted instance and add models the same way. The plugin daemon needs outbound network access to the inference.eu base URL, so check firewall and proxy settings.

Does Dify Cloud keep my data in the EU?

Model requests sent to inference.eu are processed in EU data centres. Dify Cloud stores apps, logs and conversations on Dify's own infrastructure under its terms. Self-host Dify in the EU if you need full control over that data.

Can I use inference.eu for embeddings in Dify knowledge bases?

This page covers language models. Knowledge bases use separate embedding and rerank models, configured under Model Provider. Choose EU-hosted or self-hosted options for them so that documents stay in Europe alongside your model requests.

Dify with EU-hosted models · OpenAI-compatible setup

Set it up in a few minutes

1
Create an API key in your inference.eu account.
2
In Dify, open Settings > Model Provider and install the OpenAI-API-compatible plugin from the marketplace.
3
Click Add Model, choose LLM, and enter the inference.eu model id, your API key and the inference.eu base URL as API endpoint URL.
4
Set Completion mode to Chat, enter the context size and max tokens, and enable Tool Call if the model supports function calling.
5
Select the model in a chatbot, agent or workflow LLM node and run a test conversation.

yaml

# Settings > Model Provider > OpenAI-API-compatible > Add Model
# Values to enter in the form, one entry per model:
Model Type: LLM
Model Name: glm-5.3
Model display name: GLM-5.3 (inference.eu)
API Key: <your INFERENCE_EU_API_KEY>
API endpoint URL: https://api.inference.eu/v1
Completion mode: Chat
Model context size: 131072
Upper bound for max tokens: 8192
Function calling: Tool Call
Stream function calling: Supported
Vision support: Not supported

What Dify is

Dify is an open-source platform for building language model applications. In a visual editor you create chatbots, agents with tools, text generators and multi-step workflows, then publish them as web apps or APIs. It includes knowledge bases with document retrieval, prompt management, logs and annotations, and runs as Dify Cloud or as a self-hosted Docker deployment.

Models are added through model provider plugins. The official OpenAI-API-compatible plugin, maintained by the Dify team, connects to any service that implements the OpenAI API. That is how you make inference.eu models available to every app in your workspace.

How the configuration works

After installing the plugin, add each model separately. Choose LLM as model type, enter the inference.eu model id as Model Name, your key as API Key, and the inference.eu base URL ending in /v1 as API endpoint URL. Set Completion mode to Chat.

The remaining fields tell Dify what the model can do. Model context size and the upper bound for max tokens control how much text Dify sends and requests. Set Function calling to Tool Call for models that support tools, so Dify agents use native function calling. Once saved, the model appears in every model selector in the workspace.

Which models to choose

For chatbots, text generation and classification nodes, gpt-oss-120b is a fast default that keeps apps responsive. For agents and workflows that call tools, add kimi-k3 or glm-5.3 with Tool Call enabled. Both handle multi-step reasoning and long context well.

Knowledge base apps that answer from long documents benefit from glm-5.3 or kimi-k3 with a large context size. For apps that generate or review code, add qwen3-coder-480b-a35b-instruct. You can add several inference.eu models and choose a different one in each workflow node, for example a fast model for routing and a stronger one for the answer.

Troubleshooting

If saving the model fails with 401, the API key is wrong. A 404 usually means the endpoint URL is missing /v1 for LLM models or has an extra path. "Model not found" means the Model Name differs from the inference.eu id.

If an agent describes tool calls in text instead of running them, set Function calling to Tool Call and choose a model with dependable tool support. If answers are cut off, raise the upper bound for max tokens. If knowledge base answers miss content, check the context size matches the model. Self-hosted Dify needs outbound network access from the plugin daemon.

Keeping data in the EU

With inference.eu as the model provider, the prompts, retrieved knowledge and tool results your Dify apps send to the model are processed in EU data centres and never used for training. We sign a data processing agreement under Article 28 GDPR, publish our subprocessor list and bill in euros.

Dify stores apps, conversations, logs and knowledge bases in its own database and storage. Self-host Dify in the EU for full control, or check where your Dify Cloud workspace is hosted. Embedding and rerank models for knowledge bases are configured separately, so choose EU-hosted options for them too.

Use Dify with open-source models hosted in the EU