Do open models support function calling?

Yes. Most current instruction-tuned open models, including `gpt-oss-120b`, are trained for tool use, and the inference.eu API exposes it in the standard OpenAI and Anthropic formats. Each model page notes whether tool calling is supported. Quality varies, so test your own tools before choosing.

Is tool calling compatible with the OpenAI format?

Yes. `tools`, `tool_choice`, `parallel_tool_calls`, `message.tool_calls` and `role: "tool"` messages work exactly as in the OpenAI chat completions API. Frameworks that build on it, such as LangChain, LlamaIndex, the Vercel AI SDK and the OpenAI Agents SDK, work without changes.

Does the model run my functions?

No. The model only returns the function name and arguments. Your code decides whether to run the function, runs it, and sends back the result. That gives you full control over what happens and which data leaves your systems.

How do I force the model to call a tool?

Set `tool_choice` to `required` to force some tool call, or to `{"type": "function", "function": {"name": "get_order_status"}}` to force a specific one. Use `none` to prevent tool calls on a turn where you only want text.

Can the model call several tools at once?

Yes. A single response can contain several entries in `tool_calls`, for example looking up three orders in parallel. Run them, possibly concurrently, and append one `tool` message per call with the matching `tool_call_id`. Set `parallel_tool_calls` to `false` if calls must happen one at a time.

What if the model returns invalid arguments?

Parse `arguments` with a JSON parser and validate it against your schema. If it fails, return a `tool` message that explains the problem, such as a missing field, and let the model try again. Using `strict: true` in the function definition makes arguments follow the schema more reliably.

How is tool calling different from MCP?

With tool calling you define each function and run it in your own code. With MCP, the tools live on an MCP server, and the API can call that server for you when you add it to a Responses request. Both can be combined in one request.

Does tool calling work with the Anthropic format?

Yes. In `/v1/messages`, define tools with `name`, `description` and `input_schema`. The model replies with `tool_use` blocks and `stop_reason: "tool_use"`; you answer with `tool_result` blocks. Coding agents built for Anthropic use exactly this loop.

Function calling with open models · OpenAI-compatible tool use

tools.py

import json, os
from openai import OpenAI

client = OpenAI(base_url="https://api.inference.eu/v1", api_key=os.environ["INFERENCE_EU_API_KEY"])

def get_order_status(order_id: str) -> dict:
    return {"order_id": order_id, "status": "shipped", "carrier": "DHL", "eta": "2026-11-05"}

tools = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Look up the shipping status of a customer order by its id.",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string", "description": "Order id, e.g. A-10293"}},
            "required": ["order_id"],
        },
    },
}]

messages = [{"role": "user", "content": "Where is my order A-10293?"}]
reply = client.chat.completions.create(model="gpt-oss-120b", messages=messages, tools=tools)
msg = reply.choices[0].message

if msg.tool_calls:
    messages.append(msg)
    for call in msg.tool_calls:
        result = get_order_status(**json.loads(call.function.arguments))
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
    reply = client.chat.completions.create(model="gpt-oss-120b", messages=messages, tools=tools)

print(reply.choices[0].message.content)

How tool calling works

You pass a tools array with the request. Each tool has type: "function" and a function object with a name, a description and parameters written as JSON Schema. The model reads these alongside the conversation. If it decides a tool is needed, it does not answer in text; it returns message.tool_calls, each with an id, the function name and arguments as a JSON string, and finish_reason is tool_calls.

The model never runs anything. Your code parses the arguments, calls the real function, and sends the result back as a message with role: "tool" and the matching tool_call_id. The model then uses the result to answer, or asks for another tool.

Controlling when tools are used

tool_choice controls the decision. auto, the default, lets the model choose between answering and calling a tool. required forces at least one tool call, which is useful for extraction steps. none disables tools for one turn, and naming a specific function forces that function. parallel_tool_calls set to false limits the model to one call per turn, if your tools must run in order.

Good descriptions matter more than anything else. Say what the tool does, when to use it and what each parameter means, with an example value. Keep the number of tools per request manageable; a model choosing between a handful of clear tools makes fewer mistakes than one choosing between fifty.

The agent loop in practice

Real tasks often need several rounds: search, then fetch details, then act. Run a loop. Send the request; if the response has tool calls, run them, append the assistant message and each tool result, and send again; stop when the model answers in text or after a maximum number of rounds you choose.

Validate arguments before you act on them, because the model can produce a wrong id or a missing field. Treat tools that change things, such as payments or emails, with care: ask a person to confirm, or restrict them to safe values. Return errors from tools as results with a clear message, so the model can correct itself.

Formats, models and edge cases

The same tools work through the Responses API, where function calls and outputs are typed items, and through the Anthropic Messages format, where tools use input_schema and calls come back as tool_use blocks answered with tool_result. Choose the format your framework uses.

Not every model is equally good at tool use. Large instruction-tuned models such as gpt-oss-120b handle multi-step tool loops well; small models can miss calls or invent arguments. If arguments is not valid JSON, send an error back and let the model retry, or combine tools with strict schemas. Tool definitions count as input tokens on every request.

Tool calling: let open models call your functions

How tool calling works

Controlling when tools are used

The agent loop in practice

Formats, models and edge cases

Tool calling: common questions

Related

Move your AI workloads to EU infrastructure