---
title: "Function calling with open models · OpenAI-compatible tool use"
description: "Let open models call your functions and APIs with OpenAI-compatible tool calling. Define tools, run the loop, handle arguments, processed in the EU."
canonical: "https://inference.eu/api/tool-calling/"
language: "en"
---

Guides · Tool calling

# Tool calling: let open models call your functions

Tool calling, also called function calling, lets a model ask your code to do something it cannot do itself: look up an order, query a database, send an email or call another API. You describe the tools, the model decides when to use them and with which arguments, and your code runs them. It is the basis of every agent, and it works in the OpenAI and Anthropic formats.

[Contact sales](https://inference.eu/contact/)[Example request](#example)

1.  01API format: OpenAI and Anthropic
2.  02EU data centres
3.  03Covered by one DPA

tools.py

```
import json, os
from openai import OpenAI

client = OpenAI(base_url="https://api.inference.eu/v1", api_key=os.environ["INFERENCE_EU_API_KEY"])

def get_order_status(order_id: str) -> dict:
    return {"order_id": order_id, "status": "shipped", "carrier": "DHL", "eta": "2026-11-05"}

tools = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Look up the shipping status of a customer order by its id.",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string", "description": "Order id, e.g. A-10293"}},
            "required": ["order_id"],
        },
    },
}]

messages = [{"role": "user", "content": "Where is my order A-10293?"}]
reply = client.chat.completions.create(model="gpt-oss-120b", messages=messages, tools=tools)
msg = reply.choices[0].message

if msg.tool_calls:
    messages.append(msg)
    for call in msg.tool_calls:
        result = get_order_status(**json.loads(call.function.arguments))
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
    reply = client.chat.completions.create(model="gpt-oss-120b", messages=messages, tools=tools)

print(reply.choices[0].message.content)
```

## How tool calling works

You pass a `tools` array with the request. Each tool has `type: "function"` and a `function` object with a `name`, a `description` and `parameters` written as JSON Schema. The model reads these alongside the conversation. If it decides a tool is needed, it does not answer in text; it returns `message.tool_calls`, each with an `id`, the function `name` and `arguments` as a JSON string, and `finish_reason` is `tool_calls`.

The model never runs anything. Your code parses the arguments, calls the real function, and sends the result back as a message with `role: "tool"` and the matching `tool_call_id`. The model then uses the result to answer, or asks for another tool.

## Controlling when tools are used

`tool_choice` controls the decision. `auto`, the default, lets the model choose between answering and calling a tool. `required` forces at least one tool call, which is useful for extraction steps. `none` disables tools for one turn, and naming a specific function forces that function. `parallel_tool_calls` set to `false` limits the model to one call per turn, if your tools must run in order.

Good descriptions matter more than anything else. Say what the tool does, when to use it and what each parameter means, with an example value. Keep the number of tools per request manageable; a model choosing between a handful of clear tools makes fewer mistakes than one choosing between fifty.

## The agent loop in practice

Real tasks often need several rounds: search, then fetch details, then act. Run a loop. Send the request; if the response has tool calls, run them, append the assistant message and each tool result, and send again; stop when the model answers in text or after a maximum number of rounds you choose.

Validate arguments before you act on them, because the model can produce a wrong id or a missing field. Treat tools that change things, such as payments or emails, with care: ask a person to confirm, or restrict them to safe values. Return errors from tools as results with a clear message, so the model can correct itself.

## Formats, models and edge cases

The same tools work through the Responses API, where function calls and outputs are typed items, and through the Anthropic Messages format, where tools use `input_schema` and calls come back as `tool_use` blocks answered with `tool_result`. Choose the format your framework uses.

Not every model is equally good at tool use. Large instruction-tuned models such as `gpt-oss-120b` handle multi-step tool loops well; small models can miss calls or invent arguments. If `arguments` is not valid JSON, send an error back and let the model retry, or combine tools with `strict` schemas. Tool definitions count as input tokens on every request.

## Tool calling

Category

Guides

API format

OpenAI and Anthropic

Base URL

api.inference.eu/v1

Processing

EU data centres

Training on your data

Never

## Tool calling: common questions

The questions buyers, developers and legal teams ask us most. Something missing? Ask us directly.

Yes. Most current instruction-tuned open models, including \`gpt-oss-120b\`, are trained for tool use, and the inference.eu API exposes it in the standard OpenAI and Anthropic formats. Each model page notes whether tool calling is supported. Quality varies, so test your own tools before choosing.

Yes. \`tools\`, \`tool\_choice\`, \`parallel\_tool\_calls\`, \`message.tool\_calls\` and \`role: "tool"\` messages work exactly as in the OpenAI chat completions API. Frameworks that build on it, such as LangChain, LlamaIndex, the Vercel AI SDK and the OpenAI Agents SDK, work without changes.

No. The model only returns the function name and arguments. Your code decides whether to run the function, runs it, and sends back the result. That gives you full control over what happens and which data leaves your systems.

Set \`tool\_choice\` to \`required\` to force some tool call, or to \`{"type": "function", "function": {"name": "get\_order\_status"}}\` to force a specific one. Use \`none\` to prevent tool calls on a turn where you only want text.

Yes. A single response can contain several entries in \`tool\_calls\`, for example looking up three orders in parallel. Run them, possibly concurrently, and append one \`tool\` message per call with the matching \`tool\_call\_id\`. Set \`parallel\_tool\_calls\` to \`false\` if calls must happen one at a time.

Parse \`arguments\` with a JSON parser and validate it against your schema. If it fails, return a \`tool\` message that explains the problem, such as a missing field, and let the model try again. Using \`strict: true\` in the function definition makes arguments follow the schema more reliably.

With tool calling you define each function and run it in your own code. With MCP, the tools live on an MCP server, and the API can call that server for you when you add it to a Responses request. Both can be combined in one request.

Yes. In \`/v1/messages\`, define tools with \`name\`, \`description\` and \`input\_schema\`. The model replies with \`tool\_use\` blocks and \`stop\_reason: "tool\_use"\`; you answer with \`tool\_result\` blocks. Coding agents built for Anthropic use exactly this loop.

[Ask a question](https://inference.eu/contact/)

## Related

[API overview](https://inference.eu/api/)

-   [Structured outputsGet valid JSON that matches your schema from open models with OpenAI-compatible structured outputs. JSON mode, json\_schema, strict mode and pitfalls.Guides](https://inference.eu/api/structured-outputs/)
-   [Remote MCP serversConnect remote MCP servers to model requests with the OpenAI Responses mcp tool. The API lists and calls the server's tools for you, processed in the EU.Tools](https://inference.eu/api/mcp/)
-   [StreamingStream tokens from open models as they are generated, with OpenAI and Anthropic server-sent events. Examples, chunk format and error handling.Guides](https://inference.eu/api/streaming/)
-   [ResponsesUse the OpenAI Responses API format with open-weight models on inference.eu. Typed output items, built-in tools and multi-turn state, processed in the EU.Endpoints](https://inference.eu/api/responses/)

## Move your AI workloads to EU infrastructure

Tell us which models you use and roughly how much traffic you have. We reply within one working day with pricing and next steps.

-   Reply within one working day
-   Pricing for your volume
-   DPA ready to sign