---
title: "Structured outputs and JSON schema · OpenAI-compatible, EU"
description: "Get valid JSON that matches your schema from open models with OpenAI-compatible structured outputs. JSON mode, json_schema, strict mode and pitfalls."
canonical: "https://inference.eu/api/structured-outputs/"
language: "en"
---

Guides · Structured outputs

# Structured outputs: JSON that matches your schema

Structured outputs make a model answer with JSON that follows a schema you define, instead of free text you have to parse. Use them whenever a program reads the result: extracting fields from documents, classifying tickets, routing requests or filling a form. The request format is the OpenAI \`response\_format\` field, and the output is constrained while it is generated.

[Contact sales](https://inference.eu/contact/)[Example request](#example)

1.  01API format: OpenAI
2.  02EU data centres
3.  03Covered by one DPA

extract\_invoice.py

```
import json, os
from openai import OpenAI

client = OpenAI(base_url="https://api.inference.eu/v1", api_key=os.environ["INFERENCE_EU_API_KEY"])

schema = {
    "type": "object",
    "properties": {
        "supplier": {"type": "string"},
        "invoice_number": {"type": "string"},
        "issue_date": {"type": "string", "description": "ISO 8601 date"},
        "currency": {"type": "string", "enum": ["EUR", "GBP", "CHF", "PLN", "SEK"]},
        "total": {"type": "number"},
        "vat_id": {"type": ["string", "null"]},
    },
    "required": ["supplier", "invoice_number", "issue_date", "currency", "total", "vat_id"],
    "additionalProperties": False,
}

reply = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[
        {"role": "system", "content": "Extract invoice fields. Use null when a field is missing."},
        {"role": "user", "content": open("invoice.md").read()},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "invoice", "schema": schema, "strict": True},
    },
)
invoice = json.loads(reply.choices[0].message.content)
```

## How structured outputs work

Add `response_format` with `type: "json_schema"` and a `json_schema` object containing a `name`, your `schema` written as JSON Schema, and `strict: true`. While the model generates, the API restricts each token to those that keep the output valid against the schema. The result in `message.content` is a JSON string you can parse directly.

This is more reliable than asking for JSON in the prompt. The model cannot add a sentence before the opening brace, leave out a required field or invent a property you did not define. You still describe the task in the prompt; the schema controls the shape, the prompt controls the content.

## JSON mode and schema design

`response_format: {"type": "json_object"}` is the looser option, called JSON mode. The output is guaranteed to be valid JSON, but not to follow any particular structure, so mention the expected fields in the prompt. Prefer `json_schema` whenever you know the shape.

Keep schemas simple and explicit. List every property in `required` and set `additionalProperties` to `false`, which strict mode expects. Represent optional values as a type that allows `null`, as with `vat_id` in the example, so the model can say "not found" instead of guessing. Use `enum` for fixed categories, and add a `description` to fields whose meaning is not obvious from the name.

## A practical extraction pipeline

A common pattern turns documents into data. Convert each PDF to Markdown with the document conversion tool, send the text with a short system prompt and your schema, and parse the result. Validate the parsed object with the same schema in your code, then check business rules the schema cannot express, such as totals adding up.

For thousands of documents, put the same requests into a batch job. Keep a field for the model's confidence or a short `notes` string if reviewers need to know why a value was chosen. With the Responses API, the same schema goes into `text.format` instead of `response_format`.

## Errors and edge cases

Very large or deeply nested schemas, and unusual keywords such as complex `pattern` expressions or `oneOf` with overlapping branches, may be rejected with a 400 error that names the unsupported part. Split such schemas or simplify them.

The schema guarantees the format, not the facts. A model can still put a wrong date in a valid date field, so check important values. If generation stops at `max_tokens`, the JSON is cut off and will not parse; check `finish_reason` and raise the limit. Reasoning models think before they write the JSON, so leave room for both. Refusals for unsafe requests are returned as text, not as schema output.

## Structured outputs

Category

Guides

API format

OpenAI

Base URL

api.inference.eu/v1

Processing

EU data centres

Training on your data

Never

## Structured outputs: common questions

The questions buyers, developers and legal teams ask us most. Something missing? Ask us directly.

Pass \`response\_format\` with \`type: "json\_schema"\`, a \`name\`, your JSON Schema and \`strict: true\` in a chat completions request. The model's reply in \`message.content\` is then JSON that matches the schema. For a looser guarantee, use \`type: "json\_object"\` and describe the fields in the prompt.

Yes. \`response\_format\` with \`json\_schema\` and \`json\_object\` follows the OpenAI chat completions API, and \`text.format\` follows the Responses API. SDK helpers that build the schema from a Pydantic or Zod model, such as \`client.chat.completions.parse\` in Python, work when pointed at inference.eu.

JSON mode guarantees that the output is valid JSON but not which keys it contains. Structured outputs with a \`json\_schema\` guarantee that the output follows your schema: required fields present, types correct and no extra properties. Use structured outputs whenever you know the shape you need.

Objects, arrays, strings, numbers, integers, booleans, \`null\`, \`enum\`, \`required\`, nested objects, \`additionalProperties: false\` and descriptions all work. Very complex constructs can be rejected with a 400 error that names the problem. Keep schemas flat and explicit for the most reliable results.

Yes. In Python, define a Pydantic model and pass it as \`response\_format\` to \`client.chat.completions.parse\`. The SDK converts it to a JSON Schema, sends it, and returns a parsed object in \`message.parsed\`. Zod does the same in TypeScript with the SDK's helper.

The model reached \`max\_tokens\` before it finished, and \`finish\_reason\` will be \`length\`. Raise \`max\_tokens\`, shorten long text fields in the schema, or extract fewer items per request. Reasoning models need extra room because their thinking also counts towards the limit.

Yes. Tools can use \`strict: true\` in their function definitions, so their arguments follow the parameter schema, and the final answer can use \`response\_format\`. A common setup lets the model call lookup tools first and then return a structured summary.

No. It guarantees that the answer has the right shape, not that every value is true. A model can still misread a document or guess a value. Allow \`null\` for missing data, validate important fields in code, and review a sample of results by hand.

[Ask a question](https://inference.eu/contact/)

## Related

[API overview](https://inference.eu/api/)

-   [Tool callingLet open models call your functions and APIs with OpenAI-compatible tool calling. Define tools, run the loop, handle arguments, processed in the EU.Guides](https://inference.eu/api/tool-calling/)
-   [Document conversionConvert PDF, Word, PowerPoint and Excel files to clean Markdown for RAG and LLM prompts. Tables, headings and scanned pages, processed in the EU.Tools](https://inference.eu/api/document-conversion/)
-   [Chat completionsSend chat messages to open-weight models with the OpenAI chat completions format. Requests are processed in EU data centres and never used for training.Endpoints](https://inference.eu/api/chat-completions/)
-   [BatchesRun thousands of chat or embedding requests as one asynchronous job with an OpenAI-compatible batch API. Upload JSONL, collect results, all in the EU.Endpoints](https://inference.eu/api/batches/)

## Move your AI workloads to EU infrastructure

Tell us which models you use and roughly how much traffic you have. We reply within one working day with pricing and next steps.

-   Reply within one working day
-   Pricing for your volume
-   DPA ready to sign