--- title: "API rate limits and error codes · handling 429 and retries" description: "How rate limits and errors work on the inference.eu API. HTTP status codes, the error object, rate-limit headers and a retry pattern that holds up." canonical: "https://inference.eu/api/rate-limits-and-errors/" language: "en" --- Guides · Rate limits and errors # Rate limits and errors: status codes, headers and retries Every production integration meets errors sooner or later: a malformed request, an expired key, a burst of traffic above your limit or a brief interruption. This guide explains how the inference.eu API reports each case, which errors are worth retrying, how to read rate-limit headers, and how to build a retry loop that copes with all of them. [Contact sales](https://inference.eu/contact/)[Example request](#example) 1. 01API format: OpenAI and Anthropic 2. 02EU data centres 3. 03Covered by one DPA retry.py ``` import os, random, time import openai from openai import OpenAI # The SDK already retries some errors; max_retries=0 here so the loop below is explicit. client = OpenAI(base_url="https://api.inference.eu/v1", api_key=os.environ["INFERENCE_EU_API_KEY"], max_retries=0) def complete(messages, attempts=6): for attempt in range(attempts): try: return client.chat.completions.create(model="gpt-oss-120b", messages=messages) except (openai.RateLimitError, openai.InternalServerError, openai.APIConnectionError) as err: if attempt == attempts - 1: raise retry_after = getattr(getattr(err, "response", None), "headers", {}).get("retry-after") delay = float(retry_after) if retry_after else min(2 ** attempt, 30) + random.random() time.sleep(delay) except openai.BadRequestError as err: # 400s will fail again unchanged: log and fix the request instead of retrying print("Invalid request:", err.message) raise ``` ## Status codes and the error object Errors use standard HTTP status codes and, on OpenAI-format endpoints, a JSON body with an `error` object containing `message`, `type`, `param` and `code`. The message is written for developers and names the field or value at fault. Anthropic-format endpoints return `{"type": "error", "error": {"type", "message"}}` instead. The codes you will see: 400 for an invalid request, such as a bad field or a prompt over the context window; 401 for a missing or invalid key; 403 when the key is not allowed to use a model or feature; 404 for an unknown model or object; 413 for a body that is too large; 422 for a valid request that cannot be processed; 429 for rate limits; and 500, 502, 503 or 529 for temporary server-side problems. ## How rate limits work Limits are applied per organisation or project and per model, and usually cover both requests per minute and tokens per minute. The values depend on your plan, and we agree them with you rather than publishing fixed tiers. If you expect a launch or a large migration, tell us in advance and we will raise them. Responses carry headers that show where you stand: `x-ratelimit-limit-requests`, `x-ratelimit-remaining-requests` and `x-ratelimit-reset-requests`, with matching `-tokens` headers. When you go over a limit, the API returns 429 with a `Retry-After` header giving the number of seconds to wait. Read these headers to slow down before you hit the limit rather than after. ## A retry pattern that holds up Retry 429, 5xx and connection errors; do not retry 400, 401, 403 or 404, because the same request will fail again. Wait before each retry: use `Retry-After` when present, otherwise exponential backoff with random jitter, as in the example, so that many clients do not retry at the same moment. Cap the number of attempts and the total wait. The OpenAI and Anthropic SDKs already retry some errors by default; either rely on that and set `max_retries`, or disable it and use your own loop, but not both. For large volumes, use a queue with a fixed concurrency instead of firing requests in parallel, or move the work into a batch. ## Edge cases worth handling A 400 for context length is common with long documents. Count tokens first, trim history or split the input. A successful response can still be incomplete: check `finish_reason` or `status` before using the result. During streaming, an error can arrive mid-stream as an event; treat partial output as unfinished. Set client timeouts long enough for reasoning models and large outputs, and make retried operations safe to repeat, for example by deduplicating on your own request id. Log the request id returned in the response headers with every error, so that when you contact us we can find the exact request. ## Rate limits and errors Category Guides API format OpenAI and Anthropic Base URL api.inference.eu/v1 Processing EU data centres Training on your data Never ## Rate limits and errors: common questions The questions buyers, developers and legal teams ask us most. Something missing? Ask us directly. It means you have exceeded a rate limit, either requests or tokens per minute, for your organisation, project or model. Wait for the number of seconds in the \`Retry-After\` header and try again. If you hit 429 often under normal load, ask us to review the limits on your plan. Rate limits depend on your plan and the models you use, and we agree them with you instead of publishing fixed tiers. Limits can be raised for production traffic and planned peaks. The \`x-ratelimit-\*\` response headers show your current limit and remaining capacity at any time. Retry 429, 500, 502, 503 and 529 responses and network errors such as timeouts and dropped connections, with exponential backoff. Do not retry 400, 401, 403, 404 or 422 unchanged, because they describe a problem with the request, the key or permissions that a retry will not fix. On OpenAI-format endpoints, the body is \`{"error": {"message": "...", "type": "...", "param": "...", "code": "..."}}\`. On the Anthropic Messages endpoints it is \`{"type": "error", "error": {"type": "...", "message": "..."}}\`. The SDKs turn these into typed exceptions such as \`RateLimitError\` or \`BadRequestError\`. The prompt plus \`max\_tokens\` is larger than the model's context window. The error message says so. Count tokens before sending, shorten or summarise earlier conversation turns, split long documents into chunks, or choose a model with a longer context. Read the \`x-ratelimit-remaining-\*\` headers and slow down as they approach zero. Limit concurrency with a queue, spread non-urgent work over time, use batches for bulk jobs, and cache results you request repeatedly. Tell us before a launch so we can raise your limits in time. Check that the key is sent correctly: as \`Authorization: Bearer\` on OpenAI-format endpoints, or as \`x-api-key\` on Anthropic-format endpoints. Make sure there are no extra spaces or quotes, that the environment variable is set where the code runs, and that the key has not been revoked. Include the time of the request, the model, the HTTP status, the error message and the request id from the response headers. Do not send your API key or the full content of sensitive prompts. With the request id we can trace what happened on our side. [Ask a question](https://inference.eu/contact/) ## Related [API overview](https://inference.eu/api/) - [Chat completionsSend chat messages to open-weight models with the OpenAI chat completions format. Requests are processed in EU data centres and never used for training.Endpoints](https://inference.eu/api/chat-completions/) - [BatchesRun thousands of chat or embedding requests as one asynchronous job with an OpenAI-compatible batch API. Upload JSONL, collect results, all in the EU.Endpoints](https://inference.eu/api/batches/) - [StreamingStream tokens from open models as they are generated, with OpenAI and Anthropic server-sent events. Examples, chunk format and error handling.Guides](https://inference.eu/api/streaming/) - [Migrate from OpenAIMove an application from OpenAI to open models processed in the EU. Change the base URL, key and model, then check tools, JSON output and embeddings.Guides](https://inference.eu/api/migrate-from-openai/) ## Move your AI workloads to EU infrastructure Tell us which models you use and roughly how much traffic you have. We reply within one working day with pricing and next steps. - Reply within one working day - Pricing for your volume - DPA ready to sign