--- title: "Rerank API · better RAG results, hosted in the EU" description: "Re-order search results by relevance with a rerank API for RAG. Cross-encoder reranker models such as bge-reranker-v2-m3, processed in EU data centres." canonical: "https://inference.eu/api/rerank/" language: "en" --- POST /v1/rerank # Rerank API: sort search results by relevance, processed in the EU The rerank endpoint takes a query and a list of candidate documents and returns them sorted by how well each one answers the query. It sits between vector search and the language model in a RAG pipeline, and is often the cheapest way to make answers noticeably more accurate. Queries and documents are processed in European data centres. [Contact sales](https://inference.eu/contact/)[Example request](#example) 1. 01API format: OpenAI 2. 02EU data centres 3. 03Covered by one DPA POST /v1/rerank ``` curl https://api.inference.eu/v1/rerank \ -H "Authorization: Bearer $INFERENCE_EU_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "bge-reranker-v2-m3", "query": "Can an employee carry unused holiday into next year?", "documents": [ "Unused annual leave may be carried over until 31 March of the following year.", "The office is closed between Christmas and New Year.", "Urlaubsansprüche verfallen nicht, wenn der Arbeitgeber nicht auf den Verfall hingewiesen hat.", "Expense claims must be submitted within 30 days." ], "top_n": 2 }' # {"results": [{"index": 0, "relevance_score": 0.94}, {"index": 2, "relevance_score": 0.81}], ...} ``` ## What the endpoint does `POST /v1/rerank` takes a `model`, a `query` and an array of `documents` as strings. A reranker such as `bge-reranker-v2-m3` reads the query and each document together, which lets it judge relevance far more precisely than comparing two separately computed embeddings. The cost is speed, which is why you rerank a short list of candidates rather than a whole collection. The response contains a `results` array sorted from most to least relevant. Each result has the `index` of the document in your original array and a `relevance_score`. Use the index to look up the document and its metadata on your side. ## Key request fields `top_n` limits how many results come back; leave it out to receive a score for every document. Set `return_documents` to `true` if you want each result to include the document text, which saves a lookup but makes the response larger. The request shape follows the rerank format used by most search libraries and RAG frameworks, so integrations that support a custom rerank endpoint can usually be pointed at inference.eu by changing the URL, key and model. `bge-reranker-v2-m3` is multilingual, so a query in English can rank German or French passages correctly. ## Embeddings plus rerank, the standard RAG pattern First, embed the user's question and fetch around 30 to 50 nearest chunks from your vector database. This step is fast and has high recall but imprecise ordering. Second, send the question and those chunks to the rerank endpoint and keep the top three to eight results. Third, put those chunks into the prompt of a chat model and ask it to answer with references to them. You can also use the score as a threshold. If even the best result scores low, tell the user that the knowledge base has no good answer instead of letting the model guess. Tune that threshold on a set of real questions. ## Errors and edge cases Each query and document pair must fit the reranker's input length. Long documents are truncated to that length on the model's side, so rerank chunks of similar size to the ones you embedded, not whole files. An empty `documents` array or an empty query returns a 400 error. Scores are only comparable within one model; do not mix thresholds across rerankers. Very large candidate lists slow the request down without improving results much, because a good first-stage search already puts the answer in the top 50. The number of documents per request depends on your plan, and we confirm it with you. ## Rerank Endpoint POST /v1/rerank Category Endpoints API format OpenAI Base URL api.inference.eu/v1 Processing EU data centres Training on your data Never ## Rerank: common questions The questions buyers, developers and legal teams ask us most. Something missing? Ask us directly. A reranker is a model that scores how relevant each candidate passage is to a query by reading both together. In retrieval-augmented generation it re-orders the results of vector search, so the chat model receives the most useful passages first. It usually improves answer quality more than switching to a larger chat model. Yes. Reranking every document in a large collection would be far too slow. Embeddings or keyword search find a short list of candidates quickly; the reranker then orders that short list precisely. The two steps work together, and both run on the inference.eu API. \`bge-reranker-v2-m3\` is a strong multilingual default that handles German, French, English and many other languages. Check \`GET /v1/models\` for the rerankers available to your key, and compare them on a sample of your own questions by measuring whether the right passage lands in the top three. The limit per request depends on the model and your plan, and we confirm it with you. In practice, 20 to 100 candidates is the useful range: fewer risks missing the answer, more adds time without much gain. Split larger lists into several requests and merge the scores. The request takes \`model\`, \`query\`, \`documents\`, \`top\_n\` and \`return\_documents\`, and returns \`results\` with \`index\` and \`relevance\_score\`. That is the shape most rerank clients and frameworks expect, so many work after you change the URL, key and model id. Check the field names your client sends. It is the model's estimate of how relevant the document is to the query, where higher means more relevant. Scores let you sort results and set a minimum threshold, but they are not probabilities and cannot be compared between different reranker models. Yes, with a multilingual reranker such as \`bge-reranker-v2-m3\`. It can score a German contract clause against an English question, which is common in European companies whose documents mix languages. Keep chunk sizes similar across languages for consistent scores. They are processed in EU data centres to compute the scores and are never used to train models. Rerank requests are used to answer the request only. If your contract needs specific logging or retention terms, we confirm them with you in writing. [Ask a question](https://inference.eu/contact/) ## Related [API overview](https://inference.eu/api/) - [EmbeddingsTurn text into vectors for search, RAG and clustering with an OpenAI-compatible embeddings API. Open embedding models, processed in EU data centres.Endpoints](https://inference.eu/api/embeddings/) - [Document conversionConvert PDF, Word, PowerPoint and Excel files to clean Markdown for RAG and LLM prompts. Tables, headings and scanned pages, processed in the EU.Tools](https://inference.eu/api/document-conversion/) - [Chat completionsSend chat messages to open-weight models with the OpenAI chat completions format. Requests are processed in EU data centres and never used for training.Endpoints](https://inference.eu/api/chat-completions/) ## Move your AI workloads to EU infrastructure Tell us which models you use and roughly how much traffic you have. We reply within one working day with pricing and next steps. - Reply within one working day - Pricing for your volume - DPA ready to sign