Rerank API: sort search results by relevance, processed in the EU

POST /v1/rerank takes a model, a query and an array of documents as strings. A reranker such as bge-reranker-v2-m3 reads the query and each document together, which lets it judge relevance far more precisely than comparing two separately computed embeddings. The cost is speed, which is why you rerank a short list of candidates rather than a whole collection.

The response contains a results array sorted from most to least relevant. Each result has the index of the document in your original array and a relevance_score. Use the index to look up the document and its metadata on your side.

top_n limits how many results come back; leave it out to receive a score for every document. Set return_documents to true if you want each result to include the document text, which saves a lookup but makes the response larger.

The request shape follows the rerank format used by most search libraries and RAG frameworks, so integrations that support a custom rerank endpoint can usually be pointed at inference.eu by changing the URL, key and model. bge-reranker-v2-m3 is multilingual, so a query in English can rank German or French passages correctly.

First, embed the user's question and fetch around 30 to 50 nearest chunks from your vector database. This step is fast and has high recall but imprecise ordering. Second, send the question and those chunks to the rerank endpoint and keep the top three to eight results. Third, put those chunks into the prompt of a chat model and ask it to answer with references to them.

You can also use the score as a threshold. If even the best result scores low, tell the user that the knowledge base has no good answer instead of letting the model guess. Tune that threshold on a set of real questions.

Each query and document pair must fit the reranker's input length. Long documents are truncated to that length on the model's side, so rerank chunks of similar size to the ones you embedded, not whole files. An empty documents array or an empty query returns a 400 error.

Scores are only comparable within one model; do not mix thresholds across rerankers. Very large candidate lists slow the request down without improving results much, because a good first-stage search already puts the answer in the top 50. The number of documents per request depends on your plan, and we confirm it with you.