Skip to main content

Model reference — Documents

AI models for Semantic search

Semantic search finds documents by meaning rather than by matching words, so a search for "staff leaving early" also returns the policy titled "early departure procedure" and the one that never uses either phrase.

It works by turning every passage into a numeric fingerprint — a list of numbers representing its meaning — and doing the same to the query, then returning the passages whose fingerprints are closest. The fingerprints are computed once, when documents are indexed; searching afterwards is fast and cheap. In practice the best results come from combining this with ordinary keyword search, which is still better at exact terms like part numbers, and then re-sorting the combined results with a reranking model.

Semantic search service AI models for documents

Input type — Documents

Models in this group take text or whole documents as input: plain text, PDFs, scanned pages and office files. Each specification table gives three hardware tiers — Minimum, the smallest setup on which the model runs correctly; Medium, the usual production configuration; and High, a configuration sized for peak volume. The rate is the number of sample inputs processed per hour on that hardware. Use these rates for initial sizing. Before production, benchmark your own data to validate accuracy, latency, throughput and cost. The sample input here is one A4 page of about 500 words indexed, or one query answered against the index.

BGE-M3

Vendor: Beijing Academy of Artificial Intelligence

What it does: turns passages and queries into fingerprints, covering over a hundred languages and passages up to about 8,000 words. It also produces keyword-style scores at the same time, which is what makes a blended search possible from one model.

RequirementMinimumMediumHigh
GPU typeRTX 3060RTX 4090A100 80 GB
VRAM8 GB24 GB80 GB
vCPUs61224
RAM16 GB32 GB64 GB
Server1× RTX 3060 12 GB1× RTX 4090 24 GB1× A100 SXM 80 GB
Rate (pages/hour)≈ 6,000≈ 30,000≈ 90,000

E5-large / multilingual-E5

Vendor: Microsoft

What it does: a well-proven fingerprinting model, small and quick. A dependable default for English archives, with a multilingual version for mixed collections.

RequirementMinimumMediumHigh
GPU typeRTX 3060RTX 4090A100 80 GB
VRAM6 GB24 GB80 GB
vCPUs4824
RAM16 GB32 GB64 GB
Server1× RTX 3060 12 GB1× RTX 4090 24 GB1× A100 SXM 80 GB
Rate (pages/hour)≈ 9,000≈ 45,000≈ 140,000

all-MiniLM-L6

Vendor: Microsoft (Sentence-Transformers)

What it does: a very small fingerprinting model that runs on almost any hardware. Accuracy is below the larger models, but for a modest archive it is fast, cheap and adequate.

RequirementMinimumMediumHigh
GPU typeRTX 3060RTX 4090A100 80 GB
VRAM4 GB24 GB80 GB
vCPUs4824
RAM8 GB32 GB64 GB
Server1× RTX 3060 12 GB1× RTX 4090 24 GB1× A100 SXM 80 GB
Rate (pages/hour)≈ 30,000≈ 120,000≈ 400,000

SimCSE

Vendor: Princeton NLP

What it does: a compact model that produces fingerprints suited to sentence-level matching, useful where the unit being searched is a single clause or line rather than a passage.

RequirementMinimumMediumHigh
GPU typeRTX 3060RTX 4090A100 80 GB
VRAM6 GB24 GB80 GB
vCPUs4824
RAM16 GB32 GB64 GB
Server1× RTX 3060 12 GB1× RTX 4090 24 GB1× A100 SXM 80 GB
Rate (pages/hour)≈ 20,000≈ 80,000≈ 240,000

BGE Reranker v2-M3

Vendor: Beijing Academy of Artificial Intelligence

What it does: reads a query and a candidate passage together and scores how well the passage answers it. Applied to the top few dozen results, it is usually the single largest improvement in search quality.

RequirementMinimumMediumHigh
GPU typeRTX 3090RTX 4090A100 80 GB
VRAM12 GB24 GB80 GB
vCPUs81224
RAM32 GB48 GB96 GB
Server1× RTX 3090 24 GB1× RTX 4090 24 GB1× A100 SXM 80 GB
Rate (query-passage pairs/hour)≈ 40,000≈ 150,000≈ 400,000

ColPali

Vendor: Illuin Technology

What it does: fingerprints the page image itself, so slides, scanned forms and charts are retrievable even when text extraction would lose their layout.

RequirementMinimumMediumHigh
GPU typeRTX 4090A100 80 GB2× A100 80 GB
VRAM20 GB80 GB160 GB combined
vCPUs81632
RAM32 GB64 GB128 GB
Server1× RTX 4090 24 GB1× A100 SXM 80 GB2× A100 SXM 80 GB
Rate (pages/hour)≈ 900≈ 3,500≈ 7,000

Qwen2.5 32B

Vendor: Alibaba Cloud

What it does: turns a vague question into a better query and writes a direct answer over the retrieved passages, which is what turns a list of results into an answer.

RequirementMinimumMediumHigh
GPU typeRTX 4090 (reduced precision)L40S 48 GB2× H100 80 GB
VRAM22 GB48 GB160 GB combined
vCPUs121648
RAM48 GB64 GB256 GB
Server1× RTX 4090 24 GB1× L40S 48 GB2× H100 SXM 80 GB
Rate (pages/hour)≈ 350 answers/hour≈ 1,100 answers/hour≈ 4,200 answers/hour

Choosing between them

Search quality depends on how your documents are divided into passages, whether keyword search is blended in, and whether a reranker is worth its cost at your volume. Our consultants review your archive and the queries your staff type, then recommend the indexing model, the blend and the reranking step.

At the start of a project we may run a short proof of concept on a sample of your own data, measuring the accuracy and the throughput the model actually achieves on your material. That replaces the estimates on this page with real figures, so the cost and the schedule for the full engagement are known before it is committed.

Semantic search service AI models for documents Pricing

From benchmark to production

Share a representative sample, expected volume, latency target and deployment location for Semantic search. We benchmark the shortlisted models, recommend the lowest-cost GPU configuration that meets the target, and scale it from pilot capacity to a dedicated production cluster.