Document question answering lets someone ask a question in ordinary language — “what notice period applies to the Frankfurt lease?” — and get the answer back with the passage and page it came from, instead of searching a folder of PDFs by hand.
Two kinds of model are needed. An embedding model turns every passage in your archive into a numeric fingerprint so the passages closest in meaning to a question can be found; the same is done to the question at the moment it is asked. A language model then reads the retrieved passages and writes the answer, quoting its source. A third, optional model — a reranker — re-sorts the retrieved passages by relevance before the language model sees them, which measurably improves the answers on large archives.
Models in this group take documents and questions as input: PDFs, scanned pages, office files and the plain text extracted from them. Each specification table gives three hardware tiers — Minimum, the smallest setup on which the model runs correctly; Medium, the usual production configuration; and High, a configuration sized for peak volume. The rate is the number of sample inputs processed per hour on that hardware. Use these rates for initial sizing. Before production, benchmark your own data to validate accuracy, latency, throughput and cost. The sample input here is stated per model — one A4 page of about 500 words for indexing models, or one question answered from retrieved passages for answering models.
BGE-M3
Vendor: Beijing Academy of Artificial Intelligence
What it does: turns passages and questions into numeric fingerprints so that passages matching a question’s meaning can be found even when they share no words with it. It handles over a hundred languages and passages up to about 8,000 words, which suits long contracts and reports.
Requirement
Minimum
Medium
High
GPU type
RTX 3060
RTX 4090
1× A100 80 GB
VRAM
8 GB
24 GB
80 GB
vCPUs
6
12
24
RAM
16 GB
32 GB
64 GB
Server
1× RTX 3060 12 GB
1× RTX 4090 24 GB
1× A100 SXM 80 GB
Rate (pages/hour)
≈ 6,000
≈ 30,000
≈ 90,000
E5-large / multilingual-E5
Vendor: Microsoft
What it does: a well-proven fingerprinting model for retrieval, small and quick to run. A dependable default for English archives, with a multilingual version for mixed-language collections.
Requirement
Minimum
Medium
High
GPU type
RTX 3060
RTX 4090
1× A100 80 GB
VRAM
6 GB
24 GB
80 GB
vCPUs
4
8
24
RAM
16 GB
32 GB
64 GB
Server
1× RTX 3060 12 GB
1× RTX 4090 24 GB
1× A100 SXM 80 GB
Rate (pages/hour)
≈ 9,000
≈ 45,000
≈ 140,000
BGE Reranker v2-M3
Vendor: Beijing Academy of Artificial Intelligence
What it does: reads a question and a candidate passage together and scores how well the passage actually answers it. Run over the top few dozen candidates, it pushes the genuinely relevant passage to the top, which is often the single biggest accuracy gain in the pipeline.
Requirement
Minimum
Medium
High
GPU type
RTX 3090
RTX 4090
1× A100 80 GB
VRAM
12 GB
24 GB
80 GB
vCPUs
8
12
24
RAM
32 GB
48 GB
96 GB
Server
1× RTX 3090 24 GB
1× RTX 4090 24 GB
1× A100 SXM 80 GB
Rate (question-passage pairs/hour)
≈ 40,000
≈ 150,000
≈ 400,000
Llama 3.3 70B
Vendor: Meta
What it does: reads the retrieved passages and writes the answer with citations. The larger size is worth its cost when questions require combining facts from several documents rather than quoting one.
Requirement
Minimum
Medium
High
GPU type
2× RTX 4090 (reduced precision)
1× H100 80 GB
4× H100 80 GB
VRAM
48 GB combined
80 GB
320 GB combined
vCPUs
16
24
64
RAM
64 GB
128 GB
512 GB
Server
2× RTX 4090 24 GB
1× H100 SXM 80 GB
4× H100 SXM 80 GB
Rate (answers/hour)
≈ 120
≈ 600
≈ 3,000
Qwen2.5 32B
Vendor: Alibaba Cloud
What it does: a mid-sized answering model that handles very long retrieved contexts, so it can be given a whole chapter rather than a few paragraphs. Strong across European and Asian languages.
Requirement
Minimum
Medium
High
GPU type
RTX 4090 (reduced precision)
1× L40S 48 GB
2× H100 80 GB
VRAM
22 GB
48 GB
160 GB combined
vCPUs
12
16
48
RAM
48 GB
64 GB
256 GB
Server
1× RTX 4090 24 GB
1× L40S 48 GB
2× H100 SXM 80 GB
Rate (answers/hour)
≈ 180
≈ 560
≈ 2,200
Mistral Small 3
Vendor: Mistral AI
What it does: a compact answering model that fits on one card and keeps latency low, which matters when staff expect a reply while they are still on the phone to a customer.
Requirement
Minimum
Medium
High
GPU type
RTX 4090
1× L40S 48 GB
1× H100 80 GB
VRAM
24 GB
48 GB
80 GB
vCPUs
12
16
32
RAM
48 GB
64 GB
128 GB
Server
1× RTX 4090 24 GB
1× L40S 48 GB
1× H100 SXM 80 GB
Rate (answers/hour)
≈ 260
≈ 700
≈ 2,200
ColPali
Vendor: Illuin Technology
What it does: fingerprints the page image itself rather than extracted text, so it retrieves the right page from scans, slides and charts where text extraction would lose the layout. Useful for archives full of presentations and forms.
Requirement
Minimum
Medium
High
GPU type
RTX 4090
1× A100 80 GB
2× A100 80 GB
VRAM
20 GB
80 GB
160 GB combined
vCPUs
8
16
32
RAM
32 GB
64 GB
128 GB
Server
1× RTX 4090 24 GB
1× A100 SXM 80 GB
2× A100 SXM 80 GB
Rate (pages/hour)
≈ 900
≈ 3,500
≈ 7,000
Choosing between them
The choice is really a choice of pipeline: how your archive is indexed, whether a reranker earns its keep at your volume, and how large the answering model needs to be. Our consultants review the archive, the questions your staff actually ask and the accuracy you need on citations, then recommend the combination that fits.
At the start of a project we may run a short proof of concept on a sample of your own data, measuring the accuracy and the throughput the model actually achieves on your material. That replaces the estimates on this page with real figures, so the cost and the schedule for the full engagement are known before it is committed.
Share a representative sample, expected volume, latency target and deployment location for Document question answering. We benchmark the shortlisted models, recommend the lowest-cost GPU configuration that meets the target, and scale it from pilot capacity to a dedicated production cluster.