Model reference — Pictures
Open-vocabulary detection finds objects you describe in words rather than objects a model was trained to know. You write "a red fire extinguisher", "a forklift without a driver", "a cardboard box with a torn corner", and the model returns boxes around what matches.
The practical significance is that no labelling is required. An ordinary detector must be shown a few thousand labelled examples before it can find anything new; these models find it from a phrase, immediately. The trade-off is accuracy and cost: they are slower per image and less precise than a detector fitted to the same object. So the common pattern is to use them to prove a use case and to generate initial labels, then fit a fast detector on those labels once the definition has settled.
Models in this group take a single image as input: a photograph, a scan or a screenshot. Each specification table gives three hardware tiers — Minimum, the smallest setup on which the model runs correctly; Medium, the usual production configuration; and High, a configuration sized for peak volume. The rate is the number of sample inputs processed per hour on that hardware. Use these rates for initial sizing. Before production, benchmark your own data to validate accuracy, latency, throughput and cost. The sample input here is one photograph at about 2 megapixels.
Vendor: IDEA Research
What it does: finds objects from a written phrase with the best accuracy of this class of model, including descriptions involving colour, condition and relative position. The usual first choice for proving a use case.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3090 | RTX 4090 | A100 80 GB |
| VRAM | 12 GB | 24 GB | 80 GB |
| vCPUs | 8 | 12 | 24 |
| RAM | 32 GB | 48 GB | 96 GB |
| Server | 1× RTX 3090 24 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (images/hour) | ≈ 2,000 | ≈ 8,000 | ≈ 20,000 |
Vendor: Google
What it does: detects from either a phrase or an example image, which is valuable when the thing you are looking for is easier to show than to describe.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3090 | RTX 4090 | A100 80 GB |
| VRAM | 12 GB | 24 GB | 80 GB |
| vCPUs | 8 | 12 | 24 |
| RAM | 32 GB | 48 GB | 96 GB |
| Server | 1× RTX 3090 24 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (images/hour) | ≈ 2,400 | ≈ 9,000 | ≈ 22,000 |
Vendor: Tencent AI Lab
What it does: a much faster model of the same kind, accepting a written list of categories at close to ordinary detector speed. The practical option when open-vocabulary detection must run in production at volume.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3060 | RTX 4090 | A100 80 GB |
| VRAM | 8 GB | 24 GB | 80 GB |
| vCPUs | 6 | 12 | 24 |
| RAM | 16 GB | 32 GB | 64 GB |
| Server | 1× RTX 3060 12 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (images/hour) | ≈ 20,000 | ≈ 80,000 | ≈ 220,000 |
Vendor: Microsoft
What it does: detects, describes and captions from open prompts in one small model, useful when you want a description of the scene as well as boxes.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3060 | RTX 4090 | A100 80 GB |
| VRAM | 8 GB | 24 GB | 80 GB |
| vCPUs | 6 | 12 | 24 |
| RAM | 16 GB | 32 GB | 64 GB |
| Server | 1× RTX 3060 12 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (images/hour) | ≈ 1,800 | ≈ 6,000 | ≈ 14,000 |
Vendor: Meta
What it does: outlines exactly the object a detector has boxed, which turns an approximate box from a phrase into a precise mask suitable for measurement or for labelling training data.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3090 | RTX 4090 | A100 80 GB |
| VRAM | 12 GB | 24 GB | 80 GB |
| vCPUs | 8 | 12 | 24 |
| RAM | 32 GB | 48 GB | 96 GB |
| Server | 1× RTX 3090 24 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (images/hour) | ≈ 3,000 | ≈ 12,000 | ≈ 30,000 |
These models are usually the right way to start and rarely the right way to finish. Our consultants use them to establish what your images can support, then recommend whether to run them in production or to use them to bootstrap a fitted detector.
At the start of a project we may run a short proof of concept on a sample of your own data, measuring the accuracy and the throughput the model actually achieves on your material. That replaces the estimates on this page with real figures, so the cost and the schedule for the full engagement are known before it is committed.
Share a representative sample, expected volume, latency target and deployment location for Open-vocabulary object detection. We benchmark the shortlisted models, recommend the lowest-cost GPU configuration that meets the target, and scale it from pilot capacity to a dedicated production cluster.