Model reference — Video
This service takes narration you have already recorded and lays it onto a video correctly: cleaned, level-matched, timed to the picture, and mixed with the existing sound rather than simply replacing it.
The work is in the fit. Narration recorded separately rarely matches the length of the section it belongs to, so it must be aligned to the picture and, where it runs long or short, the timing adjusted — by trimming pauses, by nudging cut points, or by slightly changing pace without altering pitch. The existing soundtrack usually has to remain audible underneath, which means ducking the music and effects under the voice. Where the video shows a person speaking, the lip movement can also be matched to the new audio.
Models in this group take a recorded video file as input and are applied frame by frame. Each specification table gives three hardware tiers — Minimum, the smallest setup on which the model runs correctly; Medium, the usual production configuration; and High, a configuration sized for peak volume. The rate is the number of sample inputs processed per hour on that hardware. Use these rates for initial sizing. Before production, benchmark your own data to validate accuracy, latency, throughput and cost. The sample input here is one minute of 1080p video at 25 frames per second.
Vendor: University of Oxford (VGG)
What it does: aligns the supplied narration to the video by matching its words against the timeline, which is what places each sentence against the right shot instead of by hand.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3090 | RTX 4090 | A100 80 GB |
| VRAM | 12 GB | 24 GB | 80 GB |
| vCPUs | 8 | 12 | 24 |
| RAM | 32 GB | 48 GB | 96 GB |
| Server | 1× RTX 3090 24 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (video minutes processed per hour) | ≈ 900 | ≈ 3,300 | ≈ 8,400 |
Vendor: SYSTRAN
What it does: transcribes both the narration and the original audio so the two can be matched and the correct insertion points found automatically.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3060 | RTX 4090 | A100 80 GB |
| VRAM | 8 GB | 24 GB | 80 GB |
| vCPUs | 6 | 12 | 24 |
| RAM | 16 GB | 32 GB | 64 GB |
| Server | 1× RTX 3060 12 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (video minutes processed per hour) | ≈ 1,200 | ≈ 4,200 | ≈ 10,800 |
Vendor: Meta
What it does: separates the original soundtrack into voice, music and effects so the existing dialogue can be removed while the score and effects are kept underneath the new narration.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3090 | RTX 4090 | A100 80 GB |
| VRAM | 12 GB | 24 GB | 80 GB |
| vCPUs | 8 | 12 | 24 |
| RAM | 32 GB | 48 GB | 96 GB |
| Server | 1× RTX 3090 24 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (video minutes processed per hour) | ≈ 360 | ≈ 1,400 | ≈ 3,600 |
Vendor: Friedrich-Alexander-Universität
What it does: removes room noise from the supplied narration in real time, which matters because narration is often recorded outside a studio.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3060 | RTX 4090 | A100 80 GB |
| VRAM | 4 GB | 24 GB | 80 GB |
| vCPUs | 4 | 8 | 24 |
| RAM | 8 GB | 32 GB | 64 GB |
| Server | 1× RTX 3060 12 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (video minutes processed per hour) | ≈ 12,000 | ≈ 42,000 | ≈ 108,000 |
Vendor: Resemble AI
What it does: repairs narration recorded on poor equipment, raising it to a quality that can sit against professionally produced picture.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3090 | RTX 4090 | A100 80 GB |
| VRAM | 12 GB | 24 GB | 80 GB |
| vCPUs | 8 | 12 | 24 |
| RAM | 32 GB | 48 GB | 96 GB |
| Server | 1× RTX 3090 24 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (video minutes processed per hour) | ≈ 480 | ≈ 1,800 | ≈ 4,800 |
Vendor: International Institute of Information Technology Hyderabad
What it does: adjusts the lip movement of a person on screen to match the new audio, for cases where the narrator is visible.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 3090 | RTX 4090 | A100 80 GB |
| VRAM | 12 GB | 24 GB | 80 GB |
| vCPUs | 8 | 12 | 24 |
| RAM | 32 GB | 48 GB | 96 GB |
| Server | 1× RTX 3090 24 GB | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB |
| Rate (video minutes processed per hour) | ≈ 60 | ≈ 220 | ≈ 550 |
Vendor: ByteDance
What it does: a higher-quality lip synchronisation model, better on close-ups and high-resolution footage where the older model’s output is visibly soft.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | RTX 4090 | A100 80 GB | 2× A100 80 GB |
| VRAM | 20 GB | 80 GB | 160 GB combined |
| vCPUs | 8 | 16 | 32 |
| RAM | 32 GB | 64 GB | 128 GB |
| Server | 1× RTX 4090 24 GB | 1× A100 SXM 80 GB | 2× A100 SXM 80 GB |
| Rate (video minutes processed per hour) | ≈ 20 | ≈ 80 | ≈ 200 |
Vendor: FFmpeg project
What it does: performs the actual mix and mux — levels, ducking, channel layout, delivery format — and needs no GPU.
| Requirement | Minimum | Medium | High |
|---|---|---|---|
| GPU type | No GPU required | No GPU required | GPU-accelerated decode (NVENC/NVDEC) |
| VRAM | — | — | 8 GB |
| vCPUs | 2 | 8 | 16 |
| RAM | 4 GB | 16 GB | 32 GB |
| Server | CPU instance, 2 vCPU | CPU instance, 8 vCPU | 1× RTX 4090 24 GB |
| Rate (video minutes processed per hour) | ≈ 6,000 | ≈ 20,000 | ≈ 60,000 |
What is needed depends on how your narration was recorded, whether the original sound must survive underneath, and whether anyone on screen is speaking. Our consultants review your material and recommend the alignment, cleaning and mixing steps, and whether lip synchronisation is warranted.
At the start of a project we may run a short proof of concept on a sample of your own data, measuring the accuracy and the throughput the model actually achieves on your material. That replaces the estimates on this page with real figures, so the cost and the schedule for the full engagement are known before it is committed.
Share a representative sample, expected volume, latency target and deployment location for Narrator to video - add supplied narrator audio. We benchmark the shortlisted models, recommend the lowest-cost GPU configuration that meets the target, and scale it from pilot capacity to a dedicated production cluster.