What matters is inside an MP4, and it is not the picture

A recorded call, a lecture, a demo. Everything worth having was said out loud, and you are charged for minutes of audio, not for the weight of the video.

The problem

Someone hands you the recording. It is an MP4: a Zoom or Meet call, a lecture, a customer demo, a screen share with a person talking over it. Everything anyone needs out of it was said out loud, and nobody is ever going to watch the picture again.

The obvious move is to give the video to the model, and the obvious move does not exist. Most clients cannot put a video in front of a model at all, and a recording of any length is far too large to travel inside a tool call. What you actually want is much smaller than the file: the words, in order, with who said them and when.

So the video has to become audio, and the audio has to become text. Two steps stand between your agent and the one thing it was asked to do.

Why it fails locally

Extracting the audio track is one command with one more tool, and that is precisely the cost: another binary to install, to keep current, and to have present on whatever machine the agent happens to be running on. It is a dependency you acquired for a step that produces nothing anyone wanted, because a WAV file is not an answer.

Video containers are not one format either. MP4, MOV, MKV, WebM, each carrying whatever audio codec the recorder chose, sometimes more than one track, sometimes with the speech on a channel your default extraction ignores. The step handles the files you tested it on and fails on the one your user sends.

And when it works you are exactly where you started: holding audio you still cannot transcribe. The extra tool did not solve the problem, it added a stage to it. The hard part, turning an hour of speech into attributed text, is still entirely ahead of you.

The code

listen accepts video and works on its audio track. Nothing is extracted on your side. Anything longer than a few minutes is queued and answers with a handle instead.

upload(filename="demo.mp4", content_type="video/mp4")
  -> key, upload_url

curl -T demo.mp4 "<upload_url>"

listen(key="<key>", language="en")
  -> handle, engine, used_fallback, cost_eur, structure

result(handle="<handle>", outline=true)
  -> who spoke, and when

result(handle="<handle>", find="pricing")
  -> excerpts, each with its timestamp

What it costs

listen costs 0.005 EUR per minute of audio in balanced mode, so a forty-five minute recorded call is about 0.23 EUR. exact costs 0.008 EUR per minute and uses the strongest model: worth it on a room microphone, on crosstalk and on heavy accents. Both separate speakers.

You are charged for minutes, not for megabytes. A video weighs on the order of a hundred times its own audio track, and none of that weight reaches the bill. The duration is measured from the file itself rather than guessed from its size, and a declared duration that disagrees with the file's weight is not trusted.

You pay once, on the real duration, whether or not you ever collect the result. The same file sent twice is recognised by its contents and not by its name, so a second attempt is one charge and not two. Querying the transcript afterwards with result is free, however many times you do it.

What this does not do

The picture is not looked at. Slides on a shared screen, a whiteboard, anything written rather than spoken does not appear in the transcript. For a document, use read.
The audio track is what gets transcribed. A screen recording with nobody talking comes back with nothing in it, and that is the correct answer.
Anything longer than a few minutes is queued: listen answers immediately with a handle and status: running, and result serves the transcript once it is ready.
Four hours is the ceiling on a single file. Longer than that, split it.

Without markup

This page in markdown: /problems/transcribe-the-audio-of-a-video.md. All 20 of them in one file: /llms-full.txt.