How local transcription works: from a file in a folder to a speaker-labelled transcript

Drop a file into a watched folder and, some seconds or minutes later, unottr shows a transcript with speakers, timestamps and a search index. There is no start button. This post is about what happens in between, because the design choices there decide how reliable the result is.

Stages, not a single job

The pipeline is a series of persisted states. Each one is written to the database before the next begins:

1. Discover. The watcher notices a new file and waits until the writer has stopped and ffprobe reports a duration. A file still being copied is not a recording yet.
2. Probe. Duration, streams and codec details are recorded.
3. Extract. ffmpeg produces 16 kHz mono PCM, the format Whisper expects.
4. Detect speech. A voice-activity model marks where people are actually talking.
5. Transcribe. Whisper runs over roughly 30-second chunks aligned to those speech boundaries. Each chunk is written to the database as it lands.
6. Diarize. A speaker model looks at the whole file and decides who spoke when.
7. Merge. Speaker turns are laid over transcript segments.
8. Finalize. The full-text index is updated and the library row turns green.

The reason for all this bookkeeping is simple: a crash resumes at a boundary instead of starting over. If your laptop goes to sleep twenty minutes into an hour-long file, the next run picks up at the last finished chunk.

Compute runs in a separate forked process. If a native library crashes, it takes one job down, not the app.

Whisper, and which one

unottr uses whisper.cpp through a native addon, with Vulkan on Linux and Metal on Apple Silicon for GPU acceleration. The model choice is made for you on first run, based on the resolved device, and can be changed later in Settings.

The measurements that drove the defaults, on five minutes of a real recording:

Turbo is slower than realtime on CPU, so the default follows the device: Turbo on a GPU, Small on CPU. A 33-minute meeting transcribes in about 83 seconds on that GPU.

Who said what

Transcription tells you the words. Diarization tells you the speakers, and it is the harder half.

On Linux with a Vulkan GPU, unottr uses Sortformer v2.1, which finds up to four speakers without being told a count. sherpa-onnx handles CPU fallback, separate microphone tracks and runs where you know the exact number of participants, from 1 to 20. Apple Silicon uses sherpa on CPU for this stage.

Speakers are then matched against voiceprints you have named before, so the same person is labelled across weeks of recordings after you name them once. When the split is wrong, you can merge two speakers, reassign one segment, or run identification again without transcribing the file a second time.

Corrections that survive

Whisper gets product names and acronyms wrong in predictable ways. unottr keeps the original Whisper text and applies your glossary rules on top, so changing or deleting a rule is reversible. Editing a transcript turn by hand takes precedence over the rules, updates search and exports, and leaves timestamps and speakers untouched.

Everything above runs without a network call. The model weights are fetched once and cached. After that, the only thing the pipeline needs from the outside world is the next file in the folder.