All posts
CRAFT5 min read

Why word-level timing beats sentence blocks

Karaoke-style highlighting isn't decoration. Timing every word instead of every sentence changes how captions read — and it's what makes the rest of the editor possible.

FathiCo-founder

Most caption tools hand you a sentence at a time. A block of text appears, sits there for three seconds, disappears, and the next one takes its place. It's the simplest thing to build, it's what the subtitle formats were designed for, and for a film it's the right answer.

For a forty-second vertical video it isn't, and the reason has less to do with captions than with what someone is doing while they read them.

The sentence block asks you to wait

A viewer scrolling a feed gives you a second or two before deciding. A caption that appears fully formed asks them to read it at their own pace, which means they finish it before you've said it, and then wait. Do that four times in a row and the video feels slower than it is.

Timing every word separately means the caption arrives at the speed of the voice. There's nothing to read ahead to. That sounds like a small difference and it changes the whole rhythm of watching with the sound off.

A caption you finish reading before the sentence is spoken isn't helping you follow along. It's giving you time to leave.

What word-level timing actually requires

Speech-to-text gives us a start and an end for every word, not just every sentence. We keep all of it. A caption segment is a group of words that each still know when they were said, so at any frame we can tell which word is being spoken right now.

That single fact is what makes several things possible at once:

  • Highlighting the spoken word. Karaoke-style colouring needs to know which word is live, to the frame.
  • Re-flowing after an edit. Change the text and the timings redistribute across what's left, because they were never welded to the sentence.
  • Choosing how much to show. Three words at a time or a full line is a setting, not a re-transcription.
  • Flagging what the model wasn't sure of. Confidence comes per word too, so a doubtful word can be marked where it actually is.

None of that is reachable once you've thrown the word boundaries away and kept a sentence with two timestamps on it.

The part that nearly ruined it

Here is the bug that cost us the most time, because it looked like a design problem and was a layout one.

Our "Pop" animation scaled each word up slightly as it was spoken. Obvious, cheap, looks great in a mockup. In motion the entire caption twitched — every word, on every word, for the length of the video.

The cause is that we were animating font size. Changing a word's font size changes its measured width, which changes where the next word sits, which re-flows the line. Every word that popped nudged its neighbours, and with three words a line something moved constantly.

The fix is a rule rather than a tweak: an animation may only touch transform, opacity and clip-path. Those three are composited — the browser moves pixels that are already drawn and never re-measures the text. Anything that changes the box the text occupies is off limits, however good it looks in a still frame. Every one of the twelve animations in the editor obeys that, and the line no longer moves.

Where it doesn't help

Word-level timing is worse for some things and it's worth saying so. A long interview read as a transcript is easier to follow in whole sentences. A caption file exported to a platform that only shows one cue at a time gains nothing from it. And a video where nobody speaks is unaffected either way.

It's a technique for short video with a voice in it, watched silently, at speed. That happens to be most of what people caption — which is why it's the default here rather than an option.

Fathi · Co-founder

Builds Add Captions. Writes here about the parts of captioning that turn out to be harder than they look.

More writing

All posts

Captions, timed to the word

Everything above is how Add Captions works. Upload a video and see it — five dollars a month, and you can caption as many as you like before paying anything.

Add captions free