Frame-accurate playback in JavaScript: clocks, requestAnimationFrame and A/V sync

Frame-accurate video playback in JavaScript means one simple promise: when your code says the playhead is at a particular moment, the picture on screen is the frame that belongs to that moment, and the sound stays in step with it. That promise sounds easy until you build a browser video editor, where a person can scrub, pause, loop and cut at any point and expect the preview to match the exported file.

This article starts from first principles. We define a frame, a frame rate, a clock and a presentation timestamp, then explain why the ordinary video element is not enough for editing. From there we build a playback loop one step at a time, using real browser APIs: requestAnimationFrame, performance.now(), AudioContext.currentTime and requestVideoFrameCallback. The Azelify editor, a browser-native video editor we build, makes the same choices described here, so we use it as a worked example near the end.

You do not need a background in video engineering. You need to be comfortable with JavaScript functions and with the idea that a browser repaints the screen many times a second. Every term is defined the first time it appears.

What is a frame, and what does frame accurate mean?#

A video is a sequence of still pictures, called frames, shown quickly enough that the eye reads them as motion. The number of pictures shown each second is the frame rate, usually written as fps for frames per second. Film has traditionally used 24 fps, many television broadcasts use 25 or 30, and many screens now refresh 60 times a second or more.^[1]

Frames are numbered from zero. Frame 0 is the first picture, frame 1 the second, and so on. In a 24 fps clip, frame n is meant to be on screen from n divided by 24 seconds until (n + 1) divided by 24 seconds. So the time 1.02 seconds falls inside frame 24, because 1.02 multiplied by 24 is 24.48, and the frame is the whole number below it. In code, that rule looks like this:

// Which frame is on screen at a given time?
// Frame n covers the time from n / fps up to (n + 1) / fps.
function frameIndexAt(seconds, fps) {
  // The tiny nudge stops values like 0.29 * 100 (which JavaScript
  // stores as 28.999999999999996) from rounding down a whole frame.
  return Math.floor(seconds * fps + 1e-9);
}

frameIndexAt(1.02, 24); // 24
frameIndexAt(0.29, 100); // 29

Frame accuracy is the property that this function always decides the picture. A player is frame accurate when, for every time t, the picture it draws is the frame whose index is frameIndexAt(t, fps). A player that shows the wrong picture for a few milliseconds after a seek is not frame accurate, even if the error is small.

Each frame also carries a presentation timestamp, abbreviated PTS, which says when the frame should appear on the stream's clock.^[2] Compressed video often stores frames in a different order from display order, because some frames are predicted from frames that come later. A decoder therefore has to output frames by timestamp, not in the order they arrived. We return to timestamps in the WebCodecs decoding article, which covers decoding in detail.

Why is the video element not enough on its own?#

The HTML video element is built for watching. You set its source, call play(), and the browser handles decoding, buffering and painting. To find out where playback is, you read video.currentTime. The older way to react to each frame was to listen to the timeupdate event:

const video = document.querySelector("video");

video.addEventListener("timeupdate", () => {
  // The browser decides when this fires. It is not tied to painting,
  // and it is not one call per picture.
  const frame = frameIndexAt(video.currentTime, 24);
  updateOverlay(frame);
});

MDN describes this older technique as one that did not give access to the actual video frames.^[3] That leaves two problems for an editor. First, timeupdate is not synchronised with the screen. Your overlay can lag behind the picture or repeat a value. Second, currentTime is a position, not a promise about the picture. web.dev states plainly that the video element does not guarantee frame-accurate seeking, and points to WebCodecs for applications that need it.^[4] When you set currentTime to a new value, the picture that appears first may not be the exact frame you asked for.

For a viewer, those small errors are invisible. For an editor, they are the whole job. A user who sets an in point, scrubs back and presses play expects the first picture to be the one at that point, every time. So the playback loop needs to ask the same question on every tick and get the same answer for the same moment.

Which clock should you trust?#

A clock, in this context, is a number that moves forward with real time. Browsers offer several, and each one is honest about different things. The table below compares the three that matter for playback.

"Monotonic" means the value never goes backwards, even if the system clock is adjusted. performance.now() has that property. Date.now() does not, which is why it is a poor choice for timing a playback loop.

The audio clock deserves a closer look, because many tutorials recommend it. The usual reason is that ears are less forgiving than eyes: a stutter in sound is more noticeable than a dropped picture. A common rule of thumb is therefore to make the audio clock the master and let the pictures follow it. The trouble is that AudioContext.currentTime counts audio the hardware has processed. If the audio thread falls behind, the counter advances more slowly than real time. Tie the timeline to that counter and a busy processor turns into slow motion for the whole picture. Think of it as an odometer: it tells you how far the wheels have turned, which is not the same as telling you the time.

Which frame API should you use?#

Before we build anything, it helps to know which browser API fits the job. The decision tree below is the one we use when we choose. It is illustrative, not a benchmark.

Fig. 01

1. Need every decoded frame?
   1. yes → Decode with WebCodecs
   2. no → Only draw what plays?
      1. yes → Video frame callback (rVFC)
      2. no → Plain video element

Illustrative, not data[mdn-rvfc][wicg-rvfc][web-dev-rvfc]

For a live editor, the answer is almost always the first branch: the preview needs the exact frame for a given time, which means decoding the file yourself. The second branch is useful when you are only drawing a video that is already playing, such as a thumbnail strip or a live overlay.

How does requestAnimationFrame pace the loop?#

requestAnimationFrame, usually shortened to rAF, asks the browser to run your function before its next repaint.^[8] It is one-shot: the callback has to request the next frame itself, or the loop stops. The callback receives a timestamp that describes the end of the previous frame. On a screen that refreshes 60 times a second, callbacks arrive roughly every 16.7 milliseconds, which is the time budget for one frame at 60 Hz.

The important lesson is that rAF tells you when the screen can be painted. It does not tell you how much time has passed. If you advance the video by one frame on every callback, the speed of the video depends on the speed of the monitor. A 24 fps clip on a 120 Hz laptop would advance five times too fast. Here is the mistake and the fix side by side:

// Wrong: advance one frame per callback.
// On a 120 Hz screen this plays a 24 fps clip five times too fast.
let frame = 0;
function badLoop() {
  frame += 1;
  drawFrame(frame);
  requestAnimationFrame(badLoop);
}

// Right: ask the clock what time it is, then derive the frame.
let startMs = null;
function goodLoop(timestamp) {
  if (startMs === null) startMs = timestamp; // the first tick sets time zero
  const seconds = (timestamp - startMs) / 1000;
  drawFrame(frameIndexAt(seconds, 24));
  requestAnimationFrame(goodLoop);
}
requestAnimationFrame(goodLoop);

Build the playback loop in six steps#

The following steps build a loop that works for a single clip. Each step adds one responsibility. The first is the clock, because everything else depends on it. The diagram shows the shape of the finished loop: every tick runs the same four jobs, and the last one hands control back to the clock.

Fig. 02

1. 01 → Read clockperformance.now()
2. 02 → Snap to framefloor(t x fps)
3. 03 → Draw if changedskip repeats
4. 04 → Check audioresync if far off

Illustrative, not data[mdn-raf][mdn-perf-now]

Step 1: Anchor the clock#

A clock that counts ticks drifts, because each tick is an estimate. A better design stores one anchor, a moment where both the timeline position and the wall time are known, and derives the current time from the difference. Nothing accumulates, so nothing drifts.

// A clock that never counts ticks. It stores one anchor and
// derives "now" from the wall clock each time it is read.
class PlaybackClock {
  constructor() {
    this.anchorTimeline = 0; // seconds on the timeline at the anchor
    this.anchorWall = 0;     // performance.now() at the anchor, in ms
    this.playing = false;
  }
  play(fromSeconds) {
    this.anchorTimeline = fromSeconds;
    this.anchorWall = performance.now();
    this.playing = true;
  }
  pause() {
    this.anchorTimeline = this.now(); // freeze the current position
    this.playing = false;
  }
  now() {
    if (!this.playing) return this.anchorTimeline;
    const elapsedSeconds = (performance.now() - this.anchorWall) / 1000;
    return this.anchorTimeline + elapsedSeconds;
  }
}

Seeking is the same operation as play: move the anchor. Because the clock is a function of the anchor and the wall clock, a seek cannot leave a stale counter behind.

Step 2: Read the clock on every tick#

The loop is a rAF callback that reads the clock, converts the time to a frame index, and schedules itself again. It does not decide what the picture looks like yet. That is the next step.

const clock = new PlaybackClock();
const FPS = 24;
let lastFrame = -1;

function onTick() {
  if (clock.playing) {
    const seconds = clock.now();                // 1. read the clock
    const frame = frameIndexAt(seconds, FPS);   // 2. snap to a frame
    if (frame !== lastFrame) {                  // 3. only act on change
      presentDue(frame);
      lastFrame = frame;
    }
  }
  requestAnimationFrame(onTick);                // 4. ask for the next tick
}
requestAnimationFrame(onTick);

Notice that the loop runs whether or not the frame changed. That is intentional. A 120 Hz screen asks five times for every new 24 fps frame, and four of those ticks should do nothing.

Step 3: Snap time to the frame grid#

Snapping means rounding the time down to the frame that is on screen. We already defined frameIndexAt in the first section. The grid matters for more than the picture. If every part of the editor uses the same frame index, the overlay, the playhead and the decoder all agree about which frame they are discussing.

The grid also sets the lowest rate at which the picture can change. A project at 24 fps is shown at 24 distinct pictures per second, however fast the screen refreshes. That is the correct behaviour for a 24 fps film. The figure shows how a ten-second timeline maps onto frame numbers at that rate.

Fig. 03

Clip A0–4s

Clip B4–10s

0s5s10s

4s · Cut

1. 1Clip A 0–4sframes 0 to 95
2. 2Clip B 4–10sframes 96 to 239

Illustrative, not data[wiki-frame-rate]

Because the cut at four seconds lands exactly on a frame boundary, frame 96 belongs to Clip B alone. A cut that fell between frame boundaries would need a rule for which clip owns the partial frame, and that rule is a design decision, not a browser default.

Step 4: Present only what is due#

Once the frame index is known, the loop needs a picture for it. Decoding is slow compared with a 16.7 ms budget, so the picture is usually decoded ahead of time and kept in a small cache. The rule is simple: if the due frame is ready, draw it; if it is not, keep showing the last good picture and ask for the frame.

const canvas = document.querySelector("canvas");
const ctx = canvas.getContext("2d");
const decoded = new Map(); // frame index -> ImageBitmap

function presentDue(frame) {
  const bitmap = decoded.get(frame);
  if (bitmap) {
    ctx.drawImage(bitmap, 0, 0, canvas.width, canvas.height);
    return true;       // the picture for this moment is on screen
  }
  requestDecode(frame); // ask for it; keep the previous picture for now
  return false;
}

The last good picture is only a fallback. It should never be presented as if it were the requested frame, otherwise a missing decode looks like correct playback. Keeping that distinction clear is one of the design decisions the Azelify editor makes, which we describe below.

Step 5: Check audio drift and resync#

Drift is the gap between two clocks that should agree. The audio clock advances as sound is rendered, and the playback clock advances with the wall clock. Even when both are healthy, they will not match exactly. A correction rule keeps the gap small. The rule has two numbers: how far apart the clocks may be, and how often the audio may be restarted to close the gap.

const DRIFT_LIMIT_S = 0.12;  // resync when audio is more than this far off
const MIN_RESYNC_MS = 2000;  // and never more often than this
let lastResyncMs = -Infinity;

// audioAnchor holds the audio time and timeline time captured
// when the audio graph was last started.
function checkAudioDrift(audioCtx, audioAnchor, timelineNow) {
  const audioElapsed = audioCtx.currentTime - audioAnchor.audioTime;
  const timelineElapsed = timelineNow - audioAnchor.timelineTime;
  const drift = audioElapsed - timelineElapsed;

  const now = performance.now();
  if (Math.abs(drift) > DRIFT_LIMIT_S && now - lastResyncMs > MIN_RESYNC_MS) {
    restartAudioAt(timelineNow); // move the sound, never the clock
    lastResyncMs = now;
  }
}

The correction moves the sound and leaves the timeline alone. That is the point. A resync is audible if it happens too often, so the second constant exists to rate-limit it.

Step 6: Handle the loop wrap and stop#

A loop region repeats: when the playhead reaches the end, it jumps back to the start. The jump is a seek, and a seek invalidates anything prepared for the old position. The frame that was decoded ahead for the end of the loop belongs to the wrong side of the wrap, so it must be discarded before it is shown.

function onTick() {
  if (clock.playing) {
    if (clock.now() >= loopEnd) {
      clock.play(loopStart);          // seek by moving the anchor
      lookahead.clear();              // drop frames from the wrong side
      restartAudioAt(loopStart);      // keep sound following the wrap
    }
    // ... the frame logic from Step 2 continues here
  }
  requestAnimationFrame(onTick);
}

Stopping is simpler: pause the clock, and stop calling the decoder for new frames. The last picture stays on screen, which is what a viewer expects from a paused player.

How does requestVideoFrameCallback help?#

requestVideoFrameCallback, usually shortened to rVFC, is a method on the video element. It registers a callback that runs when a new video frame is sent to the compositor, which is the part of the browser that assembles what is painted.^[3] Unlike rAF, it is tied to the video's own frames, so it runs at the lower of the video's frame rate and the screen's refresh rate.^[4]

Its callback also receives metadata about the frame that was just presented. The most useful field is mediaTime, the frame's own timestamp in seconds on the video's timeline. presentedFrames counts every frame sent for composition, so a jump of more than one between callbacks means you missed frames.^[9]

const video = document.querySelector("video");
let lastPresented = 0;

function onVideoFrame(now, metadata) {
  const frame = frameIndexAt(metadata.mediaTime, 24);
  const missed = metadata.presentedFrames - lastPresented - 1;
  lastPresented = metadata.presentedFrames;
  if (missed > 0) console.warn(`missed ${missed} frame(s) before ${frame}`);

  // Re-register: the callback is one-shot, like rAF.
  video.requestVideoFrameCallback(onVideoFrame);
}
video.requestVideoFrameCallback(onVideoFrame);

The API has limits that matter for an editor. The callback runs on the main thread while compositing happens elsewhere, so the specification gives no strict timing guarantee. The WICG draft says a callback can occasionally fire one vertical sync late, and that drawing from it may appear one sync after the frame does.^[9] That is acceptable for a thumbnail or an analysis overlay. It is not enough, on its own, to promise that the frame you asked for is the frame you drew.

How do you keep audio and video in sync?#

Sync is not a state you reach once. It is a relationship that you maintain by measuring and correcting. When you start the audio at a position, the two clocks agree. Within a few seconds, the audio clock may run slightly fast or slow relative to the wall clock because of rounding, device differences or a busy thread. The drift check from Step 5 finds that gap and corrects it.

The second half of sync is scheduling. Web Audio lets you start a source at a specific time on the audio clock, which means you can line the sound up precisely at the moment you begin playback. The Web Audio article explains how that scheduling works in a mixing graph.

In our experience, viewers forgive a dropped picture far more readily than sound that slips out of step with the picture. That is why the rule favours moving the sound. A late frame is an inconvenience. A voice that lags behind a mouth is a fault that the viewer notices for the whole clip.

How the Azelify editor handles playback time#

The Azelify editor makes a deliberate choice that differs from the common rule of thumb. It keeps one process-wide clock derived from the wall clock, and it never derives timeline time from decoded audio. Audio is treated as an observer: it is compared with the clock, and it is restarted when it drifts too far. The picture therefore stays on schedule even when the audio briefly re-anchors.

The table summarises the decisions and what each one costs.

The preview grid deserves a note. A 24 fps project is still shown on a grid of at least 60 steps per second. That lets interactive overlays, the playhead and transforms move smoothly on a fast screen, without changing the authored frame rate. Export always renders on the project's own grid, so a 24 fps export contains 24 frames per second.

When a render takes longer than one frame period, the editor keeps only the newest pending request. A new request overwrites an older one in a single slot instead of joining a queue. That means latency never builds up behind a backlog of frames nobody will see. The cost is that an old request is quietly superseded, which is the correct result for a scrub.

The editor also keeps a rolling window of about ten seconds of decoded audio around the playhead, so it never holds the whole project in memory. A loop wrap discards any frame prepared for the wrong side of the wrap, and the frame at the landing time is rendered fresh.

Common mistakes#

Counting ticks instead of reading time. A loop that adds one to a counter on each callback runs at the speed of the screen. Always derive the frame from elapsed time, as the Step 2 loop does.

Rounding instead of flooring. Math.round moves a frame early for the second half of each frame interval. Floor matches the rule that frame n is on screen from n over fps until the next frame begins.

Trusting the audio clock under load. An audio clock is accurate while the audio thread keeps up. When it falls behind, timeline time slows down with it, and nothing in the picture explains why.

Assuming seek lands on the exact frame. The video element does not guarantee frame-accurate seeking.^[4] After a seek, wait for the frame you asked for before you trust the picture.

Frequently asked questions#

How do I play a video frame by frame in JavaScript?#

Read a time from a clock, convert it to a frame index with floor and the frame rate, and draw that frame only when the index changes. Use requestAnimationFrame to run the loop. For frame-exact output, decode the frames yourself rather than relying on the video element's currentTime.

Why does my video drift out of sync with the audio?#

Two clocks that run on different hardware never agree forever. Measure the gap between the audio clock and your playback clock, and restart the audio when the gap passes a small limit. Rate-limit the restart so it does not click.

What is the difference between requestAnimationFrame and requestVideoFrameCallback?#

requestAnimationFrame runs before each screen repaint, typically about 60 times a second. requestVideoFrameCallback runs when a new video frame is sent for composition, so it follows the video's frame rate and includes the frame's own timestamp. Use the first to drive a timeline and the second to observe a playing video.

Should I use the audio clock or performance.now() for playback?#

Use a single clock that you control, and derive the others from it. performance.now() is steady and never goes backwards, so it is a good master for pictures. Use the audio clock as a drift observer, so that a busy audio thread does not slow the whole timeline.

How do I handle a 120 Hz screen with 24 fps footage?#

Derive the frame from the clock, not from the number of callbacks. The screen will ask for five ticks per frame, and four of them will change nothing. This is correct behaviour, and it is why the loop compares the new frame index with the last one before it draws.

Why is the seek in my video player not frame accurate?#

The video element does not promise frame-accurate seeking, so the picture after a seek may be a nearby frame. Decode the target frame yourself with a decoder that is designed for frame access, and show it only when its timestamp matches the request.

Where to go next#

If you want to see how the decoded frames reach the loop, read how to decode video frames with WebCodecs. It explains timestamps, keyframes and the decode queue in the same bottom-up style.

For the sound side of sync, read the Web Audio API for video editors, which covers scheduling, gain and waveform drawing. Together they show how the two clocks in this article are kept in step.

Sources

1. Wikipedia, “Frame rate,” en.wikipedia.org (opens in a new tab) (accessed 2026-10-13). ↩

2. Wikipedia, “Presentation timestamp,” en.wikipedia.org (opens in a new tab) (accessed 2026-10-13). ↩

3. MDN Web Docs, “HTMLVideoElement: requestVideoFrameCallback() method,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩ ↩^[2]

4. web.dev, “Perform efficient per-video-frame operations,” web.dev (opens in a new tab) (accessed 2026-10-13). ↩ ↩^[2] ↩^[3]

5. MDN Web Docs, “Performance: now() method,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

6. MDN Web Docs, “High precision timing,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

7. MDN Web Docs, “BaseAudioContext: currentTime property,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

8. MDN Web Docs, “Window: requestAnimationFrame() method,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩ ↩^[2]

9. WICG, “HTMLVideoElement.requestVideoFrameCallback(),” wicg.github.io (opens in a new tab) (accessed 2026-10-13). ↩ ↩^[2]