How to build a video editor in the browser: the complete architecture
A plain-language map of a browser video editor: the project file, timeline, decoding, compositing, audio and export, and how each piece connects.
A fifteen-part series on building a video editor that runs in the browser: project files, undo, timelines, WebCodecs, compositing, audio and export.
Azelify's editor runs in a browser tab: it decodes footage, draws every frame and mixes the sound on your own machine, with nothing to install. This series explains how an editor like that is built, one part at a time, for a reader who knows some JavaScript but has never worked with video.
The parts follow the order you would build them in. The architecture overview maps the whole system first. Then come the document every feature reads and writes: the project file, undo and redo and the timeline data model. The middle parts turn media into pictures: importing files, decoding frames with WebCodecs, frame-accurate playback, compositing, text and on-canvas handles. The last parts cover sound, exporting an MP4, keyframe animation, testing a renderer and a comparison of FFmpeg.wasm, WebCodecs and MediaRecorder.
Each part defines its terms the first time it uses them, includes code you can run, and cites the specification or browser documentation behind each claim. Where we describe our own editor, we say what we chose and what it cost, so you can make a different call for yours.
A plain-language map of a browser video editor: the project file, timeline, decoding, compositing, audio and export, and how each piece connects.
Design the JSON document behind a web video editor: what a project stores, how to version and migrate it, and how to autosave without losing work.
Implement undo and redo for a browser editor with the command pattern: commands, history stacks, grouping a drag into one step, and common traps.
Learn the data model behind a non-linear video timeline: tracks, clips, in and out points, trims, ripple edits and how overlapping clips are resolved.
Handle video uploads in a web app: read container metadata, check codec support, build lighter proxy files and store large media without crashing a tab.
Decode video frame by frame with the WebCodecs VideoDecoder: demux samples, configure the decoder, seek to keyframes and release VideoFrame memory.
Build a playback loop for a web video editor: one master clock, requestAnimationFrame pacing, dropped-frame handling and keeping audio and video in sync.
Composite video layers in the browser: Canvas 2D versus WebGL, transform matrices, blend modes, shader effects and drawing each frame in the right order.
Render titles and word-by-word captions in a web video editor: canvas text versus HTML and CSS, font loading, timing words to speech and exporting text.
Build drag, resize and rotate handles for a canvas editor: hit testing, pointer capture, screen-to-canvas coordinates, aspect locks and rotated bounds.
Use the Web Audio API in a video editor: an audio graph for live mixing, OfflineAudioContext for export, waveform drawing and loudness normalisation.
Export an MP4 from a web video editor: render frames, encode with the WebCodecs VideoEncoder, mux audio and video, and know when to render on a server.
Implement keyframe animation for a web editor: storing keyframes, interpolating values, cubic Bézier easing, and pinning graphics to tracked motion.
Test a browser video renderer properly: golden image comparisons, pixel diff tolerances, independent oracles, property-based tests and their blind spots.
Compare FFmpeg.wasm, WebCodecs and MediaRecorder for browser video: speed, codec support, file size, frame accuracy and which one to choose for an editor.