Importing video files in the browser: probing, codecs, proxies and storage

If you want to upload video in browser JavaScript, the import step decides more than most people expect. It decides whether a phone clip opens at all, whether the editor can scrub it smoothly, and whether a ten-minute project survives a page reload. The work is not just reading a file from a drop zone. It is working out what the file contains, whether this browser can decode it, and where its bytes should live while the user edits.

This article walks through that pipeline from the bottom up. It defines the terms you need, shows real browser code, and explains why each step exists. We have built an import path for our own browser editor, so the later sections describe the decisions we made, along with the costs of those decisions.

What happens when someone drops a video file into a page?#

When a user drags a file onto a page, the browser delivers it through a drag-and-drop event. A page can only receive that drop if it first cancels the dragover event, which tells the browser that this element accepts the drop.^[1] The dropped files arrive as a list on the event's data transfer object.^[2] Here is the minimum needed to receive one.

// A drop zone that accepts a dropped video file.
const zone = document.getElementById("drop-zone");

zone.addEventListener("dragover", (e) => {
  e.preventDefault(); // without this, the browser never fires "drop"
});

zone.addEventListener("drop", (e) => {
  e.preventDefault();
  const file = e.dataTransfer.files[0]; // a File object
  if (file) importVideo(file);
});

The object you receive is a File, which is a kind of Blob. A Blob is a file-like object of immutable, raw data, with a size in bytes and a type string such as video/mp4.^[3] It does not have to be a file on disk. The browser can hand you a blob from memory, from a network response or from a drop, and the same API reads all of them. Notice the word immutable. You cannot edit the bytes of a blob in place. You can only read slices of it or make a new one, and that matters when we talk about storage later.

Nothing in that snippet has decoded a single frame yet. The import has only received bytes. Everything that follows is about deciding what those bytes are.

Why is the file extension not the codec?#

Most people think of a video file as a single thing with a name like clip.mp4. In practice a video file is two layers. The container is the wrapper: it holds the streams, the timing information and the metadata. The codec is the compression method used for the video inside the wrapper. A container is like a box, and a codec is the way the contents were packed into it.^[4]

A codec is a hardware or software component that encodes or decodes a data stream, and the name is short for coder and decoder.^[5] An MP4 container can hold H.264 video with AAC audio, or H.265 video with other audio, or other combinations. The extension .mp4 tells you about the box. It tells you almost nothing about what is packed inside. A .mov from a phone and an .mp4 from a screen recorder can hold the same codec in two different boxes, or two different codecs in one box.

This is why the extension is a poor guide. A renamed file would fool any check based only on the name. The better test is the file signature: a short sequence of bytes at the start of the file that identifies its format. The Wikipedia list of file signatures documents them for many formats.^[6] For ISO base media files, which include MP4 and MOV, the four characters ftyp appear at byte offset 4. WebM and Matroska files instead start with the four bytes 1A 45 DF A3 at offset 0.^[6]

// Read the first 12 bytes and decide which container family this is.
async function sniffContainer(file) {
  const head = new Uint8Array(await file.slice(0, 12).arrayBuffer());
  const text = String.fromCharCode(...head.slice(4, 8)); // bytes 4 to 7

  if (text === "ftyp") return "iso-bmff";                   // MP4, MOV, M4V
  if (head[0] === 0x1a && head[1] === 0x45 &&
      head[2] === 0xdf && head[3] === 0xa3) return "matroska"; // WebM, MKV
  return "unknown";
}

The function reads only twelve bytes, no matter how large the file is. That is the key habit. file.slice returns a new blob that refers to a byte range, and arrayBuffer() reads only that range. A two-gigabyte file costs the same to sniff as a small one.

The table below shows how the layers combine. It is a simplified summary, and real files vary, but it shows why one name can hide several different decoding problems.

The container also carries the timing information, which matters for editing. A phone may record with a variable frame rate, meaning the time between frames is not constant. Wikipedia notes that many smartphone cameras record this way, sometimes causing audio and video sync difficulties during editing.^[7] We will come back to that in the step-by-step section.

How do you read video metadata in JavaScript?#

Metadata means everything the container says about the file without decoding it: the number of tracks, each track's codec string, its dimensions, its duration and its timing. Reading it is called demuxing, which means splitting the container into its separate streams. The browser's built-in video element can report a few facts, but it cannot hand you the encoded chunks you need for careful editing. For that, a demuxer library is the standard tool.

A common choice is mp4box.js, a JavaScript version of the MP4Box tool from the GPAC project. It reads MP4 files incrementally in the browser and reports each track's details.^[8] The basic flow is to create a file object, wait for the onReady callback that fires once the header is parsed, and then read the track list.

import MP4Box from "mp4box";

// Parse an MP4 header and report each track's codec and size.
function readMetadata(file) {
  return new Promise((resolve, reject) => {
    const mp4 = MP4Box.createFile();

    mp4.onReady = (info) => {
      // info.tracks lists every stream; each has an id and a codec string.
      resolve(info.tracks.map((t) => ({
        id: t.id,
        codec: t.codec,            // e.g. "avc1.42c00d" for H.264
        timescale: t.timescale,    // ticks per second for this track
        duration: t.duration / t.timescale, // in seconds
      })));
    };
    mp4.onError = (e) => reject(new Error(String(e)));

    // Feed the file in pieces. Each piece needs a fileStart offset.
    const reader = file.stream().getReader();
    let offset = 0;
    (async function pump() {
      const { done, value } = await reader.read();
      if (done) { mp4.flush(); return; }
      const buffer = value.buffer.slice(value.byteOffset, value.byteOffset + value.byteLength);
      buffer.fileStart = offset;
      offset += buffer.byteLength;
      mp4.appendBuffer(buffer);
      pump();
    })();
  });
}

Two details in that code matter. First, the codec string is the browser's own name for the codec and its profile. The string avc1.42c00d tells you H.264 with a particular profile and level, and it is the exact value a decoder needs.^[8] Second, timestamps in a container are counted in ticks, and the timescale converts ticks to seconds. Dividing duration by timescale gives seconds. Mixing those up is one of the most common import bugs.

Can this browser decode it? isConfigSupported vs canPlayType#

Knowing the codec string is only half the question. The other half is whether this particular browser, on this device, can decode it. The older tool for that is canPlayType, which lives on the media element. It returns one of three strings: an empty string if the media cannot be played, probably if it likely can, and maybe if there is not enough information to decide.^[9]

The problem is that maybe is the common answer. A video/mp4 check on a fresh element returns maybe, which tells you little. The newer tool is VideoDecoder.isConfigSupported, which asks a specific question: can a decoder be configured with this codec and these dimensions? It resolves to an object with a supported boolean and a copy of the config.^[10]

// Ask the browser whether it can decode this exact stream, before we commit to it.
async function canDecodeTrack(track) {
  if (typeof VideoDecoder === "undefined") return false; // no WebCodecs at all

  const result = await VideoDecoder.isConfigSupported({
    codec: track.codec,          // the string from the demuxer, e.g. "avc1.42c00d"
    codedWidth: track.video.width,   // pixel size, so the check is for this size
    codedHeight: track.video.height,
    hardwareAcceleration: "prefer-hardware",
  });
  return result.supported === true;
}

Notice the three inputs. The codec string says what the stream is. The dimensions matter because a decoder can support a codec up to a certain size and not beyond it. The hardware preference matters because a device can decode a codec in dedicated hardware, which is fast and efficient, or in software, which is slower and uses more battery.

Codec support also varies by browser and platform. The MDN codec guide notes that HEVC, also called H.265, is a proprietary format covered by patents, and that its support in browsers depends on the operating system and on whether a hardware decoder is present.^[11] A phone clip recorded in HEVC can therefore import cleanly on one machine and fail on another. That is the reason the check is per device, not a fixed list of formats in your code.

Fig. 01

1. Can the browser decode it?
   1. yes → Is the source heavy to edit?
      1. yes → Build proxy, keep original
      2. no → Edit the original directly
   2. no → Reject with a clear message

Illustrative, not data[codecs]

The decision tree above is the core of the import flow. It has only two questions because most imports need only two answers. The first answer can stop the import. The second answer decides how the editor will work with the file, and we turn to that next.

Where should the bytes live?#

A video file can be hundreds of megabytes, and a browser page has limited memory. You should never keep the whole file in a JavaScript object, and you should never store it inside the project document that gets saved as JSON. That second rule is worth a small demonstration, because it is an easy trap.

// A project object that references a video blob.
const project = { name: "Promo", clips: [{ id: "c1", blob: videoBlob }] };

// Saving the project as JSON, as many editors do.
console.log(JSON.stringify(project));
// {"name":"Promo","clips":[{"id":"c1","blob":{}}]}

The blob has become an empty object. I checked this in Node, where JSON.stringify writes only an object's own enumerable properties. A blob keeps its size and type as read-only properties rather than as own properties, so the saved project contains no bytes at all.^[3] The project looks fine until you reload it and find the clip has nothing to play. The fix is to keep bytes in a store built for binary data and keep only an identifier in the project.

The browser offers two main places for that. IndexedDB is a low-level API for storing significant amounts of structured data, including files and blobs.^[12] The origin private file system is a storage area that is private to the page's origin and not visible to the user, and it offers fast byte-level access.^[13] Both count against the same origin quota, and the storage quota rules decide what the browser can keep.^[14] Those rules matter. Data stored by default is best-effort: it can be evicted under storage pressure, and the browser removes the least recently used origins first. Requesting persistent storage with navigator.storage.persist() protects the data from that eviction, though the browser may still refuse.^[14]

Here is a small write into the origin private file system, the way an import would store a copy of the bytes:

// Copy a dropped file into the origin private file system and return its name.
async function storeOriginal(file, mediaId) {
  const root = await navigator.storage.getDirectory();
  const handle = await root.getFileHandle(`${mediaId}.bin`, { create: true });
  const writable = await handle.createWritable();
  await writable.write(file);   // streams the blob's bytes into the file
  await writable.close();       // the data is committed only on close
  return handle.name;
}

The project then holds mediaId and the file name, not the bytes. Each time the editor needs a frame, it looks up the file by that name. The close() call is easy to forget and important to keep: until the writable is closed, the write is not committed.

Why build a smaller proxy copy?#

A high-resolution phone clip is expensive to decode during scrubbing. Scrubbing jumps around the file, and each jump can require decoding forward from a nearby keyframe. A keyframe is a frame that can be decoded without any earlier frame, and the stretch between keyframes is called a group of pictures.^[15] A long gap between keyframes makes every jump slow. A lower-resolution copy with frequent keyframes makes the same jump cheap.

Fig. 02

GOP 10–2s

GOP 22–4s

GOP 34–6s

GOP 46–8s

0s5s

5s · Scrub to 5s

1. 1GOP 1 0–2skey at 0s
2. 2GOP 2 2–4skey at 2s
3. 3GOP 3 4–6skey at 4s
4. 4GOP 4 6–8skey at 6s

Illustrative, not data[gop]

A proxy is that smaller, easier copy. The editor plays and scrubs the proxy and shows the user a responsive timeline. Export reads the original, so the final result still has full quality. A proxy is a convenience for editing, never a replacement for the source. The trade-off is storage and background work: making a proxy takes encoder time and disk space, so it is worth building only when the source is heavy enough to need it.

Step-by-step: from drop to editable clip#

The steps below follow the same order as the import pipeline. Each one has a check you can run before moving on. The flow diagram shows the whole path first, so you can see where each check sits.

Fig. 03

1. 01 → DropFile object, a Blob
2. 02 → Snifffirst 12 bytes
3. 03 → Probetracks and codec
4. 04 → Decode checkisConfigSupported
5. 05 → Storebytes outside project
6. 06 → Proxyoptional, background

Illustrative, not data

Step 1: Accept the file on drop#

Wire the dragover and drop handlers shown earlier, and log the file's name and size. Confirm that a drop of a text file is still rejected by the later steps, so you know the pipeline is not trusting the extension.

Step 2: Sniff the file signature#

Run the sniffContainer function on the first twelve bytes. Reject anything that is not a known container family, and show a message that names the problem. Test with a renamed file, such as a ZIP saved as .mp4, to confirm the rejection happens on the bytes rather than the name.

Step 3: Probe the tracks#

Read the track list with the demuxer. Confirm that you get one video track with a codec string and dimensions, and that the duration matches what the player reports. A mismatch between the container's duration and the player's duration is a sign of variable timing, and the file deserves a closer look.

Step 4: Ask the browser whether it can decode#

Call canDecodeTrack with the codec string and the dimensions from the probe. If the answer is no, stop the import here and explain why. Do not try to build a proxy for a file the browser cannot decode at all.

Step 5: Store the bytes outside the project#

Write the original into the origin private file system, or into IndexedDB, and keep only the identifier in the project record. Then serialize the project and confirm the blob is gone and the identifier is still present. That check catches the JSON trap before a user does.

Step 6: Decide whether a proxy is worth building#

If the source is high resolution, or it uses a codec that is slow to scrub, schedule a background proxy. Keep the original for export. If the file is small and fast, skip the proxy entirely. Proxies are an optimization, so a missing proxy should never block editing.

How the Azelify editor handles import#

The Azelify editor follows the same pipeline, with a few decisions that are worth stating as trade-offs. The first is that the server stores the uploaded bytes and checks their file signature before trusting the declared type. We cap file size by media type, so a single oversized upload cannot exhaust the storage budget. The check happens early, which costs a round trip on a bad file but keeps invalid bytes out of the rest of the system.

The second decision is that the editor probes each file once and records the result as a small set of policy verdicts: whether the clip fits the project, whether it needs a proxy, whether it uses a color range that must be converted, and whether it can be exported. Every later stage reads those verdicts instead of working out codec rules again. The cost is a single probing step that must stay correct as codecs change. The benefit is that a rule such as "this codec cannot export" lives in one place.

The third decision is about proxies. Re-encoding runs in a background worker, one job at a time, because two hardware encoders running together compete for the same resource. The trade-off is that a long queue of imports takes longer to finish. Playback starts immediately on the original, and export always reads the original. If a proxy cannot be built, the editor falls back to the original without an error. Editing stays possible even when the background work fails.

The fourth decision concerns what the editor shows the user. Thumbnails, filmstrips and waveforms are generated from the media, but they are never added to the user's media library. Only a real import appears there. Without that rule, a library fills up with entries the user never chose. The trade-off is that the editor must track derived files separately, and it must clean them up when the source is removed.

A GIF is a useful case to end on. In the document, an animated GIF behaves like a video clip, so it can be trimmed and sped up on a video track. The decoding library cannot read GIF files, so the editor chooses its decode path by looking at the bytes, not the clip's type. That is the same lesson as the file signature: the bytes are the source of truth, and the label is only a convenience.

Frequently asked questions#

Why does my phone video fail to import?#

The most common reasons are an HEVC codec that this browser cannot decode, a high dynamic range color space that needs conversion, or a variable frame rate that the timing code does not expect. Run a per-track check with isConfigSupported and print the codec string. The message tells you which of these applies.

Can I read video metadata in JavaScript without uploading the file?#

Yes. A demuxer such as mp4box.js reads the file locally, from a File or a blob the user already selected. Only the bytes you choose to send to a server ever leave the device. Reading the header touches only a small part of the file, so it stays fast even for large clips.

How large a video can a browser handle?#

There is no single number. The limit depends on memory on the device, the origin's storage quota and the decoder budget of the hardware. The storage quota rules explain how the browser decides what to keep, and a well-built editor treats the limit as a budget it checks rather than a constant it assumes.

Is canPlayType enough to check support?#

Usually not. It often returns maybe, which is not a yes. A codec and size check through VideoDecoder.isConfigSupported answers the specific question your pipeline needs.

Can I keep the video in localStorage?#

No. Web Storage has a small per-origin limit, about 5 MiB for each of local and session storage, and it stores strings rather than binary blobs.^[14] A video needs IndexedDB or the origin private file system.

Do I need a proxy file for every video?#

No. A proxy helps when the source is heavy to scrub, such as high-resolution footage with sparse keyframes. For a small, fast clip it adds encoder time and storage without a visible benefit. Build proxies on demand and keep the original as the export source.

Where to go next#

For the decoding step that follows an import, read WebCodecs decoding of video frames, which turns the chunks you probed here into frames on a canvas. The project-document side of this pipeline, which decides what the saved record holds, is covered in the video editor project file format.

Sources

1. MDN Web Docs, “HTML Drag and Drop API,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

2. MDN Web Docs, “DataTransfer: files property,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

3. MDN Web Docs, “Blob,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩ ↩^[2]

4. Wikipedia, “Container format,” en.wikipedia.org (opens in a new tab) (accessed 2026-10-13). ↩

5. Wikipedia, “Codec,” en.wikipedia.org (opens in a new tab) (accessed 2026-10-13). ↩

6. Wikipedia, “List of file signatures,” en.wikipedia.org (opens in a new tab) (accessed 2026-10-13). ↩ ↩^[2]

7. Wikipedia, “Variable frame rate,” en.wikipedia.org (opens in a new tab) (accessed 2026-10-13). ↩

8. GitHub, “gpac/mp4box.js,” github.com (opens in a new tab) (accessed 2026-10-13). ↩ ↩^[2]

9. MDN Web Docs, “HTMLMediaElement: canPlayType() method,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

10. MDN Web Docs, “VideoDecoder: isConfigSupported() static method,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

11. MDN Web Docs, “Web video codec guide,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

12. MDN Web Docs, “IndexedDB API,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

13. MDN Web Docs, “Origin private file system,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

14. MDN Web Docs, “Storage quotas and eviction criteria,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩ ↩^[2] ↩^[3]

15. Wikipedia, “Group of pictures,” en.wikipedia.org (opens in a new tab) (accessed 2026-10-13). ↩