Web Audio API for video editors: mixing tracks, drawing waveforms and loudness

This web audio API tutorial explains how a browser video editor plays, mixes and exports sound. You do not need an audio background to follow it. We start with what a sound looks like as numbers, then build a small graph of nodes, then schedule clips on a clock, then render a mix faster than real time, and finally measure loudness the way broadcasters do.

The Web Audio API is the browser's built-in audio system. It lets JavaScript create sources, route them through effects and volume controls, and send the result to the speakers or into a buffer. A video editor needs all of those things. Preview has to be live and responsive, export has to be exact and faster than the clip's own length, and both must sound the same. The Azelify editor is our worked example. It is a browser-native video editor that keeps those two paths in agreement, and the last large section explains the design choices we made and the costs they carried.

We will define each term when it first appears. If you already know the basics, skip to the section on offline rendering, which is where a video editor's audio code starts to differ from a music toy.

What is the Web Audio API?#

Sound is a pressure wave in air. A microphone turns that wave into a voltage, and an analogue-to-digital converter measures the voltage many times each second. Each measurement is a sample, which is a plain number giving the signal's amplitude at one instant. The number of samples per second is the sample rate, commonly 44,100 or 48,000 hertz. A list of samples in this raw form is called PCM, for pulse-code modulation. Video files usually carry compressed audio, which the browser decodes into PCM before it can process anything.^[1]

The Web Audio API works on those PCM samples inside an audio context. The AudioContext (opens in a new tab) interface is the home of one audio graph and owns the clock that every scheduled sound uses. A graph is made of nodes. Each node does one job, such as playing a buffer, multiplying samples by a number, filtering, panning or measuring. Nodes connect output to input, so samples flow from sources, through the nodes you choose, to a destination, which is the speakers.^[2]

Two more words are needed. A channel is one stream of samples, so stereo has two channels, left and right. A buffer is a block of samples held in memory, represented by the AudioBuffer (opens in a new tab) interface. A video editor's job is largely to decide which buffers play, when they play, and through which nodes.

The smallest useful graph has a source, a gain node and the destination. The code below plays one file once through a volume control. Comments explain each line.

// A minimal graph: one decoded file -> gain node -> speakers.
const ctx = new AudioContext();                // one context per application
const gain = ctx.createGain();                 // a node that multiplies samples by a number
gain.connect(ctx.destination);                 // the gain's output goes to the speakers

async function playOnce(url) {
  const bytes = await (await fetch(url)).arrayBuffer();  // download the file as bytes
  const buffer = await ctx.decodeAudioData(bytes);       // decode it to PCM samples
  const src = ctx.createBufferSource();                  // a one-shot player node
  src.buffer = buffer;                                   // the samples to play
  src.connect(gain);                                     // send it through the gain
  src.start();                                           // start immediately
}

Volume changes are scheduled through an AudioParam (opens in a new tab), which is the object behind every adjustable number on a node. The linearRampToValueAtTime (opens in a new tab) method is the straight-line fade used below.^[3] A gain's gain property is an AudioParam. You can set it to a value now, or ask it to ramp to a value by a given time. Scheduling on the context's clock is what keeps a fade sample-accurate, which a JavaScript timer cannot do.

// Fade a clip in over half a second, starting at the context's own clock.
const t0 = ctx.currentTime;                    // seconds on the audio clock right now
gain.gain.setValueAtTime(0, t0);               // silence at the start
gain.gain.linearRampToValueAtTime(1, t0 + 0.5); // straight line up to full volume

How do you schedule clips with the Web Audio API?#

A timeline is a list of clips, each with a start on the timeline, an in-point and an out-point inside its source file, and a volume. Playing the timeline means turning that list into start times on the audio clock. The source node's start method accepts three arguments: when, the audio time to begin, offset, the point inside the buffer to begin from, and duration, how much of the buffer to play.^[4]

JavaScript timers are not precise enough to start audio on time. A garbage collection or a busy frame can delay a setTimeout by tens of milliseconds, and that is audible. The standard fix is look-ahead scheduling: a timer wakes often, asks which clips begin within the next fraction of a second, and schedules them on the audio clock. The audio clock never stalls, so the sound lands on time even when the page is busy.

// Schedule one clip on the audio clock. timelineNow is the playhead, in timeline seconds,
// at the moment ctxNow (audio clock seconds) was read. Both readings are taken together.
function scheduleClip(ctx, buffer, clip, timelineNow, ctxNow) {
  let when = ctxNow + (clip.start - timelineNow);  // audio time the clip is first heard
  let offset = clip.sourceIn;                      // where to start inside the file
  let duration = clip.end - clip.start;            // how much of the file to play
  if (when < ctxNow) {                             // the clip has already begun
    const late = ctxNow - when;                    // seconds already past its start
    if (late >= duration) return;                  // it has already finished
    offset += late;                                // join the clip midway
    duration -= late;
    when = ctxNow;                                 // begin right away
  }
  const src = ctx.createBufferSource();
  src.buffer = buffer;
  src.connect(ctx.destination);                    // a real editor routes through its mixer
  src.start(when, offset, duration);
}

A seek is a jump of the playhead. The simplest correct response is to stop every source that is already scheduled and schedule again from the new position. The stop is immediate and the restart is on time, so the listener hears a clean jump rather than a click or a double sound. The code above handles the case of a clip that has already started, which is what a seek into the middle of a clip produces.

How do you decode a file and draw an audio waveform in JavaScript?#

A waveform is a picture of the samples. The raw data is far too dense to draw directly. A clip of ten minutes at 48,000 samples per second has 28.8 million samples per channel, and a screen is about two thousand pixels wide. The usual answer is to reduce the samples to one minimum and one maximum for each screen column. The eye then sees the loudness envelope, and drawing costs only a few thousand line segments.

decodeAudioData (opens in a new tab) turns the file's bytes into an AudioBuffer, and the buffer's channel data is read as a typed array of floating-point numbers. The reduction is a simple loop:

// Reduce one channel of samples to a min and a max per screen column.
// Output layout: [min0, max0, min1, max1, ...]. Compute once per file and cache it.
function peaksFor(channel, columns) {
  const perColumn = Math.floor(channel.length / columns); // samples under one column
  const peaks = new Float32Array(columns * 2);
  for (let c = 0; c < columns; c++) {
    let lo = 1;                                           // no sample is above 1 in PCM
    let hi = -1;                                          // ...nor below -1
    const start = c * perColumn;
    for (let i = start; i < start + perColumn; i++) {
      const v = channel[i];
      if (v < lo) lo = v;
      if (v > hi) hi = v;
    }
    peaks[c * 2] = lo;
    peaks[c * 2 + 1] = hi;
  }
  return peaks;
}

To draw the result, loop over the columns and draw a vertical line from min to max on a canvas (opens in a new tab). Two rules keep it fast. Compute the peaks once when a file is imported, not on every frame, and store them by a content hash so a reopened project does not decode the file again. For live meters that move with playback, the Web Audio API offers an analyser node that passes the signal through unchanged while exposing its current samples.^[5][6]

What is OfflineAudioContext and why use it for export?#

An OfflineAudioContext (opens in a new tab) builds the same kind of graph as the live context, but it does not play anything. It generates the output "as fast as it can" into an AudioBuffer, so the result can be encoded into a file.^[7] Its constructor takes the number of channels, the length in samples and the sample rate. Calling startRendering returns a promise for the finished buffer.

// Render a mix of any length to an AudioBuffer, faster than real time.
// build(ctx) connects the same graph shape that the live preview uses.
async function renderMix(seconds, sampleRate, build) {
  const ctx = new OfflineAudioContext(2, Math.ceil(seconds * sampleRate), sampleRate);
  build(ctx);                    // create nodes and schedule every clip on ctx
  return ctx.startRendering();   // resolves with the rendered AudioBuffer
}

The key idea is that the graph is built by a function that accepts a context. The live path and the export path call the same builder, once with a live context and once with an offline one. If a builder is written twice, the two copies drift apart, and a clip that sounds right in preview becomes wrong in the file. A single builder, or a single plan that both builders read, prevents that drift. The next section returns to this, because the Azelify editor uses exactly this approach.

Why do crossfades dip in volume?#

A crossfade is a transition where one clip fades out while the next fades in, so both are audible for a moment. The obvious way to write it is two linear ramps, one going down from 1 to 0 and one going up from 0 to 1. At the midpoint each ramp is at 0.5.

The trouble is that the ear responds more to power than to amplitude. Power is the square of amplitude, and it is what the ear's sense of loudness tracks most closely. For two unrelated sounds, their powers add. At the midpoint, each clip contributes a power of 0.5 squared, which is 0.25, and the two sum to 0.5. A power of 0.5 is about minus three decibels, so the listener hears a dip in the middle of the transition. An equal-power crossfade uses a quarter-circle instead of a straight line. The outgoing gain is the cosine of an angle and the incoming gain is its sine. At every point, the two powers add to exactly 1.

// Equal-power crossfade curves, sampled at n points. Pass them to setValueCurveAtTime.
function equalPowerCurves(n) {
  const outgoing = new Float32Array(n);
  const incoming = new Float32Array(n);
  for (let i = 0; i < n; i++) {
    const theta = (i / (n - 1)) * (Math.PI / 2);  // from 0 to a right angle
    outgoing[i] = Math.cos(theta);                 // falls from 1 to 0
    incoming[i] = Math.sin(theta);                 // rises from 0 to 1
  }
  return { outgoing, incoming };
}
// Usage: gainOut.gain.setValueCurveAtTime(curves.outgoing, start, length);

The setValueCurveAtTime (opens in a new tab) method plays a list of values across a time span and interpolates between them, which is how a curved fade is scheduled.^[8] The table compares the two laws. Neither is always right. Linear fades suit a single clip fading in or out with nothing else under it, and equal power suits a crossfade where two signals overlap.

The figure below shows a crossfade on a ten-second timeline. The overlap is the accent band.

Fig. 01

Clip A0–4s

Crossfade4–6s

Clip B6–10s

0s5s10s

4s · Cross starts6s · Cross ends

1. 1Clip A 0–4sfades out
2. 2Crossfade 4–6sboth audible
3. 3Clip B 6–10sfades in

Illustrative, not data[mdn-curve]

What is loudness, and how do you measure LUFS?#

Peak level is the largest sample. Loudness is what a listener hears, and the two are not the same thing. A quiet drum hit can have a high peak and sound soft, while a sustained vocal with a low peak can sound loud. A loudness meter therefore weights frequencies the way the ear does and averages over time.

The standard method is ITU-R BS.1770, which defines the measurement and calls the result a loudness in LKFS. Broadcasting in Europe uses the same measurement under the name LUFS, and the Wikipedia article on LUFS explains the naming.^[9][10] The method has four steps. First, K-weighting applies two filters that approximate how loud each frequency sounds to a listener. Second, the signal is cut into short overlapping blocks of 400 milliseconds, and each block's power is averaged across channels. Third, a two-stage gate discards silence: blocks below minus 70 LUFS are dropped first, and then blocks more than 10 LU below the average of what remains. Fourth, the surviving blocks are averaged into one integrated number for the whole programme.

Finally, true peak guards against clipping that a sample meter misses. A reconstructed analogue waveform can exceed the largest sample between two samples, so the method oversamples the signal, four times in our editor, before taking the peak.^[9]

The editor's targets follow the usual European conventions. Its broadcast target is minus 23 LUFS, which matches the reference level in EBU R 128, and it also offers a social target of minus 14 and a podcast target of minus 16.^[11]

How do you normalise loudness in two passes?#

Normalising means applying one gain so that the programme reaches a target loudness. The gain is simple arithmetic. If a programme measures minus 18 LUFS and the target is minus 14, the difference is plus 4 dB, which is a linear multiplier of about 1.58. The hard part is that the measurement needs the whole programme.

The gating in BS.1770 looks at every block across the entire programme, so a long export cannot be measured one chunk at a time and averaged. Averaging the chunk readings would gate each chunk against its own neighbours and give a wrong answer. The fix is two passes. Pass one renders every chunk and feeds all of them into a single loudness analyser, which gives one integrated reading for the whole programme. Pass two applies the gain and renders again.

// Pass 1 measures the whole programme. Pass 2 applies one gain to every chunk.
// measured and target are integrated loudness values in LUFS.
function gainForTarget(measured, target) {
  const differenceDb = target - measured;      // how many dB to add or remove
  return 10 ** (differenceDb / 20);            // decibels to a linear multiplier
}
// gainForTarget(-18, -14) is about 1.58, which is plus 4 dB.

Two rules keep the second pass honest. The gain is applied before the encoder, so every chunk receives the same multiplier. And the true-peak ceiling is checked afterwards, because a gain that reaches the target can still push peaks over the limit. A limiter then catches the peaks that would exceed the ceiling, which costs a little dynamic range but prevents clipping in the file.

Why does audio sometimes refuse to start?#

Browsers stop websites from making sound before a person has interacted with the page. This rule, the autoplay policy, is documented in MDN's guide, which covers the media element and the Web Audio API.^[12] In practice, an AudioContext can be created in a state called suspended, and it stays silent until the page resumes it from a user gesture such as a click.

// Create the context up front, then resume it from a click so the first sound is allowed.
const ctx = new AudioContext();
playButton.addEventListener("click", async () => {
  if (ctx.state === "suspended") await ctx.resume(); // the click counts as a user gesture
  // ...schedule and play the timeline here...
});

This is the first thing to check when a video editor makes no sound on load. The context is often created while the page is still starting, and nothing plays until the first click. A clear "press play to hear audio" state is kinder than a silent preview that looks broken.

Step-by-step: building an export mix#

The export path combines everything above. Each step has one input and one output, so each can be tested separately. The figure shows the order: the plan comes first, the audio is rendered twice, and the encoder receives the result.

Fig. 02

1. 01 → Shared planwho is audible, when
2. 02 → Render passOfflineAudioContext
3. 03 → Measure loudnessone analyser, all chunks
4. 04 → Apply gainone gain, every chunk
5. 05 → Encode audioto the encoder

Illustrative, not data[itu-bs1770]

Step 1: Resolve the plan#

Before any node is created, decide what is audible. Work out which tracks are muted or soloed, which clips are active at each moment, and each clip's source time, volume, pan, fades and speed. Write that answer down once, as data.

Step 2: Build the offline graph#

Create an OfflineAudioContext of the right length and channel count. Build the graph from the resolved plan, not from the live editor state, so that the export cannot pick up a change the user made during rendering.

Step 3: Render in chunks#

Render the programme in fixed windows of, say, fifteen seconds. Between chunks, yield to the browser for a moment, so it can reclaim the previous buffer before the next one starts. Keep the fades in absolute clip time so that a fade does not restart at each chunk boundary.

Step 4: Measure the programme#

Feed every chunk into one loudness analyser. Read the integrated loudness once all chunks have been measured. If no target is set, skip this step's gain but still report the measurement, because a user who exports should learn what the file measures.

Step 5: Apply the gain and limit#

Compute the gain from the target, render the chunks again, and multiply each one by the gain. Check the true peak afterwards and limit any peak that would exceed the ceiling.

Step 6: Hand the audio to the encoder#

Pass each finished chunk to the audio encoder in order. The encoder and the muxer then combine the audio with the video into one file. Muxing puts the encoded streams into a single container, which the next article in this series covers in detail.

How the Azelify editor handles audio#

The Azelify editor has two complete implementations of the same audio rules. The preview uses a realtime Web Audio graph, and the export uses an offline render. The hard part of the design is keeping those two in agreement, so that a fade, a mute or a crossfade sounds the same in the file as it did on the timeline. We chose one shared plan that both paths read. Once the plan says which clip is audible and at what level, neither path makes that decision again.

Each track in the live graph follows a fixed chain: a source for each scheduled clip, a clip gain that carries volume and fades, a track input gain, an optional effect chain of compressor, equaliser, reverb and delay, a stereo panner, a track output gain, and a master gain before the destination. Analysers sit on the track outputs so that the meters move with the sound. The offline graph mirrors that chain node for node, so the export has the same structure as the preview.

Fig. 03

1. 01 → Source bufferone scheduled clip
2. 02 → Clip gainvolume and fades
3. 03 → Effect chaincompressor, EQ, reverb
4. 04 → Stereo pannerleft and right balance
5. 05 → Track gainmute and solo
6. 06 → Destinationthe speakers

Illustrative, not data[mdn-webaudio]

On the way out, the export applies a fixed finishing policy to the whole mix: a limiter on the master so that peaks never clip and, when asked, the two-pass loudness normalisation described above.

Scheduling uses look-ahead with a window of about a fifth of a second. A seek tears down the scheduled sources and schedules them again from the new position, which is the pause-free reschedule described above. The editor also keeps only a sliding window of decoded buffers around the playhead, topped up once a second, so a long project does not decode every file when it opens. Sources longer than two minutes are streamed in segments rather than decoded whole. A ninety-minute podcast at 48 kHz stereo in 32-bit floats would take roughly 2 GB as one buffer, which is why the editor streams it.

Reverb is deterministic. The impulse response that makes a room sound is generated from seeded noise, not true randomness, so the same settings always produce the same samples. We chose that because random noise would make every export sound slightly different and would make byte-for-byte comparison of exports meaningless. Procedural sounds follow the same rule: each one is a pure function of its voice, its parameters, its seed and the sample rate, so the preview and the export hear the same samples.

Crossfades get their own fade laws at each end of a clip. The end that takes part in a crossfade uses equal power, while an ordinary authored fade stays linear. A clip that is outgoing at its tail and fading in at its head needs both laws at once, so the law belongs to each end rather than to the whole clip. The plan also extends each clip a little past its own edges, so that the audio under a video transition actually moves on both sides, instead of one side holding a still.

Export renders and encodes audio in fifteen-second chunks, with a short yield between them, and runs the two-pass loudness process described above. One limitation is worth stating plainly. The realtime graph and the offline graph share their semantics, but the sample output of a live graph is not guaranteed to match an offline render bit for bit, because the browser's realtime scheduling is not deterministic. The code says so rather than promising a match it cannot keep. The shared plan is what keeps the two sounding the same.

Frequently asked questions#

What is the Web Audio API used for in a video editor?#

It plays decoded clips on a precise clock, routes them through gain and effect nodes, and renders the final mix into a buffer for export. A video editor uses the live context for preview and an offline context for the file, both built from the same description of the timeline.

What is LUFS and how is it different from peak level?#

LUFS is a loudness measurement that weights frequencies the way the ear does and averages over the programme. Peak level is simply the largest sample. Two clips with the same peak can sound very different in loudness, so streaming and broadcast standards use LUFS for normalisation.

Can I render audio faster than real time in the browser?#

Yes. An OfflineAudioContext generates its output as fast as the processor allows, so a ten-minute mix can finish in a fraction of its length. The speed depends on the graph's complexity and on the device.

Why is my audio silent until I click something?#

The browser's autoplay policy keeps an AudioContext suspended until a user gesture. Resume the context from a click handler, and show a clear play state so the user knows why nothing is heard.

How do I draw a waveform for a long audio file?#

Reduce each screen column to a minimum and a maximum sample, compute those peaks once when the file is imported, and cache them by a hash of the file. Drawing a few thousand segments is fast, while drawing millions of raw samples is not.

Why does audio start late after a seek?#

The usual cause is scheduling on a JavaScript timer rather than on the audio clock. Stop every scheduled source on a seek, then schedule again from the new position, and always pass the start time to the source node's start method.

Where to go next#

Audio has to stay in step with the picture, and the clock that drives playback is covered in frame-accurate video playback in JavaScript. When the mix is ready, exporting an MP4 from the browser explains how the encoded audio and video are written into one file.

Sources

1. MDN Web Docs, “Basic concepts behind Web Audio API,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

2. MDN Web Docs, “Web Audio API,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

3. MDN Web Docs, “AudioParam: linearRampToValueAtTime() method,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

4. MDN Web Docs, “AudioBufferSourceNode: start() method,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

5. MDN Web Docs, “Visualizations with Web Audio API,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

6. MDN Web Docs, “AnalyserNode: getFloatTimeDomainData() method,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

7. MDN Web Docs, “OfflineAudioContext,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

8. MDN Web Docs, “AudioParam: setValueCurveAtTime() method,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩

9. ITU, “Algorithms to measure audio programme loudness and true-peak audio level, Recommendation ITU-R BS.1770-4,” www.itu.int (opens in a new tab) (accessed 2026-10-13). ↩ ↩^[2]

10. Wikipedia, “LUFS,” en.wikipedia.org (opens in a new tab) (accessed 2026-10-13). ↩

11. EBU, “Loudness normalisation and permitted maximum level of audio signals,” tech.ebu.ch (opens in a new tab) (accessed 2026-10-13). ↩

12. MDN Web Docs, “Autoplay guide for media and Web Audio APIs,” developer.mozilla.org (opens in a new tab) (accessed 2026-10-13). ↩