Building In-Browser Video Conversion with WebCodecs | Why I Skipped ffmpeg.wasm

The moment anyone brings up converting video inside the browser, ffmpeg.wasm is the first name that comes up. Almost every implementation you find by searching uses it, it has earned its place as the default choice, and since your existing knowledge of the ffmpeg command carries straight over, the appeal is obvious.

When I built a set of tools that convert images, audio and video inside the browser, I passed on it and went with WebCodecs and mediabunny instead.

This article covers why I left ffmpeg.wasm out, what I used in its place, the things that tripped me up while implementing it, and the processing times I actually measured. This is not a story about picking whichever one is faster. The reason I chose it was licensing; the difference in speed is something I only found out afterwards by measuring.

One thing to establish up front: this reasoning only holds under the condition that the source is not public and the processing lives on the client. If you convert on the server, you can use ffmpeg as-is, and if your project is open source, the GPL is no obstacle either. Change the conditions and the conclusion changes with them, so please read on with that in mind.

• Who this is for: anyone weighing up an implementation that converts video inside the browser
• Assumed: TypeScript, and a browser with WebCodecs support (Chrome or Edge)

Why I left ffmpeg.wasm out 

Ship it to the client and the GPL applies 

Licensing is the biggest reason, and since this is the spot where people tend to assume that "the GPL has nothing to do with a web service", let me separate the two cases first.

FFmpeg itself is LGPL 2.1 or later by default, but building it with --enable-gpl and --enable-libx264 turns the whole thing GPL. The @ffmpeg/core package published on npm is built exactly that way, and its package.json states GPL-2.0-or-later as the license. The wrapper @ffmpeg/ffmpeg is MIT, which makes this easy to mix up, but the codecs live in core.

In other words, the exact same feature would have been free to use had I run ffmpeg on the server. The decision to keep everything inside the browser is itself what put me on the wrong side of the GPL. What I build is not open source, so I dropped it here.

There is a way out: build your own LGPL version with libx264 excluded. Do that and H.264 goes with it, and a video converter that cannot produce MP4 is not a video converter, so that road was closed from the start.

The memory ceiling 

ffmpeg.wasm keeps both the input and the output file in the virtual file system inside wasm. That means the size of the file you are converting eats into memory directly, which puts a practical ceiling on what you can handle.

The tools I run advertise no time limit and no usage limit, so a design that caps out on input size does not sit well with that promise. This was another reason to leave it out.

The multi-threaded build affects the whole site 

ffmpeg.wasm does ship a multi-threaded build, and it is faster. It needs SharedArrayBuffer though, which means setting COOP and COEP headers across the entire site. Those headers reach as far as the third-party embeds on your pages, so retrofitting them onto an existing site is not a small job.

What I used instead 

The implementation rests on WebCodecs and mediabunny, and the split between them is clean.

WebCodecs is an API that calls the encoders and decoders the browser already has, straight from JavaScript, and because no codec is shipped with the app, the GPL problem disappears right there. What WebCodecs deals in is only frames of video and audio, though. Reading and writing containers such as MP4 or WebM sits outside its scope, and that is the gap mediabunny fills.

Choosing mediabunny was closer to elimination than selection. Its author previously published mp4-muxer and webm-muxer, and both have since been folded into mediabunny and deprecated, which leaves effectively one option for this job.

Installing it, and the smallest thing that works 

pnpm add mediabunny

If all you need is a conversion to MP4, this is the whole thing.

import {
  ALL_FORMATS, BlobSource, BufferTarget, Conversion,
  Input, Mp4OutputFormat, Output, Quality, canEncodeVideo,
} from "mediabunny";

export async function convertToMp4(file: File): Promise<Blob> {
  // Encoder support differs per browser, so ask before doing any work
  if (!(await canEncodeVideo("avc"))) {
    throw new Error("This environment cannot write H.264");
  }

  const input = new Input({ source: new BlobSource(file), formats: ALL_FORMATS });
  const output = new Output({ format: new Mp4OutputFormat(), target: new BufferTarget() });

  const conversion = await Conversion.init({
    input,
    output,
    video: { codec: "avc", quality: new Quality("medium") },
  });
  await conversion.execute();

  return new Blob([output.target.buffer], { type: "video/mp4" });
}

Conversion takes care of demuxing, decoding, encoding and muxing together, so you never write code that walks frames one at a time. Changing the resolution or frame rate is a matter of adding height or frameRate to video.

Passing BlobSource is the part that matters: instead of reading the whole file into memory, it reads only the range it needs as the conversion moves along. That structure is why handing it a two-hour video does not pin your memory.

What tripped me up 

What follows is the set of things reading the documentation would not have saved me from.

Try mediabunny before Web Audio when decoding 

When expanding audio to PCM, Web Audio’s decodeAudioData is tempting. The catch is that this API resamples to the sample rate of the AudioContext (48kHz in most environments) whether you asked for it or not. Load a 44.1kHz source and 48kHz is what you get back, which rules it out whenever you need the original rate preserved.

So I try mediabunny’s decoder first and only fall back to Web Audio when that fails, and I have not changed that order since.

Encoder support varies by browser and OS 

This is where WebCodecs caught me out the hardest. Even within Chrome, which codecs you get depends on the OS. Concretely, the Linux build of Chromium has no AAC encoder.

That is why I query canEncodeVideo() and canEncodeAudio() beforehand and reject unsupported formats before any processing starts. When a video can be encoded but its audio cannot, I write out the video alone and put "audio was dropped" on screen. Dropping it silently means the user finds out only after downloading a silent video.

The result of canEncodeAudio() is memoized 

This behaviour is not written down anywhere. mediabunny’s canEncodeAudio() memoizes its result, so calling it once before you register an encoder locks that false in place.

The MP3 encoder comes from @mediabunny/mp3-encoder and has to be registered separately. Because I had a support check running ahead of that registration, the app went on reporting "MP3 cannot be written" long after the encoder was in place. It now asks WebCodecs directly.

React’s development mode runs useEffect twice, so the registration fires twice as well. I share the registration promise and await that instead.

There is a sequel to this, and I ran into it after publishing. What gets memoized is not only the settled result: the in-flight promise itself is shared (see mediabunny’s encode.ts). One stalled probe therefore drags down every other piece of code waiting on the same question.

On my setup, AudioEncoder.isConfigSupported({ codec: "mp3" }) did not return for about 43 seconds. Every WebCodecs probe on the page stalled behind it, and the video conversion was dragged along with them. Neither CPU nor bandwidth was in use during that window, so I spent a long time assuming the processing was heavy when it was simply waiting.

The probes now have a time limit and move on when nothing comes back. If the format really is unsupported, encoder initialization fails anyway, so there is nothing to gain from waiting on a probe forever.

// 能力判定が返ってこないことがあるので、待ち時間に上限を設ける
async function withProbeTimeout<T>(probe: Promise<T>, fallback: T, ms = 5000): Promise<T> {
  let timer: ReturnType<typeof setTimeout> | undefined;
  try {
    return await Promise.race([
      probe,
      new Promise<T>((resolve) => {
        timer = setTimeout(() => resolve(fallback), ms);
      }),
    ]);
  } finally {
    clearTimeout(timer);
  }
}

Measured: how far apart are they? 

Speed was not my reason for choosing, but measuring made the gap plain.

The source is a file generated with ffmpeg: 20 seconds, 1280x720, 30fps, H.264 + AAC, 8.2 MB. The machine is an Ubuntu server on a Ryzen 8000 series chip (16 cores, integrated GPU), the browser is Chromium, and every run used the same conditions.

A 20-second video finishing in 0.82 seconds works out to roughly 24 times real time. Running the output through ffprobe shows the video bitrate moved from 3.13 Mbps to 1.49 Mbps, so this is a genuine re-encode rather than a container remux, done in that time.

The gap opens up this wide because WebCodecs calls the encoder built into the browser, while ffmpeg.wasm runs the x264 code on top of wasm. The former can lean on hardware; the latter is computation inside a JavaScript engine no matter what.

Do note that the ffmpeg.wasm figure depends heavily on which preset you pick. What took 20 seconds at veryfast had not finished after 10 minutes at medium. Whenever you come across a speed comparison, check which preset it was measured with.

Frequently asked questions 

So should nobody use ffmpeg.wasm?

It is fine when the conditions fit. For an open-source project, or an internal tool where nothing counts as distribution, the GPL obligation is unlikely to be a problem. Mine was closed source and served the wasm to users, which is why it was out.

Which browsers support WebCodecs?

Chrome and Edge handle it without trouble. Safari support is coming along, but which codecs you get shifts with the browser and OS combination, so treat the canEncodeVideo() check as mandatory.

Is mediabunny free to use?

It is published under MPL-2.0, so commercial projects are fine. How the license text applies still varies by use, so check it against your own setup.

Would converting on the server be faster?

The processing itself would be, yes. It also brings upload time, server cost, and the responsibility of holding other people’s files. I wanted to avoid all three, which is why this lives in the browser.

Can it write formats other than H.264?

It can. WebM (VP8 / VP9), MKV and MOV are supported, and for audio there is MP3, AAC, Opus, FLAC and PCM. Which of them actually work comes down, again, to the browser and OS you are running on.

Will a two-hour video blow up memory?

Not with a streaming setup like this one. Load the whole file into memory instead and that is where you hit the wall.

Summary 

For converting video inside the browser, I went with WebCodecs and mediabunny rather than ffmpeg.wasm. Not because it is faster, but because shipping to the client means the GPL applies, and that single point decided it.

• Running ffmpeg on the server carries no GPL distribution obligation; serving wasm to the browser does
• WebCodecs uses the codecs already in the browser, so the app ships none of its own
• Leave container read/write to mediabunny; the alternatives have been folded into it
• Encoder support differs by browser and OS, so always query before processing
• The speed gap was real: 20.1 seconds with ffmpeg.wasm against 0.82 seconds with WebCodecs, on a 20-second video

Media Tools — image, audio and video tools that never upload your files

Convert, compress and edit images, audio and video entirely in your browser, with AI upscaling and background removal. Free, no upload, no size limit, no sign-up.

media.tools.ryusei.io

Mediabunny — A complete JavaScript media toolkit for the browser

A JavaScript library for reading, writing, and converting media files. Directly in the browser, and faster than anybunny else.

mediabunny.dev

How to Add Subtitles to a Video in Your Browser | Free, No Install, No Upload | Ryusei.IO

No install, no upload. Transcribe a video, fix the SRT, and burn the subtitles into the picture — all inside your browser, free and without an account.

ryusei.io