DEV Community 1h ago

Removing Video Watermarks Entirely in the Browser with MI-GAN, ONNX Runtime Web, and WebGPU

Why browser-local repair?

A server-side pipeline usually requires:

Uploading the original media
Waiting for a large video transfer
Paying for storage and GPU compute
Sending potentially sensitive footage to another system
Depending on a reliable network connection

Browser-local inference moves the initial cost to downloading the model. Once the runtime and model artifacts are cached, repeated edits are much faster to start, and the original media remains local.

Our implementation uses:

MI-GAN for lightweight image inpainting
ONNX Runtime Web for browser inference
WebGPU as the preferred execution backend
Web Workers to keep inference off the UI thread
Canvas / OffscreenCanvas for frame extraction, masks, and compositing
WebCodecs or a compatibility encoder path for the final video
Cache Storage to avoid downloading the model again

The complete pipeline

Source media
User defines repair regions and time ranges
Resolve regions active at the current timestamp
Decode a video frame
Build a mask
Run MI-GAN inference
Composite the repaired pixels over the source frame
Commit the progressive preview
Encode the processed frame
Create a new media asset

The model is only one stage. A production-quality experience also requires the canvas, playhead, processed-frame count, and progress indicator to share the same clock.

Image repair with normalized regions

In image mode, the user draws one or more rectangles over the canvas. We store each rectangle in normalized coordinates so the selection remains correct across different preview sizes.

function createMask ( width , height , regions ) {
  const canvas = new OffscreenCanvas ( width , height );
  const context = canvas . getContext ( " 2d " );
  context . fillStyle = " #000 " ;
  context . fillRect ( 0 , 0 , width , height );
  context . fillStyle = " #fff " ;
  for ( const region of regions ) {
    context . fillRect ( region . x * width , region . y * height , region . width * width , region . height * height );
  }
  return context . getImageData ( 0 , 0 , width , height );
}

Each region uses values between 0 and 1. The same rectangle can therefore be mapped back to the full-resolution source instead of the CSS-sized preview.

After inference, we do not immediately commit the result. The editor displays a draggable vertical comparison line:

<ComparisonView
  before={sourceUrl}
  after={repairedUrl}
  position={comparePosition}
  onPositionChange={setComparePosition}
/>

That distinction matters because inpainting does not recover a hidden ground truth. It synthesizes plausible pixels from the surrounding context, so users need a clear way to inspect the result before applying it.

Video repair is a time-range problem

Our first video prototype repaired only the frame currently visible in the editor. It looked correct during a single-frame test but was obviously wrong for a video. A watermark exists for a time range, not for one frame. It may also move: for example, it can appear in the bottom-left corner for the first few seconds and later switch to the top-right.

We represent every repair region as an independent time-scoped object:

const repairRegion = {
  id: " region-1 ",
  startTime: 0,
  endTime: 4,
  keyframes: [
    { time: 0, x: 0.72, y: 0.82, width: 0.22, height: 0.10 },
    { time: 4, x: 0.08, y: 0.08, width: 0.22, height: 0.10 }
  ]
};

The rectangle between two recorded positions is interpolated:

function interpolateRegion ( a , b , ratio ) {
  return {
    x: a.x + (b.x - a.x) * ratio,
    y: a.y + (b.y - a.y) * ratio,
    width: a.width + (b.width - a.width) * ratio,
    height: a.height + (b.height - a.height) * ratio
  };
}

This supports fixed watermarks, watermarks that jump to another corner, and slowly moving overlays without creating a separate repair job for every frame.

Multiple regions for both images and videos

Real media may contain a logo and a text overlay at the same time. Both image and video workflows therefore need multiple selections. At each video timestamp, we resolve all active regions and combine them into one mask:

const activeRegions = regions
  .filter((region) => {
    return time >= region.startTime && time <= region.endTime;
  })
  .map((region) => resolveRegionAtTime(region, time));

This data model also gives us a practical editing interface:

Every region has its own start and end time.
A selected region can be moved and resized independently.
Positions can be recorded at different timestamps.
Individual regions can be deleted.
Region edits can participate in undo and redo history.

Keeping MI-GAN off the main thread

Creating the model session or running inference on the main thread can freeze the entire editor. We keep a long-lived model session inside a Web Worker and transfer frame data to it.

const session = await ort.InferenceSession.create(modelUrl, {
  executionProviders: [ " webgpu " , " wasm " ]
});

WebGPU is preferred. If it is unavailable or initialization fails, we explicitly fall back to WASM and report the actual backend in the UI.

worker.postMessage(
  { type: " repair ", requestId, imageBitmap, regions },
  [imageBitmap]
);

The Worker and inference session remain alive for later requests. Model and runtime assets are stored in a versioned Cache Storage entry, so a second repair should never present itself as another model download.

Preprocessing and compositing without degrading the full frame

Inpainting models commonly expect a fixed input size. Resizing and regenerating the entire video frame for a small watermark can unintentionally change faces, text, and fine texture outside the selected area.

Our processing rules are:

Transform the frame and mask into the model's expected input.
Include enough context around the masked area.
Restore only the required patch at source resolution.
Preserve every pixel outside the repair mask.
Blend the mask boundary to avoid a visible seam.

context.drawImage(originalFrame, 0, 0);
context.save();
context.globalCompositeOperation = " source-over ";
context.drawImage(repairedPatch, patchX, patchY, patchWidth, patchHeight);
context.restore();

Keeping regions reasonably tight improves both performance and output stability.

The subtle bug: progress and preview were out of sync

An early version used three independent notions of time: Progress followed the frame-processing loop. Preview followed the frame the browser had actually decoded and rendered. Timeline followed a separate playback clock. The result was confusing: the progress bar might show 40% while the preview was still near 25%.

The fix was to make the committed visible frame the single UI clock:

for (let frameIndex = 0; frameIndex < totalFrames; frameIndex += 1) {
  const frame = await decodeFrame(frameIndex);
  const repairedFrame = await repairFrameIfNeeded(frame, frameIndex);
  await onFrameCommitted(repairedFrame, frameIndex);
  reportProgress({ completedFrames: frameIndex + 1, totalFrames });
}

Frames outside every repair range must still call onFrameCommitted. Otherwise, the preview and playhead appear frozen whenever inference is skipped.

We also decode the progressive preview image and give the browser one paint before advancing the completed-frame count:

await image.decode();
setPreviewFrame(image.src);
await new Promise(requestAnimationFrame);
setCompletedFrame(frameIndex + 1);

During processing, the left side now displays the source frame for the current timestamp while the right side displays the repaired frame for that same timestamp. The comparison line stays interactive throughout the process.

Separate repair progress from encoding progress

Reaching 100% repaired frames does not mean the output file is ready. We still need to initialize the encoder, encode the frames, and create the final media asset. We expose these as distinct phases:

Preparing the model
Repairing frames
Loading the video encoder
Encoding the repaired video
Creating the new asset
Complete

Trying to compress all of these into a single unexplained percentage makes the UI appear stuck at the end. A phase label tells the user what the browser is actually doing.

Cancellation must release real resources

For long videos, cancellation is not optional. Changing the button text to "Cancelling..." is not enough. A real cancel path must:

Stop scheduling new frame decodes.
Invalidate pending Worker inference requests.
Close VideoFrame and ImageBitmap objects.
Revoke temporary Blob URLs.
Stop encoders and release media resources.
Avoid downloading or inserting a partial result.

if (abortSignal.aborted) {
  throw new DOMException(" Repair cancelled ", " AbortError ");
}

We disable duplicate cancel requests immediately, but return to an editable state only after cleanup has completed. The original media remains unchanged until the user explicitly applies the new result.

A dedicated repair workspace works better than a crowded inspector

Our first UI placed all controls in the editor's right-side property panel. That panel quickly became too small for region drawing, time ranges, keyframes, preview, comparison, and progress. The final product keeps AI Repair as a lightweight capability entry point and opens the actual work in a larger modal:

Top toolbar: move, redraw, presets, undo, redo
Center: source/repaired comparison canvas
Bottom: playback, time scale, range handles, playhead
Right side: repair-region list and time/position controls
Footer: cancel, single-frame repair/test, process range, apply result

The mobile layout uses the same mental model in a vertical workspace.

Applying closes both the repair modal and the properties drawer; cancelling closes only the modal.

Performance lessons

Browser AI performance varies widely with GPU, resolution, codec, and browser implementation. Rather than promising a universal frame rate, we focused on these principles:

Download model artifacts in parallel and store them in a versioned cache.
Create ONNX sessions serially and only once.
Reuse the Worker and session for repeated jobs.
Skip inference for frames without an active mask.
Keep repair regions as small as practical.
Give the UI a chance to paint between committed frames.
Revoke temporary URLs and close GPU/video resources promptly.
Fall back to WASM clearly when WebGPU is unavailable.

The first model load is still a cost, but cached repeat operations become a much better fit for an interactive editor.

Current limitations

MI-GAN is lightweight and fast enough to be practical in a browser, but it is not perfect:

A watermark covering a face or detailed text may produce unnatural content.
Very large selected areas can generate unstable structures.
Fast motion may reveal temporal inconsistency between frames.
Processing can be significantly slower without WebGPU.
Total time includes decoding and encoding, not just model inference.
Frame-by-frame inpainting does not guarantee temporal consistency.

Optical flow, region tracking, and temporally aware restoration models are natural next steps.

Final thoughts

The biggest lesson from building this feature was: Successful model inference is only the beginning. The real product is the editing flow that lets users inspect, trust, cancel, and apply the result. Repairing one image in a demo and shipping multi-region, time-aware, progressively previewed video repair are completely different engineering problems.

This implementation runs locally in the browser and inserts the result as a new media asset, leaving the original recoverable.

GitHub: https://github.com/MartinDelophy/ai-video-editor
Live demo: https://video-editor.ai-creator.top/

If you are experimenting with WebGPU, ONNX Runtime Web, browser-side media processing, or video-editor UX, I hope these implementation details save you a few iterations.

Read on DEV Community ↗ ← Back to News