How Vocal Removal Works: Phase Cancellation vs AI Masking (with Python benchmarks)

Every karaoke app, remix tool and "make it instrumental" button is solving the same problem: getting one sound out of a finished stereo mix.

There are two very different ways to do it. One is a clever trick from the 1970s that fits in five lines of NumPy. The other is what modern AI separators do.

I built a small benchmark to compare them on a synthetic mix where every source is known, so we can measure what works instead of guessing. Here's what I found, with code.

The setup: a mix with ground truth

Real songs don't ship with their stems, so I synthesised a 6-second arrangement: a harmonic vocal with vibrato (center), a bass line (center), a kick (center), plucked guitar chords (panned 70% left) and a keys pad (panned 70% right).

Because every source is stored separately, each method can be scored with SI-SDR (scale-invariant signal-to-distortion ratio), the standard metric in source-separation research:

def si_sdr(est, ref):
    est, ref = est.ravel(), ref.ravel()
    alpha = est @ ref / (ref @ ref)
    target = alpha * ref
    return 10 * np.log10((target @ target) / ((target - est) @ (target - est)))

We score the instrumental (mix minus vocal), since that's what a karaoke track needs. Higher is better.

Method 1: the classic L − R trick

A lead vocal is usually mixed dead center, so it's identical in the left and right channels. In mid/side terms:

mid = (L + R) / 2 # everything in the center: vocal, kick, bass, snare
side = (L - R) / 2 # everything panned away from the center

Throw away the mid, keep the side, and the vocal is gone:

naive = np.stack([side, -side], axis=1)

Result: −13.9 dB instrumental SI-SDR. That's worse than doing nothing (+4.4 dB). The vocal is gone, but so are the kick and bass, because they're centered too. What's left sounds thin and phasey.

Method 2: smarter phase cancellation

Two cheap fixes make a big difference:

• Band-limited cancellation: only cancel the part of the mid where vocals live (about 200 Hz to 5 kHz).
• Bass preservation: add the low end back so the kick and bass survive.

from scipy.signal import butter, sosfiltfilt

def bandpass(x, lo, hi, sr):
    return sosfiltfilt(butter(4, [lo, hi], "band", fs=sr, output="sos"), x)

def lowpass(x, fc, sr):
    return sosfiltfilt(butter(4, fc, "low", fs=sr, output="sos"), x)

center = bandpass(mid, 200, 5000, sr)
new_mid = mid - center + lowpass(center, 150, sr)
out = np.stack([new_mid + side, new_mid - side], axis=1)

Note the sosfiltfilt: zero-phase filtering matters. When I ported this to the browser with Web Audio's BiquadFilterNode, which is not zero-phase, the filtered center was phase-shifted and simply didn't cancel. Running the filter chain forward and then backward (filtfilt) fixed it and cut the test vocal by about 53 dB.

Result: +7.4 dB. Big improvement. Kick and bass are back.

Breaking it on purpose

Real productions rarely have a dry, perfectly centered vocal, so I tested two common cases:

With stereo reverb the left and right reverb tails differ, so cancellation can't touch them. You get the famous "ghost vocal".

No amount of filtering fixes this, because phase cancellation only knows where a sound sits in the stereo field, not what it is.

Method 3: time-frequency masking (what AI separators do)

Modern separators such as Open-Unmix, Demucs and MDX-style models work (fully or partly) on the spectrogram. For every time-frequency bin they estimate how much energy belongs to each source, then apply that mask to the mixture.

To see the ceiling of this approach, I computed the ideal ratio mask from the known sources. This is what a perfectly trained network would try to predict:

from scipy.signal import stft, istft

M = stft(mix_ch, sr, nperseg=1024, noverlap=768)[2]
V = stft(vocal_ch, sr, nperseg=1024, noverlap=768)[2]
I = M - V
mask = np.abs(I)**2 / (np.abs(I)**2 + np.abs(V)**2 + 1e-12)
instrumental = istft(mask * M, sr, nperseg=1024, noverlap=768)[1]

The mask is almost 1 everywhere except along the thin harmonic lines of the vocal. It removes the voice by timbre, not by position, and leaves the rest of the mix alone.

It also holds up in the hard cases: 20.8 dB with the off-center vocal and 16.3 dB with stereo reverb, where phase cancellation collapsed to 0.6 dB.

One honest caveat: this row is an oracle, because it builds the mask from the true vocal. A trained network has to predict the mask from the mixture alone, so real models land below it. But a 13-16 dB gap over the best hand-built method explains why every serious separator today is learned.

Takeaways

• L − R is free and instant, and nearly useless on its own because it deletes the kick and bass.
• Band-limited phase cancellation with bass preservation is a decent zero-cost karaoke maker if the vocal is dry and centered. Use zero-phase filters.
• Spectrogram masking separates by what a sound is, so it survives panning and reverb. That's the core idea behind neural vocal removers.

Try it yourself

Everything here is open source:

• 🎛️ Live in-browser demo: Classic Vocal Remover on Hugging Face Spaces (Web Audio, nothing uploaded)
• 🐍 Python CLI: classic-vocal-remover
• 📓 Full notebook with audio examples: vocal-removal-tutorial
• 🎚️ Remix separated stems into karaoke or practice tracks: stem-mixer

If you want real AI separation without training anything, that's what we build at Remove Vocal. Upload a song or a video, pick 2, 4 or 6 stems (vocals, instrumental, drums, bass, other), preview them in the browser and download WAV or MP3.

Questions or ideas? Drop a comment, or reach us at s******@remove-vocal.com. I'm especially curious which separation edge cases you've run into.