Spectral subtraction is one of the classic techniques used in digital audio noise reduction and speech enhancement.

The basic idea is simple: estimate the noise spectrum and subtract it from the spectrum of the noisy signal.

Although modern audio systems increasingly use adaptive filtering, microphone arrays, and AI-based speech enhancement, spectral subtraction remains important because it is relatively simple, computationally efficient, and easy to understand.

It also provides an excellent foundation for understanding more advanced noise reduction systems.

In this article, we explain how spectral subtraction works, how it is implemented with FFT and STFT, why it can produce musical noise, and how engineers can improve its performance in practical audio systems.

What Is Spectral Subtraction?

Spectral subtraction is a frequency-domain audio noise reduction technique.

The underlying assumption is that the microphone captures a combination of desired sound and unwanted noise:

y(t) = x(t) + n(t)

where:

• y(t) = observed noisy signal
• x(t) = desired signal
• n(t) = noise

The goal is to estimate x(t) from y(t).

Instead of attempting to remove the noise directly from the waveform, spectral subtraction transforms the audio into the frequency domain.

The general processing chain is:

Noisy Audio

→ Framing

→ Windowing

→ FFT / STFT

→ Noise Spectrum Estimation

→ Spectral Subtraction

→ Spectral Gain / Floor

→ Inverse FFT

→ Overlap-Add

→ Enhanced Audio

This approach is particularly useful when the background noise has relatively stable spectral characteristics.

Why Use the Frequency Domain?

Many audio signals contain different types of information at different frequencies.

For example:

• low-frequency energy may come from HVAC systems or engines;
• narrow-band peaks may come from electrical or mechanical sources;
• speech contains changing harmonic and formant structures;
• high-frequency components may contain consonant information and environmental noise.

A time-domain waveform does not make these components easy to separate.

The frequency domain provides a more useful representation for analyzing and modifying individual frequency regions.

This is why FFT and STFT are fundamental to many traditional audio noise reduction algorithms.

For non-stationary signals such as speech, STFT is particularly useful because it allows the system to analyze the spectrum frame by frame.

How Does Spectral Subtraction Work?

A practical spectral subtraction system normally follows several steps.

Step 1: Capture the Noisy Audio

The microphone captures:

desired signal + background noise

For example:

Speaker + air conditioning + computer fan

The microphone output is the noisy signal:

y(t)

Step 2: Divide the Signal Into Frames

The continuous audio stream is divided into short overlapping frames.

For example, a speech-processing system might use a frame duration in the range of approximately 20–32 ms.

Each frame is processed separately.

This is important because the statistical characteristics of speech and noise can change over time.

Step 3: Apply a Window Function

A window such as the Hann window is applied to each frame.

The purpose is to reduce spectral leakage caused by analyzing a finite segment of the signal.

The windowed frame can be represented as:

yₘ[n] = y[n + mH]w[n]

where:

• m = frame index
• H = hop size
• w[n] = window function

Step 4: Calculate the FFT

The FFT converts the time-domain frame into a frequency-domain representation.

For each frequency bin:

Yₘ[k] = FFT{yₘ[n]}

The system can then calculate the magnitude or power spectrum.

For example:

|Yₘ[k]|

or:

|Yₘ[k]|²

Step 5: Estimate the Noise Spectrum

This is one of the most important parts of spectral subtraction.

The algorithm needs an estimate of the background noise spectrum.

If the noise is relatively stationary, the system may estimate it during periods when speech is absent.

For example:

Microphone signal

→ Voice Activity Detection

→ Identify non-speech frames

→ Estimate noise spectrum

A simple approach is to average the power spectrum of several noise-only frames.

The estimated noise power can be represented as:

P̂n[k]

The quality of this estimate strongly affects the final noise reduction performance.

If the noise estimate is too low, residual noise remains.

If it is too high, parts of the desired speech can be removed.

Step 6: Subtract the Estimated Noise

A simplified power-spectrum spectral subtraction equation is:

P̂x[k] = max(Py[k] − αP̂n[k], βPmin)

where:

• P̂x[k] = estimated clean-signal power
• Py[k] = noisy-signal power
• P̂n[k] = estimated noise power
• α = noise over-subtraction factor
• β = spectral-floor coefficient
• Pmin = reference spectral level or floor term

The exact formulation varies between implementations.

The important concept is:

Estimate the noise contribution and reduce the frequency components where noise is dominant.

What Is Over-Subtraction?

Simply subtracting the estimated noise once may not provide sufficient suppression when the noise estimate is imperfect.

Engineers can therefore introduce an over-subtraction factor.

If:

α = 1

the algorithm performs approximately direct subtraction.

If:

α > 1

the algorithm removes more estimated noise.

This can increase noise suppression, but aggressive over-subtraction can also damage speech.

For example, if speech and noise overlap strongly at a particular frequency, the algorithm cannot perfectly distinguish them.

Increasing α too much may therefore remove useful speech energy.

This is one of the fundamental trade-offs in spectral subtraction:

More noise suppression can mean more speech distortion.

What Is the Spectral Floor?

After subtraction, some frequency bins may become zero or even mathematically negative.

Negative power is physically meaningless, so the algorithm needs a lower bound.

A spectral floor prevents the gain from becoming excessively small.

Conceptually:

Estimated spectrum < minimum threshold

→ replace with minimum allowed level

This can help prevent excessive attenuation and reduce unpleasant processing artifacts.

The exact floor value is an important tuning parameter.

Magnitude Spectral Subtraction vs Power Spectral Subtraction

There are several formulations of spectral subtraction.

A system may operate on:

• magnitude spectrum
• power spectrum
• amplitude spectrum
• complex spectrum

For example, a magnitude-based formulation may be written as:

|X̂[k]| = max(|Y[k]| − α|N̂[k]|, Gmin)

while a power-spectrum approach uses squared magnitude.

The implementation should clearly define which spectral quantity is being estimated and subtracted.

This distinction matters because the parameter interpretation and resulting gain behavior can differ.

Why Does Spectral Subtraction Produce Musical Noise?

One of the best-known limitations of spectral subtraction is musical noise.

Musical noise consists of artificial, tonal-sounding residual components that can appear after aggressive spectral processing.

It can sound like:

• small whistles
• random tones
• metallic artifacts
• rapidly changing tonal components

Why does this happen?

Imagine a noisy spectrum containing many frequency bins.

For one frame, the algorithm may estimate that a particular bin is dominated by noise and strongly suppress it.

In the next frame, because the noise estimate and signal fluctuate slightly, that same bin may not be suppressed as strongly.

The resulting pattern changes over time.

After inverse transformation, these isolated time-frequency components can become perceptually noticeable tonal artifacts.

This is one reason why simply increasing the noise subtraction factor does not necessarily produce better audio.

How Can Musical Noise Be Reduced?

Several techniques can be used to reduce musical noise.

1. Spectral Smoothing

Instead of processing each frequency bin independently, neighboring frequency bins can be smoothed.

This reduces rapid variations in the gain spectrum.

2. Temporal Smoothing

The gain applied to each frequency bin can also be smoothed across successive frames.

This reduces abrupt changes in spectral attenuation.

3. Gain Flooring

A minimum gain can prevent individual frequency components from being completely eliminated.

This can reduce unnatural spectral holes.

4. More Stable Noise Estimation

A better noise estimator reduces fluctuations in the estimated noise spectrum.

This can significantly improve the stability of spectral subtraction.

5. Voice Activity Detection

A reliable VAD can help identify speech-free regions for noise estimation.

Poor VAD decisions can contaminate the noise model with speech, causing the algorithm to suppress parts of the desired signal.

6. Post-Filtering

Spectral subtraction can be combined with additional filtering stages.

For example:

STFT → Spectral Subtraction → Post-Filter → ISTFT

This allows the system to refine the result after the initial noise reduction stage.

Spectral Subtraction for Stationary and Non-Stationary Noise

Spectral subtraction tends to work most naturally when the background noise is relatively stable.

More suitable examples

• fan noise
• air-conditioning noise
• some mechanical background noise
• relatively stable broadband noise

More challenging examples

• keyboard clicks
• intermittent impacts
• changing traffic
• competing speech
• sudden environmental sounds

The reason is straightforward:

If the noise spectrum changes faster than the algorithm can estimate it, the noise model becomes inaccurate.

This is one reason modern systems often combine spectral methods with adaptive or AI-based approaches.

Spectral Subtraction and Non-Stationary Noise

This does not mean spectral subtraction cannot be used with changing noise.

The algorithm can update its noise estimate continuously.

Possible approaches include:

• minimum-statistics estimation
• recursive noise estimation
• VAD-assisted estimation
• adaptive smoothing
• noise tracking

The goal is to make:

N̂(k, t)

follow the changing noise spectrum without accidentally tracking the desired speech.

This creates a fundamental engineering trade-off:

Fast adaptation

vs.

Speech preservation

If the noise estimator adapts too slowly, changing noise remains.

If it adapts too quickly, it may interpret speech as noise.

Spectral Subtraction in a Real-Time Audio System

A practical real-time implementation can be represented as:

Microphone
    ↓
Audio Input
    ↓
Frame Buffer
    ↓
Windowing
    ↓
FFT
    ↓
Noise Estimation
    ↓
Spectral Subtraction
    ↓
Gain Smoothing
    ↓
IFFT
    ↓
Overlap-Add
    ↓
Enhanced Audio

This process repeats continuously for every incoming frame.

The implementation must therefore consider:

• frame size
• FFT size
• hop size
• window function
• noise estimation method
• smoothing
• spectral floor
• processing latency
• CPU/DSP load

This is where an algorithm that works well in an offline experiment can encounter problems when deployed in an embedded product.

Example: Spectral Subtraction for Speech in an Office

Consider a conference microphone operating in an office.

The target speaker is talking while:

• an air conditioner is running;
• computers are operating;
• people are moving around;
• occasional keyboard noise is present.

The system might perform:

Microphone

→ STFT

→ Noise Estimation

→ Spectral Subtraction

→ Gain Smoothing

→ ISTFT

During speech-free periods, the system estimates the background spectrum.

During speech, it compares the observed spectrum with the estimated noise spectrum and attenuates frequency regions where noise dominates.

The result can be a cleaner signal for:

• voice communication
• speech recognition
• transcription
• conferencing
• audio recording

However, if another person begins speaking, the system may have difficulty determining whether that speech is desired voice or interference.

This is where multi-microphone processing or AI-based speech enhancement can provide additional information.

Spectral Subtraction vs Wiener Filtering

Both methods operate in the frequency domain and can use noise-spectrum estimation, but their underlying approaches differ.

Neither method is universally optimal.

The correct choice depends on:

• noise characteristics
• processor resources
• latency requirements
• speech-quality requirements
• available reference information

Spectral Subtraction vs AI Noise Reduction

Modern AI-based noise reduction can learn complex relationships between speech and noise from training data.

The difference can be summarized as follows:

This does not make classical DSP obsolete.

In many practical products, DSP remains useful for:

• preprocessing
• filtering
• echo cancellation
• gain control
• noise estimation
• spatial processing

AI can then provide additional enhancement.

A hybrid architecture can therefore combine the strengths of both approaches.

Can Spectral Subtraction Be Used With MEMS Microphones?

Yes.

Spectral subtraction operates on the digital representation of an audio signal, so it can be used with digital MEMS microphones or with analog MEMS microphone systems followed by appropriate analog-to-digital conversion.

A simplified system could be:

MEMS Microphone

→ ADC / Digital Microphone Interface

→ STFT

→ Noise Estimation

→ Spectral Subtraction

→ Speech Enhancement

The quality of the input signal remains important.

A noise reduction algorithm cannot fully recover information that has already been lost because of:

• microphone clipping
• insufficient SNR
• poor acoustic isolation
• excessive reverberation
• poor ADC performance
• mechanical vibration
• severe acoustic interference

Therefore, microphone selection and acoustic design should be considered together with DSP.

Spectral Subtraction in Microphone Array Systems

Spectral subtraction becomes more interesting when multiple microphones are available.

A microphone array provides spatial information in addition to the time-frequency information provided by STFT.

For example:

Multiple MEMS Microphones

↓

Synchronized Channels

↓

STFT

↓

Beamforming

↓

Spectral Noise Reduction

↓

Speech Enhancement

This allows the system to use two different types of information:

Spectral information

Where is the energy located in frequency?

Spatial information

Where is the sound coming from?

Combining these two dimensions can be more powerful than relying on single-channel spectral subtraction alone.

This is particularly relevant to:

• far-field voice capture
• conference microphones
• classroom audio
• smart speakers
• AI voice terminals
• professional microphone arrays

SISTC’s microphone-array solutions similarly combine multi-channel microphone acquisition with digital signal processing and noise-reduction capabilities. For example, the GY6228AEC-based array documentation describes synchronized multi-channel digital microphone input and functions including noise suppression and array signal processing.

Practical Parameters for Spectral Subtraction

There is no universal set of parameters.

However, for speech applications, engineers often begin with a configuration such as:

These are starting points, not universal specifications.

The optimum configuration depends on:

• target speech bandwidth
• noise characteristics
• processor capability
• acceptable latency
• desired suppression level
• speech quality requirements

A Simple Python Implementation

A basic spectral-subtraction experiment can be implemented using STFT processing.

The following example illustrates the core concept rather than providing a production-ready noise reduction algorithm.

import numpy as np
from scipy.io import wavfile
from scipy.signal import stft, istft

fs, audio = wavfile.read("noisy_audio.wav")

if audio.ndim > 1:
    audio = audio[:, 0]

audio = audio.astype(np.float32)

# STFT
f, t, Zxx = stft(
    audio,
    fs=fs,
    window="hann",
    nperseg=512,
    noverlap=256
)

magnitude = np.abs(Zxx)
phase = np.angle(Zxx)

# Example: estimate noise from the first 0.5 seconds
noise_frames = t < 0.5

noise_power = np.mean(
    magnitude[:, noise_frames] ** 2,
    axis=1,
    keepdims=True
)

signal_power = magnitude ** 2

# Oversubtraction factor
alpha = 1.0

# Spectral subtraction
clean_power = np.maximum(
    signal_power - alpha * noise_power,
    0.01 * signal_power
)

clean_magnitude = np.sqrt(clean_power)

# Restore phase
Z_clean = clean_magnitude * np.exp(1j * phase)

# Inverse STFT
_, enhanced = istft(
    Z_clean,
    fs=fs,
    window="hann",
    nperseg=512,
    noverlap=256
)

wavfile.write(
    "enhanced_audio.wav",
    fs,
    enhanced.astype(np.float32)
)

This example assumes that the first 0.5 seconds contain a representative noise-only segment.

That assumption is often unrealistic in real products.

A production system therefore needs a more robust noise-estimation strategy.

Limitations of Basic Spectral Subtraction

Spectral subtraction is attractive because of its simplicity, but several limitations should be considered.

1. Dependence on Noise Estimation

If the noise estimate is incorrect, suppression performance decreases.

2. Musical Noise

Independent time-frequency attenuation can create artificial tonal artifacts.

3. Speech Distortion

Aggressive subtraction can remove speech components that overlap with background noise.

4. Changing Noise

Rapidly changing noise is difficult to track accurately.

5. Reverberation

Spectral subtraction is primarily designed around additive-noise assumptions. Reverberation is a different acoustic problem and may require additional processing.

6. Competing Speech

Another speaker is not simply stationary background noise.

Separating two speech sources generally requires spatial processing, source separation, or learned models rather than basic spectral subtraction alone.

When Should Engineers Use Spectral Subtraction?

Spectral subtraction can be a practical choice when:

• computational resources are limited;
• low power consumption is important;
• the background noise is relatively stable;
• a transparent algorithm is preferred;
• deterministic processing is desirable;
• rapid implementation is required;
• the system can tolerate some residual artifacts.

It may be less suitable as the only noise reduction method when:

• noise changes rapidly;
• multiple speakers are present;
• strong reverberation exists;
• the target voice is very weak;
• extremely natural speech quality is required.

In those cases, engineers may consider combining spectral processing with:

• microphone arrays
• beamforming
• adaptive filtering
• acoustic echo cancellation
• AI speech enhancement
• source separation

The Role of Spectral Subtraction in Modern Audio Systems

It is important not to view spectral subtraction as an outdated technique.

Its greatest value today may be as one component of a broader signal-processing architecture.

For example:

MEMS Microphone

↓

Acoustic Front End

↓

STFT

↓

Noise Estimation

↓

Beamforming

↓

Spectral Noise Reduction

↓

AI Speech Enhancement

↓

Voice Recognition

A hybrid system can use classical DSP where it is efficient and predictable, while using AI where complex acoustic patterns require learned representations.

This architecture is especially relevant to modern intelligent microphone systems.

Frequently Asked Questions

What is spectral subtraction?

Spectral subtraction is a frequency-domain noise reduction method that estimates the noise spectrum and subtracts it from the noisy audio spectrum.

How does spectral subtraction work?

The audio is divided into short frames, transformed using FFT or STFT, analyzed to estimate background noise, and then frequency components associated with the estimated noise are attenuated before reconstructing the audio.

What is the spectral subtraction formula?

A simplified power-spectrum formulation is:

P̂x[k] = max(Py[k] − αP̂n[k], βPmin)

The exact equation varies according to the implementation.

What is musical noise in spectral subtraction?

Musical noise is an artificial tonal or whistle-like artifact that can occur when individual time-frequency components are suppressed inconsistently across frequency and time.

How can musical noise be reduced?

Common approaches include spectral smoothing, temporal smoothing, gain flooring, improved noise estimation, VAD-assisted processing, and post-filtering.

Does spectral subtraction work for speech?

Yes. It can be effective for speech when background noise is relatively stable and the noise spectrum can be estimated reliably.

Does spectral subtraction work for non-stationary noise?

It can, but performance depends heavily on how quickly and accurately the noise estimator can track changes.

Can spectral subtraction be used with MEMS microphones?

Yes. It can process digital audio captured from digital MEMS microphones or digitized signals from analog MEMS microphones.

Can spectral subtraction replace a microphone array?

No. Spectral subtraction primarily uses time-frequency information. A microphone array provides spatial information that can be used for beamforming and directional noise suppression.

Is AI noise reduction better than spectral subtraction?

They use different approaches. Spectral subtraction is computationally efficient and interpretable, while AI models can learn more complex noise and speech characteristics but generally require training data and greater computational resources.

Conclusion

Spectral subtraction is one of the fundamental algorithms in digital audio noise reduction.

Its concept is straightforward:

Estimate the noise spectrum and reduce it from the noisy signal in the frequency domain.

However, effective implementation requires much more than the subtraction operation itself.

Noise estimation, FFT/STFT parameters, spectral flooring, smoothing, VAD, latency, and speech preservation all affect the final result.

The biggest challenge is finding the right balance between:

Noise suppression

and

Speech quality

For relatively stable background noise and resource-constrained systems, spectral subtraction can still be useful.

For complex environments, it can also serve as one component of a larger system combining microphone arrays, beamforming, adaptive filtering, and AI-based speech enhancement.

For engineers developing modern microphone and acoustic systems, understanding spectral subtraction provides an important foundation for understanding more advanced noise reduction technologies.