Limitations of Traditional Vocoders

Before the development of deep learning based approaches, vocoding relied primarily on signal processing techniques. These methods take the acoustic feature representation, typically a magnitude spectrogram (like the mel-spectrograms generated by TTS front-ends), and attempt to reconstruct a time-domain audio waveform.

A prominent example is the Griffin-Lim algorithm (GLA). The core challenge these traditional methods face is the phase reconstruction problem. TTS acoustic models usually predict only the magnitude component of the spectrogram, discarding the phase information. While magnitude captures much of the spectral content, phase is essential for accurately reconstructing the waveform's temporal structure and perceived quality.

Think of the Short-Time Fourier Transform (STFT) which converts a signal segment into magnitude and phase components for different frequencies. Reversing this process (Inverse STFT or ISTFT) requires both. Given only the magnitude $|X(t, f)|$, how do we find the correct phase $\phi(t, f)$ to reconstruct the original signal $x(n)$?

$$x(n) \xrightarrow{\text{STFT}} X(t, f) = |X(t, f)| e^{j \phi(t, f)}$$ $$\hat{X}(t, f) = |X_{\text{target}}(t, f)| e^{j \hat{\phi}(t, f)} \xrightarrow{\text{ISTFT}} \hat{x}(n)$$

The Griffin-Lim algorithm tackles this iteratively. It starts with the target magnitude spectrogram and an initial guess for the phase (often random noise or zero phase). It then alternates between two steps:

1. Time-Domain Synthesis: Combine the target magnitude with the current phase estimate and apply the Inverse STFT (ISTFT) to get a time-domain signal.
2. Frequency-Domain Refinement: Apply the STFT to the synthesized signal. Keep the phase from this STFT result, but replace its magnitude with the original target magnitude spectrogram.

This process is repeated, aiming to find a signal whose STFT magnitude matches the target while satisfying the consistency constraints inherent in the STFT/ISTFT process. The underlying assumption is that enforcing consistency will implicitly guide the phase towards a reasonable estimate.

However, this process has significant drawbacks:

• Suboptimal Phase Estimation: The iterative procedure does not guarantee finding the original or perceptually best phase. Phase reconstruction from magnitude alone is fundamentally an ill-posed problem, meaning multiple phase configurations can correspond to the same magnitude spectrogram, but lead to different perceptual results. GLA often converges to a locally optimal solution that isn't ideal.
• Audible Artifacts: The most noticeable limitation is the quality of the synthesized audio. Waveforms generated using methods like Griffin-Lim frequently exhibit characteristic artifacts. These can include:
  • Buzziness or robotic sound: A lack of natural smoothness.
  • Metallic timbre: Unnatural resonances or spectral characteristics.
  • Muffled quality: Loss of clarity, especially in higher frequencies.
  • "Phasey" sound: Perceptual distortions arising from inconsistent or incorrect phase relationships across frequency bins and time frames.
• Slow Convergence: Griffin-Lim requires multiple iterations (often 50-100 or more) to converge, which can be computationally intensive, especially for long audio sequences, hindering real-time applications.

Consider the difference in spectrograms. While a GLA-vocoded spectrogram might closely match the target magnitude spectrogram, its underlying phase structure leads to the perceptual degradation.

Illustration: While the Griffin-Lim reconstructed magnitude (center) might closely match the target magnitude (left), the resulting audio quality suffers compared to audio derived from the original waveform (right, whose magnitude might differ slightly but whose phase is correct) because of suboptimal phase estimation.

These limitations motivated the development of neural vocoders. By learning complex mappings directly from acoustic features to waveforms, or by modeling the conditional distribution of audio samples, neural networks can implicitly or explicitly learn the correct phase relationships, leading to significantly more natural and higher-fidelity synthesized speech. We will examine these advanced techniques in the following sections.