Methods For Neural Network-Based Voice Enhancement And Systems Thereof

What is this patent about?

’496 is related to the field of audio processing, specifically voice enhancement. Background noise and unclear speech (e.g., mumbling, slurring) can significantly degrade the quality and intelligibility of audio, hindering communication and the accuracy of speech recognition systems. Traditional voice enhancement techniques often focus on noise reduction, which can inadvertently distort speech features, leading to further inaccuracies. This patent addresses the need for improved voice enhancement that preserves speech characteristics while effectively suppressing noise and enhancing clarity.

The underlying idea behind ’496 is to use a two-stage neural network approach to enhance voice quality in real-time. The first neural network reduces the dimensionality of the input audio, focusing on relevant speech content and characteristics while discarding noise. The second neural network then reconstructs the audio from this reduced representation, generating enhanced speech that is clearer and more intelligible.

The claims of ’496 focus on a voice enhancement system, a method for real-time voice enhancement, and a non-transitory computer-readable medium. The independent claims cover the process of fragmenting input audio into frames, converting these frames into low-dimensional representations using a first neural network (which also removes non-content elements like noise), applying a second neural network to generate enhanced target speech frames, and combining these frames into output audio.

In practice, the system first takes an audio stream and divides it into short frames. The first neural network, having been trained on a large dataset of speech and noise, then transforms each frame into a compact, lower-dimensional representation. This representation is designed to capture the essential features of the speech while discarding irrelevant noise and artifacts. The second neural network then takes these compact representations and reconstructs them into enhanced speech frames, which are then stitched together to form the final output audio.

This approach differs from prior methods by explicitly using a low-dimensional representation to filter out noise and preserve speech characteristics. Instead of directly manipulating the raw audio signal, the system operates on a compressed representation, allowing it to more effectively separate speech from noise and enhance clarity. The two neural networks are trained to work together, with the first network learning to extract relevant features and the second network learning to reconstruct high-quality speech from these features.