System and method for hybrid processing in a natural language voice services environment

What is this patent about?

'541 is related to the field of natural language processing, specifically voice services environments. The background involves the increasing complexity of consumer electronic devices and the challenges users face in fully utilizing their features. Existing voice user interfaces often require specific command sequences and suffer from inaccurate speech recognition, limiting their ability to engage users in productive dialogues and leverage information across different devices and applications.

The underlying idea behind '541 is to enable hybrid processing of natural language utterances by distributing the interpretation and processing workload across multiple devices. This involves a central system, acting as a virtual router, that receives audio and contextual data from various user devices (e.g., a phone and a computer), selects the best audio sample, and coordinates with other devices to determine the user's intent and resolve the request.

The claims of '541 focus on a computer system receiving a natural language utterance from one device and a related non-voice input from another device. The system processes the non-voice input to determine context, then uses this context to interpret the utterance. Based on this interpretation, the system generates a user request and selects a user processing device (which could be the original devices or a third one) that has the content needed to fulfill the request, and transmits the request to that device.

In practice, the invention allows a user to interact with multiple devices simultaneously to complete a task. For example, a user might speak a command into their phone while simultaneously highlighting text on their computer screen. The system combines the voice input with the highlighted text to understand the user's intent more accurately. The virtual router then intelligently routes the request to the device best equipped to handle it, such as a server with a large database or the user's own computer if the relevant data is stored locally.

This approach differs from prior solutions by not being limited to a single device or a pre-defined set of commands. Instead, it leverages the capabilities of multiple devices in a coordinated manner to provide a more intuitive and efficient user experience. By selecting the 'cleanest' audio sample and incorporating contextual information from non-voice inputs, the system aims to improve the accuracy of speech recognition and intent determination, leading to a more seamless and natural interaction.