Distributed endpointing for speech recognition
First Claim
1. A system for performing distributed speech recognition, the system comprising:
- a local device comprising at least one processor coupled to a memory, the memory including instructions operable to be executed by the processor to perform a set of actions, configuring the processor;
to receive audio using at least one microphone;
to monitor audio data corresponding to the audio to detect voice activity in the audio data,to determine that the audio data comprises a wakeword,to begin transmission of the audio data to a server device in response to determining the audio data comprises the wakeword,to receive, from the server device, a confirmation that the audio data includes the wakeword,to receive, from the server device, an indication to stop the transmission of the audio data;
to stop the transmission of the audio data in response to receiving the indication; and
to continue to receive further audio using the at least one microphone following receipt of the indication;
the server device comprising at least one processor coupled to a memory, the memory including instructions operable to be executed by the processor to perform a set of actions, configuring the processor;
to begin receiving the audio data,to confirm the wakeword in the audio data,to transmit the confirmation to the local device,to determine an end of the voice activity in the audio data, andto transmit the indication to the local device in response to determining the end of the voice activity.
1 Assignment
0 Petitions
Accused Products
Abstract
An efficient audio streaming method and apparatus includes a client process implemented on a client or local device and a server process implemented on a remote server or server(s). The client process and server process each have speech recognition components and communicate over a network, and together efficiently manage the detection of speech in an audio signal streamed by the local device to the server for speech recognition and potentially further processing at the server. The client process monitors audio input and in a first detection stage, implements endpointing on the local device to determine when speech is detected. The client process may further determine if a “wakeword” is detected, and then the client process opens a connection and begins streaming audio to the server process via the network. The server process receives the speech audio stream and monitors the audio, implementing endpointing in the server process, to determine when to tell the client process to close the connection and stop streaming audio. The client process continues streaming audio to the server until the server process determines disconnect criteria have been met and tells the client process to stop streaming audio.
-
Citations
10 Claims
-
1. A system for performing distributed speech recognition, the system comprising:
-
a local device comprising at least one processor coupled to a memory, the memory including instructions operable to be executed by the processor to perform a set of actions, configuring the processor; to receive audio using at least one microphone; to monitor audio data corresponding to the audio to detect voice activity in the audio data, to determine that the audio data comprises a wakeword, to begin transmission of the audio data to a server device in response to determining the audio data comprises the wakeword, to receive, from the server device, a confirmation that the audio data includes the wakeword, to receive, from the server device, an indication to stop the transmission of the audio data; to stop the transmission of the audio data in response to receiving the indication; and to continue to receive further audio using the at least one microphone following receipt of the indication; the server device comprising at least one processor coupled to a memory, the memory including instructions operable to be executed by the processor to perform a set of actions, configuring the processor; to begin receiving the audio data, to confirm the wakeword in the audio data, to transmit the confirmation to the local device, to determine an end of the voice activity in the audio data, and to transmit the indication to the local device in response to determining the end of the voice activity. - View Dependent Claims (2, 3)
-
-
4. A computer-implemented method, comprising:
-
receiving, by a local device, audio using at least one microphone; monitoring, by the local device, audio data corresponding to the audio to detect voice activity in the audio data; determining, by the local device, that the audio data comprises a wakeword; starting, by the local device, transmission of the audio data to a remote device in response to determining the audio data comprises the wakeword; receiving, by the local device, a confirmation from the remote device that the transmitted audio data includes the wakeword; receiving, by the local device, an indication to stop the transmission of the audio data, from the remote device, in response to the remote device determining an end of the voice activity in the audio data; stopping, by the local device, the transmission of the audio data in response to receiving the indication; continuing to receive further audio using the at least one microphone following receipt of the indication. - View Dependent Claims (5)
-
-
6. A computing device, comprising:
-
a processor; a memory device including instructions operable to be executed by the processor to perform a set of actions, configuring the processor; to receive audio using at least one microphone; to monitor audio data corresponding to the audio to detect voice activity in the audio data; to determine that the audio data comprises a wakeword; to start transmission of the audio data to a remote device in response to determining the audio data comprises the wakeword; to receive, from the remote device, a confirmation that the transmitted audio data includes the wakeword; to receive an indication to stop the transmission of the audio data, from the remote device, in response to the remote device determining an end of the voice activity in the audio data; to stop the transmission of the audio data in response to receiving the indication; and to continue to receive further audio using the at least one microphone following receipt of the indication. - View Dependent Claims (7, 8, 9, 10)
-
Specification