Methods and systems for detecting and processing speech signals

• US 10,255,920 B2
• Filed: 06/16/2017
• Issued: 04/09/2019
• Est. Priority Date: 02/24/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Civelli, Jay Pierre, Shemer, Mikhal, Shabestary, Turaj Zakizadeh, Tapuska, David
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US15/625,685
• Publication Number:
  US 20170287486A1
• Time in Patent Office: 662 Days
• Field of Search: 704232, 704255, 704275, 704270, 706 20
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

Provided are methods, systems, and apparatuses for detecting, processing, and responding to audio signals, including speech signals, within a designated area or space. A platform for multiple media devices connected via a network is configured to process speech, such as voice commands, detected at the media devices, and respond to the detected speech by causing the media devices to simultaneously perform one or more requested actions. The platform is capable of scoring the quality of a speech request, handling speech requests from multiple end points of the platform using a centralized processing approach, a de-centralized processing approach, or a combination thereof, and also manipulating partial processing of speech requests from multiple end points into a coherent whole when necessary.

Claims

1. A computer-implemented method comprising:

receiving, by a computing device that (i) is operating in a low power mode, (ii) is configured to exit the low power mode upon determining that a particular hotword has likely been spoken, and (iii) is in proximity of other computing devices that are each also configured to exit the low power mode upon determining that the particular hotword has been spoken, audio data corresponding to a user uttering the particular hotword;

based on comparing a hotword confidence score generated by the computing device with one or more respective hotword confidence scores received from one or more of the other computing devices, determining, by the computing device, to remain operating in the low power mode despite determining that the particular hotword has likely been spoken.

2. The computer-implemented method of claim 1, comprising:

transmitting, by the computing device, the hotword confidence score to the other computing devices in response to determining the hotword confidence score.

3. The computer-implemented method of claim 1, comprising:

transmitting, by the computing device, the hotword confidence score to a server in response to receiving other hotword confidence scores from the other computing devices.

4. The computer-implemented method of claim 1, wherein the hotword confidence score is based on a location of the user relative to the computing device.

5. The computer-implemented method of claim 1, wherein determining, by the computing device, to remain operating in the low power mode despite determining that the particular hotword has likely been spoken comprises:

determining, by the computing device, to remain operating in the low power mode despite determining that the particular hotword has likely been spoken in response to determining the hotword confidence score generated by the computing device is less than the one or more respective hotword confidence scores received from the one or more of the other computing devices.

6. The computer-implemented method of claim 3, wherein transmitting the hotword confidence score to a server comprises:

transmitting, by the computing device, the hotword confidence score to the server in response to determining the hotword confidence score is greater than the other hotword confidence scores.

7. The computer-implemented method of claim 3, comprising:

waiting, by the computing device, a predetermined amount of time to receive the other hotword confidence scores from the other computing devices in response to transmitting the hotword confidence score to the other computing devices.

8. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a computing device that (i) is operating in a low power mode, (ii) is configured to exit the low power mode upon determining that a particular hotword has likely been spoken, and (iii) is in proximity of other computing devices that are each also configured to exit the low power mode upon determining that the particular hotword has been spoken, audio data corresponding to a user uttering the particular hotword;

based on comparing a hotword confidence score generated by the computing device with one or more respective hotword confidence scores received from one or more of the other computing devices, determining, by the computing device, to remain operating in the low power mode despite determining that the particular hotword has likely been spoken.

9. The system of claim 8, wherein the operations further comprise:

transmitting, by the computing device, the hotword confidence score to the other computing devices in response to determining the hotword confidence score.

10. The system of claim 8, wherein the operations further comprise:

transmitting, by the computing device, the hotword confidence score to a server in response to receiving other hotword confidence scores from the other computing devices.

11. The system of claim 8, wherein the hotword confidence score is based on a location of the user relative to the computing device.

12. The system of claim 8, wherein determining, by the computing device, to remain operating in the low power mode despite determining that the particular hotword has likely been spoken comprises:

determining, by the computing device, to remain operating in the low power mode despite determining that the particular hotword has likely been spoken in response to determining the hotword confidence score generated by the computing device is less than the one or more respective hotword confidence scores received from the one or more of the other computing devices.

13. The system of claim 10, wherein transmitting the hotword confidence score to a server comprises:

transmitting, by the computing device, the hotword confidence score to the server in response to determining the hotword confidence score is greater than the other hotword confidence scores.

14. The system of claim 10, wherein the operations further comprise:

waiting, by the computing device, a predetermined amount of time to receive the other hotword confidence scores from the other computing devices in response to transmitting the hotword confidence score to the other computing devices.

15. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a computing device that (i) is operating in a low power mode, (ii) is configured to exit the low power mode upon determining that a particular hotword has likely been spoken, and (iii) is in proximity of other computing devices that are each also configured to exit the low power mode upon determining that the particular hotword has been spoken, audio data corresponding to a user uttering the particular hotword;

based on comparing a hotword confidence score generated by the computing device with one or more respective hotword confidence scores received from one or more of the other computing devices, determining, by the computing device, to remain operating in the low power mode despite determining that the particular hotword has likely been spoken.

16. The computer-readable medium of claim 15, wherein the operations comprise:

transmitting, by the computing device, the hotword confidence score to the other computing devices in response to determining the hotword confidence score.

17. The computer-readable medium of claim 15, wherein the operations comprise:

transmitting, by the computing device, the hotword confidence score to a server in response to receiving other hotword confidence scores from the other computing devices.

18. The computer-readable medium of claim 15, wherein the hotword confidence score is based on a location of the user relative to the computing device.

19. The computer-readable medium of claim 15, wherein determining, by the computing device, to remain operating in the low power mode despite determining that the particular hotword has likely been spoken comprises:

determining, by the computing device, to remain operating in the low power mode despite determining that the particular hotword has likely been spoken in response to determining the hotword confidence score generated by the computing device is less than the one or more respective hotword confidence scores received from the one or more of the other computing devices.

20. The computer-readable medium of claim 15, wherein transmitting the hotword confidence score to a server comprises:

transmitting, by the computing device, the hotword confidence score to the server in response to determining the hotword confidence score is greater than the other hotword confidence scores.