Coordination among multiple voice recognition devices

• US 10,665,232 B2
• Filed: 05/24/2017
• Issued: 05/26/2020
• Est. Priority Date: 05/24/2017
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: Thomsen, Brant D.
• Primary Examiner(s): Hang, Vu B
• Application Number:
  US15/604,536
• Publication Number:
  US 20180342244A1
• Time in Patent Office: 1,098 Days
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/21 the extracted parameters being power information
• CPC Class Codes: G10L 25/60 for measuring the quality of voice signals

Abstract

One embodiment of the invention sets forth a mechanism for selecting a voice recognition device included in a network of such devices for processing audio commands corresponding to a detected speech event. The voice recognition devices in the network individually determine which one of the devices is best suited to process the audio commands corresponding to the speech event. In operation, each of the voice recognition devices that detected the same speech event independently selects the same device for processing the audio command. Because each of the voice recognition devices, including the selected device, selects the same device for processing the audio command, the voice recognition devices not need to share information related to the selected device with one another.

Claims

1. A method for processing speech-based commands, the method comprising:

at a first device included in a plurality of devices;

detecting a speech event via locally received audio signals;

recognizing content of the speech event;

determining a number of other devices included in the plurality of devices that recognized the content of the speech event;

generating a set of characteristics associated with the speech event;

receiving, from a second device included in the plurality of devices, a second set of characteristics associated with the speech event detected by the second device;

determining, based on a comparison between the set of characteristics and the second set of characteristics and based on the number of other devices exceeding a threshold, that an action responsive to the speech event should be performed locally; and

performing the action locally.

2. The method of claim 1, wherein the set of characteristics includes a frequency of a locally received audio signal corresponding to the speech event, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the frequency falls within a frequency range associated with human speech.

3. The method of claim 1, wherein the set of characteristics includes a measure of energy of a locally received audio signal corresponding to the speech event, and the second set of characteristics includes a second measure of energy of an externally detected audio signal corresponding to the speech event, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the measure of energy of the locally received audio signal is greater than the second measure of energy of the externally detected audio signal.

4. The method of claim 1, wherein the set of characteristics includes a measure of quality of locally detected audio corresponding to the speech event, and the second set of characteristics includes a second measure of quality of externally detected audio corresponding to the speech event, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the measure of quality of the locally detected audio is higher than the second measure of quality of the externally detected audio.

5. The method of claim 1, wherein the set of characteristics includes a timestamp corresponding to when the speech event was detected locally, and the second set of characteristics includes a timestamp corresponding to when the speech event was detected by the second device, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the speech event was detected locally before the speech event was detected by the second device based on the timestamps.

6. The method of claim 1, wherein the set of characteristics includes a confidence score associated with spoken content recognized from the speech event detected locally, and the second set of characteristics includes a second confidence score associated with spoken content recognized from the speech event detected by the second device, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the confidence score of the locally detected audio is higher than the second confidence score.

7. The method of claim 1, wherein the set of characteristics includes a confidence score associated with spoken content recognized from the speech event, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the confidence score is greater than a threshold value.

8. The method of claim 1, wherein the set of characteristics includes spoken content recognized from the speech event, and the second set of characteristics includes second spoken content recognized from the speech event, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the spoken content substantially matches the second spoken content.

9. The method of claim 1, further comprising:

generating a third set of characteristics associated with a second speech event that has been detected locally;

receiving, from the second device, a fourth set of characteristics associated with the second speech event detected by the second device;

determining, based on a comparison between the third set of characteristics and the fourth set of characteristics, that an action responsive to the speech event should be performed by the second device; and

terminating local processing of the second speech event.

10. A computer readable medium storing instructions that, when executed by a processor of a first device in a plurality of devices, cause the processor to process speech-based commands, by performing the steps of:

detecting a speech event via locally received audio signals;

recognizing content of the speech event;

determining a number of devices included in the plurality of devices that recognized the content of the speech event;

generating a set of characteristics associated with the speech event;

receiving, from a second device included in the plurality of devices, a second set of characteristics associated with the speech event detected by the second device;

determining, based on a comparison between the set of characteristics and the second set of characteristics and based on the number of devices included in the plurality of devices exceeding a threshold, that an action responsive to the speech event should be performed locally; and

performing the action locally.

11. The computer readable medium of claim 10, wherein the set of characteristics includes a frequency of a locally received audio signal corresponding to the speech event, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the frequency falls within a frequency range associated with human speech.

12. The computer readable medium of claim 10, wherein the set of characteristics includes a measure of energy of a locally received audio signal corresponding to the speech event, and the second set of characteristics includes a second measure of energy of an externally detected audio signal corresponding to the speech event, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the measure of energy of the locally received audio signal is greater than the second measure of energy of the externally detected audio signal.

13. The computer readable medium of claim 10, wherein the set of characteristics includes a measure of quality of locally detected audio corresponding to the speech event, and the second set of characteristics includes a second measure of quality of externally detected audio corresponding to the speech event, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the measure of quality of the locally detected audio is higher than the second measure of quality of the externally detected audio.

14. The computer readable medium of claim 10, wherein the set of characteristics includes a timestamp corresponding to when the speech event was detected locally, and the second set of characteristics includes a timestamp corresponding to when the speech event was detected by the second device, and wherein determining that the action responsive to the speech event should be performed locally comprises determining that the speech event was detected locally before the speech event was detected by the second device based on the timestamps.

15. The computer readable medium of claim 14, wherein the timestamps are determined based on a synchronized clock.

16. The computer readable medium of claim 10, wherein the instructions further cause the processor to perform the step of determining that the speech event detected by the second device matches the speech event that has been detected locally.

17. The computer readable medium of claim 16, wherein determining that the speech event detected by the second device matches the speech event that has been detected locally comprises comparing an external timestamp corresponding to when the second device detected the speech event with a local timestamp corresponding to when the speech event was detected locally.

18. The computer readable medium of claim 16, wherein determining that the speech event detected by the second device matches the speech event that has been detected locally comprises comparing first content recognized based on the speech event detected by the second device with second content recognized based on the speech event that has been detected locally.

19. A voice recognition system, comprising, at a first device included in a plurality of devices:

a microphone; and

a computer processor, executing instructions, to perform the steps of;

detecting a speech event via audio signals locally received via the microphone;

recognizing content of the speech event;

determining a number of devices included in the plurality of devices that recognized the content of the speech event;

generating a set of characteristics associated with the speech event,

receiving a second set of characteristics associated with the speech event that has been detected externally by another device included in the plurality of devices,

determining, based on a comparison between the set of characteristics and the second set of characteristics and based on the number of devices included in the plurality of devices exceeding a threshold, that an action responsive to the speech event should be performed locally, and

performing the action locally.

20. The voice recognition system of claim 19, wherein the instructions further cause the processor to perform the steps of:

generating a third set of characteristics associated with a second speech event that has been detected locally via the microphone;

receiving, from the another device included in the plurality of devices, a fourth set of characteristics associated with the second speech event that has been detected externally by the another device included in the plurality of devices;

determining, based on a comparison between the third set of characteristics and the fourth set of characteristics, that an action responsive to the speech event should be performed by the another device included in the plurality of devices; and

terminating local processing of the second speech event.