Multi-microphone speech recognition systems and related techniques

• US 10,304,462 B2
• Filed: 01/15/2018
• Issued: 05/28/2019
• Est. Priority Date: 06/06/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Ramprashad, Sean A., Thornburg, Harvey D., Krishnaswamy, Arvindh, Lindahl, Aram M.
• Primary Examiner(s): Singh, Satwant K
• Application Number:
  US15/871,836
• Publication Number:
  US 20180137864A1
• Time in Patent Office: 498 Days
• Field of Search: None
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A speech recognition system for resolving impaired utterances can have a speech recognition engine configured to receive a plurality of representations of an utterance and concurrently to determine a plurality of highest-likelihood transcription candidates corresponding to each respective representation of the utterance. The recognition system can also have a selector configured to determine a most-likely accurate transcription from among the transcription candidates. As but one example, the plurality of representations of the utterance can be acquired by a microphone array, and beamforming techniques can generate independent streams of the utterance across various look directions using output from the microphone array.

Claims

1. A system for resolving utterances, the system comprising a processor and a memory, wherein the memory contains instructions that, when executed by the processor, cause the system to:

receive a plurality of utterance representations corresponding to an utterance, wherein each utterance representation corresponds to a respective version of the utterance observed by an acoustic device in a plurality of acoustic devices exposed to the utterance;

jointly process the plurality of utterance representations to determine recognized speech corresponding to the utterance; and

coordinate control of one or more of the plurality of acoustic devices responsive to the recognized speech.

2. The system according to claim 1, further comprising a communication connection, wherein the plurality of acoustic devices comprises at least one audio appliance, and wherein the instructions, when executed by the processor, further cause the system to receive at least one of the plurality of utterance representations from the at least one audio appliance over the communication connection.

3. The system according to claim 2, wherein at least one of the plurality of acoustic devices further comprises a microphone transducer, and wherein the plurality of utterance representations further comprises an utterance representation corresponding to an output from the microphone transducer.

4. The system according to claim 3, wherein:

the microphone transducer is a first microphone transducer;

the at least one of the plurality of acoustic devices further comprises a second microphone transducer; and

the plurality of utterance representations further comprises an utterance representation corresponding to an output from the second microphone transducer.

5. The system according to claim 3, wherein the microphone transducer is a first microphone transducer and at least one other of the plurality of acoustic devices comprises a second microphone transducer.

6. The system according to claim 1, wherein the instructions, when executed by the processor, further cause the system to determine at least one intermediate transcription candidate from each of the plurality of utterance representations.

7. The system according to claim 6, wherein the instructions, when executed by the processor, further cause the system to update one or more of the intermediate transcription candidates responsive to a comparison of at least two of the intermediate transcription candidates with each other.

8. The system according to claim 6, wherein the instructions, when executed by the processor, further cause the system to combine one or more recognition parameters from a recognition-parameter store with at least one of the intermediate transcription candidates.

9. The system according to claim 1, wherein the instructions, when executed by the processor, further cause the system to determine a plurality of intermediate transcription candidates from at least one observed version of the utterance.

10. The system according to claim 9, wherein the instructions, when executed by the processor, further cause the system to update one or more of the intermediate transcription candidates responsive to a comparison of at least two of the intermediate transcription candidates with each other.

11. The system according to claim 9, wherein the instructions, when executed by the processor, further cause the system to combine one or more recognition parameters from a recognition-parameter store with at least one of the intermediate transcription candidates.

12. The system according to claim 1, wherein the instructions, when executed by the processor, further cause the system to determine one or more acoustic features and associated likelihoods corresponding to each utterance representation.

13. The system according to claim 12, wherein the instructions, when executed by the processor, further cause the system to determine the most-likely transcription candidate from a fusion of the one or more acoustic features and associated likelihoods determined among the plurality of utterance representations.

14. The system according to claim 12, wherein each acoustic feature has a corresponding posterior probability, and wherein the instructions, when executed by the processor, further cause the system to identify the acoustic features having a highest-probability of correctly representing the utterance, and to combine the one or more recognition parameters from a recognition parameter store with the highest probability acoustic features.

15. The system of claim 14, wherein the recognition parameter store comprises an acoustic feature dictionary, a language model, or both.

16. The system of claim 15, wherein the plurality of acoustic features comprises a plurality of phonemes and the acoustic feature dictionary comprises a phonetic dictionary.

17. An acoustic appliance comprising a processor and a memory, wherein the memory stores instructions which, when executed by the processor, cause the acoustic appliance to, in real-time:

receive a plurality of versions of an utterance from a corresponding plurality of spatially distributed acoustic devices;

process each version of the utterance to determine an utterance representation corresponding to each version of the utterance;

over a communication connection, transmit the plurality of utterance representations to a computing component; and

over the communication connection, receive from the computing component, recognized speech corresponding to the utterance.

18. The appliance according to claim 17, wherein the instructions, when executed by the processor, further cause the acoustic appliance to invoke a task responsive to the recognized speech received from the computing component.

19. The appliance according to claim 18, wherein the task, when invoked, causes the acoustic appliance to issue a command to another appliance.

20. The appliance according to claim 18, wherein the task, when invoked, causes the acoustic appliance to coordinate control of one or more other appliances over the communication connection.

21. The appliance according to claim 17, wherein the acoustic appliance comprises a first appliance, and the plurality of spatially distributed acoustic devices comprises a second appliance, and wherein the instructions, when executed by the processor, further cause the acoustic appliance to communicate information corresponding to the recognized speech to the second appliance over the communication connection.

22. The appliance according to claim 17, wherein the plurality of representations of the utterance comprises encoded audio or encoded speech.

23. The appliance according to claim 17, wherein the plurality of representations of the utterance comprises a stream of acoustic features and corresponding statistical measures.

24. The appliance according to claim 17, wherein the instructions, when executed by the processor, further cause the appliance to synchronize the plurality of representations of the utterance with each other.

25. The appliance according to claim 17, wherein the communication connection is further configured to communicate with another appliance and wherein the instructions, when executed by the processor, cause the acoustic appliance to receive at least one representation of the utterance from the other appliance, and wherein the plurality of representations of the utterance further comprises the at least one representation of the utterance received from the other appliance.

26. The appliance according to claim 17, wherein each spatially distributed acoustic device observes at least one of the plurality of utterance versions.

27. An electronic device comprising a processor, a memory, and a plurality of microphone transducers, wherein the memory contains instructions that, when executed by the processor, cause the electronic device to, in real-time:

observe an utterance with each of the plurality of microphone transducers and determine an observed version of the utterance for each microphone transducer;

determine a plurality of utterance representations from the plurality of observed versions of the utterance, wherein each utterance representation corresponds to a respective observed version of the utterance;

communicate the plurality of utterance representations to a speech-recognition component/module over a communication connection;

receive over the communication connection recognized speech corresponding to the utterance; and

invoke a task responsive to the recognized speech.

28. The electronic device according to claim 27, wherein the task, when invoked, causes the electronic device to issue a command over the communication connection to another electronic device.

29. The electronic device according to claim 27, wherein the instructions, when executed by the processor, further cause the electronic device to receive, over the communication connection, at least one utterance representation from another electronic device exposed to the utterance, and wherein the plurality of utterance representations comprises the at least one utterance representation received over the communication connection.

30. The electronic device according to claim 27, wherein the communication connection comprises a network connection.

31. The electronic device according to claim 27, wherein the communication connection comprises a bus to interconnect components of the electronic device.

32. The electronic device according to claim 27, wherein the instructions, when executed by the processor, further cause the electronic device to selectively encode the plurality of utterance representations to generate a plurality of encoded utterance representations in correspondence with a measure of quality of the communication connection.

33. The electronic device according to claim 27, wherein the task, when invoked, causes the electronic device to coordinate control of one or more other electronic devices over the communication connection.