Background speech recognition assistant using speaker verification

• US 8,768,707 B2
• Filed: 12/16/2011
• Issued: 07/01/2014
• Est. Priority Date: 09/27/2011
• Status: Active Grant

• Current Assignee: Sensory Incorporated
• Original Assignee: Sensory Incorporated
• Inventors: Mozer, Todd F.
• Primary Examiner(s): Godbold, Douglas
• Application Number:
  US13/329,017
• Publication Number:
  US 20130080167A1
• Time in Patent Office: 928 Days
• Field of Search: 704246- 50, 704270-2701
• US Class Current: 704/270
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

In one embodiment, a method includes receiving an acoustic input signal at a speech recognizer. A user is identified that is speaking based on the acoustic input signal. The method then determines speaker-specific information previously stored for the user and a set of responses based on the recognized acoustic input signal and the speaker-specific information for the user. It is determined if the response should be output and the response is outputted if it is determined the response should be output.

Claims

1. A method comprising:

receiving, by a computing device, an acoustic input signal at a speech recognizer;

identifying, by the computing device, a user that is speaking based on the acoustic input signal;

determining, by the computing device, speaker-specific information previously stored for the user;

determining, by the computing device, a set of classifications, wherein the set of classifications are determined based on the speaker-specific information;

classifying, by the computing device, portions of the acoustic input signal into different classifications in the set of classifications;

selecting, by the computing device, a classification in the set of classifications based on a criterion associated with the classification;

determining, by the computing device, a set of responses based on the recognized acoustic input signal, the classification, and the speaker-specific information for the user;

determining, by the computing device, if the response should be output; and

outputting, by the computing device, the response if it is determined the response should be output,

wherein classifying portions is performed in an always on mode, and wherein identifying the user that is speaking is performed after receiving a trigger phrase to activate the speech recognizer.

2. The method of claim 1, wherein the speech recognizer is configured to recognize the acoustic input signal in the always on mode and the response is outputted without touching computing device or speaking the trigger phrase to activate the speech recognizer.

3. The method of claim 1, wherein the response is outputted after a user speaks the trigger phrase to activate the speech recognizer.

4. The method of claim 1, wherein the wherein the speaker-specific information is associated with previous speech recognition of speech from the user.

5. The method of claim 1, wherein the speaker-specific information is used to modify a classification in the set of classifications based on a preference of the user in the speaker-specific information.

6. The method of claim 5, wherein a set of keywords associated with the speaker-specific information is used in the classification.

7. The method of claim 1, wherein classifying portions is not performed until receiving the trigger phrase to activate the speech recognizer.

8. The method of claim 1, further comprising training the speech recognizer to recognize different user'"'"'s speech signature.

9. The method of claim 1, further comprising storing speaker-specific information for the user based on the response for use in determining additional responses.

10. The method of claim 1, wherein determining the set of responses comprises:

determining user preferences in the speaker-specific information; and

performing a search using the user preferences and the recognized acoustic input signal.

11. The method of claim 10, further comprising:

determining the set of responses; and

ranking the responses based on the user preferences.

12. The method of claim 1, further comprising:

ranking the set of responses based on criteria and the speaker-specific information;

determining if the response should be output based on a ranking of the response;

determining an output method in a plurality of output methods based on the ranking of the response; and

outputting the response using the output method.

13. A method comprising:

receiving, by a computing device, a signal from a first stage recognizer based on recognition of an acoustic input signal and classification of portions of the acoustic input signal into a classification in a plurality of classifications using a first speech recognition algorithm, the first stage recognizer being configured to recognize the acoustic input signal in an always on mode;

activating, by the computing device, the second stage recognizer upon receiving the signal to recognize the acoustic input signal, the second stage recognizer configured to use a second speech recognition algorithm;

identifying, by the computing device, a user that is speaking based on the acoustic input signal;

determining, by the computing device, speaker-specific information previously stored for the user;

determining, by the computing device, a response to the recognized acoustic input signal based on the speaker-specific information;

determining, by the computing device, if the response should be output based on a ranking of the response; and

outputting, by the computing device, the response if it is determined the response should be output.

14. The method of claim 13, wherein determining the response comprises:

determining a plurality of responses based on the recognized acoustic input signal;

ranking the plurality of responses based on criteria including the speaker-specific information; and

selecting a response based on the ranking.

15. The method of claim 13, wherein the ranking is based on the speaker-specific information, a relevance factor, urgency factor, and an importance factor assigned to the response.

16. The method of claim 13, further comprising:

determining an output method in a plurality of output methods based on the ranking and the speaker-specific information; and

outputting the response based on the output method.

17. The method of claim 13, wherein the first stage recognizer is triggered to turn on and send the signal based on the speaker-specific information.

18. A system comprising:

a first stage recognizer configured to recognize the acoustic input signal using a first speech recognition algorithm in an always on mode, the first stage recognizer configured to;

receive an acoustic input signal;

identify a user that is speaking based on the acoustic input signal;

determine speaker specific information previously stored for the user;

classify portions of the acoustic input signal into different classifications using a first speech recognition algorithm;

determine a second stage recognizer should be triggered based on a selection of a classification based on classified portions being classified with the selected classification and the speaker-specific information; and

a second stage recognizer configured to;

receive a signal from the first stage recognizer to activate the second stage recognizer;

activate the second stage recognizer upon receiving the signal to recognize the acoustic input signal, the second stage recognizer configured to use a second speech recognition algorithm different from the first speech recognition algorithm to recognize the acoustic input signal;

determine a response to the recognized acoustic input signal using the speaker-specific information;

determine if the response should be output based on a ranking of the response; and

output the response if it is determined the response should be output.

19. The system of claim 18, wherein the second stage recognizer determines an output method to output the response based on the speaker-specific information.

20. The system of claim 19, wherein the first stage recognizer classifies portions of the acoustic input signal into different classifications, wherein the different classifications are determined based on the speaker-specific information.

21. A method comprising:

receiving, by a computing device, a trigger phrase;

activating, by the computing device, a speech recognizer based on receiving the trigger phrase;

receiving, by the computing device, an acoustic input signal at the speech recognizer;

identifying, by the computing device, a user that is speaking based on the acoustic input signal or the trigger phrase;

determining, by the computing device, speaker-specific information previously stored for the user;

determining, by the computing device, a set of classifications, wherein the set of classifications are determined based on the speaker-specific information;

classifying, by the computing device, portions of the acoustic input signal into different classifications in the set of classifications;

selecting, by the computing device, a classification in the set of classifications based on a criterion associated with the classification;

determining, by the computing device, a set of responses based on the recognized acoustic input signal, the classification, and the speaker-specific information for the user; and

outputting, by the computing device, the response if it is determined the response should be output wherein classifying portions is performed in an always on mode, and wherein identifying the user that is speaking is performed after receiving the trigger phrase to activate the speech recognizer.

22. The method of claim 21, further comprising verifying who is speaking after receiving the trigger phrase to determine if the identified user that is speaking is still speaking.

23. The method of claim 22, wherein the verifying is performed periodically.

24. The method of claim 22, wherein a second verifying who is speaking occurs when a higher security is deemed necessary.

25. The method of claim 24 where a manual log in is not required in a secure situation because the second verifying is performed.