Context-aware query recognition for electronic devices

• US 10,147,423 B2
• Filed: 09/29/2016
• Issued: 12/04/2018
• Est. Priority Date: 09/29/2016
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel IP Corporation (Intel Corporation)
• Inventors: Georges, Munir Nikolai Alexander, Stemmer, Georg, Hofer, Joachim
• Primary Examiner(s): Desir, Pierre-Louis
• Assistant Examiner(s): WANG, YI SHENG
• Application Number:
  US15/280,809
• Publication Number:
  US 20180090140A1
• Time in Patent Office: 796 Days
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/142 Hidden Markov Models [HMMs]
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

A method for context-aware query recognition in an electronic device includes receiving user speech from an input device. A speech signal is generated from the user speech. It is determined if the speech signal includes an action to be performed and if the electronic device is the intended recipient of the user speech. If the recognized speech signal include the action and the intended recipient of the user speech is the electronic device, a command is generated for the electronic device to perform the action.

Claims

1. An electronic device to execute a context-aware query recognition, the device comprising:

an input device to receive user speech;

memory to store instructions for performing a context-aware query; and

a controller, coupled to the memory and the input device, the controller being configured to;

generate a speech signal including applying the user speech to a statistically-based speech recognition algorithm,

determine, using a first machine learning process employing at least one of a condition random field, hidden Markov model, or neural network, if the speech signal includes words implying an action and whether the words implying the action request the action to be performed,

determine, using a second machine learning process employing at least one of a condition random field, Gaussian mixture model, or neural network, the second machine learning process running in parallel with the first machine learning process, if the speech signal includes words associated with an intended recipient of the user speech, and

if the speech signal requests the action to be performed and if the intended recipient of the user speech is the electronic device, generate a command for the electronic device to perform the action, wherein the user speech and the speech signal do not include a wake-up phrase.

2. The device of claim 1, further comprising a sensor coupled to the controller to generate a sensor input to the controller.

3. The device of claim 2, wherein the sensor comprises at least one of an image sensor to generate image data;

an audio sensor to receive the user speech;

or a biometric sensor to generate biometric sensor data.

4. The device of claim 3, wherein the controller is further configured to determine the intended recipient based on at least one of the image data or the biometric sensor data.

5. The device of claim 3, wherein the controller is further configured to determine if the speech signal includes the words implying the action and if the words implying the action request the action to be performed based on at least one of the image data or the biometric sensor data.

6. The device of claim 1, wherein the controller is further configured to perform a database search based on the command.

7. The device of claim 1;

wherein the controller is further configured to perform at least one of a World Wide Web search based on the command, input the command to a dialog engine, or input the command to a chat-bot.

8. The device of claim 1;

wherein the controller is further configured to pass the command to an application program interface for execution.

9. The device of claim 1, wherein the electronic device is a kitchen appliance, a computer, a control system in a vehicle, a personal electronic device, or a home controller.

10. A computer-implemented method for context-aware query recognition in an electronic device, the method comprising:

receiving user speech from an input device;

generating a speech signal including applying the user speech to a statistical-based speech recognition algorithm, wherein the user speech and the speech signal do not include a wake-up phrase;

determining, using a first machine learning process employing at least one of a condition random field, hidden Markov model, or neural network, if the speech signal includes words implying an action and determining whether the words implying the action request the action to be performed;

determining an intended recipient of the user speech using a second machine learning process employing at least one of a condition random field, Gaussian mixture model, or neural network, the second machine learning process running in parallel with the first machine learning process; and

if the speech signal requests the action to be performed and the intended recipient of the user speech is the electronic device, generating a command for the electronic device to perform the action.

11. The method of claim 10;

further comprising receiving a sensor input comprising an image, audio or biometric data.

12. The method of claim 11, further comprising:

determining if the speech signal includes the words implying the action and if the words implying the action request the action to be performed based on the sensor input.

13. The method of claim 11, further comprising:

determining the intended recipient of the user speech based on the sensor input.

14. The method of claim 10, wherein determining if the speech signal includes the words implying the action and if the words implying the action request the action to be performed comprises:

receiving at least one of words or phonemes from the speech signal;

determining using the first machine learning process, if the words or the phonemes are action words or sounds; and

outputting the action to be performed if the words or the phonemes request the action to be performed.

15. The method of claim 14, further comprising outputting a negative or a probabilistic indication if the words or the phonemes are not action words or sounds or do not request the action to be performed.

16. The method of claim 10, wherein determining the intended recipient of the user speech comprises:

receiving at least one of words or phonemes from the speech signal;

determining using the second machine learning process, if the words or the phonemes are associated with the electronic device; and

if the words or phonemes are associated with the electronic device, outputting the an affirmative indication that the intended recipient of the user speech is the electronic device.

17. The method of claim 10, wherein determining the intended recipient of the user speech comprises:

receiving at least one of words or phonemes from the speech signal;

determining, using the statistically-based speech recognition algorithm, whether the words or the phonemes are associated with the electronic device; and

if the words or phonemes are not associated with the electronic device, outputting a negative indication.

18. A computer-implemented method for context-aware query recognition in an electronic device, the method comprising:

receiving at the electronic device user speech from an input device;

generating a speech signal in response to the user speech by applying the user speech to a statistically-based speech recognition algorithm, wherein the user speech and the speech signal do not include a wake-up phrase;

analyzing the speech signal using a first machine learning process employing at least one of a condition random field, hidden Markov model, or neural network to determine if the speech signal includes at least one of words or phonemes indicating that the user speech was intended for the electronic device;

analyzing the speech signal using a second machine learning process employing at least one of a condition random field, Gaussian mixture model, or neural network, the second machine learning process running in parallel with the first machine learning process to determine if the speech signal includes at least one of words or phonemes implying an action and determining whether the words implying the action request an action; and

generating a command for the electronic device based on the requested action.

19. The method of claim 18, further comprising forwarding the command to an application programming interface to execute the command in one of a kitchen appliance, a computer, a control system in a vehicle, a personal electronic device, or a home controller.

20. The method of claim 18, further comprising transmitting the command over an Internet connection to a server for performing a database search or a World Wide Web search.

21. At least one non-transitory computer-readable medium comprising instructions for executing context-aware query recognition in an electronic device that, when executed by a computer, cause the computer to:

receive user speech from an input device;

generate a speech signal including applying the user speech to a statistically-based speech recognition algorithm, wherein the user speech and the speech signal do not include a wake-up phrase;

determine using a first machine learning process employing at least one of a condition random field, hidden Markov model, or neural network, if the speech signal includes words implying an action and determine using the first machine learning process whether the words implying the action request the action to be performed;

determine an intended recipient of the user speech using a second machine learning process employing at least one of a condition random field, Gaussian mixture model, or neural network, the second machine learning process running in parallel with the first machine learning process; and

if the speech signal requests the action to be performed and the intended recipient of the user speech is the electronic device, generate a command for the electronic device to perform the action.

22. The computer-readable medium of claim 21, wherein, when the instructions cause the computer to determine that the speech signal includes the words implying the action and that the words implying the action request the action to be performed, the instructions cause the computer to:

receive at least one of words or phonemes from the speech signal;

determine, using the first machine learning process, if the words or the phonemes are action words or sounds; and

output the action to be performed if the words or the phonemes comprise the action to be performed.

23. The computer-readable medium of claim 21, wherein, when the instructions cause the computer to determine the intended recipient of the user speech, the instructions cause the computer to:

receive at least one of words or phonemes from the speech signal;

determine, using the second machine learning process, if the words or the phonemes are words or sounds associated with the electronic device; and

if the words or phonemes are associated with the electronic device, output an affirmative indication that the intended recipient of the user speech is the electronic device.

24. The computer-readable medium of claim 21;

wherein the instructions further cause the computer to determine the intended recipient of the user speech based on a sensor input.

25. The computer-readable medium of claim 24, wherein the instructions further cause the computer to determine the intended recipient of the user speech based on images from the sensor input.