System and associated method for speech keyword detection enhanced by detecting user activity

• US 9,747,894 B2
• Filed: 01/29/2014
• Issued: 08/29/2017
• Est. Priority Date: 12/09/2013
• Status: Active Grant

• Current Assignee: MediaTek, Inc.
• Original Assignee: MediaTek, Inc.
• Inventors: Hsu, Chao-Ling, Cheng, Yiou-Wen, Sun, Liang-Che, Wang, Yuanyuan
• Primary Examiner(s): SARPONG, AKWASI
• Application Number:
  US14/166,881
• Publication Number:
  US 20150161989A1
• Time in Patent Office: 1,308 Days
• Field of Search: 704251, 358418
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/228 of application context

Abstract

The invention provides a system for speech keyword detection and associated method. The system includes a speech keyword detector, an activity predictor and a decision maker. The activity predictor obtains sensor data provided by a plurality of sensors, and processes the sensor data to provide an activity prediction result indicating a probability for whether a user is about to give voice keyword. The decision maker processes the activity prediction result and a preliminary keyword detection result of the speech keyword detection to provide a keyword detection result.

Claims

1. A system for speech keyword detection of a device, comprising:

a voice detector for evaluating informativeness of a received incoming sound according to a current volume and a past average volume of the incoming sound, so as to selectively enable the speech keyword detector according to the informativeness;

a speech keyword detector for generating a most probable keyword as a preliminary keyword detection result according to the received incoming sound;

an activity predictor for obtaining sensor data provided by a plurality of sensors, and processing the sensor data to provide an activity prediction result indicating occurrence of indicative activities which reflect whether a user is about to give a voice keyword; and

a decision maker coupled to the activity predictor and the speech keyword detector, for providing a keyword detection result by executing;

when the activity prediction result indicates absence of indicative activities, lowering a sensitivity to admit that the most probable keyword is said, so as to suppress false alarm error; and

when the activity prediction result indicates presence of indicative activities, raising the sensitivity to admit that the most probable keyword is said, so as to suppress miss error;

wherein the voice detector comprises;

a first estimator for computing a current sample of a first sequence as a weighted sum of the current volume of the incoming sound and a preceding sample of the first sequence;

a second estimator for computing a current sample of a second sequence as a weighted sum of the current volume of the incoming sound and a preceding sample of the second sequence; and

a comparator, coupled to the first estimator and the second estimator, comparing the first sequence and the second sequence to indicate the informativeness of the incoming sound.

2. The system of claim 1 further comprising:

an activity database comprising a plurality of activity lists;

each activity list being associated with a target application, and comprising a plurality of activity templates,

wherein the activity predictor executes;

obtaining a selected one of the activity lists, wherein the target application associated with the selected activity list matches a currently running application of the device; and

comparing the sensor data with each activity template of the selected activity list to provide the activity prediction result.

3. The system of claim 1 further comprising:

an activity database comprising a plurality of activity lists;

each activity list being associated with a target application, and comprising a plurality of activity templates,

wherein the activity predictor executes;

obtaining a selected one of the activity lists, wherein the target application associated with the selected activity list matches a currently running application of the device;

providing extracted data by extracting features of the sensor data;

and

comparing the extracted data with each activity template of the selected activity list to provide the activity prediction result.

4. The system of claim 1 further comprising:

a keyword database comprising a plurality of keyword lists;

each keyword list being associated with a target application, and comprising a plurality of candidate keywords,

wherein the speech keyword detector executes;

obtaining a selected one of the keyword lists, wherein the target application associated with the selected keyword list matches a currently running application of the device; and

comparing the received incoming sound with each candidate keyword of the selected keyword list to provide the preliminary keyword detection result.

5. The system of claim 1, wherein:

the first estimator computes the current sample of the first sequence by summing a first product and a second product, the first product multiplying the preceding sample of the first sequence with a first weighting, and the second product multiplying the current volume of the incoming sound with a first complementary weighting; and

the second estimator computes the current sample of the second sequence by summing a third product and a fourth product, the third product multiplying the preceding sample of the second sequence with a second weighting, and the fourth product multiplying the current volume of the incoming sound with a second complementary weighting.

6. The system of claim 5, wherein the first weighting is less than the second weighting.

7. The system of claim 1, wherein the decision maker applies a probability fusion algorithm to process the activity prediction result and the preliminary keyword detection result for providing the keyword detection result.

8. The system of claim 1, wherein the decision maker adopts a machine learning algorithm to process the activity prediction result and the preliminary keyword detection result for proving the keyword detection result.

9. The system of claim 1, wherein the decision maker calculates a linear combination of the activity prediction result and the preliminary keyword detection result as the keyword detection result.

10. The system of claim 1, wherein the sensors include at least one of the following:

an accelerometer, a gyroscope, a magnetometer, a barometer, a proximity sensor, a light sensor, a touch screen, a receiver of a positioning system, a wireless receiver and a camera.

11. A method for speech keyword detection of a device, comprising:

performing a voice detection by evaluating informativeness of a received incoming sound according to a current volume and a past average volume of the incoming sound, so as to selectively enable the speech keyword detection according to the informativeness;

generating a most probable keyword as a preliminary keyword detection result according to the received incoming sound;

obtaining sensor data provided by a plurality of sensors;

according to the sensor data, computing an activity prediction result indicating occurrence of indicative activities which reflect whether a user is about to give voice keyword; and

computing a keyword detection result by executing;

when the activity prediction result indicates absence of indicative activities, lowering a sensitivity to admit that the most probable keyword is said, so as to suppress false alarm error; and

when the activity prediction result indicates presence of indicative activities, raising the sensitivity to admit that the most probable keyword is said, so as to suppress miss error;

wherein the step of performing the voice detection comprises;

computing a current sample of a first sequence as a weighted sum of the current volume of the incoming sound and a preceding sample of the first sequence;

computing a current sample of a second sequence as a weighted sum of the current volume of the incoming sound and a preceding sample of the second sequence; and

comparing the first sequence and the second sequence to indicate the informativeness of the incoming sound.

12. The method of claim 11 further comprising:

providing an activity database which comprises a plurality of activity lists;

each activity list being associated with a target application, and comprising a plurality of activity templates;

identifying currently running application of the device;

selecting one of the activity lists by comparing the currently running application with each target application of each activity list; and

comparing the sensor data with each activity template of the selected activity list to provide the activity prediction result.

13. The method of claim 11 further comprising:

providing an activity database which comprises a plurality of activity lists;

each activity list being associated with a target application, and comprising a plurality of activity templates;

identifying currently running application of the device;

by comparing the currently running application with each target application of each activity list, selecting one of the activity lists;

providing extracted data by extracting features of the sensor data;

comparing the extracted data with each activity template of the selected activity list and accordingly providing an activity matching result to be included in the activity prediction result.

14. The method of claim 11 further comprising:

providing a keyword database which comprises a plurality of keyword lists;

each keyword list being associated with a target application, and comprising a plurality of candidate keywords;

identifying currently running application of the device;

selecting one of the keyword lists by comparing the currently running application with each target application of each keyword list; and

providing the preliminary keyword detection result by comparing the received incoming sound with each candidate keyword of the selected keyword list.

15. The method of claim 11, wherein:

the step of computing the current sample of the first sequence by comprises summing a first product and a second product, the first product multiplying the preceding sample of the first sequence with a first weighting, and the second product multiplying the current volume of the incoming sound with a first complementary weighting; and

the step of computing the current sample of the second sequence comprises summing a third product and a fourth product, the third product multiplying the preceding sample of the second sequence with a second weighting, and the fourth product multiplying the current volume of the incoming sound with a second complementary weighting.

16. The method of claim 15, wherein the first weighting is less than the second weighting.

17. The method of claim 11, wherein computing the keyword detection result comprises:

adopting a probability fusion algorithm to process the activity prediction result and the preliminary keyword detection result.

18. The method of claim 11, wherein computing the keyword detection result comprises:

adopting a machine learning algorithm to process the activity prediction result and the preliminary keyword detection result.

19. The method of claim 11, wherein the step of computing the second keyword detection result calculates a linear combination of activity prediction result and the preliminary keyword detection result as the keyword detection result.

20. The method of claim 11, wherein the sensors includes at least one of the following:

an accelerometer, a gyroscope, a magnetometer, a barometer, a proximity sensor, a light sensor, a touch screen, a receiver of a positioning system, a wireless receiver and a camera.