Speech recognition apparatus and method

• US 9,443,510 B2
• Filed: 07/05/2013
• Issued: 09/13/2016
• Est. Priority Date: 07/09/2012
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Jung, Dukyung
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US14/411,627
• Publication Number:
  US 20150161992A1
• Time in Patent Office: 1,166 Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/083 Recognition networks G10L15/142, G10L15/16 take precedence
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology

Abstract

The present specification relates to a speech recognition apparatus and method capable of accurately recognizing the speech of a user in an easy and convenient manner without the user having to operate a speech recognition start button or the like. The speech recognition apparatus according to embodiments of the present specification comprises: a camera for capturing a user image; a microphone; a control unit for detecting a preset user gesture from the user image, and, if a nonlexical word is detected from the speech signal which is input through the microphone from the point in time at which the user gesture was detected, determining the speech signal detected after the detected nonlexical word as an effective speech signal; and a speech recognition unit for recognizing the effective speech signal.

Claims

1. A speech recognition apparatus, comprising:

a camera configured to capture a plurality of images of a user;

a microphone;

a control unit configured to;

track at least one eye of the user based on the plurality of images of the user;

determine a reference time at which the tracked at least one eye of the user is directed toward the microphone;

determine whether a nonlexical word is detected in a first speech signal received via the microphone during a period of time beginning from the reference time at which the tracked at least one eye of the user is directed toward the microphone; and

based on a determination that the nonlexical word is detected in the first speech signal during the period of time beginning from the reference time, determine a second speech signal received via the microphone subsequent to the detected nonlexical word; and

a speech recognition unit configured to recognize a speech of the user from the second speech signal.

2. The speech recognition apparatus of claim 1, wherein the control unit is configured to determine whether the nonlexical word is detected in the first speech signal by determining whether an interjection is detected in the first speech signal.

3. The speech recognition apparatus of claim 1, wherein the control unit is further configured to activate the speech recognition unit based on a determination that the tracked at least one eye of the user is directed toward the microphone.

4. The speech recognition apparatus of claim 3, wherein the control unit is further configured to:

based on a determination that the nonlexical word is not detected in the first speech signal, deactivate the speech recognition unit.

5. The speech recognition apparatus of claim 1, wherein:

the camera and the microphone are provided in proximity to each other; and

the control unit is further configured to determine that the tracked at least one eye of the user is directed toward the microphone by determining that the tracked at least one eye of the user is directed toward the camera.

6. The speech recognition apparatus of claim 1, wherein the control unit is further configured to track a shape of a mouth of the user based on the plurality of images of the user.

7. The speech recognition apparatus of claim 6, wherein the control unit is configured to track the shape of the mouth of the user by tracking at least one of a movement of the mouth of the user or an open state of the mouth of the user.

8. The speech recognition apparatus of claim 6, wherein the control unit is further configured to:

determine a first tracked shape of the mouth that corresponds to the nonlexical word;

determine whether the first tracked shape of the mouth is detected in the plurality of images of the user during the period of time beginning from the reference time at which the tracked at least one eye of the user is directed toward the microphone; and

based on a determination that the first tracked shape of the mouth is detected in the plurality of images of the user during the period of time beginning from the reference time, determine the second speech signal received via the microphone subsequent to the detected first tracked shape of the mouth.

9. The speech recognition apparatus of claim 1, wherein the control unit is further configured to:

determine, based on the plurality of images of the user, that one of an arm or a hand of the user moves near a face of the user; and

based on the determination that the one of the arm or the hand of the user moves near the face of the user, activate the speech recognition unit.

10. A speech recognition method, comprising:

capturing, via a camera, a plurality of images of a user;

tracking at least one eye of the user based on the plurality of images of the user;

determining a reference time at which the tracked at least one eye of the user is directed toward a microphone;

determining whether a nonlexical word is detected in a first speech signal received via the microphone during a period of time beginning from the reference time at which the tracked at least one eye of the user is directed toward the microphone;

based on a determination that the nonlexical word is detected in the first speech signal during the period of time beginning from the reference time, determine a second speech signal received via the microphone subsequent to the detected nonlexical word; and

recognizing, via a speech recognition unit, a speech of the user from the second speech signal.

11. The speech recognition method of claim 10, wherein determining whether the nonlexical word is detected in the first speech signal comprises determining whether an interjection is detected in the first speech signal.

12. The speech recognition method of claim 10, further comprising:

activating the speech recognition unit based on a determination that the tracked at least one eye of the user is directed toward the microphone.

13. The speech recognition method of claim 12, further comprising:

deactivating the speech recognition unit based on a determination that the nonlexical word is not detected in the first speech signal after a predetermined period of time subsequent to a most recent activation of the speech recognition unit.

14. The speech recognition method of claim 10, further comprising determining that the tracked at least one eye of the user is directed toward the microphone by determining that the tracked at least one eye of the user is directed toward the camera.

15. The speech recognition method of claim 10, further comprising:

tracking a shape of a mouth of the user based on the plurality of images of the user.

16. The speech recognition method of claim 15, wherein tracking the shape of the mouth of the user comprises tracking at least one of a movement of the mouth of the user or an open state of the mouth of the user.

17. The speech recognition method of claim 15, further comprising:

determining a first tracked shape of the mouth that corresponds to the nonlexical word;

determining whether the first tracked shape of the mouth is detected in the plurality of images of the user during the period of time beginning from the reference time at which the tracked at least one eye of the user is directed toward the microphone; and

based on a determination that the first tracked shape of the mouth is detected in the plurality of images of the user during the period of time beginning from the reference time, determining the second speech signal received via the microphone subsequent to the detected first tracked shape of the mouth.

18. The speech recognition method of claim 10, further comprising:

determining, based on the plurality of images of the user, that one of an arm or a hand of the user moves near a face of the user; and

based on the determination that the one of the arm or the hand of the user moves near the face of the user, activating the speech recognition unit.