SYSTEMS AND METHODS OF PERFORMING SPEECH RECOGNITION USING GESTURES

• US 20090150160A1
• Filed: 09/25/2008
• Published: 06/11/2009
• Est. Priority Date: 10/05/2007
• Status: Active Grant

• Current Assignee: Sensory Incorporated
• Original Assignee: Sensory Incorporated
• Inventors: Mozer, Todd F.
• Granted Patent: US 8,321,219 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features

Abstract

Embodiments of the present invention improve methods of performing speech recognition using human gestures. In one embodiment, the present invention includes a speech recognition method comprising detecting a gesture, selecting a first recognition set based on the gesture, receiving a speech input signal, and recognizing the speech input signal in the context of the first recognition set.

Claims

1. A speech recognition method comprising:

detecting a gesture of at least one part of a human body;

selecting a first recognition set based on the gesture;

receiving a speech input signal; and

recognizing the speech input signal in the context of the first recognition set, the recognizing resulting in a recognition result.

2. The method of claim 1 further comprising initiating a program corresponding to the gesture.

3. The method of claim 1 wherein the gesture is a hand gesture, wherein the step of detecting the gesture includes detecting a portion of a human hand.

4. The method of claim 1 wherein the gesture includes motion.

5. The method of claim 1 wherein the step of detecting includes the use of infrared light emissions.

6. The method of claim 1 wherein the gesture is from a set of gestures corresponding to a plurality of recognition sets,

wherein each recognition set corresponds to at least one gesture in the set of gestures.

7. The method of claim 1 wherein the recognizing includes using the first recognition set to weight a set of likelihoods, wherein the first recognition set includes segments of speech.

8. The method of claim 1 further comprising reducing a level of sound generation in response to the step of detecting the gesture.

9. The method of claim 1 further comprising initiating a state of a computer program corresponding to the gesture.

10. The method of claim 9 further comprising detecting a second gesture.

11. The method of claim 10 further comprising adjusting a parameter of the program corresponding to the second gesture.

12. The method of claim 11 further comprising providing a voice prompt corresponding to the gesture.

13. The method of claim 12 further comprising selecting a new state of the computer program based on the recognition result.

14. The method of claim 13 further comprising providing a visual cue corresponding to the new state of the program.

21. The method of claim 1 further comprising initiating a program corresponding to the gesture,

wherein the program includes the function associated with the gesture.

22. The method of claim 1 wherein the gesture is a hand gesture, wherein the step of detecting the gesture includes detecting a portion of a human hand.

23. The method of claim 1 wherein the gesture includes motion.

24. The method of claim 1 wherein the step of detecting includes the use of infrared light emissions.

25. The method of claim 1 wherein the gesture is from a set of gestures corresponding to a plurality of functions,

wherein each function corresponds to at least one gesture in the set of gestures.

15. A speech recognition system comprising:

a controller;

a gesture detector coupled to provide signals to the controller, the signals corresponding to a gesture;

a speech recognizer having access to a plurality recognition sets, the speech recognizer coupled to provide recognition results to the controller;

an audio user interface coupled to receive a speech input and coupled to provide audio signals to the speech recognizer, the audio signals corresponding to the speech input; and

a plurality of output devices coupled to the controller,

wherein a recognition set is selected from the plurality of recognition sets, the recognition set corresponding to the gesture,

wherein the controller provides signals to at least one output device of the plurality of output devices, the signals corresponding to the gesture and the recognition results.

16. The system of claim 15 wherein the gesture detector includes a plurality of infrared detectors arranged to provide signals corresponding to the gesture.

17. The system of claim 15 wherein at least a portion of the plurality of output devices are coupled through a wireless network.

18. The system of claim 15 wherein the controller along with the gesture detector recognize the gesture from a set of gestures, wherein each gesture corresponds to a recognition set,

wherein each recognition set corresponds to at least one output device of the plurality of output devices.

19. The system of claim 15 wherein at least one of the output devices of the plurality of output devices is a speech synthesizer that provides a verbal prompt corresponding to the gesture.

20. A speech recognition method comprising:

detecting a gesture of at least one part of a human body;

receiving a speech input signal; and

recognizing the speech input signal in the context of a recognition set, the recognizing resulting in a first recognition result;

weighting the first recognition result according to a function associated with the gesture; and

determining a second recognition result based on the weighting.

26. A system comprising:

an automobile; and

a speech recognition system in a cabin of said automobile, the speech recognition system comprising,

a controller,

a gesture detector coupled to provide signals to the controller, the signals corresponding to a gesture,

a speech recognizer having access to a plurality recognition sets, the speech recognizer coupled to provide recognition results to the controller,

an audio user interface coupled to receive a speech input and coupled to provide audio signals to the speech recognizer, the audio signals corresponding to the speech input, and

a plurality of output devices coupled to the controller,

wherein a recognition set is selected from the plurality of recognition sets, the recognition set corresponding to the gesture, and

wherein the controller provides signals to at least one output device of the plurality of output devices corresponding to the gesture and the recognition results.