Speech recognition

• US 20040243416A1
• Filed: 06/02/2003
• Published: 12/02/2004
• Est. Priority Date: 06/02/2003
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Gardos, Thomas R.
• Application Number:
  US10/453,447
• Publication Number:
  US 20040243416A1
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

An apparatus that includes an image capture device and a support. The image capture device captures images of a user'"'"'s lips, and the support holds the image capture device in a position substantially constant relative to the user'"'"'s lips as the user'"'"'s head moves.

Claims

1. An apparatus comprising:

an image capture device to capture images of a speech articulation portion of a user; and

a support to hold the image capture device in a position substantially constant relative to the speech articulation portion as a head of the user moves.

2. The apparatus of claim 1 in which the speech articulation portion comprises upper and lower lips of the user.

3. The apparatus of claim 1 in which the speech articulation portion comprises a tongue of the user.

4. The apparatus of claim 1 in which the image capture device is configured to capture images of the speech articulation portion from a distance that remains substantially constant as the user'"'"'s head moves.

5. The apparatus of claim 4 in which the field of view of the image capture device is confined to upper and lower lips of the user.

6. The apparatus of claim 1 further comprising an audio sensor to sense a voice of the user.

7. The apparatus of claim 6 in which the audio sensor is mounted on the support.

8. The apparatus of claim 1 in which the support comprises a headset.

9. The apparatus of claim 1 further comprising a data processor to recognize speech based on images captured by the image capture device.

10. The apparatus of claim 9 in which the data processor recognizes speech also based on the voice.

11. The apparatus of claim 1 in which the support comprises a mouthpiece to support the image capture device at a position facing lips of the user.

12. The apparatus of claim 1 in which the image capture device comprises a camera.

13. The apparatus of claim 12 in which the image capture device comprises a lens facing lips of the user.

14. The apparatus of claim 12 in which the image capture device comprises a light guide to transmit an image of lips of the user to the camera.

15. The apparatus of claim 12 in which the image capture device comprises a mirror facing lips of the user.

16. The apparatus of claim 1 further comprising a display to show animated lips based on images of the speech articulation portion captured by the image capture device.

17. The apparatus of claim 1 further comprising a motion sensor to detect motions of the user'"'"'s head.

18. The apparatus of claim 17 further comprising a data processor to generate images of animated lips, the data processor controlling the orientation of the animated lips based in part on signals generated by the motion sensor.

19. The apparatus of claim 18 in which the data processor also controls an orientation of an animated talking head that contains the animated lips based in part on signals generated by the motion sensor.

20. The apparatus of claim 1 further comprising an orientation sensor to detect orientations of the user'"'"'s head.

21. The apparatus of claim 1 in which the image capture device captures images of at least a portion of an eyebrow or an eye of the user.

22. The apparatus of claim 21 further comprising a data processor to recognize speech based on images captured by the image capture device.

23. An apparatus comprising:

a motion sensor to detect a movement of a user'"'"'s head;

a headset to support the motion sensor at a position substantially constant relative to the user'"'"'s head; and

a data processor to generate a signal indicating a type of movement of the user'"'"'s head based on signals from the motion sensor, the type of movement being selected from a set of pre-defined types of movements.

24. The apparatus of claim 23 in which at least one of the pre-defined types of movements include tilting.

25. The apparatus of claim 24 in which the pre-defined types of movements include tilting left, tilting right, tilting forward, tilting backward, head nod, or head shake.

26. The apparatus of claim 23 in which the signal indicating the type of movement also indicates an amount of movement.

27. The apparatus of claim 26, further comprising a data processor configured to recognize speech based on voice signal and signals from the motion sensor.

28. An apparatus comprising:

an image capture device to capture images of lips of a user;

a motion sensor to detect a movement of a head of the user and generate a head action signal;

a processor to process the images of the lips and the head action signal to generate lip position parameters and head action parameters;

a headset to support the image capture device and the motion sensor at positions substantially constant relative to the user'"'"'s head as the user'"'"'s head moves; and

a transmitter to transmit the lip position and head action parameters.

29. The apparatus of claim 28 in which the image capture device comprises a mirror positioned in front of the user'"'"'s lips.

30. The apparatus of claim 29 in which the image capture device comprises a camera placed in front of the user'"'"'s lips.

31. A method comprising:

recognizing speech of a user based on images of lips of the user obtained by a camera positioned at a location that remains substantially constant relative to the user'"'"'s lips as a head of the user moves.

32. The method of claim 31 further comprising measuring a distance between an upper lip and a lower lip of the user.

33. The method of claim 31 further comprising generating time-stamped lip position parameters from images of the user'"'"'s lips.

34. The method of claim 31 further comprising recognizing speech of the user based on images of at least a portion of the user'"'"'s eye or eyebrow.

35. The method of claim 31 further comprising controlling a process for recognizing speech based on images of at least a portion of the user'"'"'s eye or eyebrow.

36. A method comprising at least one of recognizing speech of a user and controlling a machine based on information derived from movements of a head of the user sensed by a motion sensor attached to the user'"'"'s head.

37. The method of claim 36 further comprising confirming accuracy of speech recognition based on information derived from movements of the user'"'"'s head sensed by the motion sensor.

38. The method of claim 36 further comprising selecting between different modes of speech recognition based on different head movements sensed by the motion sensor.

39. A method comprising:

obtaining successive images of a speech articulation portion of a face of a user from a position that is substantially constant relative to the user'"'"'s face as a head of the user moves.

40. The method of claim 39 further comprising detecting a voice of the user.

41. The method of claim 40 further comprising recognizing speech based on the voice and the images of the speech articulation portion.

42. A method comprising:

measuring movement of a user'"'"'s head to generate a head motion signal;

detecting a voice of the user; and

recognizing speech based on the voice and the head motion signal.

43. The method of claim 42, further comprising processing the head motion signal to generate a head motion type signal.

44. The method of claim 42, further comprising selecting a head motion type from a set of pre-defined head motion types based on the head motion signal, the pre-defined head motion types including at least one of tilting left, tilting right, tilting forward, tilting backward, head nod, and head shake.

45. The method of claim 42 further comprising using recognized speech to control actions of a computer game.

46. The method of claim 42 further comprising generating an animated head within a computer game based on the head motion signal.

47. A method comprising:

generating an animated talking head to represent a speaker; and

adjusting an orientation of the animated talking head based on a head motion signal generated from a motion sensor that senses movements of a head of the speaker.

48. The method of claim 47 further comprising receiving the head motion signal from a network.

49. The method of claim 47 further comprising generating animated lips based images of lips of the speaker captured from a position that is substantially constant relative to the lips as the speaker'"'"'s head moves.

50. A method comprising:

confirming accuracy of recognition of a speech of a user based on a head action parameter derived from measurements of movements of a head of the user.

51. The method of claim 50 in which the head action parameter comprises a head-nod parameter.

52. The method of claim 50 further comprising measuring movements of the user'"'"'s head using a motion sensor attached to the user'"'"'s head.

53. A machine-accessible medium, which when accessed results in a machine performing operations comprising:

recognizing speech of a user based on images of lips of the user obtained by a camera positioned at a location that remains substantially constant relative to the user'"'"'s lips as a head of the user moves.

54. The machine-accessible medium of claim 53, which when accessed further results in the machine performing operations comprising measuring a distance between an upper lip and a lower lip of the user.

55. The machine-accessible medium of claim 53, which when accessed further results in the machine performing operations comprising generating time-stamped lip position parameters from images of the user'"'"'s lips.

56. A machine-accessible medium, which when accessed results in a machine performing operations comprising:

measuring movement of a head of a user to generate a head motion signal;

detecting a voice of the user; and

recognizing speech based on the voice and the head motion signal.

57. The machine-accessible medium of claim 56, which when accessed further results in the machine performing operations comprising generating an animated head within a computer game based on the head motion signal.

58. The machine-accessible medium of claim 56, which when accessed further results in the machine performing operations comprising using recognized speech to control actions of a computer game.

59. A machine-accessible medium, which when accessed results in a machine performing operations comprising:

generating an animated talking head to represent a speaker; and

adjusting an orientation of the animated talking head based on a head motion signal generated from a motion sensor that senses movements of a head of the speaker.

60. The machine-accessible medium of claim 59, which when accessed further results in the machine performing operations comprising receiving the head motion signal from a network.

61. The machine-accessible medium of claim 59, which when accessed further results in the machine performing operations comprising generating animated lips based images of lips of the speaker captured from a position that is substantially constant relative to the lips as the speaker'"'"'s head moves.