Head-worn, trimodal device to increase transcription accuracy in a voice recognition system and to process unvocalized speech

• US 20020194005A1
• Filed: 03/27/2002
• Published: 12/19/2002
• Est. Priority Date: 03/27/2001
• Status: Active Grant

• Current Assignee: RAST Associates LLC
• Original Assignee: RAST Associates LLC
• Inventors: Lahr, Roy J.
• Granted Patent: US 7,082,393 B2
• Time in Patent Office: Days
• US Class Current: 704/271
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features

Abstract

A voice recognition device and method allows position-stabilized capture of spoken sounds with great repeatability and accuracy. The voice recognition device may additionally provide two channels of lip movement information to supplement the usual audible speech component recognition system in selecting the proper pairing of data input to text output. The voice recognition device may provide a further channel of information about the speech generating motions via an ultrasonic injection of sound into the vocal cavity and subsequent decoding of the emitted sound after injection. The ultrasonic injection and decoding may also used to provide audible clues as to the unvoiced sound formed by speaking when the vocal cords are not energized. The ensemble of electronic equipment upon the bail band may be in microcircuit form, including placing the components on a copper layer polyimide flexible strip. The side camera and “other side” illuminator LED may be on thin copper polyimide strips attached to the main electronics ensemble, and a set of thin polyimide conductors would conduct power into the ensemble and the signals out of the ensemble through one of the bail band ends, into the ear piece and down the connector to the associated computer equipment and may also supply the power for the electronic ensemble. The electronic ensemble may be potted with a thin layer of elastomer, such as translucent silicone, and provide a moisture barrier and physical protection for the ensemble, while still offering a very light visual weight to the combination of the electronic ensemble and the bail band.

Claims

1. An head-worn apparatus comprising:

a bail band configured to pivot, the bail band being wearable on a user'"'"'s head using the user'"'"'s ears as reference points for pivoting,

wherein at least a first pivot point on the bail band is situated opposite a second pivot point on the bail band so that stable measurements made of at least one of sound data and spoken data are recognizable by a device.

2. The head-worn apparatus of claim 1, wherein the first and second pivot points include ear surround pads.

3. The head-worn apparatus of claim 1, wherein the first and second pivot points are manipulatable by the user so that the bail band can be situated directly in front of lips of the user to get a best record of spoken word emitted from the lips.

4. The head-worn apparatus of claim 3, wherein the first and second pivot points are configured to be used to pivot the bail band repeatedly near at least one of the lips of the user, a forehead region of the user, a chin of the user, and a physical location away from the lips of the user, so that the bail band is configured to be movable away from the lips and back near the lips in a previous position.

5. The head-worn apparatus of claim 4, wherein the bail band has a length and a shape to allow the bail band to be pivoted near a user head region so that sufficient clearance is available between the bail band and the user head region.

6. The head-worn apparatus of claim 5, wherein the user head region is at least one of the lips of the user, the forehead region of the user, the chin of the user, an eyewear worn on the head of the user, a headwear worn adjacent to an eye of the user, and a computer vision accessory worn on the head of the user.

7. The head-worn apparatus of claim 5, wherein the bail band includes a first end situated near the first pivot point and a second end situated near the second pivot point, the bail band being pivoted near the user head region by adjustment of the first and second ends of the bail band.

8. The head-worn apparatus of claim 3, wherein at least one of the first pivot point and the second pivot point is adjustable to provide and fix to any desired distance between the lips of the user and the bail band.

9. The head-worn apparatus of claim 8, further comprising a reference bar having a predetermined length is pivotable from at least one of the first end and the second end of the bail band, the reference bar being positionable directly in front of the lips of the user.

10. The head-worn apparatus of claim 9, wherein the reference bar includes a first reference bar end and a second reference bar end, where at least one of the first reference bar end and the second reference bar end is at least one of a rounded surface, a smoothed surface, and a spherical-shape end.

11. The head-worn apparatus of claim 10, wherein the reference bar is pivotable at an orthogonal angle and at least one additional angle in reference to a frontal region of the bail band, a maximum distance between the reference bar and the lips of the user being achieved when the reference bar is pivoted at the orthogonal angle.

12. The head-worn apparatus of claim 11, further comprising a swinging bar being pivotable from a region near a center of the reference bar, the center of the reference bar being situated between the first reference bar end and the second reference bar end.

13. The head-worn apparatus of claim 11, wherein the bail band is constructed of at least one of thin material and translucent material so as to at least one of present minimum visual apparent size when the reference bar is in use and position the reference bar essentially parallel to the frontal region of the bail band when the reference bar is being stored.

14. The head-worn apparatus of claim 1, further comprising a microphone disposed in a central region on the bail band and being pointed toward the lips of the user.

15. The head-worn apparatus of claim 14, wherein the bail band is configured augmented in which any sound pickup from a direction away from the lips effectively provides cancellation of adjacent noise by at least one of mechanical subtraction and electronic subtraction.

16. The head-worn apparatus of claim 1, further comprising a plurality of individual microphones disposed in a central region on the bail band and being pointed toward the lips of the user, each individual microphone of the plurality of microphones having respective phase-adjusted output signals to provide an accurate pickup of spoken word from a region near the lips of the user.

17. The head-worn apparatus of claim 14, wherein the microphone includes a porous cover of at least one of foam and wire so that the microphone is protected from any undesired projection from the lips.

18. The head-worn apparatus of claim 16, wherein the individual microphones of the plurality of individual microphones each include a porous cover of at least one of foam and wire so that the plurality of individual microphones is protected from any undesired projection from the lips.

19. The head-worn apparatus of claim 1, wherein the bail band is formed to utilize at least one specific regional shape of a head of the user as at least one reference point from which to mount at least one of the first pivot point, the second pivot point and another pivot point, so that the bail band is configured to be repeatedly placed on the head in a manner allowing a precise positioning in reference to the lips of the user.

20. The head-worn apparatus of claim 14, further comprising a first camera, the first camera being disposed on a nearly central location on a lip side of the bail band, the lip side of the bail band being that part of the bail band closest to the lips of the user, wherein the first camera is positioned towards the lips so as to provide a frontal lip camera view.

21. The head-worn apparatus of claim 20, further comprising a light source disposed adjacent to the first camera on an outer surface of the bail band, so that a central beam of the light source illuminates a surface of the lips of the user.

22. The head-worn apparatus of claim 21, wherein the light source is configured to provide a variable intensity output.

23. The head-worn apparatus of claim 21, further comprising a system for recording settings of the light source and of any video gain of the first camera for the user so that the system adjusts the light source and the video gain of the first camera when the user again uses the head-worn apparatus.

24. The head-worn apparatus of claim 20, wherein the first camera includes a lens having circular area with an about 1.0 inch to about 1.5 inch diameter.

25. The head-worn apparatus of claim 20, further comprising a second camera, the second camera being disposed on the bail band and being adjacent to the first camera, the second camera viewing at least one of a wide angle view of the lips and an about rectangular area having an about 1.25 inch height and about 3.50 width so that the lips are central in the about rectangular area.

26. The head-worn apparatus of claim 20, wherein an output of the first camera provides visual image of the lips moving, the first camera output being displayable at user option on an associated device for review.

27. The head-worn apparatus of claim 26, wherein the visual image of the lips is recordable as at least one of a video data stream and a time-associated recording of any spoken sounds.

28. The head-worn apparatus of claim 25, wherein an output of the second camera provides visual image of the lips moving, the second camera output being displayable at user option on an associated device for review.

29. The head-worn apparatus of claim 28, wherein the visual image of the lips is recordable as at least one of a video data stream and a time-associated recording of any spoken sounds.

30. The head-worn apparatus of claim 25, further comprising a third camera, the third camera being disposed on a facial side of the bail band and being positioned lateral to the lips of the user.

31. The head-worn apparatus of claim 25, further comprising an illumination source disposed on an opposing side to the second camera on the bail band, so that illumination from the illumination source falls upon an interior surface of the bail band to provide a backdrop illumination for the second camera.

32. The head-worn apparatus of claim 25, further comprising an IR emitting LED to provide backdrop illumination for the second camera so as to give at least one of high contrast image of front-back and of vertical movement of the lips during speaking and “

lip reading”

data input to a voice recognition system.

33. The head-worn apparatus of claim 31, further comprising a system for adjusting an illumination level provided by the illumination source to provide an optimum silhouette view of movement of the lips during speaking.

34. The head-worn apparatus of claim 31, further comprising a system for recording settings of the illumination source and of any video gain of the second camera for the user so that the system adjusts the illumination source and the video gain of the second camera when the user again uses the head-worn apparatus.

35. The head-worn apparatus of claim 29, wherein an output of the third camera provides visual image of the lips moving, the third camera output being displayable at user option on an associated device for review.

36. The head-worn apparatus of claim 28, wherein the visual image of the lips is recordable as at least one of a video data stream and a time-associated recording of any spoken sounds.

37. The head-worn apparatus of claim 30, further comprising a system for recording the optimum amplification settings for the microphone so that settings for the user are maintained.

38. The head-worn apparatus of claim 14, further comprising a system for setting amplification settings so that the microphone provides optimum input levels for the user.

39. The head-worn apparatus of claim 1, further comprising a high frequency sound emitting source disposed in a central location on the bail band, so that any emitted sound from the high frequency sound emitting source is directed towards the lips.

40. The head-worn apparatus of claim 37, wherein the high frequency sound emitting source emits within a range of about 38 kHz frequency to about 100 kHz.

41. The head-worn apparatus of claim 37, wherein the emitted sound is of nearly constant ultrasonic frequency.

42. The head-worn apparatus of claim 37, wherein the emitted sound is configured to be automatically altered so as to vary continuously between lower and upper ultrasonic limit frequencies.

43. The head-worn apparatus of claim 42, wherein the emitted sound is within a range of about 38 kHz to about 44 kHz.

44. The head-worn apparatus of claim 1, wherein a separate receiving microphone for ultrasonic frequencies is provided on a central location on the bail band, on the facial side, to convert ultrasound emitting from a vocal tract of the user into electrical signals.

45. The head-worn apparatus of claim 35, further comprising an emitter, the emitter being located on the bail band, wherein ultrasound generation is conveyed to the emitted and a reflected signal from a vocal tract of the user is recovered by the microphone.

46. The head-worn apparatus of claim 42, wherein a downconverted signal is used as input to a voice recognition system for system training on a provided text, then the downconverted signal is used as input in actual voice recognition operation, so as to provide a text transcription of non-vocalized speech.

47. The head-worn apparatus of claim 46, wherein a downconverted signal is used as input to a voice recognition system on a provided text, so as to provide a data transcription of non-vocalized speech so that a text-to-synthetic speech output can be provided.

48. The head-worn apparatus of claim 46, wherein the downconverted signal is used as a training input for a voice recognition system, aided by a processed video signal from the first camera.

51. A method for training a voice recognition system using the apparatus of claim 30.

49. The head-worn apparatus of 46, wherein the downconverted signal is used as a training input for a voice recognition system, aided by a processed video signal from the second camera.

50. The head-worn apparatus of 46, wherein the downconverted signal is used as a training input for a voice recognition system, aided by processed video signal from the third camera.