Methods and devices for enhancing fluency in persons who stutter employing visual speech gestures

• US 7,031,922 B1
• Filed: 11/20/2000
• Issued: 04/18/2006
• Est. Priority Date: 11/20/2000
• Status: Expired due to Fees

• Current Assignee: East Carolina University (University of North Carolina System)
• Original Assignee: East Carolina University (University of North Carolina System)
• Inventors: Stuart, Andrew, Kalinowski, Joseph, Rastatter, Michael
• Primary Examiner(s): Chawan, Vijay B.
• Application Number:
  US09/718,222
• Time in Patent Office: 1,975 Days
• Field of Search: 704270-278, 704/235, 434/185, 434/178, 434/156, 434/169, 600/23, 600/559, 600372-378, 623/10, 514/546, 607/57
• US Class Current: 704/271
• CPC Class Codes: A61F 5/58 Apparatus for correcting stammering or stuttering

Abstract

Methods, devices, and computer program products display visual choral speech to a patient who stutters or has a speech impediment or impairment as a visual stimulus for the patient to enhance the fluency of the patient. The visual choral speech is incongruous with the speech produced by the stutterer and is provided by a visual display of the articulatory movements of a person other than the patient (or a simulated representation thereof) of the person'"'"'s lips and mouth as the person speaks. The visual speech gestures can be displayed to the patient in advance of a speaking event or speech production by the stutterer and/or concurrently with a speaking event (either intermittently or continuous during the speaking event). The visual choral speech gestures can be based on a string of coherent words to provide the visual speech gestures signal such that it is relayed to the user without the attendant auditory component allowing the user to speak at a substantially normal pace with enhanced fluency.

Claims

1. A method for enhancing the fluency of persons who stutter, comprising steps of:

displaying visual speech gestures associated with the articulatory movements of a person'"'"'s mouth on a display while a patient having a stuttering or speech impediment is speaking so that the patient is able to visually perceive the articulatory movements of the person'"'"'s mouth provided on the display such that the patient is able to refer to the display at desired times to thereby enhance the fluency of the speech of the patient.

2. A method according to claim 1, wherein said displaying step is carried out while the patient speaks at a substantially normal pace.

3. A method according to claim 2, wherein said displaying step is carried out substantially continuously during the speech of the patient.

4. A method according to claim 1, wherein said displaying step is carried out in advance of and temporally proximate to when the patient speaks.

5. A method according to claim 1, wherein said displaying step is performed such that any attendant auditory sound associated with the visual speech gestures of said displaying step is inaudible to the patient.

6. A method according to claim 1, wherein said displaying step is silent.

7. A method according to claim 1, wherein the visual speech gestures of said displaying step are generated by a person silently mouthing word passages of text with the displaying carried out substantially without attendant audible output of the words.

8. A method according to claim 1, wherein the linguistic content of the visual speech gestures is incongruous with the content of the speech output by the patient.

9. A method according to claim 1, wherein the visual speech gestures of said displaying step are performed at a substantially normal speech pace without displaying associated textual words and symbols.

10. A method according to claim 1, wherein the visual speech gestures of said displaying step correspond to articulatory movements of a person generating coherent speech, and wherein the displaying step is carried out without prominently displaying textual words and symbols associated with the coherent speech.

11. A method according to claim 1, wherein the visual speech gestures of said displaying step correspond to articulatory movements of a person generating incoherent speech.

12. A method according to claim 1, wherein said displaying step is directed directly into the retina of at least one eye of the patient.

13. A method according to claim 1, further comprising the steps of accepting user input to adjust the visual presentation of the displayed output of the visual speech gestures of said displaying step;

and adjusting the visual presentation responsive to the user input.

14. A method according to claim 13, wherein the step of adjusting comprises zooming to enlarge the mouth region of the person making the visual gestures.

15. A method according to claim 13, wherein said displaying step is carried out in response to the onset of a stuttering event.

16. A method according to Claim 1, wherein said visual speech gestures of said displaying step are generated by at least one person.

17. A method according to claim 16, wherein the visual speech gestures are generated by a plurality of different persons speaking silently or with an inaudible speech output and generally concurrently.

18. A method according to claim 16, wherein the visual speech gestures are generated by a plurality of persons with different coherent speech output the persons being serially displayed to generate the visual speech gestures without audible attendant word sounds.

19. A method according to claim 18, wherein the visual speech gestures of the different persons of said displaying step are selectable by the patient during the displaying operation.

20. A method according to claim 1, wherein said visual speech gestures of said displaying step are simulated representations of at least a person'"'"'s mouth and lips generating articulatory movements.

21. A method according to claim 1, wherein the display is carried on the frames of eyeglasses.

22. A method according to claim 1, wherein the display is operably associated with a telephone.

23. A method according to claim 22, wherein the telephone is wireless.

24. A method according to claim 1, wherein the visual speech gestures correspond to the articulatory movements of at least one person speaking textual matter recognizable to the patient in the language of the patient, and wherein the recognizable textual matter is incongruous with the speech content of the patient.

25. A method according to claim 24, wherein the visual speech gestures of the at least one person comprises articulatory movements of visual speech gestures associated with words corresponding to one or more of reciting nursery rhymes, poems, the lyrics of songs, speeches, national pledges, biblical passages, passages of books, and prayers, and wherein the displaying is carried out so that auditory output related to the words is generally inaudible to the user.

26. A method according to claim 25, wherein said visual speech gestures are displayed such that any auditory sound associated therewith is suppressed, wherein the visual speech gestures are incongruous with the speech of the patient, and wherein the visual speech gestures are generated based on normal paced fluent speech.

27. A method according to claim 1, wherein said displaying step is carried out such that the prominent image in the display is the person generating the visual speech gestures so that the articulatory movements are readily discernable by the patient.

28. A method according to claim 1, further comprising the step of generating an auditory stimulus configured to enhance speaking fluency of the patient which is audible to the patient, the auditory stimulus being unrelated to the visual speech gestures of said displaying step.

29. A method according to claim 28, wherein said generating step is separately selectable responsive to user input to initiate auditory output of the auditory stimulus which is audible to the patient based on said generating step.

30. A method according to claim 29, wherein the auditory stimulus is an independent and incongruent exogenously generated auditory spoken speech stimulus signal in the form of an entrained vowel or consonant.

31. A method according to claim 1, wherein said displaying step is carried out by the display which is integrated into one of a portable hand-held device, a general purpose computer, a wireless communication device, a watch, a head mounted display, and a telephone.

32. A method according to claim 1, wherein the displaying visual speech gestures associated with the articulatory movements of a person'"'"'s mouth on a display is carried out to be devoid of text and word symbols in proximity to the mouth making the articulatory movements.

33. A device to enhance the fluency of persons who stutter, comprising:

a display device configured to display at least one visual speech gesture stimulus associated with the articulatory movements of a person reciting coherent language which is incongruous with the speech production of a user.

34. A device according to claim 33, wherein the device further comprises a display controller that is configured to repeatedly output the visual speech gesture stimulus to a user at desired times upon activation by the user corresponding to at least one of an episodic stuttering event on the part of the user, in advance of the production of speech by the user, and during the production of speech by the user.

35. A device according to claim 33, wherein said display device comprises a first operative standby mode and a second active display mode, and wherein said display device includes a user input circuit operably associated with the display device to cause the display device to enter the active display mode and display the visual speech gestures substantially immediate of a user activating said user input circuit.

36. A device according to claim 33, wherein said display device is configured to suppress attendant auditory sound associated with the visual speech gestures such that said visual speech gestures are displayed on said display device devoid of an audible attendant auditory output.

37. A device according to claim 33, wherein said visual speech gestures are substantially silently displayed on said display device to the user.

38. A device according to claim 33, wherein said device further comprises a speaker configured to output an auditory stimulus which is unrelated to the visual speech gesture stimulus, wherein the auditory stimulus is configured to be selectively activated by the user to be output separately or concurrently with the visual speech gesture stimulus.

39. A device according to claim 38, wherein the auditory stimulus is a natural spoken speech signal comprising a sustained speech sound, and wherein the natural spoken speech signal and the visual speech gestures are incongruous with the content of the user'"'"'s speech.

40. A device according to claim 38, wherein said device is configured with a user activation circuit which allows the device to output in response to activation thereof, the auditory stimulus speech signal.

41. A device according to claim 33, wherein said device further comprises a user input zoom adjustment whereby the user can adjust the presentation of the visual speech gestures on the display in a manner which allows the user to readily discern the articulatory movements of the visual speech gesture stimulus while speaking at a substantially normal speech pace.

42. A device according to claim 33, wherein the device is configured to present on the display, in serialorder, a plurality of different persons providing a plurality of different visual speech gestures, each of the different visual speech gestures corresponding to coherent speech spoken at a substantially normal pace with the display device configured to display the visual speech gestures of the coherent speech without attendant audible output of words corresponding to the coherent speech.

43. A device according to claim 33, wherein said device is configured such that it is portable, and the display device is configured and sized to be removeably mountable to the head of the patient, and wherein, in use, the display device is sized and configured to reside proximate at least one eye of the user such that said visual speech gestures are visually relayed to the retina of the user.

44. A device according to claim 33, wherein the display device is incorporated into the body of a residence telephone or a wireless portable telephone.

45. A device according to claim 33, wherein the display device is incorporated into one of a portable device, a handheld device, a general purpose computer, a personal digital assistant, a wireless communication device, a watch, and a telephone.

46. A device according to claim 33, wherein said device is configured to be worn on the head such that the display device is positioned proximate one or both of the eyes and does not impede the entire field of vision of the user.

47. A device according to claim 33, wherein the display device is configured with a frame which is adapted to span across the face of the user and be supported by the nose and ears of the user such that the display device is configured as with two displays, each extending downwardly from the frame to overlie a portion of the eyes of the user.

48. A device according to claim 33, wherein said display device is configured to display the visual speech gesture stimulus such that the person making the articulatory movements is the prominent image in the display so that the articulatory movements are substantially continuously presented at the desired times in a manner in which the articulatory movements are readily discernable by the user during speech, wherein the display device is configured to provide the image of the articulatory movements without prominently displaying textual words and symbols associated with the coherent speech.

49. A device according to claim 48, wherein the visual speech gestures are presented on the display so that a mouth region of a person mouthing the coherent speech to provide the articulatory movements substantially fills the viewing area of the display.

50. A device according to claim 49, wherein the display device is configured to display the visual speech gesture stimulus so that any peripheral images surrounding the person generating the articulatory movements are arranged to inhibit distraction from the articulatory movements to reduce the likelihood of visual distraction from background images in the display.

51. A device according to claim 33, wherein the display is configured to generate the at least one visual speech gesture stimulus associated with the articulatory movements of a person reciting coherent language which is incongruous with the speech production of a user so that the display does not provide corresponding textual word or symbolic output.

52. A computer program product for enhancing the fluency of persons who stutter, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied in said medium, said computer-readable program code comprising;

computer readable program code for displaying at least one visual speech gesture stimulus associated with the articulatory movements of a person'"'"'s mouth on a display while a patient having a stuttering or speech impediment is speaking so that the patient is able to visually perceive the articulatory movements of the person'"'"'s mouth provided on the display such that the patient is able to refer to the display at desired times, wherein the at least one visual speech gesture stimulus is provided by a person other than the patient.

53. A computer program product according to claim 52, wherein the at least one visual speech gesture stimulus is a plurality of different visual speech gesture stimuli, the different visual speech gestures stimuli corresponding to at least one of:

(a) different persons generating the visual speech gestures associated with words of coherent speech; and

(b) different articulatory movements on the part of a person providing the visual speech gestures associated with words of different coherent speech, and wherein said computer program product further comprises computer readable program code for visually displaying in serial order the different visual speech gesture stimuli.

54. A computer program product according to claim 53, wherein the visual speech gesture stimulus corresponds to coherent textual passages which are recognizable by the patient and incongruous wit the speech of the patient, and wherein said computer program product further comprises computer readable program code for accepting input from the patient to select the visual speech gesture stimuli to be displayed.

55. A computer program product according to claim 52, wherein the visual speech gesture stimuli corresponds to coherent language which is incongruous with the speech of the patient, and wherein the linguistic content of the visual speech gesture stimuli is recognizable by the patient and is output so that the visual speech gesture stimulus is substantially inaudible to the patient.

56. A computer program product according to claim 52, wherein the at least one visual speech gesture stimulus corresponds to a plurality of different visual speech gesture stimuli, and wherein said computer program product further comprises computer readable program code for code for serially displaying in a patient selectable format, the articulatory movements of at least one person speaking at least one of (a) the lyrics of a song (b) a poem, (c) a prayer (d) a passage from the Bible, (e) a passage from a story (f) a speech (g) and the Pledge of Allegiance to thereby enhance the fluency of a patient who stutters or has a speech impairment.

57. A computer program product according to claim 52, further comprising computer readable program code for accepting user input to adjust the visual presentation of the articulatory movements associated with the visual speech gestures.

58. A computer program product according to claim 57, wherein the computer program code for accepting user input to adjust the visual presentation includes computer program code accepting user input to zoom the image to enlarge the visual presentation of the articulatory movements displayed.

59. A computer program product according to claim 52, wherein the computer program code for displaying the visual speech gestures is configured so that the visual speech gestures of the articulatory movements of the person on the display is the prominent viewable image on the display and is associated with words of coherent speech such that the visual speech gestures are readily discernable to the patient while the patient is speaking and so that the visual speech gestures are substantially continuously displayed white the patient is speaking, the visual speech gestures being displayed generally without attendant audible verbal output.

60. A computer program product according to claim 52, further comprising computer program code for displaying the visual speech gesture images without (a) attendant audible speech sound, and (b) proximate textual or symbolic representation of the coherent speech.

61. A computer program product according to claim 52, further comprising computer program code for repeatedly displaying the visual speech gesture stimulus to the patient at desired times corresponding to one of an episodic stuttering event on the part of the patient, in advance of the production of speech on the part of the patient, and while the patient is speaking.

62. A computer program product according to claim 61, further comprising computer code for generating and providing an independent auditory stimulus that is configured to enhance speech fluency that is transmitted to the patient which is unrelated to the visual speech gesture stimulus.

63. A computer program product according to claim 62, further comprising computer program code for allowing the auditory stimulus to be selectively activated by the patient separately or concurrently with the display of the visual speech gesture stimulus.

64. A computer program product according to claim 63, wherein the auditory stimulus is a natural spoken speech signal which is incongruous with the speech of the patient.

65. A computer program product according to claim 52, wherein the computer readable program code for displaying the at least one visual speech gesture stimulus is configured to display articulatory movement of a mouth generating coherent words substantially without associated audible verbal utterances of the word and without textual word output or symbols of the coherent language proximate the mouth.

66. A portable device for enhancing fluency in stutterers, the device configured to display visual speech gestures of at least one person making articulatory movements by mouthing words associated with coherent speech, and wherein, in operation, the device is configured to provide the visual speech gestures with associated word utterances being generally inaudible to a user so that the user is able to visually perceive the articulatory movements of the person'"'"'s mouth without attendant word sounds to thereby enhance the fluency of the speech of the patient.

67. A device according to claim 66, wherein the coherent speech is incongruent with the speech of the user and, wherein the coherent speech is associated with one or more of the person on the display reciting nursery rhymes, poems, the lyrics of songs, speeches, national pledges, biblical passages, passages of books, and prayers.

68. A device according to claim 66, wherein the coherent language is incongruous with the speech production of the user, and wherein the device does not display textual word or symbolic output associated with words of the coherent speech.

69. A device according to claim 66, wherein the device comprises a circuit configured to transmit an auditory stimulus that is that is configured to enhance fluency separately from the visual speech gesture stimulus and has auditory content that is unrelated to the visual speech gesture stimulus, the auditory stimulus being output so that it is audible to the user.

70. A device according to claim 66, further comprising user input configured to allow the user to selectively display the visual speech gesture at and/or for a desired time.