System For Text Assisted Telephony

• US 20120250837A1
• Filed: 06/01/2012
• Published: 10/04/2012
• Est. Priority Date: 09/08/1997
• Status: Abandoned Application

• Current Assignee: Kevin R. Colwell, Robert M. Engleke
• Original Assignee: Kevin R. Colwell, Robert M. Engleke
• Inventors: Engleke, Robert M., Colwell, Kevin R.
• Application Number:
  US13/486,221
• Publication Number:
  US 20120250837A1
• Time in Patent Office: Days
• US Class Current: 379/52
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/21 Combinations with auxiliary equipment, e.g. with clocks or memoranda pads
• CPC Class Codes: H04M 1/2155 Acoustic coupling
• CPC Class Codes: H04M 1/2475 for a hearing impaired user
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 11/066 Telephone sets adapted for data transmision
• CPC Class Codes: H04M 11/08 specially adapted for optional reception of entertainment or informative matter
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/42161 via computer interface
• CPC Class Codes: H04M 3/42382 Text-based messaging services in telephone networks such as PSTN/ISDN, e.g. User-to-User Signalling or Short Message Service for fixed networks
• CPC Class Codes: H04M 3/42391 where the subscribers are hearing-impaired persons, e.g. telephone devices for the deaf

Abstract

A method of and system for operating a captioned telephone service, the method comprising the steps of providing words spoken by a remote user to a relay, at the relay, a call assistant listening to the words spoken by the remote user and re-voicing the words into a computer with voice recognition software trained to the voice of the call assistant to create a text stream of the words spoken by the remote user and presenting the text stream to an assisted user via a display.

Claims

1. A method of operating a relay system to facilitate communication between a deaf person and a hearing person, the method comprising the steps of:

receiving words spoken by the hearing person at a relay;

presenting, by voice, the words spoken by the hearing person to a computer at the relay;

using the computer to translate the words spoken by the hearing person into a text message stream including the spoken words;

presenting the text message stream to a call assistant via a display screen;

receiving corrections to the text message stream via an input device from the call assistant; and

transmitting the corrections to a communication device including a display screen within sight of the deaf person for display to the deaf person.

2. The method of claim 1 further including the step of displaying the text message stream in visually readable text on the display screen within sight of the deaf person.

3. The method of claim 1 wherein the step of presenting, by voice, the words spoken by the hearing person includes presenting the words spoken to the ear of the call assistant and the call assistant revoicing the words spoken into a microphone linked to the computer.

4. The method of claim 3 wherein the computer runs voice recognition software trained to the voice of the call assistant to translate the words spoken by the call assistant into the text message stream.

5. The method of claim 1 wherein the step of receiving words spoken by the hearing person includes receiving words spoken on a communication link linked to the relay and the step of transmitting the corrections to a communication device includes transmitting the corrections on a communication link.

6. The method of claim 5 wherein the step of receiving words spoken by the hearing person includes receiving words spoken on a first communication link linked to the relay and the step of transmitting the corrections to a communication device includes transmitting the corrections on a second communication link wherein the second communication link is different than the first communication link.

7. The method of claim 6 wherein the first communication link is between a device used by the hearing person and the relay and the second link is between the relay and the communication device including the display screen that is within sight of the deaf person.

8. The method of claim 5 wherein the communication links are part of a communication network.

9. The method of claim 8 wherein the communication network is a telecommunication network.

10. The method of claim 1 wherein the step of transmitting the corrections to a communication device including a display screen includes correcting the text message stream using the corrections to create a corrected text message stream and transmitting the corrected text message stream to the communication device including a display screen.

11. The method of claim 1 further including the step of buffering the received words spoken by the hearing person at the relay prior to translating the words spoken into the text message stream.

12. The method of claim 1 further including the step of presenting the words spoken by the hearing person to the ear of the call assistant.

13. The method of claim 1 wherein the step of receiving includes receiving the words spoken by the hearing person over a communication network and wherein the step of transmitting the text message stream includes transmitting the text message stream over a communication network.

14. The method of claim 1 wherein the step of transmitting includes using a modem at the relay to transmit the text message stream on a communication line.

15. The method of claim 1 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display screen within sight of the deaf person.

16. The method of claim 15 wherein the text message stream is a digital text message stream.

17. The method of claim 1 wherein the step of receiving the words spoken by the hearing person includes receiving the words spoken by the hearing person on a communication link between the relay and the communication device including a display screen within sight of the deaf person.

18. The method of claim 17 wherein the step of transmitting the corrections to a communication device includes transmitting the corrections on a communication link between the relay and the communication device including a display screen within sight of the deaf person.

19. The method of claim 18 wherein the step of receiving the words spoken and the step of transmitting the text message stream occur over the same communication link between the relay and the communication device including a display screen within sight of the deaf person.

20. A method of operating a relay system to facilitate communication between a deaf person and a hearing person, the method comprising the steps of:

receiving words spoken by the hearing person at a relay from a communication device used by the deaf person;

presenting the words spoken by the hearing person to a computer at the relay;

using the computer to translate the words spoken by the hearing person into a text message stream including the spoken words; and

transmitting the text message stream to the communication device for display to the deaf person.

21. The method of claim 20 further including the steps of presenting the text message stream to a call assistant via a display screen, receiving corrections to the text message stream via an input device from the call assistant and correcting the text message stream using the corrections to generate a corrected text message stream, the step of transmitting the text message stream including transmitting the corrected text message stream to the communication device including a display screen within sight of the deaf person.

22. The method of claim 20 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display within sight of the deaf person.

23. The method of claim 20 further including the step of displaying the text message stream in visually readable text on the display within sight of the deaf person.

24. The method of claim 20 wherein the step of presenting the words spoken by the hearing person includes presenting the words spoken to the ear of the call assistant and the call assistant revoicing the words spoken into a microphone linked to the computer.

25. The method of claim 24 wherein the computer runs voice recognition software trained to the voice of the call assistant to translate the words spoken by the call assistant into the text message stream.

26. The method of claim 20 wherein the step of receiving words spoken by the hearing person includes receiving words spoken on a communication line linked to the relay and the step of transmitting the text message stream includes transmitting the text message stream on a communication line.

27. The method of claim 26 wherein the step of receiving words spoken by the hearing person includes receiving words spoken on a first communication line linked to the relay and the step of transmitting the text message stream includes transmitting the corrections on the first communication line.

28. The method of claim 20 further including the step of buffering the received words spoken by the hearing person at the relay prior to translating the words spoken into the text message stream.

29. The method of claim 20 further including the step of presenting the words spoken by the hearing person to the ear of a call assistant.

30. The method of claim 29 further including the step of presenting the text message stream to the call assistant via a display screen at the relay.

31. The method of claim 20 wherein the step of transmitting includes using a modem at the relay to transmit the text message stream on a communication line.

32. A method of operating a relay to facilitate communication between a deaf person and a hearing person that are each remotely located from the relay, the method comprising the steps of:

receiving words spoken by the hearing person at the relay via a communication line;

presenting the words spoken by the hearing person to the ear of a call assistant;

presenting to a computer and by voice the words spoken by the hearing person;

using the computer to translate the words presented by voice into a text message stream;

displaying the text message stream on a display within sight of the call assistant; and

transmitting the text message stream on a communication line to a communication device including a display screen within sight of the deaf person.

33. The method of claim 32 wherein the step of receiving the words spoken includes receiving the words spoken on a first communication line and the step of transmitting the text message stream includes transmitting the text message stream on a second communication line where the second communication line is different than the first communication line.

34. The method of claim 32 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display screen within sight of the deaf person.

35. The method of claim 32 wherein the step of presenting to a computer and by voice the words spoken by the hearing person includes the step of the call assistant revoicing the words presented to the ear of the call assistant into a microphone linked to the computer.

36. The method of claim 34 wherein the computer runs voice recognition software trained to the voice of the call assistant to translate the words spoken by the call assistant into the text message stream.

37. The method of claim 32 wherein the step of receiving the words spoken by the hearing person includes receiving the words spoken by the hearing person on a communication link between the relay and the communication device including a display screen within sight of the deaf person.

38. The method of claim 37 wherein the step of transmitting the text message stream includes transmitting the text message stream on a communication link between the relay and the communication device including a display screen within sight of the deaf person.

39. The method of claim 38 wherein the step of receiving the words spoken and the step of transmitting the text message stream occur over the same communication link between the relay and the communication device including a display screen within sight of the deaf person.

40. The method of claim 39 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display screen within sight of the deaf person.

41. A method of operating a relay system to facilitate communication between a deaf person and a hearing person, the method comprising the steps of:

receiving at a relay a text message stream generated by a computer that translates words spoken by a hearing person to generate the text message stream;

presenting the text message stream to a call assistant via a display at the relay;

receiving corrections to the text message stream via an input device from the call assistant; and

transmitting the corrections to a communication device including a display within sight of the deaf person for display to the deaf person.

42. The method of claim 41 further including the steps of receiving the words spoken by the hearing person at the relay and using a computer at the relay to translate the words spoken into the text message stream.

43. The method of claim 42 wherein the step of receiving the words spoken by the hearing person at the relay includes receiving the words spoken by the hearing person on a communication link between the relay and the communication device including a display screen within sight of the deaf person.

44. The method of claim 41 wherein the step of transmitting the corrections includes using a modem at the relay to transmit the corrections to the communication device.

45. The method of claim 44 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display screen within sight of the deaf person.

46. The method of claim 42 further including the steps of presenting the words spoken by the hearing person to the ear of a call assistant and receiving the spoken words revoiced by the call assistant, the computer translating the revoiced words by the call assistant to generate the text message stream.

47. The method of claim 41 further including the step of displaying the corrections using the communication device including the display within sight of the deaf person.

48. The method of claim 41 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display screen within sight of the deaf person.

49. A method for operating a communication device used by a deaf person to facilitate communication with a hearing person, the method comprising the steps of:

receiving words spoken by the hearing person at the communication device used by the deaf person;

transmitting the received words spoken by the hearing person to a relay remote from the communication device used by the deaf person;

receiving a text message stream from the relay at the communication device used by the deaf person; and

displaying the text message stream on a display in sight of the deaf person.

50. The method of claim 49 wherein the step of receiving words spoken by the hearing person includes receiving words in a microphone linked to the communication device used by the deaf person.

51. The method of claim 49 wherein the step of receiving a text message stream includes receiving a digital communication stream from the relay.

52. The method of claim 49 wherein the step of transmitting the words spoken includes transmitting the words spoken to the relay via a communication link and wherein the step of receiving the text message stream includes receiving the text message stream via a communication link.

53. The method of claim 52 wherein the communication link used to transmit the words spoken to the relay includes a wireless communication link.

54. The method of claim 52 wherein the communication link used to transmit the words spoken to the relay includes a cellular communication link.

55. The method of claim 52 wherein the words spoken are transmitted via a first communication link and the text message stream is received over the first communication link.

56. A relay to facilitate communication between a deaf person and a hearing person, the relay comprising:

a visual display;

an interface input device;

a processor programmed to perform the steps of;

receiving words spoken by the hearing person;

presenting, by voice, the words spoken by the hearing person to a processor at the relay;

using the processor to translate the words spoken by the hearing person into a text message stream including the spoken words;

presenting the text message stream to a call assistant via the display;

receiving corrections to the text message stream via the input device from the call assistant; and

transmitting the corrections to a communication device including a display screen within sight of the deaf person for display to the deaf person.

57. The relay of claim 56 wherein the step of transmitting includes using a modem at the relay to transmit the text message stream to the communication device.

58. The relay of claim 56 wherein the step of presenting, by voice, the words spoken by the hearing person includes presenting the words spoken to the ear of the call assistant and the call assistant revoicing the words spoken into a microphone linked to the processor.

59. The relay of claim 58 wherein the processor runs voice recognition software trained to the voice of the call assistant to translate the words spoken by the call assistant into the text message stream.

60. The relay of claim 56 wherein the step of receiving words spoken by the hearing person includes receiving words spoken on a communication link linked to the relay and the step of transmitting the corrections to a communication device includes transmitting the corrections on a communication link.

61. The relay of claim 60 wherein the step of receiving words spoken by the hearing person includes receiving words spoken on a first communication link linked to the relay and the step of transmitting the corrections to a communication device includes transmitting the corrections on a second communication link wherein the second communication link is different than the first communication link.

62. The relay of claim 61 wherein the first communication link is between a device used by the hearing person and the relay and the second link is between the relay and the communication device including the display screen that is within sight of the deaf person.

63. The relay of claim 60 wherein the communication links are part of a communication network.

64. The relay of claim 63 wherein the communication network is a telecommunication network.

65. The relay of claim 56 wherein the step of transmitting the corrections to a communication device including a display screen includes correcting the text message stream using the corrections to create a corrected text message stream and transmitting the corrected text message stream to the communication device.

66. The relay of claim 56 further including the step of buffering the received words spoken by the hearing person at the relay prior to translating the words spoken into the text message stream.

67. The relay of claim 56 further including the step of presenting the words spoken by the hearing person to the ear of the call assistant.

68. The relay of claim 56 wherein the step of receiving includes receiving the words spoken by the hearing person over a communication network and wherein the step of transmitting the text message stream includes transmitting the text message stream over a communication network.

69. The relay of claim 56 wherein the step of transmitting includes using a modem at the relay to transmit the text message stream on a communication line.

70. The relay of claim 56 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display screen within sight of the deaf person.

71. The relay of claim 70 wherein the text message stream is a digital text message stream.

72. The relay of claim 56 wherein the step of receiving the words spoken by the hearing person includes receiving the words spoken by the hearing person on a communication link between the relay and the communication device.

73. The relay of claim 72 wherein the step of transmitting the corrections to a communication device includes transmitting the corrections on a communication link between the relay and the communication device.

74. The relay of claim 73 wherein the step of receiving the words spoken and the step of transmitting the text message stream occur over the same communication link between the relay and the communication device.

75. A relay to facilitate communication between a deaf person and a hearing person, the relay comprising:

a processor programmed to perform the steps of;

receiving words spoken by the hearing person from a communication device used by the deaf person;

using the processor to translate the words spoken by the hearing person into a text message stream including the spoken words; and

transmitting the text message stream to the communication device for display to the deaf person.

76. The relay of claim 75 further including a display and an input device, the processor further programmed to perform the steps of presenting the text message stream to a call assistant via the display, receiving corrections to the text message stream via the input device from the call assistant and correcting the text message stream using the corrections to generate a corrected text message stream, the step of transmitting the text message stream including transmitting the corrected text message stream to the communication device.

77. The relay of claim 75 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device.

78. The relay of claim 75 further including a microphone linked to the processor and a speaker, the processor further programmed to present the words spoken to the ear of the call assistant via the speaker and receive the words spoken by the hearing person revoiced by the call assistant into the microphone.

79. The relay of claim 78 wherein the processor runs voice recognition software trained to the voice of the call assistant to translate the words spoken by the call assistant into the text message stream.

80. The relay of claim 75 wherein the step of receiving words spoken by the hearing person includes receiving words spoken on a communication line linked to the relay and the step of transmitting the text message stream includes transmitting the text message stream on a communication line.

81. The relay of claim 80 wherein the step of receiving words spoken by the hearing person includes receiving words spoken on a first communication line linked to the relay and the step of transmitting the text message stream includes transmitting the corrections on the first communication line.

82. The relay of claim 75 further including a buffer device wherein the processor is further programmed to perform the step of buffering the received words spoken by the hearing person in the buffer device prior to translating the words spoken into the text message stream.

83. The relay of claim 75 further including a speaker wherein the processor is further programmed to perform the step of presenting the words spoken by the hearing person to the ear of a call assistant via the speaker.

84. The relay of claim 83 further including a display linked to the processor, the processor further programmed to perform the step of presenting the text message stream to the call assistant via the display.

85. The relay of claim 75 further including a modem, the step of transmitting including using a modem at the relay to transmit the text message stream on a communication line.

86. A relay to facilitate communication between a deaf person and a hearing person that are each remotely located from the relay, the relay comprising:

a visual display within sight of a call assistant;

a speaker;

a processor programmed to perform the steps of;

presenting the words spoken by the hearing person to the ear of a call assistant via the speaker;

receiving the words spoken by the hearing person in voice;

using a voice recognition program to translate the words spoken by the hearing person into a text message stream;

displaying the text message stream on the display; and

transmitting the text message stream on a communication line to a communication device including a display screen within sight of the deaf person.

87. The relay of claim 86 wherein the step of receiving the words spoken includes receiving the words spoken on a first communication line and the step of transmitting the text message stream includes transmitting the text message stream on a second communication line where the second communication line is different than the first communication line.

88. The relay of claim 86 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display screen within sight of the deaf person.

89. The relay of claim 86 further including a microphone linked to the processor, the step of receiving the words spoken by the hearing person in voice including the step of the call assistant revoicing the words presented to the ear of the call assistant into the microphone.

90. The relay of claim 89 wherein the processor runs voice recognition software trained to the voice of the call assistant to translate the words spoken by the call assistant into the text message stream.

91. The relay of claim 86 wherein the step of receiving the words spoken by the hearing person includes receiving the words spoken by the hearing person on a communication link between the relay and the communication device.

92. The method of claim 91 wherein the step of transmitting the text message stream includes transmitting the text message stream on a communication link between the relay and the communication device.

93. The method of claim 92 wherein the step of receiving the words spoken and the step of transmitting the text message stream occur over the same communication link between the relay and the communication device including a display screen within sight of the deaf person.

94. The method of claim 93 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display screen within sight of the deaf person.

95. A relay to facilitate communication between a deaf person and a hearing person, the relay comprising:

a visual display;

an input device;

a processor programmed to perform the steps of;

receiving a text message stream generated by a computer that translates words spoken by a hearing person to generate the text message stream;

presenting the text message stream to a call assistant via the display;

receiving corrections to the text message stream via the input device from the call assistant; and

transmitting the corrections to a communication device including a display within sight of the deaf person.

96. The relay of claim 95 wherein the processor includes the computer and is further programmed to perform the steps of receiving the words spoken by the hearing person and translating the words spoken into the text message stream.

97. The relay of claim 96 wherein the step of receiving the words spoken by the hearing person at the relay includes receiving the words spoken by the hearing person on a communication link between the relay and the communication device.

98. The relay of claim 95 further including a modem, the step of transmitting the corrections including using the modem to transmit the corrections to the communication device.

99. The relay of claim 98 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device.

100. The relay of claim 96 further including a speaker and a microphone linked to the processor, the processor further programmed to perform the steps of presenting the words spoken by the hearing person to the ear of a call assistant via the speaker and receiving the spoken words revoiced by the call assistant into the microphone, the processor translating the revoiced words by the call assistant to generate the text message stream.

101. The method of claim 95 wherein the step of transmitting includes converting the text message stream into a digital communication stream and transmitting the digital communication stream to the communication device including the display screen within sight of the deaf person.

102. A communication device used by a deaf person to facilitate communication with a hearing person, the communication device comprising:

a visual display within sight of the deaf person;

a communication device processor programmed to perform the steps of;

receiving words spoken by the hearing person;

transmitting the received words spoken by the hearing person to a relay remote from the communication device used by the deaf person;

receiving a text message stream from the relay; and

displaying the text message stream on the display.

103. The communication device of claim 102 further including a microphone, the step of receiving words spoken by the hearing person including receiving words in the microphone linked to the communication device used by the deaf person.

104. The communication device of claim 102 wherein the step of receiving a text message stream includes receiving a digital communication stream from the relay.

105. The communication device of claim 102 wherein the step of transmitting the words spoken includes transmitting the words spoken to the relay via a communication link and wherein the step of receiving the text message stream includes receiving the text message stream via a communication link.

106. The communication device of claim 105 wherein the communication link used to transmit the words spoken to the relay includes a wireless communication link.

107. The communication device of claim 105 wherein the communication link used to transmit the words spoken to the relay includes a cellular communication link.

108. The communication device of claim 105 wherein the words spoken are transmitted via a first communication link and the text message stream is received over the first communication link.