System for text assisted telephony

• US 8,908,838 B2
• Filed: 06/09/2014
• Issued: 12/09/2014
• Est. Priority Date: 08/23/2001
• Status: Expired due to Fees

• Current Assignee: Ultratech Incorporated (Veeco Instruments Incorporated)
• Original Assignee: Ultratech Incorporated (Veeco Instruments Incorporated)
• Inventors: Engelke, Robert M, Colwell, Kevin R
• Primary Examiner(s): WOO, STELLA L
• Application Number:
  US14/299,531
• Publication Number:
  US 20140270102A1
• Time in Patent Office: 183 Days
• Field of Search: 379/52, 379/93.15, 379/201.01, 704/235, 704/260, 704/271
• US Class Current: 379/52
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04M 1/21 Combinations with auxiliary equipment, e.g. with clocks or memoranda pads
• CPC Class Codes: H04M 1/2155 Acoustic coupling
• CPC Class Codes: H04M 1/2475 for a hearing impaired user
• CPC Class Codes: H04M 1/271 controlled by voice recognition
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 11/066 Telephone sets adapted for data transmision
• CPC Class Codes: H04M 11/08 specially adapted for optional reception of entertainment or informative matter
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 2201/60 Medium conversion
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00
• CPC Class Codes: H04M 3/42161 via computer interface
• CPC Class Codes: H04M 3/42382 Text-based messaging services in telephone networks such as PSTN/ISDN, e.g. User-to-User Signalling or Short Message Service for fixed networks
• CPC Class Codes: H04M 3/42391 where the subscribers are hearing-impaired persons, e.g. telephone devices for the deaf

Abstract

A system and method for providing captioned services comprising a relay, an assisted user'"'"'s captioned device including a processor programmed to perform the steps of establishing a first communication link between the captioned device and a hearing person'"'"'s device, receiving voice signals from the hearing person via the first communication link, receive an indication that an activator has been activated to invoke a captioning service and in response transmitting the hearing user'"'"'s voice signals received at the captioned device to a relay via a second communication link, receiving text back corresponding to the hearing user'"'"'s voice signals from the relay and displaying the text wherein the assisted user can invoke the captioning service either prior to or after the first communication link is established.

Claims

1. A system for providing text captioned telephony services to an assisted user communicating with a hearing person, the system comprising:

a relay;

an assisted user'"'"'s captioned device comprising;

(1) a captioned device processor linkable to first and second communication links;

(2) a visual display linked to the captioned device processor;

(3) a microphone linked to the captioned device processor for receiving voice signals from the assisted user using the assisted user'"'"'s captioned device;

(4) a speaker linked to the captioned device processor for broadcasting voice signals received by the captioned device to the ear of the assisted user as voice signals from the hearing person are received at the captioned device;

(5) an activator linked to the captioned device processor, the activator controllable by the assisted user to invoke a captioning service;

the captioned device processor configured to perform the steps of;

(i) establishing-the first communication link between the assisted user'"'"'s captioned device and the hearing person;

(ii) obtaining voice signals from the assisted user via the microphone;

(iii) transmitting the assisted user'"'"'s voice signals from the captioned device to the hearing person via the first communication link;

(iv) receiving voice signals from the hearing person via the first communication link;

(v) broadcasting the voice signals from the hearing person via the speaker to the assisted user as the voice signals from the hearing person are received at the captioned device;

(vi) receiving a signal from the activator to invoke the captioning service;

(vii) in response to receiving a signal from the activator;

(a) transmitting the hearing person'"'"'s voice signals received at the captioned device to the relay via the second communication link;

(b) receiving text corresponding to the hearing person'"'"'s voice signals from the relay via the second communication link; and

(c) displaying the text received from the relay via the visual display for the assisted user to view;

wherein the activator is activatable to establish the second communication link and invoke the captioning service subsequent to broadcasting at least some voice signals from the hearing person to the assisted user while a voice call between the assisted user and the hearing person is progressing; and

wherein the activator is further activatable to establish the second communication link and invoke the captioning service prior to broadcasting any of the hearing person'"'"'s voice signals to the assisted user.

2. The system of claim 1 wherein the activator is activatable to invoke the captioning service prior to establishing the first communication link between the assisted user'"'"'s captioned device and a hearing person.

3. The system of claim 2 wherein activation of the activator prior to establishing the first communication link causes the captioned device processor to invoke the captioning service prior to broadcast of any of the hearing person'"'"'s voice signals to the ear of the assisted user.

4. The system of claim 3 wherein activation of the activator prior to establishing the first communication link causes the captioned device processor to establish the second communication link to the relay.

5. The system of claim 4 wherein the relay is a preselected relay associated with the assisted user'"'"'s captioned device.

6. The system of claim 5 wherein the hearing person'"'"'s voice signals are transmitted to the relay via the second communication link as digital data packets and wherein the text is transmitted from the relay to the assisted user'"'"'s captioned device as digital data packets.

7. The system of claim 6 wherein the step of receiving text from the relay includes receiving the text via the second communication link.

8. The system of claim 7 wherein the relay is remote from each of the assisted user'"'"'s captioned device and the hearing person.

9. The system of claim 1 wherein the activator includes a button on the assisted user'"'"'s captioned device.

10. The system of claim 1 further including the steps of, at the relay, receiving voice signals from a call assistant that revoice the hearing person'"'"'s voice signals, using software trained to the voice of the call assistant to transcribe the voice signals from the call assistant into a text stream, presenting the text stream via a display to the call assistant, receiving corrections to the text stream from the call assistant to generate corrected text and transmitting the corrected text to the assisted user'"'"'s captioned device.

11. The system of claim 1 wherein the step of establishing the first communication link includes receiving an indication from the assisted user'"'"'s captioned device invoking the first communication link.

30. The system of claim 10 wherein the corrected text transmitted to the assisted user'"'"'s device includes at least some words form the original text stream presented on the display at the relay as well as at least some corrected words.

12. A system for providing captioned services to an assisted user communicating with a hearing person, the system comprising:

an assisted user'"'"'s captioned device comprising;

(1) a captioned device processor linkable to first and second communication links;

(2) a visual display linked to the captioned device processor;

(3) a speaker linked to the captioned device processor for broadcasting voice signals received by the captioned device to the ear of the assisted user substantially in real time as voice signals from the hearing person are received at the captioned device;

(4) an activator linked to the captioned device processor, the activator controllable by an assisted user to invoke a captioning service through the second communication link;

the captioned device processor configured to perform the steps of;

(i) establishing-the first communication link between the assisted user'"'"'s captioned device and the hearing person;

(ii) receiving voice signals from the hearing person via the first communication link;

(iii) broadcasting the voice signals from the hearing person via the speaker to the ear of the assisted user as the voice signals from the hearing person are received at the captioned device;

(iv) receiving an indication that the activator has been activated to invoke the captioning service;

(v) in response to activation of the activator;

(a) initiating the second communication link between the assisted user'"'"'s device and a relay;

(b) transmitting the hearing person'"'"'s voice signals received at the captioned device from the hearing person to the relay via the second communication link;

(c) receiving text corresponding to the hearing person'"'"'s voice signals from the relay; and

(d) displaying the text received from the relay via the visual display for the assisted user to view;

wherein the activator is selectively operable prior to establishing the first communication link to invoke the captioning service and prior to any voice communication between the assisted user and the hearing person.

13. The system of claim 12 wherein the activator is further selectable subsequent to broadcasting at least some voice signals from the hearing person to the ear of the assisted user to set up the captioning service while a voice call between the assisted user and the hearing person is progressing.

14. The system of claim 13 wherein the assisted user'"'"'s captioning device is usable without selectively operating the activator to facilitate a voice call between the assisted user and the hearing person.

15. The system of claim 12 wherein the relay is a preselected relay associated with the assisted user'"'"'s captioned device.

16. The system of claim 12 wherein the hearing person'"'"'s voice signals are transmitted to the relay via the second communication link as digital data packets and wherein the text is transmitted from the relay to the assisted user'"'"'s captioned device as digital data packets via the second communication link.

17. The system of claim 12 wherein the relay is remote from each of the assisted user'"'"'s captioned device and the hearing person.

18. The system of claim 12 further including a relay wherein, at the relay, only the hearing person'"'"'s voice signals are presented to a call assistant, the call assistant revoices the hearing person'"'"'s voice signals, software trained to the voice of the call assistant transcribes the voice signals from the call assistant into a text stream, the text stream is presented via a display to the call assistant, corrections to the text stream are received from the call assistant to generate corrected text and the corrected text is transmitted to the assisted user'"'"'s captioned device.

19. A method for providing captioned telephony services to an assisted user communicating with a hearing person, the method comprising the steps of:

(i) providing-an assisted user'"'"'s captioned device comprising;

(a) a captioned device processor linkable to first and second communication links;

(b) a visual display linked to the captioned device processor;

(c) a speaker linked to the captioned device processor for broadcasting voice signals from the hearing person received by the captioned device to the ear of the assisted user substantially in real time as the voice signals from the hearing person are received at the captioned device;

(d) an activator linked to the captioned device processor, the activator controllable by an assisted user to indicate that a captioning service should be invoked;

(ii) initiating the first communication link between the assisted user'"'"'s captioned device and a hearing person;

(iii) receiving voice signals from the hearing person via the first communication link;

(iv) broadcasting the voice signals from the hearing person via the speaker to the ear of the assisted user as the voice signals from the hearing person are received at the captioned device;

(v) receiving a signal from the actuator invoking the captioning service;

(vi) in response to the signal invoking the captioning service, transmitting the hearing person'"'"'s voice signals received at the captioned device to a relay via the second communication link;

(vii) receiving text corresponding to the hearing person'"'"'s voice signals from the relay; and

(viii) displaying the text received from the relay via the visual display for the assisted user to view;

wherein the activator is selectively operable to invoke the captioning service prior to establishing the first communication link and prior to broadcasting at least some voice signals from the hearing person to the assisted user while a voice call between the assisted user and the hearing person is progressing.

20. The method of claim 19 further including the steps of, in response to selectively operating the activator and prior to transmitting the hearing person'"'"'s voice signals to the relay, establishing the second communication link between the assisted user'"'"'s captioned device and the relay.

21. The method of claim 19 wherein the step of receiving text includes receiving the text over a third communication link separate from the first and second communication links.

22. The method of claim 19 wherein the relay is a preselected relay associated with the assisted user'"'"'s captioned device.

23. The method of claim 19 wherein the hearing person'"'"'s voice signals are transmitted to the relay via the second communication link as digital data packets and wherein the text is transmitted from the relay to the assisted user'"'"'s captioned device as digital data packets.

24. The method of claim 19 wherein the relay is remote from each of the assisted user'"'"'s captioned device and the hearing person.

25. The method of claim 19 wherein the activator includes a button on the assisted user'"'"'s captioned device.

26. The method of claim 19 further including the step of, at the relay, presenting only the hearing person'"'"'s voice signals to the ear of a call assistant, receiving voice signals from the call assistant that revoice the hearing person'"'"'s voice signals, using software trained to the voice of the call assistant to transcribe the voice signals from the call assistant into a text stream, presenting the text stream via a display to the call assistant, receiving corrections to the text stream from the call assistant to generate corrected text and transmitting the corrected text to the assisted user'"'"'s captioned device.

27. The method of claim 19 wherein the step of initiating the first communication link includes receiving an indication from the assisted user'"'"'s captioned device invoking the first communication link.

28. The method of claim 19 wherein the second communication link is an internet link.

29. The method of claim 19 wherein the activator is selected to set up the captioning service prior to initiating the first communication link.