Communication device and related methods for automatically connecting to a captioning communication service to receive text captions following an interruption during a call

• US 9,787,831 B1
• Filed: 11/16/2015
• Issued: 10/10/2017
• Est. Priority Date: 03/25/2015
• Status: Active Grant

• Current Assignee: Sorenson Ip Holdings, LLC
• Original Assignee: Sorenson Ip Holdings, LLC
• Inventors: Chevrier, Brian, Stimpson, Michael S.
• Primary Examiner(s): El-Zoobi, Maria
• Application Number:
  US14/942,905
• Time in Patent Office: 694 Days
• Field of Search: None
• CPC Class Codes: H04M 1/2475 for a hearing impaired user
• CPC Class Codes: H04M 1/72478 for hearing-impaired users
• CPC Class Codes: H04M 11/10 with dictation recording and playback systems
• CPC Class Codes: H04M 2201/40 using speech recognition speech recognition per se G10L15/00
• CPC Class Codes: H04M 3/42391 where the subscribers are hearing-impaired persons, e.g. telephone devices for the deaf
• CPC Class Codes: H04M 7/0042 where the data service is a text-based messaging service H04M7/0054 takes precedence

Abstract

Apparatuses and methods are disclosed for automatically connecting to a relay service and establish a captioning communication session during a call between a hearing-impaired user and a far-end user. The apparatus includes communication elements configured to receive captions from a relay service, and a processor configured to automatically connect to the relay service and establish a captioning communication session during a call responsive to a determination that a connection to the relay service failed during a time in which the hearing-impaired user desired captions to be enabled. A method includes determining captions are desired by the hearing-impaired user at a first time to be enabled for display to the communication device during a call, detecting a failed connection to the relay service at a second time, and automatically connecting with the relay service and establishing a new captioning communication session at a third time during the call.

Claims

1. A communication device comprising:

a processor configured to;

establish a voice communication session with a far-end communication device to communicate audio data therebetween during the voice communication session;

detect if captioning is enabled for the voice communication session, wherein captioning is selectable by a user to be one of either enabled or disabled;

attempt to establish a captioning communication session with a captioning communication service to receive text data corresponding to a text transcription of far-end audio data during the voice communication session responsive to captioning being enabled; and

automatically continue to attempt to establish a captioning communication session without user input at intervals during the voice communication session based, at least in part, upon one or more messages generated from different software modules operated by the processor for as long as the voice communication session is still established, captioning is still enabled, and until the captioning communication session is established,

wherein a main module of the different software modules is configured to establish the captioning communication session according to a state machine that includes;

a disconnected state during which the main module awaits instructions to enable captioning;

a session opening state during which a session invitation is sent to the captioning communication service responsive to receiving the instructions to enable captioning;

a connected state during which audio is sent to the captioning communication service and text captions are received from the captioning communication service responsive to receiving an acceptance message from the captioning communication service; and

a session closing state during which a message is sent to the captioning communication service responsive to receiving an indication that captioning has been disabled; and

an electronic display operably coupled with the processor, the electronic display configured to display the text captions received through the captioning communication session during the voice communication session.

2. The communication device of claim 1, wherein the processor is configured to attempt to establish the captioning communication session and automatically continue to attempt to establish the captioning communication session prior to any previous captioning communication session being successfully established during the voice communication session.

3. The communication device of claim 2, wherein captioning is enabled initially for the voice communication session by default.

4. The communication device of claim 1, wherein the processor is configured to attempt to establish the captioning communication session and automatically continue to attempt to establish the captioning communication session responsive to a previous captioning communication session being disconnected during the voice communication session without captioning being disabled by the user.

5. The communication device of claim 1, further comprising a caption button configured to receive a user input to control captioning as being either enabled or disabled.

6. The communication device of claim 5, wherein the caption button is at least one of a mechanical input or an input of a graphical user interface generated by the processor and displayed by the electronic display.

26. The communication device of claim 1, wherein the one or more messages includes an error message received by the main module during the session opening state or the connected state.

27. The communication device of claim 1, wherein different software modules further include:

a far-end communication module configured to send and receive audio data with the far-end communication device;

a captioning service session module configured to establish a captioning communication session with the captioning communication service;

a captioning service communication module configured to send and receive data between the captioning communication service; and

a user interface module configured to communicate with the electronic display.

28. The communication device of claim 27, wherein the one or more messages

are received by the main module from either the captioning service session module or the captioning service communication module.

7. A communication device comprising:

a processor configured to control operations comprising;

establish a voice communication session with a far-end communication device to communicate audio data therebetween during the voice communication session;

detect if captioning is enabled for the voice communication session, wherein captioning is selectable by a user to be one of either enabled or disabled;

attempt to establish a captioning communication session with a captioning communication service to receive text data corresponding to a text transcription of far-end audio data during the voice communication session responsive to captioning being enabled; and

automatically continue to attempt to establish a captioning communication session without user input at intervals during the voice communication session for as long as the voice communication session is still established, captioning is still enabled, and until the captioning communication session is established;

an electronic display operably coupled with the processor, the electronic display configured to display text captions received through the captioning communication session during the voice communication session; and

a memory device having instructions stored thereon for execution of software modules by the processor, including a main module configured to coordinate communication with other modules, and to control automatic attempts to establish the captioning communication session responsive to determining if captioning is enabled.

8. The communication device of claim 7, wherein the main module is configured to determine if captioning is enabled based, at least in part, on one or more messages received from the other modules.

9. The communication device of claim 8, wherein the main module is configured to determine if captioning is enabled based, at least in part, on a message received from a captioning session module configured to establish the captioning communication session with the captioning communication service.

10. The communication device of claim 8, wherein the main module is configured to determine if captioning is enabled based, at least in part, on a message received from a captioning communication module configured to transmit and receive data between the communication device and the captioning communication service during the captioning communication session.

11. The communication device of claim 8, wherein the main module is configured to determine if captioning is enabled based, at least in part, on a message received from a module other than a user interface module configured to receive user inputs through a user interface.

12. A communication device comprising:

an electronic display; and

a processor operably coupled with the electronic display, the processor programmed to control operations comprising;

communicate audio data during a call between the communication device and a far-end communication device through a first connection therebetween;

enable captioning for the call responsive to a user indication;

disable captioning for the call responsive to a second user indication;

establish a second connection between the communication device and a remote captioning service responsive to the captioning being enabled for the call;

automatically re-establish the second connection between the communication device and the remote captioning service responsive to detecting the second connection has been terminated in a manner different than being responsive to the second user indication of intentionally disabling captioning;

execute a main module configured to coordinate communication with other modules and to control automatic attempts to establish the second connection responsive to determining if captioning is enabled; and

receive text captions from the remote captioning service and provide the text captions to the electronic display for display by the electronic display while the second connection is established.

13. The communication device of claim 12, wherein the user indication and the second user indication include a respective selection and de-selection of a caption button of the communication device.

14. A communication device comprising:

an electronic display; and

a processor operably coupled with the electronic display, the processor programmed to control operations comprising;

communicate audio data during a call between the communication device and a far-end communication device through a first connection therebetween;

enable captioning for the call responsive to a user indication;

disable captioning for the call responsive to a second user indication;

establish a second connection between the communication device and a remote captioning service responsive to the captions being enabled for the call;

automatically re-establish the second connection between the communication device and the remote captioning service responsive to detecting the second connection has been terminated in a manner different than being responsive to the second user indication for intentionally disabling captioning, wherein detecting the manner of termination of the second connection is based, at least in part, upon one or more messages generated from different software modules operated by the processor; and

receive text captions from the remote captioning service and display the text captions on the electronic display while the second connection is established.

15. The communication device of claim 14, wherein the operations further include determine that the second connection has been terminated in a manner different than being responsive to the second user indication by receiving an error message from a captioning session module that is generated if no response is received from the remote captioning service within a predetermined period of time following a ping message being sent to the remote captioning service.

16. The communication device of claim 14, wherein the operations further include determine that the second connection has been terminated in a manner different than being responsive to the second user indication by receiving an error message from a captioning communication module that is generated if there is an error during transmission of audio packets from the communication device to the remote captioning service.

17. The communication device of claim 14, wherein the operations further include determine that the second connection has been terminated in a manner different than being responsive to the second user indication by receiving a termination message from the remote captioning service indicating that the remote captioning service has been terminated by a call assistant handling the call at the remote captioning service.

18. A method comprising:

communicating audio data with a far-end communication device during a voice communication session between at least two users;

receiving text data from a remote captioning service over a captioning communication session responsive to captioning being enabled during the voice communication session, wherein captioning is either enabled or disabled based, at least in part, on a user indication;

displaying text captions on an electronic display corresponding to the text data received from the remote captioning service; and

automatically attempting to re-establish the captioning communication session responsive to determining that the captioning communication session has been terminated in a manner that is different than the user indication, wherein determining the manner of termination of the captioning communication session is based, at least in part, upon one or more messages generated from different software modules coordinated by a main module executed by a processor that controls automatic attempts to establish the captioning communication session while also maintaining the voice communication session with the far-end communication device.

19. The method of claim 18, further comprising receiving the user indication responsive to a selection from a user interface.

20. The method of claim 18, wherein the user indication is set to captioning enabled by default.

22. The method of claim 18, wherein automatically attempting to re-establish the captioning communication session is responsive to detecting an error after sending a ping message to the remote captioning service.

23. The method of claim 18, wherein automatically attempting to re-establish the captioning communication session is responsive to receiving a termination message from the remote captioning service indicating that a call assistant at the remote captioning service terminated the captioning communication session.

24. The method of claim 18, wherein automatically attempting to re-establish the captioning communication session is performed without user input until a new captioning communication session is established the remote captioning service, until captioning is disabled by the user indication, or until the voice communication session is terminated.

21. A method comprising:

communicating audio data with a far-end communication device during a voice communication session between at least two users;

receiving text data from a remote captioning service over a captioning communication session responsive to captioning being enabled during the voice communication session, wherein captioning is either enabled or disabled based, at least in part, on a user indication;

displaying text captions on an electronic display corresponding to the text data received from the remote captioning service; and

automatically attempting to re-establish the captioning communication session responsive to determining that the captioning communication session has been terminated in a manner that is different than the user indication including detecting an error in transmission of audio data to the remote captioning service over the captioning communication session, while also maintaining the voice communication session with the far-end communication device.

25. A method comprising:

communicating audio data with a far-end communication device during a voice communication session between at least two users;

receiving text data from a remote captioning service over a captioning communication session responsive to captioning being enabled during the voice communication session, wherein captioning is either enabled or disabled based, at least in part, on a user indication;

displaying text captions on an electronic display corresponding to the text data received from the remote captioning service; and

automatically attempting to re-establish the captioning communication session responsive to determining that the captioning communication session has been terminated in a manner that is different than the user indication, while also maintaining the voice communication session with the far-end communication device, and including sending session requests at staggered intervals that are increased over time up to a maximum interval until the captioning communication session is re-established or until the voice communication session is terminated.