Methods and devices for automatic volume control of a far-end voice signal provided to a captioning communication service

• US 9,380,150 B1
• Filed: 11/05/2015
• Issued: 06/28/2016
• Est. Priority Date: 09/16/2015
• Status: Active Grant

• Current Assignee: Sorenson Ip Holdings, LLC
• Original Assignee: Captioncall LLC
• Inventors: Bullough, Jeffrey C., Roylance, Shane A., Chevrier, Brian
• Primary Examiner(s): Jamal, Alexander
• Application Number:
  US14/933,893
• Time in Patent Office: 236 Days
• Field of Search: 379/406.02, 379/406.07, 379/406.16, 379/88.13, 379/395
• US Class Current: 1/1
• CPC Class Codes: H04M 1/2475 for a hearing impaired user
• CPC Class Codes: H04M 1/60 including speech amplifiers
• CPC Class Codes: H04M 1/72478 for hearing-impaired users
• CPC Class Codes: H04M 2207/30 third party service providers
• CPC Class Codes: H04M 3/002 Applications of echo suppressors or cancellers in telephonic connections in two-way loud-speaking telephone systems H04M9/08, echo suppressors or cancellers per se H04B3/20
• CPC Class Codes: H04M 3/42391 where the subscribers are hearing-impaired persons, e.g. telephone devices for the deaf
• CPC Class Codes: H04M 9/082 using echo cancellers echo cancellers per se H04B3/23
• CPC Class Codes: H04W 4/16 Communication-related supplementary services, e.g. call-transfer or call-hold

Abstract

Apparatuses and methods are disclosed for automatic volume control of an audio stream reproduced by a captioning communication service for use by a call assistant in generating a text transcription of a communication session between a hearing-impaired user and a far-end user. The automatic volume control automatically adjusts a volume of the audio stream reproduced by the captioning communication service responsive to a volume control command identifying which of the far-end voice signal and the near-end voice signal is active at a given time. The system further includes an echo modifier configured to add distortion to an echo portion of the far-end voice signal when generating the audio stream.

Claims

1. A communication device specifically configured for use by a hearing-impaired user, the communication device comprising:

a microphone configured to generate a near-end voice signal;

communication elements configured to receive a received far-end voice signal through a network from a far-end communication device; and

a speaker configured to reproduce the far-end voice signal;

a processor operably coupled with the microphone and the communication elements, the processor configured to;

transmit an audio stream signal to a remote captioning service;

split the far-end voice signal into a first path to the speaker that includes an echo canceller and a second path without an echo canceller to include with the audio stream signal for transmission to remote captioning service; and

automatically control a volume level of an audio stream signal reproduced by the remote captioning communication service by causing the volume level of the audio stream signal to be reduced responsive to determining that the near-end voice signal is active and the received far-end voice signal is inactive.

2. The communication device of claim 1, wherein the processor comprises an automatic control system including an active signal detector configured to generate and send a volume control command to audio processing logic in response to determining the near-end voice signal is active and the received far-end signal is inactive.

3. The communication device of claim 2, wherein the volume control command is one of a binary flag or a numerical value.

5. The communication device of claim 2, wherein the processor further comprises an echo modifier including:

an echo estimator configured to generate an estimated echo signal for the received far-end voice signal; and

echo distortion logic configured to add distortion to the estimated echo signal to generate a modified estimated echo signal, wherein a summation block receives the modified estimated echo signal and the received far-end voice signal to generate the audio stream signal sent to the audio processing logic such that the modified echo portion of the audio stream signal is audibly distinguishable from the far-end voice signal portion of the audio stream signal when reproduced by the remote captioning communication service.

6. The communication device of claim 5, wherein the echo distortion logic is configured to add distortion to the estimated echo signal by performing at least one of frequency shifting the estimate echo signal, signal modulation of the estimate echo signal, attenuation of the estimate echo signal, adding white noise to the estimate echo signal or adding colored noise to the estimated echo signal.

7. The communication device of claim 1, wherein the processor is configured to automatically control the volume level of the audio stream signal to have:

a first volume level responsive to determining that only the near-end voice signal is active;

and a second volume level responsive to determining that only the received far-end voice signal is active or responsive to determining that both the near-end voice signal and the received far-end voice signal are simultaneously active.

4. A communication device comprising:

a microphone configured to generate a near-end voice signal;

communication elements configured to receive a received far-end voice signal through a network from a far-end communication device; and

a processor operably coupled with the microphone and the communication elements, the processor configured to;

transmit an audio stream signal including a remote captioning communication service;

automatically control a volume level of an audio stream signal reproduced by the remote captioning communication service by generating a volume control command responsive to determining which of the near-end voice signal and the received far-end voice signal is active; and

encode the audio stream signal with the volume control command in packets during transmission thereof to audio processing logic that is part of the remote captioning communication service;

where the processor is operable to split the far-end voice signal into a first path to a speaker that includes an echo canceller and a second path without an echo canceller to include with the audio stream signal for transmission to the remote captioning communication service.

8. A method of operating a captioning communication service for hearing-impaired users, the method comprising:

determining an active talker situation responsive to comparing a near-end voice signal from a near-end communication device and a received far-end voice signal from a far-end communication device;

transmitting, to a remote captioning, service, an audio stream including the far-end voice signal that is split off from a path prior to being sent to a speaker of the near-end communication device; and

automatically adjusting a volume level of the audio stream reproduced by

the remote captioning communication service based on the determined active talker situation by reducing the volume level during a near-end only situation in comparison to a far-end only situation or a double-talk situation;

where a processor in the near-end communication device canceller and a second path without an echo canceller to include with the audio stream signal for transmission to the remote captioning communication service.

9. The method of claim 8, wherein comparing the near-end voice signal and the received far-end voice signal include cross-correlating the received far-end voice signal and the near-end voice signal.

10. The method of claim 8, wherein determining the active talker situation includes generating a volume control command indicating the active talker situation.

11. The method of claim 10, further comprising processing the audio stream according to the volume control command prior to being reproduced.

13. The method of claim 11, wherein generating the volume control command and processing the audio stream are both performed by the remote captioning communication service.

14. The method of claim 11, wherein generating the volume control command and processing the audio stream are both performed by the near-end communication device.

15. The method of claim 8, further comprising:

estimating an echo portion of the received far-end voice signal; and

adding distortion to the estimated echo portion to generate the audio stream such that the audio stream is a modified received far-end voice signal without cancelling the echo portion such that the modified echo portion of the audio stream signal is audibly distinguishable from the far-end voice signal portion of the audio stream signal when reproduced by the remote captioning communication service.

19. The method of claim 15, wherein adding distortion to the estimated echo portion includes frequency shifting the estimated echo portion.

20. The method of claim 15, wherein adding distortion to the estimated echo portion includes signal modulation of the estimated echo portion.

21. The method of claim 15, wherein adding distortion to the estimated echo portion includes partial or complete attenuation of the estimated echo portion.

22. The method of claim 15, wherein adding distortion to the estimated echo portion includes adding white or colored noise to the estimated echo portion.

12. A method of operating a captioning communication service for hearing-impaired users, the method comprising:

determining an active talker situation responsive to comparing a near-end voice signal from a near-end communication device and a received far-end voice signal from a far-end communication device; and

automatically adjusting a volume level of an audio stream reproduced by a remote captioning communication service based on the determined active talker situation including generating a volume control command indicating the active talker situation; and

processing the audio stream according to the volume control command prior to being reproduced, wherein generating the volume control command is performed by the near-end communication device, and processing the audio stream is performed by the remote captioning communication service;

where a processor in the near-end communication device is operable to split the far-end voice signal into a first path to a speaker that includes an echo canceller and a second path without an echo canceller to include with the audio stream signal for transmission to the remote captioning communication service.

16. A method of operating a captioning communication service for hearing-impaired users, the method comprising;

determining an active talker situation responsive to comparing a near-end voice signal from a near-end communication device and a received far-end voice signal from a far-end communication device; and

automatically adjusting a volume level of an audio stream reproduced by a remote captioning communication service based on the determined active talker situation; and

packetizing the audio stream with a volume control command and sending the packets to the remote captioning communication service for the remote captioning service to adjust the volume level of the audio stream responsive to the volume control command;

where a processor in the near-end communication device is operable to split the far-end voice signal into a first path to a speaker that includes an echo canceller and a second path without an echo canceller to include with the audio stream signal for transmission to the remote captioning communication service.

17. A captioning communication system, comprising:

a near-end communication device including;

a microphone configured to capture a near-end voice signal during a communication session with a far-end communication device;

communication elements configured to receive a far-end voice signal from the far-end communication device during the communication session;

a speaker configured to reproduce the far-end voice signal;

an electronic display configured to display text captions during the communication session; and

a processor operably coupled with the microphone, the communication elements, the speaker, and the electronic display; and

a remote captioning communication service configured to generate a text transcription of the far-end voice signal during the communication session and transmit the text transcription in real time to the near-end communication device for the text captions to be displayed, wherein at least one of the near-end communication device and the remote captioning communication service is configured to operate;

a volume control system configured to automatically adjust a volume of an audio stream including an echo portion and a far-end signal portion, the audio stream reproduced by a speaker of the remote captioning communication service at different volume levels responsive to a volume control command identifying which of the far-end voice signal and the near-end voice signal is active at a given time; and

an echo modifier configured to add distortion to the echo portion to distort the echo portion such that the distortion causes the echo portion to be audibly distinguishable from the far when the audio stream is reproduced by the speaker of the remote captioning communication service;

where the processor is operable to split the far-end voice signal into a first path to the speaker that includes an echo canceller and a second path without an echo canceller to include with the audio stream signal for transmission to the remote captioning communication service.

18. The captioning communication system of claim 17, wherein:

the automatic control system includes an active signal detector configured to generate the volume control command responsive to a cross correlation of the near-end voice signal and the received far-end voice signal; and

the echo modifier includes an echo estimator configured to provide an estimated echo signal to echo distortion logic without cancelling the echo.