Post-teleconference playback using non-destructive audio transport

• US 10,511,718 B2
• Filed: 06/15/2016
• Issued: 12/17/2019
• Est. Priority Date: 06/16/2015
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Dickins, Glenn N., Cartwright, Richard J.
• Primary Examiner(s): Truong, Lan-Houng
• Application Number:
  US15/578,386
• Publication Number:
  US 20180295240A1
• Time in Patent Office: 1,280 Days
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 21/0316 by changing the amplitude
• CPC Class Codes: H04M 3/2281 Call monitoring, e.g. for law enforcement purposes; Call tracing; Detection or prevention of malicious calls
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30

Abstract

Teleconference audio data including a plurality of individual uplink data packet streams, may be received during a teleconference. Each uplink data packet stream may corresponding to a telephone endpoint used by one or more teleconference participants. The teleconference audio data may be analyzed to determine a plurality of suppressive gain coefficients, which may be applied to first instances of the teleconference audio data during the teleconference, to produce first gain-suppressed audio data provided to the telephone endpoints during the teleconference. Second instances of the teleconference audio data, as well as gain coefficient data corresponding to the plurality of suppressive gain coefficients, may be sent to a memory system as individual uplink data packet streams. The second instances of the teleconference audio data may be less gain-suppressed than the first gain-suppressed audio data.

Claims

1. An apparatus, comprising:

an interface system; and

a control system capable of;

receiving teleconference audio data during a teleconference, via the interface system, the teleconference audio data comprising a plurality of individual uplink data packet streams, each uplink data packet stream corresponding to a telephone endpoint used by one or more teleconference participants;

analyzing the teleconference audio data to determine a plurality of suppressive gain coefficients;

applying the suppressive gain coefficients to first instances of the teleconference audio data during the teleconference, to produce first gain-suppressed audio data;

providing the first gain-suppressed audio data to the telephone endpoints during the teleconference; and

sending to a memory system, via the interface system;

second instances of the teleconference audio data as individual uplink data packet streams, wherein the second instances of the teleconference audio data are less gain-suppressed than the first gain-suppressed audio data; and

gain coefficient data corresponding to the plurality of suppressive gain coefficients,

wherein the gain coefficient data includes one or more data types comprising;

gain coefficient data indicating gains that could be applied to audio signals before and after instances of detected voice activity;

gain coefficient data indicating gains that could be applied to level audio signals corresponding to voice activity;

gain coefficient data indicating gains that could be applied to attenuate noise;

gain coefficient data indicating gains that could be applied to attenuate conference participant nuisance audio corresponding to apparent non-voice activity;

gain coefficient data indicating gains that could be applied to attenuate sibilance caused by voice capture and coding of fricatives; and

gain coefficient data indicating gains that could be applied to attenuate reverberation.

2. The apparatus of claim 1, wherein the control system is capable of encoding the gain coefficient data and the audio data in data packets of the uplink data packet streams.

3. The apparatus of claim 1, wherein the second instances of the teleconference audio data are either unsuppressed or only partially suppressed, as compared to the first gain-suppressed audio data.

4. The apparatus of claim 1, wherein the control system is capable of:

determining that a late data packet of an incomplete uplink data packet stream has not been received from a telephone endpoint within a late packet time threshold, the late packet time threshold being greater than or equal to a mouth-to-ear latency time threshold of the teleconference; and

transmitting a request to the telephone endpoint, via the interface system, to re-send the late data packet.

5. The apparatus of claim 4, wherein the control system is capable of:

receiving the late data packet; and

adding the late data packet to the incomplete uplink data packet stream.

6. The apparatus of claim 4, wherein the late packet time threshold is greater than or equal to 1 second and wherein the mouth-to-ear latency time threshold is greater than or equal to 100 milliseconds.

7. The apparatus of claim 1, wherein at least one of the uplink data packet streams includes at least one data packet that was received after a mouth-to-ear latency time threshold of the teleconference and was not used for reproducing audio data during the teleconference.

8. A method of processing audio data, the method comprising:

receiving teleconference audio data during a teleconference, the teleconference audio data comprising a plurality of individual uplink data packet streams, each uplink data packet stream corresponding to a telephone endpoint used by one or more teleconference participants;

analyzing the teleconference audio data to determine a plurality of suppressive gain coefficients;

applying the suppressive gain coefficients to first instances of the teleconference audio data during the teleconference, to produce first gain-suppressed audio data;

providing the first gain-suppressed audio data to the telephone endpoints during the teleconference; and

sending the following to a memory system;

second instances of the teleconference audio data as individual uplink data packet streams, wherein the second instances of the teleconference audio data are less gain-suppressed than the first gain-suppressed audio data; and

gain coefficient data corresponding to the plurality of suppressive gain coefficients,

wherein the gain coefficient data includes one or more data types comprising;

gain coefficient data indicating gains that could be applied to audio signals before and after instances of detected voice activity;

gain coefficient data indicating gains that could be applied to level audio signals corresponding to voice activity;

gain coefficient data indicating gains that could be applied to attenuate noise;

gain coefficient data indicating gains that could be applied to attenuate conference participant nuisance audio corresponding to apparent non-voice activity;

gain coefficient data indicating gains that could be applied to attenuate sibilance caused by voice capture and coding of fricatives; and

gain coefficient data indicating gains that could be applied to attenuate reverberation.

9. The method of claim 8, further comprising encoding the gain coefficient data and the audio data in data packets of the uplink data packet streams.

10. The method of claim 8, wherein the second instances of the teleconference audio data are either unsuppressed or only partially suppressed, as compared to the first gain-suppressed audio data.

11. The method of claim 8, further comprising:

determining that a late data packet of an incomplete uplink data packet stream has not been received from a telephone endpoint within a late packet time threshold, the late packet time threshold being greater than or equal to a mouth-to-ear latency time threshold of the teleconference; and

transmitting a request to the telephone endpoint, via an interface system, to re-send the late data packet.

12. The method of claim 11, further comprising:

receiving the late data packet; and

adding the late data packet to the incomplete uplink data packet stream.

13. The method of claim 11, wherein the late packet time threshold is greater than or equal to 1 second and wherein the mouth-to-ear latency time threshold is greater than or equal to 100 milliseconds.

14. The method of claim 8, wherein at least one of the uplink data packet streams includes at least one data packet that was received after a mouth-to-ear latency time threshold of the teleconference and was not used for reproducing audio data during the teleconference.

15. A non-transitory medium having software stored thereon, the software including instructions for processing audio data by controlling at least one device for:

receiving teleconference audio data during a teleconference, the teleconference audio data comprising a plurality of individual uplink data packet streams, each uplink data packet stream corresponding to a telephone endpoint used by one or more teleconference participants;

analyzing the teleconference audio data to determine a plurality of suppressive gain coefficients;

applying the suppressive gain coefficients to first instances of the teleconference audio data during the teleconference, to produce first gain-suppressed audio data;

providing the first gain-suppressed audio data to the telephone endpoints during the teleconference; and

sending the following to a memory system;

second instances of the teleconference audio data as individual uplink data packet streams, wherein the second instances of the teleconference audio data are less gain-suppressed than the first gain-suppressed audio data; and

gain coefficient data corresponding to the plurality of suppressive gain coefficients,

wherein the gain coefficient data includes one or more data types comprising;

gain coefficient data indicating gains that could be applied to audio signals before and after instances of detected voice activity;

gain coefficient data indicating gains that could be applied to level audio signals corresponding to voice activity;

gain coefficient data indicating gains that could be applied to attenuate noise;

gain coefficient data indicating gains that could be applied to attenuate conference participant nuisance audio corresponding to apparent non-voice activity;

gain coefficient data indicating gains that could be applied to attenuate sibilance caused by voice capture and coding of fricatives; and

gain coefficient data indicating gains that could be applied to attenuate reverberation.

16. The non-transitory medium of claim 15, wherein the software includes instructions for encoding the gain coefficient data and the audio data in data packets of the uplink data packet streams.

17. The method of claim 15, wherein the second instances of the teleconference audio data are either unsuppressed or only partially suppressed, as compared to the first gain-suppressed audio data.

18. The non-transitory medium of claim 15, wherein the software includes instructions for:

determining that a late data packet of an incomplete uplink data packet stream has not been received from a telephone endpoint within a late packet time threshold, the late packet time threshold being greater than or equal to a mouth-to-ear latency time threshold of the teleconference; and

transmitting a request to the telephone endpoint, via the interface system, to re-send the late data packet.

19. The non-transitory medium of claim 18, wherein the software includes instructions for:

receiving the late data packet; and

adding the late data packet to the incomplete uplink data packet stream.

20. The non-transitory medium of claim 15, wherein at least one of the uplink data packet streams includes at least one data packet that was received after a mouth-to-ear latency time threshold of the teleconference and was not used for reproducing audio data during the teleconference.