Systems and methods for improving audio conferencing services

• US 9,087,521 B2
• Filed: 07/01/2014
• Issued: 07/21/2015
• Est. Priority Date: 07/02/2013
• Status: Active Grant

• Current Assignee: Family Systems, Ltd.
• Original Assignee: Family Systems, Ltd.
• Inventors: Reynolds, Brian
• Primary Examiner(s): BAKER, MATTHEW H
• Application Number:
  US14/321,348
• Publication Number:
  US 20150012270A1
• Time in Patent Office: 385 Days
• Field of Search: 348/14, 379/202.01, 707/741
• US Class Current: 1/1
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/686 using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/19 by using information detectable on the record carrier
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04M 2203/303 Marking
• CPC Class Codes: H04M 2203/305 Recording playback features, e.g. increased speed
• CPC Class Codes: H04M 2203/5072 Multiple active speakers
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/566 relating to a participants right to speak arrangements for multi-party communication with floor control, e.g. for conferences, H04L65/4038, H04L65/4046, H04L65/4053
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04M 3/569 using the instant speaker's algorithm speech detection per se G10L25/78
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Systems and methods are disclosed herein for improving audio conferencing services. One aspect relates to processing audio content of a conference. A first audio signal is received from a first conference participant, and a start and an end of a first utterance by the first conference participant are detected from the first audio signal. A second audio signal is received from a second conference participant, and a start and an end of a second utterance by the second conference participant is detected from the second audio signal. The second conference participant is provided with at least a portion of the first utterance, wherein at least one of start time, start point, and duration is determined based at least in part on the start, end, or both, of the second utterance.

Claims

1. A method for processing audio content of a conference, the method comprising:

receiving, by at least one processor, a first audio signal from a first conference participant, wherein the first audio signal is detected by a microphone;

detecting, from the first audio signal, a start and an end of a first utterance by the first conference participant;

receiving, by the at least one processor, a second audio signal from a second conference participant;

detecting, from the second audio signal, a start and an end of a second utterance by the second conference participant;

providing, to the second conference participant, at least a portion of the first utterance;

in response to detecting the start of the second utterance,

stopping the providing, to the second conference participant, of the portion of the first utterance; and

storing, in a memory, an indication of a point in the first utterance at which the providing to the second conference participant was stopped; and

in response to detecting the end of the second utterance, resuming the providing, to the second conference participant, of the portion of the first utterance at or just prior to the point referenced by the stored indication.

2. The method of claim 1, wherein the portion of the first utterance is provided at a time corresponding to at least one of a start time for providing the portion of the first utterance, a start point of the portion of the first utterance, and a duration of the first utterance, and wherein the portion of the first utterance is provided to the second conference participant before the start of the second utterance or after the end of the second utterance.

3. The method of claim 1, wherein the start of the second utterance occurs after the start of the first utterance and before the end of the first utterance, and the portion of the first utterance is based on a previous portion of the first utterance that is provided to the second conference participant before the start of the second utterance.

4. The method claim 1, further comprising, upon detection of the start of the second utterance, switching the first and second conference participants to a mode in which utterances are played sequentially to the first and second conference participants.

5. The method of claim 1, wherein resuming the providing of the portion of the first utterance to the second conference participant at the point referenced by the stored indication comprises:

accessing, from the memory, a recorded version of the first audio signal at the point referenced by the stored indication;

playing the portion of the first utterance from the point referenced by the stored indication; and

providing conference audio to the second conference participant in real time when playback of the recorded version terminates.

6. The method of claim 5, wherein the recorded version of the first audio signal is stored as a plurality of audio clips in a playlist, each audio clip including an utterance by one of the conference participants, and wherein playing the portion of the first utterance comprises:

playing the plurality of audio clips sequentially from the point referenced by the stored indication.

7. The method of claim 5, wherein the recorded version of the first audio signal is stored as at least some of a plurality of audio clips in a playlist, each audio clip including an utterance by one of a plurality of conference participants, and the method further comprising:

playing the plurality of audio clips from the point referenced by the stored indication in the same manner in which they were recorded, wherein two or more of the plurality of audio clips are played in an overlapping manner when the corresponding conference audio included overlapping utterances from multiple conference participants.

8. The method of claim 1, wherein detecting the start of the first utterance includes receiving a first selection from the first conference participant to unmute an audio input interface, and detecting the end of the first utterance includes receiving a second selection from the first conference participant to mute the audio input interface.

9. The method of claim 1 further comprising:

storing the first utterance as an audio clip in a playlist, wherein the playlist includes a plurality of audio clips of utterances by other conference participants;

automatically categorizing the stored audio clip in the playlist under a section identifying the conference or a subject of the conference; and

automatically tagging the stored audio clip with information identifying the first conference participant.

10. The method of claim 1, wherein detecting the start of the first utterance includes receiving a first selection from the first conference participant to pause an audio output, and detecting the end of the first utterance includes receiving a second selection from the first conference participant to play the audio output.

11. A system for processing audio content of a conference, the system comprising controller circuitry configured to:

receive a first audio signal from a first conference participant, wherein the first audio signal is detected by a microphone;

detect, from the first audio signal, a start and an end of a first utterance by the first conference participant;

receive a second audio signal from a second conference participant;

detect, from the second audio signal, a start and an end of a second utterance by the second conference participant; and

provide, to the second conference participant, at least a portion of the first utterance, wherein;

when the start of the second utterance is detected, the controller circuitry stops providing the portion of the first utterance to the second conference participant;

a memory is configured to store an indication of a point in the first utterance at which the controller circuitry stops the providing of the portion of the first utterance to the second conference participant; and

when the end of the second utterance is detected, the controller circuitry resumes providing the portion of the first utterance to the second conference participant at or just prior to the point referenced by the stored indication.

12. The system of claim 11, wherein the controller circuitry provides the portion of the first utterance at a time corresponding to at least one of a start time for providing the portion of the first utterance, a start point of the portion of the first utterance, and a duration of the first utterance, and wherein the portion of the first utterance is provided to the second conference participant before the start of the second utterance or after the end of the second utterance.

13. The system of claim 11, wherein the start of the second utterance occurs after the start of the first utterance and before the end of the first utterance, and the portion of the first utterance is based on a previous portion of the first utterance that is provided to the second conference participant before the start of the second utterance.

14. The system of claim 11, wherein the controller circuitry is further configured to, upon detection of the start of the second utterance, switch the first and second conference participants to a mode in which utterances are played sequentially to the first and second participants.

15. The system of claim 11, wherein the controller circuitry resumes the providing of the portion of the first utterance to the second conference participant at the point referenced by the stored indication by:

accessing, from the memory, a recorded version of the first audio signal at the point referenced by the stored indication;

playing the portion of the first utterance from the point referenced by the stored indication; and

providing conference audio to the second conference participant in real time when playback of the recorded version terminates.

16. The system of claim 15, wherein the recorded version of the first audio signal is stored as a plurality of audio clips in a playlist in the memory, each audio clip includes an utterance by one of the conference participants, and wherein the controller circuitry plays the portion of the first utterance by playing the plurality of audio clips sequentially from the point referenced by the stored indication.

17. The system of claim 15, wherein the recorded version of the first audio signal is stored as at least some of a plurality of audio clips in a playlist in the memory, each audio clip includes an utterance by one of a plurality of conference participants, and wherein the controller circuitry plays the plurality of audio clips from the point referenced by the stored indication in the same manner in which they were recorded, wherein two or more of the plurality of audio clips are played in an overlapping manner when the corresponding conference audio included overlapping utterances from multiple conference participants.

18. The system of claim 11, wherein the controller circuitry detects the start of the first utterance by receiving a first selection from the first conference participant to unmute an audio input interface, and detecting the end of the first utterance includes receiving a second selection from the first conference participant to mute the audio input interface.

19. The system of claim 11 wherein:

the memory is configured to store the first utterance as an audio clip in a playlist, wherein the playlist includes a plurality of audio clips of utterances by other conference participants; and

the controller circuitry is further configured to;

automatically categorize the stored audio clip in the playlist under a section identifying the conference or a subject of the conference; and

automatically tag the stored audio clip with information identifying the first conference participant.

20. The system of claim 11, wherein the controller circuitry detects the start of the first utterance by receiving a first selection from the first conference participant to pause an audio output, and detecting the end of the first utterance includes receiving a second selection from the first conference participant to play the audio output.