SYSTEMS AND METHODS FOR IMPROVING AUDIO CONFERENCING SERVICES

• US 20150312518A1
• Filed: 07/06/2015
• Published: 10/29/2015
• Est. Priority Date: 07/02/2013
• Status: Active Grant

• Current Assignee: Family Systems, Ltd.
• Original Assignee: Family Systems, Ltd.
• Inventors: Reynolds, Brian
• Granted Patent: US 9,538,129 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/686 using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/19 by using information detectable on the record carrier
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04M 2203/303 Marking
• CPC Class Codes: H04M 2203/305 Recording playback features, e.g. increased speed
• CPC Class Codes: H04M 2203/5072 Multiple active speakers conducting a computer conference, e.g. Admission detection H04L12/1822
• CPC Class Codes: H04M 3/42221 Conversation recording systems at the subscriber's set H04M1/656
• CPC Class Codes: H04M 3/566 relating to a participants right to speak arrangements for multi-party communication with floor control, e.g. for conferences, H04L65/4038, H04L65/4046, H04L65/4053
• CPC Class Codes: H04M 3/568 audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants echo suppression in two-way loud-speaking telephone systems H04M9/02; sound field processing per se H04S7/30
• CPC Class Codes: H04M 3/569 using the instant speaker's algorithm speech detection per se G10L25/78
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Systems and methods are disclosed herein for improving audio conferencing services. One aspect relates to processing audio content of a conference. A first audio signal is received from a first conference participant, and a start and an end of a first utterance by the first conference participant are detected from the first audio signal. A second audio signal is received from a second conference participant, and a start and an end of a second utterance by the second conference participant is detected from the second audio signal. The second conference participant is provided with at least a portion of the first utterance, wherein at least one of start time, start point, and duration is determined based at least in part on the start, end, or both, of the second utterance.

Claims

1-30. -30. (canceled)

31. A method for processing audio content of a conference, the method comprising:

providing, by at least one processor, audio from the conference to a first conference participant, wherein the audio is detected by at least one microphone;

detecting, by the at least one processor, a start of an utterance by the first conference participant;

in response to detecting the start of the utterance, stopping the provision of the audio from the conference to the first conference participant;

storing, in a memory, an indication of a point in the audio from the conference at which the provision of the audio from the conference to the first conference participant was stopped;

detecting, by the at least one processor, an end of the utterance by the first conference participant; and

in response to detecting the end of the utterance, resuming the provision of the audio from the conference to the first conference participant at or just prior to the point referenced by the stored indication.

32. The method of claim 31, wherein:

detecting a start of an utterance by the first conference participant comprises;

monitoring a volume level of an audio stream sourced from the first conference participant;

comparing the monitored volume level of the audio stream to a first threshold value; and

determining the start of the utterance when the monitored volume level of the audio stream exceeds the first threshold value; and

detecting an end of the utterance by the first conference participant comprises;

monitoring the volume level of the audio stream sourced from the first conference participant;

comparing the monitored volume level of the audio stream to a second threshold value; and

determining the end of the utterance when the monitored volume level of the audio stream falls below the second threshold value for a predefined duration of time.

33. The method of claim 31, wherein detecting a start of an utterance by the first conference participant includes receiving a first selection from the first conference participant to unmute an audio input interface or to pause an audio output, and detecting an end of the utterance by the first conference participant includes receiving a second selection from the first conference participant to mute the audio input interface or to play the audio output.

34. The method of claim 31 further comprising:

in response to detecting the start of the utterance, initiating a recording of the utterance by the first conference participant; and

in response to detecting the end of the utterance, terminating the recording of the utterance by the first conference participant.

35. The method of claim 34 further comprising:

storing a reference to the recorded utterance as an audio clip in a playlist, wherein the playlist includes a plurality of audio clips of utterances by the first conference participant and other conference participants;

automatically categorizing the stored audio clip in the playlist identifying the conference or a subject of the conference; and

automatically tagging the stored audio clip with information identifying the first conference participant.

36. The method of claim 35 further comprising:

receiving user input of data to associate with the stored audio clip in the playlist; and

storing the data with an association to the stored audio clip, wherein the stored data comprises at least one of a subject, description, transcription, keyword, flag, digital file, and uniform resource locator.

37. The method of claim 31, further comprising:

in response to detecting the start of the utterance, storing, in the memory, a first index point corresponding the start of the utterance; and

in response to detecting the end of the utterance, storing, in the memory, a second index point corresponding to the end of the utterance.

38. The method of claim 31, wherein resuming the provision of the audio from the conference to the first conference participant at the point referenced by the stored indication comprises:

accessing a recorded version of the audio from the conference at the point referenced by the stored indication;

playing the recorded version of the audio from the conference from the point referenced by the stored indication at an accelerated rate; and

providing the audio from the conference to the first conference participant in real time when playback of the recorded version terminates.

39. The method of claim 38, wherein the recorded version of the audio from the conference is stored as a plurality of audio clips in a playlist, each audio clip including an utterance by one of a plurality of conference participants, and wherein playing the recorded version of the audio from the conference comprises:

playing the plurality of audio clips sequentially from the point referenced by the stored indication.

40. The method of claim 38, wherein the recorded version of the audio from the conference is stored as a plurality of audio clips in a playlist, each audio clip including an utterance by one of a plurality of conference participants, and wherein playing the recorded version of the audio from the conference comprises:

playing the plurality of audio clips from the point referenced by the stored indication in the same manner in which they were recorded, wherein two or more of the plurality of audio clips are played in an overlapping manner when the corresponding audio from the conference included overlapping utterances from multiple conference participants.

41. The method of claim 31, wherein the utterance is a first utterance, and audio from the conference includes an audio signal recorded from a second conference participant in a location remote from the first conference participant, the method further comprising:

detecting a start of a second utterance by the second conference participant; and

stopping, in response to the detecting the start of the second utterance, providing of the audio from the conference to the second conference participant.

42. The method of claim 41, wherein the first utterance and the second utterance at least partially overlap during a time interval, and the audio from the conference is stopped for both the first conference participant and the second conference participant during the time interval, the method further comprising recording the first utterance during the time interval to obtain a first recording, and recording the second utterance during the time interval to obtain a second recording.

43. The method of claim 42, further comprising:

providing the first recording to the second conference participant in response to detecting an end of the second utterance; and

providing the second recording to the first conference participant in response to detecting the end of the first utterance.

44. The method of claim 31, wherein:

the audio from the conference that is provided to the first conference participant before the detecting of the start of the utterance is live conference audio provided in real time, and

the audio from the conference that is provided to the first conference participant after the detecting of the end of the utterance is recorded conference audio that is played to the first conference participant at an accelerated rate until the first conference participant is caught up to the live conference, the method further comprising;

resuming providing the live conference audio in real time to the first conference participant when the first conference participant is caught up to the live conference.

52. The system of claim 35, wherein the controller circuitry is configured to resume the provision of the audio from the conference to the first conference participant at the point referenced by the stored indication by:

accessing a recorded version of the audio from the conference at the point referenced by the stored indication;

playing the recorded version of the audio from the conference from the point referenced by the stored indication at an accelerated rate; and

providing the audio from the conference to the first conference participant in real time when playback of the recorded version terminates.

53. The system of claim 52, wherein the recorded version of the audio from the conference is stored as a plurality of audio clips in a playlist, each audio clip including an utterance by one of a plurality of conference participants, and wherein the controller circuitry plays the recorded version of the audio from the conference by:

playing the plurality of audio clips sequentially from the point referenced by the stored indication.

54. The system of claim 52, wherein the recorded version of the audio from the conference is stored as a plurality of audio clips in a playlist, each audio clip including an utterance by one of a plurality of conference participants, and wherein the controller circuitry plays the recorded version of the audio from the conference by:

playing the plurality of audio clips from the point referenced by the stored indication in the same manner in which they were recorded, wherein two or more of the plurality of audio clips are played in an overlapping manner when the corresponding audio from the conference included overlapping utterances from multiple conference participants.

45. A system for processing audio content of a conference, the system comprising controller circuitry configured to:

provide audio from the conference to a first conference participant;

detect a start of an utterance by the first conference participant;

in response to detecting the start of the utterance, stop the provision of the audio from the conference to the first conference participant;

store, in a memory, an indication of a point in the audio from the conference at which the provision of the audio from the conference to the first conference participant was stopped;

detect an end of the utterance by the first conference participant; and

in response to detecting the end of the utterance, resume the provision of the audio from the conference to the first conference participant at or just prior to the point referenced by the stored indication.

46. The system of claim 45, wherein the controller circuitry is configured to:

detect the start of the utterance by;

monitoring a volume level of an audio stream sourced from the first conference participant;

comparing the monitored volume level of the audio stream to a first threshold value; and

determining the start of the utterance when the monitored volume level of the audio stream exceeds the first threshold value; and

detect the end of the utterance by;

monitoring the volume level of the audio stream sourced from the first conference participant;

comparing the monitored volume level of the audio stream to a second threshold value; and

determining the end of the utterance when the monitored volume level of the audio stream falls below the second threshold value for a predefined duration of time.

47. The system of claim 45, wherein the controller circuitry detects the start of the utterance by receiving a first selection from the first conference participant to unmute an audio input interface or to pause an audio output, and detects the end of the utterance by receiving a second selection from the first conference participant to mute the audio input interface or to play the audio output.

48. The system of claim 45, wherein the controller circuitry is further configured to:

in response to detecting the start of the utterance, initiate a recording of the utterance by the first conference participant; and

in response to detecting the end of the utterance, terminate the recording of the utterance by the first conference participant.

49. The system of claim 48, wherein the controller circuitry is further configured to:

store a reference to the recorded utterance as an audio clip in a playlist, wherein the playlist includes a plurality of audio clips of utterances by other conference participants;

automatically categorize the stored audio clip in the playlist identifying the conference or a subject of the conference; and

automatically tag the stored audio clip with information identifying the first conference participant.

50. The system of claim 49, wherein the controller circuitry is further configured to:

receive user input of data to associate with the stored audio clip in the playlist; and

store, in the memory, the data with an association to the stored audio clip, wherein the stored data comprises at least one of a subject, description, transcription, keyword, flag, digital file, and uniform resource locator.

51. The system of claim 45, wherein the controller circuitry is further configured to:

in response to detecting the start of the utterance, store, in the memory, a first index point corresponding the start of the utterance; and

in response to detecting the end of the utterance, store, in the memory, a second index point corresponding to the end of the utterance.

55. The system of claim 45, wherein the utterance is a first utterance, and audio from the conference includes an audio signal recorded from a second conference participant in a location remote from the first conference participant, the controller circuitry being further configured to:

detect a start of a second utterance by the second conference participant; and

stop, in response to the detecting the start of the second utterance, providing of the audio from the conference to the second conference participant.

56. The system of claim 45, wherein the first utterance and the second utterance at least partially overlap during a time interval, and the audio from the conference is stopped for both the first conference participant and the second conference participant during the time interval, the controller circuitry being further configured to record the first utterance during the time interval to obtain a first recording, and record the second utterance during the time interval to obtain a second recording.

57. The system of claim 56, wherein the controller circuitry is further configured to:

provide the first recording to the second conference participant in response to detecting an end of the second utterance; and

provide the second recording to the first conference participant in response to detecting the end of the first utterance.

58. The system of claim 45, wherein:

the audio from the conference that is provided to the first conference participant before the start of the utterance is detected is live conference audio provided in real time, and

the audio from the conference that is provided to the first conference participant after the end of the utterance is detected is recorded conference audio that is played to the first conference participant at an accelerated rate until the first conference participant is caught up to the live conference, the controller circuitry further configured to;

resume providing the live conference audio in real time to the first conference participant when the first conference participant is caught up to the live conference.

59. A method for processing conference audio, the method comprising:

receiving, by at least one processor, a first audio signal from a first conference participant, wherein the first audio signal is detected by a microphone;

providing, to the second conference participant, the first audio signal;

in response to detecting a first request to pause the conference audio from the second conference participant, stopping the providing, to the second conference participant, of the first audio signal;

in response to detecting a second request to unpause the conference audio from the second conference participant, providing, to the second conference participant and at an accelerated rate that is faster than real time, a remaining portion of the first audio signal; and

providing, at the accelerated rate and to the second conference participant, a remainder of the conference audio that was recorded after detecting the first request, until the second conference participant is caught up to conference audio in real time.

60. The method of claim 59, wherein the first request to pause the conference audio from the second conference participant includes a request for the second conference participant to play an earlier portion of the conference audio, the method further comprising providing the earlier portion of the conference audio until the second request is detected.

61. The method of claim 59, wherein the conference audio that is provided to the first conference participant includes different content and same content compared to the conference audio that is provided to the second conference participant, and the same content is provided to the first conference participant and the second conference participant at different times.

62. The method of claim 59, further comprising detecting, from the first audio signal, a start and an end of a first utterance by the first conference participant, wherein:

a first portion of the first utterance is provided in real time to the second conference participant before detecting the first request to pause the conference audio; and

the remaining portion of the first audio signal includes a second portion of the first utterance.