Automatic mute control for video conferencing

• US 8,681,203 B1
• Filed: 08/20/2012
• Issued: 03/25/2014
• Est. Priority Date: 08/20/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Yin, Li, Zhang, Hong, Wang, Zhe
• Primary Examiner(s): Ramakrishnaiah, Melur
• Application Number:
  US13/589,831
• Time in Patent Office: 582 Days
• Field of Search: 348 1401- 1416, 370260-261, 379/202.01, 709/204
• US Class Current: 348/14.08
• CPC Class Codes: H04M 3/567 Multimedia conference systems
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

In one example, a device executes one or more video communication processes that receive audio streams and video streams from a plurality of computing devices participating in a video communication session associated with the one or more video communication processes. The device evaluates one or more properties of the audio streams, including the volume of an audio signal among the audio streams. The device selects a first group of the audio streams to mute in the video communication session, based at least in part on the one or more properties of the audio streams. The device distributes a second group of the audio streams in the video communication session, while muting the first group of audio streams in the video communication session.

Claims

1. A method comprising:

executing, by a first computing device, one or more video communication processes that receive audio streams and video streams from a plurality of computing devices participating in a video communication session associated with the one or more video communication processes;

evaluating, by the first computing device, one or more properties of the audio streams from the plurality of computing devices, the one or more properties comprising a volume of an audio signal among the audio streams;

receiving, by the first computing device, document data for a document application from a second computing device included in the plurality of computing devices;

selecting, by the first computing device, a first group of the audio streams from the plurality of computing devices to mute in the video communication session, based at least in part on the one or more properties of the audio streams;

outputting, by the first computing device, a second group of the audio streams from the plurality of computing devices for distribution in the video communication session, while muting the first group of audio streams from the plurality of computing devices in the video communication session; and

outputting, by the first computing device, the document data for distribution among the plurality of computing devices to enable the document data to be displayed on at least one of the computing devices;

wherein selecting the first group of the audio streams is based at least in part on selecting one or more of the audio streams other than an audio stream received from the second computing device, and wherein outputting the second group of the audio streams comprises outputting the audio stream received from the second computing device.

2. The method of claim 1, further comprising:

evaluating an average volume of an audio signal of one or more of the audio streams;

identifying one of the audio streams that has a highest average volume of its audio signal from among the audio streams evaluated;

selecting one or more remaining audio streams that do not have the highest average volume for the first group of the audio streams; and

outputting for distribution, in the video communication session, the identified audio stream that has the highest average volume, while muting in the video communication session the selected one or more remaining audio streams that do not have the highest average volume.

3. The method of claim 2, wherein evaluating the average volume of the audio signal in one or more of the audio streams comprises evaluating a volume of the audio signal in each of the audio streams over an interval of time.

4. The method of claim 3, wherein the interval of time is between one second and ten seconds inclusive.

5. The method of claim 1, further comprising:

outputting the video streams received from each of the plurality of computing devices in the video communication session.

6. The method of claim 1, further comprising:

wherein each of the audio streams is based at least in part on audio data captured by an audio input device associated with a respective one of the computing devices, and each of the video streams is based at least in part on video data captured by a video input device associated with a respective one of the computing devices.

7. The method of claim 1, further comprising:

receiving from a third computing device among the plurality of computing devices an input for manipulating a document represented by the document data; and

wherein selecting the first group of the audio streams is based at least in part on selecting one or more of the audio streams other than an audio stream received from the third computing device from which the input for manipulating the document is received, and wherein outputting the second group of the audio streams comprises outputting the audio stream from the third computing device from which the input for manipulating the document is received.

8. The method of claim 1, further comprising:

receiving data on user roles for users associated with one or more of the computing devices;

wherein selecting the first group of the audio streams is based at least in part on the user roles.

9. The method of claim 1, further comprising:

detecting a decrease in volume in an audio signal in the second group of the audio streams being distributed in the video communication session; and

in response to detecting the decrease in volume in the audio signal in the second group of the audio streams, outputting for distribution, in the video communication session, the first group of the audio streams that were previously selected to mute.

10. The method of claim 9, wherein detecting a decrease in volume in the audio signal in the second group of the audio streams comprises evaluating the audio signal in the second group of the audio streams over multiple intervals of time.

11. The method of claim 10, wherein evaluating the audio signal in the second group of the audio streams over multiple intervals of time comprises evaluating the audio signal in the second group of the audio streams over intervals of between one second and ten seconds inclusive.

12. The method of claim 1, further comprising:

detecting an increase in volume in an audio signal in a first audio stream from among the first group of the audio streams; and

in response to detecting the increase in volume in the audio signal in a first audio stream, outputting the first audio stream for distribution in the video communication session.

13. A method comprising:

executing, by a first computing device, one or more video communication processes that receive audio streams and video streams from a plurality of computing devices participating in a video communication session associated with the one or more video communication processes;

evaluating, by the first computing device, one or more properties of the audio streams from the plurality of computing devices, the one or more properties comprising a volume of an audio signal among the audio streams;

selecting, by the first computing device, a first group of the audio streams from the plurality of computing devices to mute in the video communication session, based at least in part on the one or more properties of the audio streams; and

outputting, by the first computing device, a second group of the audio streams from the plurality of computing devices for distribution in the video communication session, while muting the first group of audio streams from the plurality of computing devices in the video communication session;

wherein evaluating one or more properties of the audio streams comprises evaluating one or more of the audio streams to determine whether the audio streams are conveying human speech from a single speaker; and

wherein selecting the first group of the audio streams to mute comprises selecting at least one of the audio streams that is not conveying human speech from a single speaker.

14. The method of claim 13, further comprising:

detecting a decrease in volume in an audio signal in the second group of the audio streams being distributed in the video communication session; and

in response to detecting the decrease in volume in the audio signal in the second group of the audio streams, outputting for distribution, in the video communication session, the first group of the audio streams that were previously selected to mute.

15. A method comprising:

executing, by a first computing device, one or more video communication processes that receive audio streams and video streams from a plurality of computing devices participating in a video communication session associated with the one or more video communication processes;

evaluating, by the first computing device, one or more properties of the audio streams from the plurality of computing devices, the one or more properties comprising a volume of an audio signal among the audio streams;

evaluating, by the first computing device, one or more of the video streams to determine whether the video streams are conveying video data that represents a human figure;

selecting, by the first computing device, a first group of the audio streams from the plurality of computing devices to mute in the video communication session, based at least in part on the one or more properties of the audio streams; and

outputting, by the first computing device, a second group of the audio streams from the plurality of computing devices for distribution in the video communication session, while muting the first group of audio streams from the plurality of computing devices in the video communication session;

wherein selecting the first group of the audio streams to mute comprises selecting at least one of the audio streams from one of the plurality of computing devices with a video stream that is not conveying video data representing a human figure.

16. The method of claim 15, further comprising:

receiving data on user roles for users associated with one or more of the computing devices;

wherein selecting the first group of the audio streams is based at least in part on the user roles.

17. A first computing device, comprising:

at least one processor;

wherein the at least one processor is configured to;

receive audio streams and video streams via a network interface from a plurality of client computing devices participating in a video communication session associated with one or more video communication processes;

receive document data for a document application from a second computing device among the plurality of client computing devices;

evaluate one or more properties of the audio streams from the plurality of client computing devices;

select a first group of the audio streams from the plurality of client computing devices to refrain from outputting for distribution to the client computing devices in the video communication session, based at least in part on one or more properties of the audio streams;

output the video streams for distribution in the video communication session via the network interface;

output the document data for distribution among the plurality of client computing devices to enable the document data to be displayed on the client computing devices; and

output a second group of the audio streams from the plurality of client computing devices for distribution in the video communication session via the network interface, while refraining from distributing the first group of the audio streams in the video communication session;

wherein selecting the first group of the audio streams to refrain from outputting for distribution to the client computing devices in the video communication session is based at least in part on selecting one or more of the audio streams other than an audio stream received from the second computing device, and wherein outputting the second group of the audio streams for distribution comprises outputting the audio stream from the second computing device.

18. The first computing device of claim 17, wherein the at least one processor is further configured to:

evaluate an average volume of an audio signal of one or more of the audio streams;

identify one of the audio streams that has a highest average volume of its audio signal from among the audio streams evaluated;

select one or more remaining audio streams that do not have the highest average volume for the first group of the audio streams; and

output for distribution, in the video communication session, the identified audio stream that has the highest average volume, while refraining from outputting for distribution in the video communication session the selected one or more remaining audio streams that do not have the highest average volume.

19. The first computing device of claim 17, wherein the first computing device is a server.

20. The first computing device of claim 17, wherein the at least one processor is further configured to:

receive, from a third computing device among the plurality of client computing devices, an input for manipulating a document represented by the document data;

wherein selecting the first group of the audio streams to refrain from outputting for distribution to the client computing devices in the video communication session is based at least in part on selecting one or more of the audio streams other than an audio stream received from the third computing device from which the input for manipulating the document is received, and wherein outputting the second group of the audio streams comprises outputting the audio stream from the third computing device from which the input for manipulating the document is received.