System and method for managing a mute button setting for a conference call

• US 10,776,073 B2
• Filed: 10/08/2018
• Issued: 09/15/2020
• Est. Priority Date: 10/08/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Lenke, Nils, Montague, Eric, Ganong, III, William F.
• Primary Examiner(s): Chin, Vivian C
• Assistant Examiner(s): Odunukwe, Ubachukwu A
• Application Number:
  US16/154,439
• Publication Number:
  US 20200110572A1
• Time in Patent Office: 708 Days
• Field of Search: 381107, 381 91, 381 92, 381 941, 381110, 381 56, 381 57, 379 8802
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 51/10 Multimedia information
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A system, method and computer-readable storage device are disclosed for managing a mute and unmute feature on a device which is used to communicate data in a communication conference. The method includes detecting, when the device is set to mute, whether the user is speaking and whether the speech is meant for the conference. Background noises are distinguished from the speech of the user. If the user is speaking and the device is set to mute, the device will automatically switch to and unmute setting such that people in the indication conference can hear the user speak. Facial recognition, and gaze detection or other data can also be used to determine when to automatically mute or unmute the device and can aid in inferring an intent of the user to speak to the conference participants.

Claims

1. A method comprising:

establishing, via a network-based communication server, a communication conference between at least a first user having a first device and a second user having a second device;

setting a mute feature for the first device associated with the first user such that the mute feature is on;

detecting, via the network-based communication server and while the first device has the mute feature set, whether the first user is speaking and whether, based on data from one or more of a voice detection module, a facial recognition module, a head orientation detection module, a gaze detection module, a background noise module, a motion detection module, an audio volume module or a speaker identification module, the first user intends to communicate with the second user in the communication conference, to yield a determination; and

when the determination indicates that the first user is speaking and the determination indicates that the first user intends to speak to the second user in the communication conference, automatically setting, via the network-based communication server, the mute feature to off for the first device to enable the second user to hear the first user in the communication conference.

2. The method of claim 1, wherein the step of detecting whether the first user is speaking further comprises determining an identification of the first user.

3. The method of claim 1, wherein detecting whether the first user is speaking further comprises using a voice activation detection module which also determines whether the speaking by the first user is intended for the communication conference.

4. The method of claim 1, wherein detecting whether the first user is speaking further comprises distinguishing between speech from the first user and a background noise.

5. The method of claim 1, wherein detecting whether the first user is speaking is based at least in part on facial orientation.

6. A method comprising:

establishing, via a network-based communication server, a communication conference between at least a first user having a first device and a second user having a second device;

setting a mute feature for the first device associated with the first user such that the mute feature is off;

detecting, via the network-based communication server, while the first device has the mute feature off and based at least in part on a facial orientation, whether background noise exists in the communication conference at a predetermined threshold to yield a determination; and

when the determination indicates that the background noise exists at the predetermined threshold, automatically setting, via the network-based communication server, the mute feature to on for the first device to prevent the second user from hearing sounds from the first device in the communication conference.

7. The method of claim 6, wherein the sounds comprise the background noise and wherein automatically setting the mute feature to on to prevent the second user from hearing sounds from the first device in the communication conference further comprises initiating a filter associated with the background noise.

8. The method of claim 6, wherein the step of detecting whether the background noise exists in the communication conference at the predetermined threshold is performed on one of the first device and a network-based server.

9. The method of claim 6, wherein the step of detecting whether the background noise exists in the communication conference at the predetermined threshold further comprises applying a machine learning model to an environment of the first device.

10. The method of claim 6, wherein the step of detecting whether the background noise exists in the communication conference at the predetermined threshold further comprises distinguishing between speech from the first user and the background noise.

11. A network-based conferencing system comprising:

a processor; and

a computer-readable storage medium having instructions stored which, when executed by the processor, cause the processor to perform operations comprising;

establishing a communication conference between at least a first user having a first device and a second user having a second device;

setting a mute feature on the first device associated with the first user such that the mute feature is on;

detecting, while the first device has the mute feature set, whether the first user is speaking and whether, based on data from one or more of a voice detection module, a facial recognition module, a head orientation detection module, a gaze detection module, a background noise module, a motion detection module, an audio volume module or a speaker identification module, the first user intends to communicate with the second user in the communication conference, to yield a determination; and

when the determination indicates that the first user is speaking and the determination indicates that the first user intends to speak to the second user in the communication conference, automatically setting the mute feature to off to enable the second user to hear the first user in the communication conference.

12. The network-based conferencing system of claim 11, wherein the step of detecting whether the first user is speaking is performed on one or more of the first device and a network-based server.

13. The network-based conferencing system of claim 11, wherein detecting whether the first user is speaking further comprises using a voice activation detection module.

14. The network-based conferencing system of claim 11, wherein detecting whether the first user is speaking further comprises distinguishing between speech from the first user and a background noise.

15. The network-based conferencing system of claim 11, wherein detecting whether the first user is speaking is based at least in part on one or more of facial recognition data, gaze detection data, background noise, motion detection, and audio volume data.

16. The network-based conferencing system of claim 11, wherein automatically setting the mute feature to off to enable the second user to hear the first user in the communication conference further comprises partially, in a timed manner or fully setting the mute feature to off.

17. The network-based conferencing system of claim 16, wherein automatically setting the mute feature to off to enable the second user to hear the first user in the communication conference partially, in the timed manner or fully setting the mute feature to off is based on a confidence level associated with the determination.