Video and audio tagging for active speaker detection

• US 9,065,971 B2
• Filed: 12/19/2012
• Issued: 06/23/2015
• Est. Priority Date: 12/19/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Verthein, William George, Leorin, Simone
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US13/719,314
• Publication Number:
  US 20140168352A1
• Time in Patent Office: 916 Days
• Field of Search: 348/14.08, 348/515, 375/240.01, 375/240.16, 386/241, 704/233, 704/251, 704/8, 704/273, 725/35, 725/18, 725/34, 725/136, 340/12.51, 340/539.11, 370/493, 379/142.17, 379/168, 382/165, 463/35, 463/42
• US Class Current: 1/1
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/155 involving storage of or access to video conference sessions tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831

Abstract

A videoconferencing system is described that is configured to select an active speaker while avoiding erroneously selecting a microphone or camera that is picking up audio or video from a connected remote signal. A determination is made whether an audio signal is above a threshold level. If so, then a determination is made as to whether a tag is present in that audio signal. If so, that signal is ignored. If not, a camera is directed toward the sound source identified by the audio signal. A determination is made whether a tag is present in the video signal from that camera. If so, the camera is redirected. If not, local tag(s) are inserted into the audio signal and/or the video signal. The tagged signal(s) are transmitted. Thus, system will ignore sound or video that has an embedded tag from another videoconferencing system.

Claims

1. A transmitter system for a videoconferencing system, comprising:

a tag generator to generate an audio tag;

a combiner to combine an audio signal with the audio tag to produce a tagged audio signal; and

a transmitter to transmit the tagged audio signal and a corresponding video signal; and

a control system operative to;

determine whether the audio signal is above a threshold level;

if the audio signal has been determined to be above the threshold level, then determine whether the audio signal has an audio tag embedded therein; and

if the audio signal has been determined not to have an audio tag embedded therein, then either direct a camera toward a source of the audio signal or select a camera pointing toward a source of the audio signal, wherein the camera produces the corresponding video signal.

2. The transmitter system of claim 1 and further comprising a control system to receive a plurality of audio signals, and to select one of the received audio signals as the audio signal to be combined with the audio tag, wherein the control system selects one of the received audio signals which does not contain an audio tag from another videoconferencing system.

3. The transmitter system of claim 1 wherein distortion of the audio signal caused by the audio tag is below a predetermined level.

4. The transmitter system of claim 1 wherein the control system is further operative to:

determine whether the corresponding video signal has a video tag embedded therein; and

if the video signal has been determined to have a video tag embedded therein, then either redirect the camera to a previous source of an audio signal or select a previously-selected camera.

5. The transmitter system of claim 4 wherein the control system is further operative to:

if the corresponding video signal has been determined not to have a video tag embedded therein, then instruct the combiner to combine the audio signal with the audio tag to produce the tagged audio signal and instruct the transmitter to transmit the tagged audio signal and the corresponding video signal.

6. The transmitter system of claim 1 and further comprising:

a video tag generator;

a video combiner to combine the corresponding video signal with the video tag to produce a tagged video signal; and

wherein the transmitter also transmits the tagged video signal.

7. A method for operating a videoconferencing system, the method comprising:

receiving an audio signal;

receiving a corresponding video signal;

generating an audio tag;

determining whether the audio signal is above a threshold level;

if the audio signal has been determined to be above the threshold level, then determining whether the audio signal has an audio tag embedded therein; and

if the audio signal has been determined not to have an audio lag embedded therein, then

either directing a camera toward a source of the audio signal or selecting a camera pointing toward a source of the audio signal, wherein the camera produces the corresponding video signal;

combining the audio signal with the audio tag to produce a tagged audio signal; and

transmitting the tagged audio signal and the corresponding video signal.

8. The method of claim 7 wherein there are a plurality of received audio signals, and further comprising selecting an audio signal for combining which does not contain an audio tag from another videoconferencing system.

9. The method of claim 7 wherein distortion of the audio signal caused by the audio tag is below a predetermined level.

10. The method of claim 7 and further comprising, prior to combining the audio signal with the audio tag:

determining whether the corresponding video signal has a video tag embedded therein; and

if the video signal has been determined to have a video tag embedded therein, then either redirecting the camera to a previous source of an audio signal or selecting a previously-selected camera.

11. The method of claim 10 and further comprising, prior to combining the audio signal with the audio tag:

if the corresponding video signal has been determined not to have a video tag embedded therein, then performing the combining and transmitting.

12. The method of claim 11 and further comprising:

prior to the transmitting, generating a video tag, and combining the corresponding video signal with the video tag to produce a tagged video signal; and

wherein transmitting the corresponding video signal comprises transmitting the tagged video signal.

13. A computer storage medium having computer executable instructions stored thereon which, when executed by a computer, cause the computer to:

determine whether a received audio signal is above a threshold level;

if the received audio signal has been determined to be above the threshold level, the determine whether the received audio signal has an audio tag embedded therein;

if the received audio signal has been determined not to have an audio lag embedded therein, then

either direct a camera toward a source of the received audio signal or select a camera pointing toward a source of the received audio signal, wherein the camera produces a corresponding video signal;

generate an audio tag;

combine the received audio signal with the audio tag to produce a tagged audio signal; and

transmit the tagged audio signal and the corresponding video signal.

14. The computer storage medium of claim 13 wherein the computer storage medium further comprises computer executable instructions stored thereon which, when executed by a computer, cause the computer to detect audio tags from a remote videoconferencing system in a plurality of received audio signals, and to select, as the audio signal to be combined with the audio tag, one of the received audio signals which does not contain an audio tag from another videoconferencing system.

15. The computer storage medium of claim 13 wherein the computer storage medium further comprises computer executable instructions stored thereon which, when executed by a computer, further cause the computer, prior to combining the audio signal with the audio tag, to:

determine whether the corresponding video signal has a video tag embedded therein; and

if the corresponding video signal has been determined to have a video tag embedded therein, then either redirect the camera to a previous source of an audio signal or select a previously-selected camera.

16. The computer storage medium of claim 15 wherein the computer storage medium further comprises computer executable instructions stored thereon which, when executed by a computer, further cause the computer, prior to combining the audio signal with the audio tag, to:

if the corresponding video signal has been determined not to have a video tag embedded therein, then performing the combining and transmitting.

17. The computer storage medium of claim 13 wherein the computer storage medium further comprises computer executable instructions stored thereon which, when executed by a computer, further cause the computer to:

prior to the transmitting, generate a video tag, and combine the corresponding video signal with the video tag to produce a tagged video signal; and

wherein to transmit the corresponding video signal comprises to transmit the tagged video signal.