Implementing multiple dominant speaker video streams with manual override

• US 8,330,794 B2
• Filed: 06/10/2009
• Issued: 12/11/2012
• Est. Priority Date: 06/10/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ramanathan, Rajesh, Tandon, Ankit, Moore, Tim, Bhattacharjee, Avronil
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US12/482,442
• Publication Number:
  US 20100315484A1
• Time in Patent Office: 1,280 Days
• Field of Search: 348/14.03, 348/14.07, 348/14.08, 348/14.12, 348/14.09, 370/473, 370/503, 379/90.01, 709/231, 709/204
• US Class Current: 348/14.09
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/440263 by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

In a video conference system capable of providing multiple video streams, displayed streams for participants are configured based on automatic selection by the system (such as active speaker) and manual override by the participants. The displayed views are further enhanced by implementing duplicate detection to avoid display of duplicate streams between the automatic selections and manual selections.

Claims

1. A method to be executed at least in part in a computing device for facilitating a multiple stream video conference, the method comprising:

receiving a plurality of video streams each video stream being associated with a participant of the video conference, wherein one of the video streams is designated as current active speaker;

receiving a selection of at least one other participant based on an automatic selection algorithm;

receiving a selection of at least one other participant by a viewing participant;

controlling the video streams through one of;

a media channel and a signaling channel with the participants; and

selecting video streams to be displayed to the viewing participant in addition to the video stream of the current active speaker, wherein video streams associated with the viewing participant selection are prioritized over automatically selected participants, and wherein duplicate video streams are removed from being displayed.

2. The method of claim 1, further comprising:

updating the selected video streams to be displayed in response to one of;

designation of a new current active speaker, selection of a new participant by the viewing participant, and automatic selection of a new participant.

3. The method of claim 1, further comprising:

providing information associated with audio sources contributing to the conference to the viewing participant over an audio channel.

4. The method of claim 3, wherein the viewing participant is enabled to select the at least one other participant based on the information received over the audio channel.

5. The method of claim 1, wherein the selection of the video streams to be displayed is performed by one of a Multipoint Control Unit (MCU) of a conference system facilitating the video conference and a client device associated with the viewing participant.

6. The method of claim 5, wherein a number of video streams to be displayed to the viewing participant is determined based on at least one from a set of:

a processing capacity, an available memory size, a number of currently active programs, a number of currently active video applications, a display resolution, a decoding capability, and a downlink bandwidth related with the client device associated with the viewing participant.

7. The method of claim 1, further comprising:

predictively oversubscribing the viewing participant to multiple video streams to process duplicate detection in real time.

8. The method of claim 1, wherein at least one of the video streams is reserved for one of a panoramic view and a room view.

9. The method of claim 8, further comprising:

performing duplicate detection based on comparing participants visible in the reserved video stream and individual participant video streams.

10. The method of claim 1, wherein one of the video streams is reserved for a local preview in response to activation of the local preview by the viewing participant.

11. The method of claim 1, further comprising:

selecting initial video streams to be displayed to the viewing participant based on at least one from a set of;

selection of participants received from the viewing participant, status of a local preview feature, and a number of initially attending participants.

12. A video conference system for facilitating a multiple stream video conference, the system comprising:

a Multipoint Control Unit (MCU) configured to;

receive a plurality of video streams associated with participants of the video conference;

designate one of the video streams is as current active speaker;

select another video stream associated with another participant based on at least one of a video attribute and an audio attribute of the other participant;

provide the video stream designated as current active speaker and the other video stream to a client application associated with a viewing participant; and

a computing device executing the client application, the client application configured to;

receive a selection of a further participant from the viewing participant;

select video streams to be displayed to the viewing participant in addition to the video stream of the current active speaker, based on prioritizing a video stream associated with the further participant over the other video stream, and wherein duplicate video streams are removed from being displayed; and

bring one of the video streams in a background of the client application to a foreground of the client application subsequent to detecting the background video stream as a duplicate of one of the video streams in the foreground.

13. The system of claim 12, wherein the client application is further configured to:

enable the viewing participant select the video stream associated with the further participant based on information associated with audio sources of the video conference received through an audio channel from the MCU.

14. The system of claim 12, wherein the client application is further configured to:

display the received video streams in split panes; and

indicate the video stream associated with the current active speaker and the video stream associated with the viewing participant selected participant through a graphical scheme.

15. The system of claim 12, wherein the MCU is one of:

a standalone hardware device, a software program executed by a server facilitating the video conference, a combination of a standalone hardware device and a software program, and a decentralized MCU comprising of coordinated devices of the system.

16. A computer-readable storage medium with instructions stored thereon for facilitating a multiple stream video conference, the instructions comprising:

receiving a plurality of video streams associated with participants of the video conference;

controlling the video streams through one of;

a media channel and a signaling channel with the participants;

designating one of the video streams is as current active speaker;

automatically selecting another video stream associated with another participant based on at least one of a video attribute and an audio attribute of the other participant;

receiving a selection of a further video stream associated with a further participant from a viewing participant;

removing duplicate video streams by;

bringing one of the video streams in a background of the client application to a foreground of the client application subsequent to detecting the background video stream as a duplicate of one of the video streams in the foreground;

selecting video streams to be displayed to the viewing participant in addition to the video stream of the current active speaker;

adding a local preview video stream associated with the viewing participant if the local preview is activated by the viewing participant; and

providing the video stream designated as current active speaker and the selected video streams to a client application associated with the viewing participant.

17. The computer-readable storage medium of claim 16, the instructions further comprising:

maintaining a table of records associated with the selected video streams, wherein;

each row of the table represents a video stream with a top row representing a current dominant speaker;

a column of the table includes identifiers associated with each video stream;

another column of the table includes a view pane assigned to a video stream for display to the viewing participant; and

a further column of the table includes a channel selection assigned to the video stream.

18. The computer-readable storage medium of claim 17, wherein rows of the table are shifted down to maintain a history of viewed video streams in response to a new row being inserted to the top of the table representing a new current dominant speaker.

19. The computer-readable storage medium of claim 16, wherein the current dominant speaker is one of:

a current active speaker and a viewing participant selected participant.

20. The computer-readable storage medium of claim 16, wherein the instructions further comprise:

one of predictively subscribing and oversubscribing to multiple video streams based on a video screen sizing selected by the viewing participant.