Method and system for selectively viewing participants of a multimedia network conference

• US 7,668,907 B1
• Filed: 08/16/2001
• Issued: 02/23/2010
• Est. Priority Date: 08/16/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Velayutham, Senthil K., Larsen, Kyle A., Janakiraman, Mathrubootham
• Primary Examiner(s): Nawaz; Asad M
• Application Number:
  US09/931,211
• Time in Patent Office: 3,113 Days
• Field of Search: 709200-225, 709227-248, 370260-265, 379/90, 725/38, 725/54
• US Class Current: 709/204
• CPC Class Codes: H04L 12/1822 Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15

Abstract

In a network multimedia conference, a bridge service that functions as a proxy for a client to join the conference dynamically selects a video stream from multiple video streams originating from the other participants of the conference and sends the selected video stream to the client for viewing. The bridge service monitors conferencing events of each participant and updates conferencing activity state variables of the participant, such as whether the participant is being shown to the client, whether the participant is talking, how long the participant has been shown or not shown, etc. Periodically, a weight assigned to each of the participants is computed based on the state variables, and the video stream of the participant with the highest weight is then selected for viewing by the client.

Claims

1. A system for conducting a multimedia conference, comprising:

a plurality of participants each providing multimedia conferencing data including a video signal and an audio signal;

a client in conference with the plurality of participants, the client capable of receiving the video signal corresponding to one of the plurality of participants at a time;

a participant selection control parameter stored in a memory for tuning a video switching stream behavior, wherein the participant selection control parameter affects an outcome of a weight computation, said participant selection control parameter received when the multimedia conference is set up, said participant control selection parameter having a static display constraint on a selection of a video signal;

a participant state table stored in a memory and indicating an activity state variable for each of the plurality of participants, said activity state variable including values and statistics associated with each of the plurality of participants'"'"' video signal and audio signal; and

a bridge server connected to the plurality of participants through a network and having a point-to-point connection with the client, the bridge server assigning a predetermined weight to at least one of the plurality of participants for a duration specified by the static display constraint, receiving simultaneously the multimedia conferencing data including the video signal from each of the plurality of participants, updating the activity state variable stored in the memory for each of the plurality of participants in the participant state table according to changes in a data information and a control information of each of the plurality of participants'"'"' video signal and audio signal, periodically computing a weight of said each of the plurality of participants based on the activity state variable of said each of the plurality of participants and the participant selection control parameter, identifying a participant having a highest weight among the plurality of participants, and selecting from the received multimedia conferencing data the video signal corresponding to the identified participant having the highest weight for transmission to the client for viewing.

2. The system of claim 1, wherein the plurality of participants and the bridge server are connected through a multicast network.

3. The system of claim 1, wherein the bridge server further transmits to the client an audio stream containing a mixture of the audio signals from the participants of the network conference.

4. The system of claim 1, wherein updating the activity state variable stored in the memory for each participant includes:

determining a shown length of time for which the video signal of said each participant has been shown to the client if said each participant is currently being shown to the client; and

wherein the computing of the weight of said each participant includes;

determining if the shown length of time is less than a first participant selection control parameter of a minimum shown time, and setting the weight to a second participant selection control parameter of a maximum weight if the shown length of time is less than the first participant selection control parameter.

5. The system of claim 4, wherein updating the activity state variable stored in the memory for each participant includes:

determining a talking length of time for which said each participant has been talking; and

wherein computing the weight of said each participant includes;

if the shown length of time is more than the first participant selection control parameter, then determining if the talking length of time is less than a third participant selection control parameter of a minimum shown time if active, and setting the weight to the second participant selection control parameter if the talking length of time is less than the third participant selection control parameter.

6. The system of claim 5, wherein the computing of the weight of said each participant by the bridge server includes:

if the shown length of time is more than the first participant selection control parameter and the talking length of time is more than the third participant selection control parameter, then determining if the shown length of time is less than a fourth participant selection control parameter of an active cycle time indicating an upper limit of time for showing a video signal of an active talking participant, and if the shown length of time is less than the fourth participant selection control parameter, then setting the weight to a value increasing the likelihood that said each participant will be identified as having the highest weight.

7. The system of claim 6, wherein the computing of the weight of said each participant includes:

if the shown length of time is more than the first participant selection control parameter and the talking length of time is more than the third participant selection control parameter and the shown length of time is more than the fourth participant selection control parameter, then setting the weight to a value decreasing the likelihood that said each participant will be identified as having the highest weight.

8. The system of claim 1, wherein the multimedia conferencing data received by the bridge server include a combined video stream having substreams corresponding to the participants, and wherein the bridge server demultiplexes the combined video stream into a plurality of individual video signals each including one of the substreams in the combined video stream.

9. The system of claim 1, wherein the participant selection control parameter includes at least one member of a group comprising:

a maximum weight;

a minimum weight;

a minimum time that a selected participant'"'"'s video signal will be displayed on a client screen;

a minimum time that a selected participant'"'"'s video signal will be displayed on the client screen if the participant is still talking;

a time period between computing weights and switching video;

a time period that an actively talking participant'"'"'s video signal is displayed on the client screen if only the actively talking participant is talking;

a time period that each participant'"'"'s video signal is displayed if none of the participants is talking;

a value used to enforce a time period that the actively talking participant'"'"'s video signal is displayed on the client screen if only the actively talking participant is talking, said value being less than a maximum time that can elapse before a participant who is not talking is selected;

a value used to enforce a time period that each participant'"'"'s video signal is displayed on the client screen if none of the participants is talking, said value being less than the time period that each participant'"'"'s video signal is displayed if none of the participants is talking; and

a maximum time that can elapse before a participant who is not talking is selected.

10. The system of claim 1, wherein the activity state variables includes at least one of a group comprising:

a time value for which the participant'"'"'s video signal was last displayed on a client screen;

a time value for which the participant'"'"'s video signal has been displayed on the client screen;

a time value since the participant started talking;

a time value since the participant stopped talking;

a Boolean value indicating whether the participant is currently talking;

a Boolean value indicating whether the participant is currently sending a video signal; and

a Boolean value indicating whether the participant'"'"'s video signal is currently being displayed on the client screen.

11. A method for selecting one video signal from a plurality of video signals for forwarding to a client, each video signal corresponding to a participant of multiple participants of a multimedia conference, said method comprising:

when the multimedia conference is being set up, receiving a participant selection control parameter for the multimedia conference, said participant selection control parameter having a static display constraint of selecting the one video signal;

assigning a predetermined weight to at least one of the multiple participants for a duration specified by the static display constraint;

receiving simultaneously a multimedia conferencing data from the multiple participants, the multimedia conferencing data including the plurality of video signals from the multiple participants;

monitoring participant events of the multimedia conference, said participant events associated with the multimedia conferencing data of the multiple participants, said participant events being generated in response to changes in a data information and a control information of the multimedia conferencing data received from the multiple participants;

providing a participant state table associated with the multimedia conference indicating an activity state variable for each participant of the multimedia conference, said activity state variable including values and statistics associated with the participant'"'"'s multimedia conference data;

updating at least one of the activity state variables in the participant state table according to the participant events;

periodically computing a weight for each of the multiple participants based on the activity state variable of said each participant and the participant selection control parameter;

identifying a participant having a highest weight among the multiple participants; and

selecting from the received multimedia conferencing data the one video signal corresponding to the identified participant having the highest weight for viewing by the client.

12. The method of claim 11, wherein the multiple participants are connected to a bridge server through a multicast network.

13. The method of claim 12, further comprising transmitting to the client an audio stream containing a mixture of audio signals from the multiple participants of the network conference.

14. The method of claim 11, wherein the computing the weight comprises:

determining whether each participant'"'"'s activity state variable indicates the participant is currently being shown to the client, wherein if a first participant'"'"'s activity state variable indicates the first participant is not being shown to the client and the first participant is talking, a value for the participant selection control parameter for a maximum time that can elapse before a second participant who is not talking is selected is added to the first participant'"'"'s weight.

15. The method of claim 11, wherein the updating the activity state variable comprises:

determining a value for the activity state variable for a length of time for which said each participant has been shown to the client if said each participant is currently being shown; and

determining whether said each participant is talking;

wherein the computing the weight comprises;

if the participant is not talking and the value of the activity state variable is less than a first value for the participant selection control parameter for a time period that each participant'"'"'s video signal is displayed if none of the participants is talking, setting the weight to a second value for the participant selection control parameter for enhancing a likelihood that the participant will be selected; and

if the participant is not talking and the value of the activity state variable is greater than the first value for the participant selection control parameter, setting the weight to a third value for the participant selection control parameter for enhancing a likelihood that the participant will not be selected.

16. The method of claim 11, wherein the updating the activity state variable comprises:

determining a value for the activity state variable indicating whether said each participant is not sending video; and

wherein computing the weight for each of the participants comprises;

if the value for the activity state variable indicates said each participant is not sending video, setting the weight to a value of the participant selection control parameter for a minimum weight.

17. The method of claim 11, wherein the updating the activity state variable comprises:

determining a value for the activity state variable for a length of time for which said each participant has been shown to the client; and

wherein computing the weight for each of the participants comprises;

if the value for the activity state variable is less than a first value for the participant selection control parameter for a minimum time that a selected participant'"'"'s video signal will be displayed on the client screen, setting the weight to a second value of the participant selection control parameter for a maximum weight.

18. The method of claim 11, wherein the multimedia conference data includes a combined video stream containing multiple substreams, each substream corresponding to a video signal of one of the multiple participants, and wherein the receiving includes demultiplexing the combined video stream into the plurality of video signals, each video signal including one of the substreams of the combined video stream.

19. The method of claim 11, wherein the participant selection control parameter includes at least one member of a group comprising:

a maximum weight;

a minimum weight;

a minimum time that a selected participant'"'"'s video signal will be displayed on a client screen;

a minimum time that a selected participant'"'"'s video signal will be displayed on the client screen if the participant is still talking;

a time period between computing weights and switching video;

a time period that an actively talking participant'"'"'s video signal is displayed on the client screen if only the actively talking participant is talking;

a time period that each participant'"'"'s video signal is displayed if none of the participants is talking;

a value used to enforce a time period that the actively talking participant'"'"'s video signal is displayed on the client screen if only the actively talking participant is talking, said value being less than a maximum time that can elapse before a participant who is not talking is selected;

a value used to enforce a time period that each participant'"'"'s video signal is displayed on the client screen if none of the participants is talking, said value being less than the time period that each participant'"'"'s video signal is displayed if none of the participants is talking; and

a maximum time that can elapse before a participant who is not talking is selected.

20. The method of claim 11, wherein the activity state variables includes at least one of a group comprising:

a time value for which the participant'"'"'s video signal was last displayed on a client screen;

a time value for which the participant'"'"'s video signal has been displayed on the client screen;

a time value since the participant started talking;

a time value since the participant stopped talking;

a Boolean value indicating whether the participant is currently talking;

a Boolean value indicating whether the participant is currently sending a video signal; and

a Boolean value indicating whether the participant'"'"'s video signal is currently being displayed on the client screen.