System and method for dynamically altering videoconference bit rates and layout based on participant activity

• US 7,768,543 B2
• Filed: 03/16/2006
• Issued: 08/03/2010
• Est. Priority Date: 03/09/2006
• Status: Active Grant

• Current Assignee: GOTO Group Incorporated (LogMeIn, Inc.)
• Original Assignee: Citrix Online LLC (Cloud Software Group)
• Inventors: Christiansen, Bernd Oliver
• Primary Examiner(s): Woo; Stella L
• Application Number:
  US11/377,913
• Publication Number:
  US 20070211141A1
• Time in Patent Office: 1,601 Days
• Field of Search: 348/14.08, 348/14.09, 348/14.01, 348/14.07
• US Class Current: 348/14.08
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/4038 with floor control
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Systems and Methods to transmit and simultaneously display all video streams in a video conference by dynamically adapting video streams based on audio volume of the participants include allocating an amount of bandwidth to a participant in a videoconference; determining a level of activity exhibited by the participant during the videoconference, and changing the amount of bandwidth allocated to the participant responsive to the determined level of activity. In another aspect, a method for displaying video streams includes creating a graphical representation of a participant in a videoconference; displaying the created graphical representation; determining a level of activity exhibited by the participant during the videoconference; and altering the displayed graphical representation of the participant responsive to the determined level of activity.

Claims

1. A method of generating a dynamic graphical representation of a participant in a videoconference that reflects the participant'"'"'s level of activity during the videoconference, the method comprising:

(a) creating a graphical representation of a participant in a videoconference;

(b) causing the display of the created graphical representation on a screen;

(c) determining a level of activity exhibited by the participant during the videoconference;

(d) determining a percentage of a total activity level represented by the determined level of activity by comparing the determined level of activity to a sum of all activity levels of all participants in the videoconference; and

(e) causing the displayed graphical representation of the participant to be altered to occupy a percentage of the total screen area equal to the percentage determined in step (d).

2. The method of claim 1 wherein step (a) comprises creating a two-dimensional graphical representation of a participant in a videoconference.

3. The method of claim 1 wherein step (a) comprises creating a three-dimensional graphical representation of a participant in a videoconference.

4. The method of claim 1 wherein step (a) comprises creating a two-dimensional graphical representation of a participant in a videoconference using an image of the participant.

5. The method of claim 1 wherein step (a) comprises creating a two-dimensional video representation of a participant in a videoconference.

6. The method of claim 1 wherein step (c) comprises determining the participant'"'"'s absolute volume level during the videoconference.

7. The method of claim 1 wherein step (c) comprises determining the participant'"'"'s volume level during the videoconference relative to other participants.

8. The method of claim 1 wherein step (c) comprises determining the participant'"'"'s volubility during the videoconference.

9. The method of claim 1 wherein step (c) comprises determining the participant'"'"'s level of animation during the videoconference.

10. The method of claim 1 wherein step (c) comprises determining the participant'"'"'s level of animation during the videoconference relative to other participants.

11. The method of claim 1 further comprising changing the relative position of the displayed graphical representation of the participant responsive to the determined level of activity.

12. The method of claim 1 further comprising changing the color bit depth of the displayed graphical representation of the participant responsive to the determined level of activity.

13. The method of claim 1 further comprising replacing a color graphical representation of the participant with a grayscale representation of the participant responsive to the determined level of activity.

14. The method of claim 1 further comprising replacing a color graphical representation of the participant with a black and white representation of the participant responsive to the determined level of activity.

15. The method of claim 1 further comprising replacing a video representation of the participant with a still image responsive to the determined level of activity.

16. The method of claim 1 further comprising changing a transformation of the displayed graphical representation of the participant responsive to the determined level of activity.

17. The method of claim 1 wherein step (c) comprises determining a deviation in the participant'"'"'s volume level from a mean volume level of all participants in the videoconference.

18. The method of claim 1 wherein step (c) comprises determining an amount of motion in the participant'"'"'s mouth area during the videoconference.

19. An apparatus for generating a dynamic graphical representation of a participant in a videoconference that reflects the activity level of the participant during the videoconference, the apparatus comprising:

an activity monitor for (i) determining a level of activity exhibited by a first participant during a videoconference and (ii) determining a percentage of a total activity level represented by the determined level of activity by comparing the determined level of activity to a sum of all activity levels of all participants in the videoconference; and

a display engine, in communication with the activity monitor, for (i) causing the display of a graphical representation of the first participant on a screen and (ii) causing the displayed graphical representation of the first participant to be altered to occupy a percentage of the total screen area equal to the percentage determined by the activity monitor.

20. The apparatus of claim 19 wherein the display engine creates a two-dimensional graphical representation of the first participant.

21. The apparatus of claim 19 wherein the display engine creates a three-dimensional graphical representation of the first participant.

22. The apparatus of claim 19 wherein the display engine creates a two-dimensional graphical representation of the first participant using an image of the first participant.

23. The apparatus of claim 19 wherein the display engine creates a two-dimensional video representation of the first participant.

24. The apparatus of claim 19 wherein the activity monitor determines the first participant'"'"'s absolute volume level during the videoconference.

25. The apparatus of claim 19 wherein the activity monitor determines the first participant'"'"'s volume level during the videoconference relative to other participants.

26. The apparatus of claim 19 wherein the activity monitor determines the first participant'"'"'s volubility during the videoconference.

27. The apparatus of claim 19 wherein the activity monitor determines the first participant'"'"'s level of animation during the videoconference.

28. The apparatus of claim 19 wherein the activity monitor determines the first participant'"'"'s level of animation during the videoconference relative to other participants.

29. The apparatus of claim 19 wherein the display engine changes the relative position of the displayed graphical representation of the first participant responsive to the determined level of activity.

30. The apparatus of claim 19 wherein the display engine changes the color bit depth of the displayed graphical representation of the first participant responsive to the determined level of activity.

31. The apparatus of claim 19 wherein the display engine replaces a color graphical representation of the first participant with a grayscale representation of the first participant responsive to the determined level of activity.

32. The apparatus of claim 19 wherein the display engine replaces a color graphical representation of the first participant with a black and white representation of the first participant responsive to the determined level of activity.

33. The apparatus of claim 19 wherein the display engine replaces a video representation of the first participant with a still image responsive to the determined level of activity.

34. The apparatus of claim 19 wherein the display engine changes a transformation of the displayed graphical representation of the first participant responsive to the determined level of activity.

35. A method of allocating bandwidth to participants in a videoconference, the method comprising:

(a) allocating an amount of bandwidth to a participant in a videoconference;

(b) determining a level of activity exhibited by the participant during the videoconference;

(c) determining a percentage of a total activity level represented by the determined level of activity by comparing the determined level of activity to a sum of all activity levels of all participants in the videoconference; and

(d) changing a percentage of the total available bandwidth allocated to the participant to be equal to the percentage determined in step (c).

36. The method of claim 35 wherein step (b) comprises determining the participant'"'"'s absolute volume level during the videoconference.

37. The method of claim 35 wherein step (b) comprises determining the participant'"'"'s volume level during the videoconference relative to other participants.

38. The method of claim 35 wherein step (b) comprises determining the participant'"'"'s volubility during the videoconference.

39. The method of claim 35 wherein step (b) comprises determining the participant'"'"'s level of animation during the videoconference.

40. The method of claim 35 wherein step (b) comprises determining the participant'"'"'s level of animation during the videoconference relative to other participants.

41. The method of claim 35 wherein step (b) comprises

(b-a) receiving a communication from another participant in the video conference, the communication including an indication of the activity level exhibited by the another participant;

and

(b-b) comparing an indication of the participant'"'"'s level of activity to the another participant'"'"'s level of activity.

42. The method of claim 35 wherein step (d) comprises increasing the amount of bandwidth allocated to the participant.

43. The method of claim 35 wherein step (d) comprises decreasing the amount of bandwidth allocated to the participant.

44. The method of claim 35 further comprising changing a size of a transmitted graphical representation of the participant responsive to the change in bandwidth allocated to the participant.

45. The method of claim 35 further comprising changing the a color bit depth of the a transmitted graphical representation of the participant responsive to the change in bandwidth allocated to the participant.

46. The method of claim 35 further comprising replacing a color graphical representation of the participant with a grayscale representation of the participant responsive to the change in bandwidth allocated to the participant.

47. The method of claim 35 further comprising replacing a color graphical representation of the participant with a black and white representation of the participant responsive to the change in bandwidth allocated to the participant.

48. The method of claim 35 further comprising replacing a video representation of the participant with a still image responsive to the change in bandwidth allocated to the participant.

49. The method of claim 35 wherein step (b) comprises determining a deviation in the participant'"'"'s volume level from a mean volume level of all participants in the videoconference.

50. The method of claim 35 wherein step (b) comprises determining an amount of motion in the participant'"'"'s mouth area during the videoconference.

51. An apparatus for allocating bandwidth to participants in a videoconference, the apparatus comprising:

an activity monitor for (i) determining a level of activity exhibited by a local participant during a videoconference and (ii) determining a percentage of a total activity level represented by the determined level of activity by comparing the determined level of activity to a sum of all activity levels of all participants in the videoconference; and

a network engine, in communication with the activity monitor, for (i) transmitting data to a remote participant in the videoconference and (ii) changing a percentage of the total available bandwidth consumed in transmitting the data to be equal to the percentage determined by the activity monitor.

52. The apparatus of claim 51 wherein the activity monitor determines the local participant'"'"'s absolute volume level during the videoconference.

53. The apparatus of claim 51 wherein the activity monitor determines the local participant'"'"'s volume level during the videoconference relative to other participants.

54. The apparatus of claim 51 wherein the activity monitor determines the local participant'"'"'s volubility during the videoconference.

55. The apparatus of claim 51 wherein the activity monitor determines the local participant'"'"'s level of animation during the videoconference.

56. The apparatus of claim 51 wherein the activity monitor determines the local participant'"'"'s level of animation during the videoconference relative to other participants.

57. The apparatus of claim 51 further comprising a display engine in communication with the network engine, the display engine for changing a size of a transmitted graphical representation of the local participant responsive to the amount of bandwidth allocated to the local participant.

58. The apparatus of claim 51 further comprising a display engine in communication with the network engine, the display engine for changing a color bit depth of a transmitted graphical representation of the local participant responsive to the amount of bandwidth allocated to the local participant.

59. The apparatus of claim 51 further comprising a display engine in communication with the network engine, the display engine for replacing a color graphical representation of the local participant with a grayscale representation of the local participant responsive to the amount of bandwidth allocated to the local participant.

60. The apparatus of claim 51 further comprising a display engine in communication with the network engine, the display engine for replacing a color graphical representation of the local participant with a black and white representation of the local participant responsive to the amount of bandwidth allocated to the local participant.

61. The apparatus of claim 51 further comprising a display engine in communication with the network engine, the display engine for replacing a video representation of the local participant with a still image responsive to the amount of bandwidth allocated to the local participant.