Optimizing virtual collaboration sessions for mobile computing devices

• US 8,789,094 B1
• Filed: 03/19/2012
• Issued: 07/22/2014
• Est. Priority Date: 06/16/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Reddappagari, Param, Singh, Ujjwal, Singhal, Nikhyl P., Cortes, Marcelo Enrique Stockl
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): Tilahun, Alazar
• Application Number:
  US13/424,081
• Time in Patent Office: 855 Days
• Field of Search: 348/14.01, 348/14.02, 348/14.03, 348/14.08, 348/14.09, 709204-207, 715733-759, 725/62, 725/37, 345/661
• US Class Current: 725/37
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: H04L 12/1813 for computer conferences, e.g. chat rooms instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Aspects of the present disclosure provide techniques that may optimize a virtual collaboration session for mobile computing devices. In one example, a method includes receiving, using one or more computing devices, a first video stream sent from a first client device using a communication session, and receiving a second video stream sent from a second client device using the communication session. The method may also include receiving a request sent from the first client device using the communication session to select the first video stream as a primary video, and providing a third video stream based at least on the primary video stream and the secondary video stream to the first client device using the communication session, wherein the third video stream includes the primary video stream.

Claims

1. A method comprising:

receiving, using one or more computing devices, a first video stream sent from a first client device of a plurality of client devices using a communication session, wherein each of the plurality of client devices is associated with at least one participant;

receiving, using the one or more computing devices, a second video stream sent from a second client device of the plurality of client devices using the communication session, wherein the first client device, the second client device, and the one or more computing devices are each communicatively coupled to the communication session;

receiving, using the one or more computing devices, a request sent from the first client device using the communication session to select the second video stream as a primary video stream;

determining, using the one or more computing devices, whether an amount of bandwidth available to the first client device is less than a first threshold value and a second threshold value, wherein the first threshold value is a video threshold and the second threshold value is a still image threshold; and

providing, using the one or more computing devices, a third video stream based at least on the primary video stream and the secondary video stream to the first client device using the communication session, wherein the third video stream includes the primary video stream, wherein the third video stream is a combined stream based at least on whether the amount of bandwidth available to the first client device is less than the first threshold value and the second threshold value, wherein the communication session allows for a different display or stream from each of the plurality of client devices to be concurrently provided to the first client device, including at least two of;

a video stream display, an audio stream, and a still image display, based upon, at least in part, the amount of bandwidth available to the first client device.

2. The method of claim 1, further comprising:

receiving, using the one or more computing devices, from the first client device, a primary audio stream associated with the primary video stream using the communication session;

receiving, using the one or more computing devices, from the second client device, a secondary audio stream associated with the secondary video stream using the communication session;

providing, using the one or more computing devices, the combined stream based at least on the primary audio stream, the secondary audio stream, the primary video stream, and the secondary video stream; and

outputting, using the one or more computing devices, the combined stream to the first client device using the communication session.

3. The method of claim 2, further comprising:

determining, using the one or more computing devices, that the amount of bandwidth available to the first client device is less than the first threshold value; and

generating, using the one or more computing devices, the combined stream based on the primary audio stream and at least one of a still image associated with the primary video stream and a still image associated with the secondary video stream.

4. The method of claim 3, further comprising:

determining, using the one or more computing devices, that the amount of bandwidth available to the first client device is less than the second threshold value, wherein the second threshold value is less than the first threshold value; and

generating, using the one or more computing devices, the combined stream based only on the primary audio stream.

5. The method of claim 1, further comprising:

determining, using the one or more computing devices, the amount of bandwidth available to the first client device; and

if the amount of available bandwidth is less than the first threshold value, generating, using the one or more computing devices, the third video stream based on at least one of a still image associated with the primary video and a still image associated with the secondary video stream.

6. The method of claim 1, further comprising:

receiving, using the one or more computing devices from the first client device, a change request that indicates a selection of the second video stream as the primary video stream and the first video stream as the secondary video stream.

7. The method of claim 1, wherein providing the third video stream further comprises:

determining, using the one or more computing devices, a resolution of the primary video stream based at least on display configuration information received from the first client device, wherein the third video stream is generated at the determined resolution.

8. A method comprising:

providing, using a first computing device, a first video stream and a still image associated with a second video stream for display on a presence-sensitive screen of the first computing device, wherein the first computing device and a second computing device are each communicatively coupled to a communication session including a plurality of computing devices, wherein each of the plurality of computing devices is associated with at least one participant, and wherein the first video stream is received from the second computing device using the communication session;

detecting a gesture at the presence-sensitive screen of the first computing device, wherein the gesture corresponds to a selection of the second video stream;

determining, using the first computing device, whether an amount of bandwidth available to the first computing device is less than a first threshold value and a second threshold value, wherein the first threshold value is a video threshold and the second threshold value is a still image threshold; and

providing, using the first computing device, based at least on whether the amount of bandwidth available to the first computing device is less than the first threshold value and the second threshold value, the second video stream and a still image associated with the first video stream for display on the presence-sensitive screen, wherein the communication session allows for a different display or stream from each of the plurality of computing devices to be concurrently provided to the first computing device, including at least two of;

a video stream display, an audio stream, and a still image display, based upon, at least in part, the amount of bandwidth available to the first computing device.

9. The method of claim 8, further comprising:

responsive to detecting the gesture, sending, to a server comprising one or more computing devices, a request to select the second video stream based at least on the detected gesture, wherein the first computing device and the server are communicatively coupled using the communication session; and

receiving, using the first computing device from the server using the communication session, a third video stream that includes the second video stream and the still image associated with the first video stream, wherein the third video stream is a combined stream based at least on whether the amount of bandwidth available to the first computing device is less than the first threshold value and the second threshold value.

10. The method of claim 8, further comprising:

capturing video data with an input device of the first computing device;

sending the video data as a third video stream to a server using the communication session;

detecting, using the first computing device, a second gesture at the presence-sensitive screen;

responsive to detecting a second gesture, capturing second video data with a different second input device of the first computing device; and

sending the second video data as the third video stream to the server using the communication session, wherein the third video stream is a combined stream based at least on whether the amount of bandwidth available to the first computing device is less than the first threshold value and the second threshold value.

11. The method of claim 8, further comprising:

capturing audio data using an audio input device of the first computing device;

capturing video data using a video input device of the first computing device; and

sending the audio data and the video data as a combined stream to a server, based at least on whether the amount of bandwidth available to the first computing device is less than the first threshold value and the second threshold value, wherein the first computing device and the server are communicatively coupled using the communication session.

12. The method of claim 11, further comprising:

determining that the amount of bandwidth available to the first computing device is less than a threshold value; and

sending the audio data but not the third video data as the combined stream to the server using the communication session.

13. A non-transitory computer-readable storage medium comprising instructions for causing at least one processor of one or more computing devices to perform operations comprising:

receiving a first video stream sent from a first client device of a plurality of client devices using a communication session, wherein each of the plurality of client devices is associated with at least one participant;

receiving a second video stream sent from a second client device of the plurality of client devices using the communication session, wherein the first client device, the second client device, and the one or more computing devices are communicatively coupled to the communication session;

receiving a request sent from the first client device using the communication session to select the second video stream as a primary video stream;

determining whether an amount of bandwidth available to the first client device is less than a first threshold value and a second threshold value, wherein the first threshold value is a video threshold and the second threshold value is a still image threshold; and

providing a third video stream based at least on the primary video stream and the secondary video stream to the first client device using the communication session, wherein the third video stream includes the primary video stream, wherein the third video stream is a combined stream based at least on whether the amount of bandwidth available to the first client device is less than the first threshold value and the second threshold value, wherein the communication session allows for a different display or stream from each of the plurality of client devices to be concurrently provided to the first client device, including at least two of;

a video stream display, an audio stream, and a still image display, based upon, at least in part, the amount of bandwidth available to the first client device.

14. The non-transitory computer-readable storage medium of claim 13, the operations further comprising:

receiving, using the one or more computing devices from the first client device, a primary audio stream associated with the primary video stream using the communication session; and

receiving, using the one or more computing devices from the second client device, a secondary audio stream associated with the secondary video stream using the communication session.

15. The non-transitory computer-readable storage medium of claim 14, the operations further comprising:

determining, using the one or more computing devices, whether the amount of bandwidth available to the first client device is

     1) less than the first threshold value and greater than the second threshold value, or

     2) less than the second threshold value, wherein the second threshold value is less than the first threshold value; and

when the amount of bandwidth available to the first client device is less than the first threshold value and greater than the second threshold value, generating the combined audio and video stream based on the primary audio stream and at least one of a still image associated with the primary video stream and a still image associated with the secondary video stream;

when the amount of bandwidth available to the first client device is less than the second threshold value, generating the combined audio and video stream based only on the primary audio stream.

16. A non-transitory computer-readable storage medium comprising instructions for causing at least one processor of a first computing device to perform operations comprising:

providing, using the first computing device, a first video stream and a still image associated with a second video stream for display on a presence-sensitive screen of the first computing device, wherein the first computing device and a second computing device are each communicatively coupled to a communication session including a plurality of computing devices, wherein each of the plurality of computing devices is associated with at least one participant, and wherein the first video stream is received from the second computing device using the communication session;

detecting a gesture at the presence-sensitive screen of the first computing device, wherein the gesture corresponds to a selection of the second video stream;

determining whether an amount of bandwidth available to the first computing device is less than a first threshold and a second threshold, wherein the first threshold value is a video threshold and the second threshold value is a still image threshold; and

providing, using the first computing device, based at least on whether the amount of bandwidth available to the first computing device is less than the first threshold value and the second threshold value, the second video stream and a still image associated with the first video stream for display on the presence-sensitive screen, wherein the communication session allows for a different display or stream from each of the plurality of computing devices to be concurrently provided to the first computing device, including at least two of;

a video stream display, an audio stream, and a still image display, based upon, at least in part, the amount of bandwidth available to the first computing device.

17. The non-transitory computer-readable storage medium of claim 16, the operations further comprising:

capturing audio data using an audio input device of the first computing device;

capturing video data using a video input device of the first computing device; and

sending the audio data and the video data as a combined stream to a server, based at least on whether the amount of bandwidth available to the first computing device is less than the first threshold value and the second threshold value, wherein the first computing device and the server are communicatively coupled using the communication session.

18. A server comprising one or more computing devices, the one or more computing devices being configured to perform a method of:

receiving a first video stream sent from a first client device of a plurality of client devices using a communication session, wherein each of the plurality of client devices is associated with at least one participant;

receiving a second video stream sent from a second client device of the plurality of client devices using the communication session, wherein the first client device, the second client device, and the server are each communicatively coupled to the communication session;

receiving a request sent from the first client device using the communication session to select the second video stream as a primary video stream;

determining whether an amount of bandwidth available to the first client device is less than a first threshold value and a second threshold value, wherein the first threshold value is a video threshold and the second threshold value is a still image threshold; and

providing a third video stream based at least on the primary video stream and the secondary video stream to the first client device using the communication session, wherein the third video stream includes the primary video stream, wherein the third video stream is a combined stream based at least on whether the amount of bandwidth available to the first client device is less than the first threshold value and the second threshold value, wherein the communication session allows for a different display or stream from each of the plurality of client devices to be concurrently provided to the first client device, including at least two of;

a video stream display, an audio stream, and a still image display, based upon, at least in part, the amount of bandwidth available to the first client device.

19. The server of claim 18, the method further comprising:

receiving, from the first client device, a primary audio stream associated with the primary video stream using the communication session; and

receiving, from the second client device, a secondary audio stream associated with the secondary video stream using the communication session; and

providing a combined stream based at least on the primary audio stream, the secondary audio stream, the primary video stream, and the secondary video stream; and

outputting the combined stream to the first client device using the communication session.

20. The server of claim 19, the method further comprising:

determining, using the one or more computing devices, that the amount of bandwidth available to the first client device is less than the first threshold value; and

generating the combined stream based on the primary audio stream and at least one of a still image associated with the primary video stream and a still image associated with the secondary video stream.