Combining multiple bit rate and scalable video coding

• US 8,947,492 B2
• Filed: 06/18/2010
• Issued: 02/03/2015
• Est. Priority Date: 06/18/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lee, Ming-Chieh, Qian, Tin, Moore, Timothy
• Primary Examiner(s): Gauthier, Gerald
• Application Number:
  US12/819,092
• Publication Number:
  US 20110310216A1
• Time in Patent Office: 1,691 Days
• Field of Search: 348/14.01, 348/14.08, 370/254, 370/401, 375/240.03, 375/240.11, 375/240.12, 375/240.16, 375/240.1, 375/240.01, 379/202.01, 709/218, 709/205, 709/231, 709/224, 725/116, 725/95, 725/97, 705/26.41, 705/37, 705/46, 705/91
• US Class Current: 348/14.08
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 21/25833 involving client hardware characteristics, e.g. manufacturer, processing or storage capabilities allocation of resources considering hardware capabilities in multiprogramming arrangements G06F9/5044; allocation of resources considering software capabilities in multiprogramming arrangements G06F9/5055
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Video streams are generated using a combination of Multiple Bit Rate (MBR) encoding and Scalable Video Coding (SVC). Capabilities and requests of the clients are used in determining the video streams to generate as well as what video streams to deliver to the clients. The clients are placed into groups based on a resolution capability of the client. For each resolution grouping, MBR is used for generating spatial streams and SVC is used for generating temporal and quality streams.

Claims

1. A method for encoding video streams using a combination of Multiple Bit Rate (MBR) encoding and Scalable Video Control (SVC) encoding, comprising:

determining a capability associated with clients in a video conference using a processing unit;

wherein the capability relates to an ability of each of the clients to encode one or more video streams;

placing each of the clients into a group using the processing unit that is selected based on the determined capability for each of the clients;

receiving from at least one of the clients in the video conference different streams at different resolutions that are generated by the at least one of the clients that include an MBR stream that is used for spatial scalability and one or more SVC streams relating to at least one of temporal and quality scalability; and

sending at least one of the different streams received from the at least one of the clients in the video conference to at least one of the other clients in the video conference.

2. The method of claim 1, further comprising aggregating capabilities and requests from of each of the clients into a message and providing the message to clients in the video conference that are requested to generate streams within the video conference.

3. The method of claim 1, wherein placing each of the clients into the group comprises determining a supported resolution for each of the clients and grouping the clients according to the supported resolution.

4. The method of claim 1, wherein each layer of a SVC stream is packetized into a separate steam.

5. The method of claim 1, wherein determining the capability associated with each of the clients comprises determining a bandwidth of at least one communication medium associated with each of the clients and determining a processing power for each of the clients.

6. The method of claim 2, wherein the request that is received comprises a desired resolution, a frame rate and a bitrate.

7. The method of claim 1, wherein analyzing the capability associated with each of the clients comprises determining an encoding capability and a decoding capability.

8. The method of claim 2, further comprising determining when conditions change within the video conference and updating the message such that different streams are generated by the one or more clients.

9. The method of claim 1, wherein at least some of the clients have a different resolution.

10. A computer-readable medium, excluding a signal, which stores a set of instructions which when executed performs a method for generating video streams that combine Multiple Bit Rate (MBR) encoding and Scalable Video Coding (SVC), the method executed by the set of instructions comprising:

determining capabilities associated with clients in a video conference;

wherein the capabilities include determining for each client;

a resolution and an encoding ability;

placing each of the clients into a group that is selected based on the determined resolution for each of the clients;

aggregating the capabilities for the clients into a message;

sending the message to clients that are requested to generate streams for the video conference; and

receiving from the clients in the video conference that are requested to generate streams, streams that include an MBR stream that is used for spatial scalability and one or more SVC streams relating to at least one of temporal and quality scalability, wherein at least one of the client generates streams at different resolutions.

11. The computer-readable medium of claim 10, further comprising aggregating requests from of each of the clients into the message;

wherein the requests relate to a viewing request.

12. The computer-readable medium of claim 10, wherein placing each of the clients into the group comprises determining a maximum resolution for each of the clients and grouping the clients according to the maximum resolution.

13. The computer-readable medium of claim 10, wherein each layer of a SVC stream is packetized into a separate steam.

14. The computer-readable medium of claim 10, wherein determining the capability associated with each of the clients comprises determining an available bandwidth for each of the clients.

15. The computer-readable medium of claim 11, wherein the requests comprise a desired resolution, a frame rate and a bitrate.

16. The computer-readable medium of claim 10, wherein determining the capability associated with each of the clients comprises determining a decoding capability.

17. The computer-readable medium of claim 11, further comprising determining when conditions change within the video conference and updating the message such that different streams are generated by the one or more clients.

18. The computer-readable medium of claim 10, wherein at least some of the clients have a different resolution.

19. A system for generating streams that combines Multiple Bit Rate (MBR) encoding and Scalable Video Coding (SVC) encoding in a video conference, the system comprising:

a memory storage;

a network connection device; and

a processing unit coupled to the memory storage, wherein the processing unit is operative to;

register at least one first video conference client and at least one second video conference client operatively connected to the system via the network connection device, wherein being operative to register the at least one first video conference client and the at least one second video conference client comprises being operative to establish capabilities for the at least one first video conference client and the at least one second video conference client, wherein the capabilities comprise;

a resolution;

a frame rate; and

a bit rate;

receive a plurality of video streams at different resolutions from the at least one first video conference client, wherein the plurality of video streams are generated by the first video conference client and comprise a first set of streams that comprise an MBR stream and at least one SVC stream and a second set of video streams that comprise an MBR stream and at least one SVC stream, wherein the at least one first set of streams comprises higher resolution video streams than the second set of video streams.

20. The system of claim 19, further comprising aggregate the capabilities and requests from of each of the clients into a message and provide the message to the clients in the video conference that are requested to generate streams within the video conference.