Composite video streaming using stateless compression

• US 9,998,749 B2
• Filed: 10/19/2010
• Issued: 06/12/2018
• Est. Priority Date: 10/19/2010
• Status: Active Grant

• Current Assignee: OTOY, Inc.
• Original Assignee: OTOY, Inc.
• Inventors: Urbach, Julian Michael
• Primary Examiner(s): AN, SHAWN S
• Application Number:
  US12/907,906
• Publication Number:
  US 20120093214A1
• Time in Patent Office: 2,793 Days
• Field of Search: 37524001, 37524024, 37524002, 37524008, 37524026, 382239, 382243, 382236, 382244
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: H04N 19/507 using conditional replenishment
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A video rendering and streaming methodology that utilizes stateless video compression and video image segmentation to achieve enhanced video compression. In some implementations, the video compression and streaming techniques described herein can be deployed to allow for delivery of high-definition video games to client devices that host a standard browser.

Claims

1. A method comprising:

executing, by a processor, an application that provides a video output for transmission to a client, the client being connected to a plurality of socket connections established between the client and the processor;

generating, by the processor, the video output of the application, the video output comprising a composite video image;

dividing, by the processor, a full video frame corresponding to the composite video image into a plurality of unique frame regions, each of the divided unique regions of the full video frame corresponding to a respective socket connection of the plurality of socket connections established for the same client and the number of divided unique regions corresponding to the number of the plurality of socket connections, wherein the divided unique regions comprise a target frame region and one or more positional frame regions positioned and rendered adjacent to the target frame region;

rendering, by the processor, a plurality of separate video streams, wherein each of the separate video streams corresponds to a respective one of the plurality of divided unique frame regions of the composite video image, and wherein each of the separate video streams is executed as a separate thread to write to each of the plurality of socket connections established for the same client in parallel with other separate video streams corresponding to other divided unique frame regions;

rendering, by the processor, a movement of the target frame region relative to the composite video image, wherein rendering comprises determining changes to a height or width of the one or more positional frame regions adjacent to the target frame region and transmitting a new frame having a new height and width over respective socket connections corresponding to the one or more positional frame regions.

2. The method of claim 1 further comprising

encoding, by the processor, the video frames of the video streams using a stateless compression format.

3. The method of claim 2, wherein the stateless compression format is the M-JPEG format.

4. The method of claim 1, wherein the socket connections are Transmission Control Protocol (TCP) socket connections.

5. The method of claim 1, wherein determining if differences between a current video frame region and a previously rendered frame region corresponding to the divided unique region exceed a perceptual difference threshold further comprises:

performing, by the processor, a pixel by pixel XOR (exclusive-or) operation on component values of respective bit maps of the current video frame region and the previously rendered video frame region; and

determining, by the processor, if there are differences at each pixel based on the XOR operation.

6. The method of claim 1, wherein determining if differences between a current video frame region and a previously rendered frame region corresponding to the divided unique region exceed a perceptual difference threshold further comprises:

executing, by the processor, a memory comparison function for comparing the unique regions of the composite video image to detect if the differences between the current video frame region and a previously rendered frame region corresponding to the divided unique region exceed the difference threshold.

7. The method of claim 1, wherein determining if differences between a current video frame region and previously rendered frame region corresponding to the divided unique region exceed a perceptual difference threshold further comprises:

performing, by the processor, a pixel by pixel comparison on component values of respective bit maps of the current video frame region and the previously rendered video frame region;

computing, by the processor, a distance value between component values of the respective bitmaps; and

determining, by the processor, one of an average or a mean of the distance values;

comparing, by the processor, one of the average or the mean of the distance values to a threshold distance value; and

determining, by the processor, the differences of the current video frame region from the previously rendered frame region based on the comparison.

8. An apparatus, comprising:

one or more processors;

a memory;

a network interface; and

computer program code stored in a non-transitory computer-readable storage medium and comprising computer-executable instructions for;

executing an application that provides a video output for transmission to a client, the client being connected to a plurality of socket connections established between the client and the processor;

generating the video output of the application, the video output comprising a composite video image;

dividing a full video frame corresponding to the composite video image into a plurality of unique frame regions, each of the divided unique regions of the full video frame corresponding to a respective socket connection of the plurality of socket connections established for the same client and the number of divided unique regions corresponding to the number of the plurality of socket connections, wherein the divided unique regions comprise a target frame region and one or more positional frame regions positioned and rendered adjacent to the target frame region;

rendering a plurality of separate video streams, wherein each of the separate video streams corresponds to a respective one of the plurality of divided unique frame regions of the composite video image, and wherein each of the separate video streams is executed as a separate thread to write to each of the plurality of socket connections established for the same client in parallel with other separate video streams corresponding to other divided unique frame regions;

rendering a movement of the target frame region relative to the composite video image, wherein rendering comprises determining changes to a height or width of the one or more positional frame regions adjacent to the target frame region and transmitting a new frame having a new height and width over respective socket connections corresponding to the one or more positional frame regions.

9. The apparatus of claim 8 wherein the computer program code further comprises instructions for:

encoding the video frames of the video streams using a stateless compression format.

10. The apparatus of claim 9 wherein the stateless compression format is the M-JPEG format.

11. The apparatus of claim 8 wherein the socket connections are Transmission Control Protocol (TCP) socket connections.

12. A non-transitory computer readable storage medium having stored thereon computer programming instructions for:

executing an application that provides a video output for transmission to a client, the client being connected to a plurality of socket connections established between the client and the processor;

generating the video output of the application, the video output comprising a composite video image;

dividing a full video frame corresponding to the composite video image into a plurality of unique frame regions, each of the divided unique regions of the full video frame corresponding to a respective socket connection of the plurality of socket connections established for the same client and the number of divided unique regions corresponding to the number of the plurality of socket connections, wherein the divided unique regions comprise a target frame region and one or more positional frame regions positioned and rendered adjacent to the target frame region;

rendering a plurality of separate video streams, wherein each of the separate video streams corresponds to a respective one of the plurality of divided unique regions of the composite video image, and wherein each of the separate video streams is executed as a separate thread to write to each of the plurality of socket connections established for the same client in parallel with other separate video streams corresponding to other divided unique frame regions;

rendering a movement of the target frame region relative to the composite video image, wherein rendering comprises determining changes to a height or width of the one or more positional frame regions adjacent to the target frame region and transmitting a new frame having a new height and width over respective socket connections corresponding to the one or more positional frame regions.

13. The non-transitory computer readable storage medium of claim 12 further comprising computer programming instructions for:

receiving from the client, a separate request for each of the plurality of video streams.

14. The non-transitory computer-readable storage medium of claim 13, further comprising computer programming instructions for:

transmitting to the client, an output providing access to the plurality of video streams.