System and method for exploiting scene graph information in construction of an encoded video sequence

• US 9,294,785 B2
• Filed: 04/25/2014
• Issued: 03/22/2016
• Est. Priority Date: 06/06/2013
• Status: Active Grant

• Current Assignee: ACTIVEVIDEO NETWORKS, LLC
• Original Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Inventors: Brockmann, Ronald Alexander, Dev, Anuj, Hoeben, Maarten
• Primary Examiner(s): Bayat, Ali
• Application Number:
  US14/262,674
• Publication Number:
  US 20140362930A1
• Time in Patent Office: 697 Days
• Field of Search: 382/236
• US Class Current: 1/1
• CPC Class Codes: G06T 9/004 Predictors, e.g. intraframe, interframe coding
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression

Abstract

A system method and computer program product for creating a composited video frame sequence for an application. A current scene state for the application is compared to a previous scene state wherein each scene state includes a plurality of objects. A video construction engine determines if properties of one or more objects have changed based upon a comparison of the scene states. If properties of one or more objects have changed based upon the comparison, the delta between the object'"'"'s states is determined and this information is used by a fragment encoding module if the fragment has not been encoded before. The information is used to define, for example, the motion vectors for use by the fragment encoding module in construction of the fragments to be used by the stitching module to build the composited video frame sequence.

Claims

1. A method for creating a composited video frame sequence, comprising:

at a system including one or more processors and memory storing instructions for execution by the processor;

comparing a current scene state with a previous scene state, wherein the current scene state includes a first plurality of objects having respective properties, and wherein the previous scene state includes a second plurality of objects having respective properties;

detecting a difference between the respective properties of the first plurality of objects and the respective properties of the second plurality of objects;

in accordance with the difference between the respective properties being detected, retrieving one or more pre-encoded first video fragments based on the detected difference, wherein each of the one or more pre-encoded first video fragments is a portion of a full frame of video; and

compositing the video frame sequence, wherein the video frame sequence includes at least one of the one or more pre-encoded first video fragments.

2. The method of claim 1, wherein the one or more pre-encoded first video fragments are retrieved from a memory.

3. The method of claim 2, wherein the memory is non-volatile memory.

4. The method of claim 2, wherein the memory is volatile memory.

5. The method of claim 1, wherein the difference detected between the respective properties of the first plurality of objects and the respective properties of the second plurality of objects corresponds to at least one property from a group consisting of:

a position, transformation matrix, texture, and translucency, of a respective object.

6. The method of claim 1, wherein detecting the difference between the respective properties of the first plurality of objects and the respective properties of the second plurality includes:

tessellating a first bounding rectangle, corresponding to at least one object of the first plurality of objects, with a second bounding rectangle, corresponding to at least one object of the second plurality of objects.

7. The method of claim 1, the method further comprising:

in accordance with the difference between the respective properties being detected, encoding one or more second video fragments based on the detected difference, wherein the video frame sequence further includes at least one of the one or more encoded second video fragments.

8. A computer system for creating a composited video frame sequence for an application, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

comparing a current scene state with a previous scene state, wherein the current scene state includes a first plurality of objects having respective properties, and wherein the previous scene state includes a second plurality of objects having respective properties;

detecting a difference between the respective properties of the first plurality of objects and the respective properties of the second plurality of objects;

in accordance with the difference between the respective properties being detected, retrieving one or more pre-encoded first video fragments based on the detected difference, wherein each of the one or more pre-encoded first video fragments is a portion of a full frame of video; and

compositing the video frame sequence, wherein the video frame sequence includes at least one of the one or more pre-encoded first video fragments.

9. A non-transitory computer readable storage medium, storing one or more programs for execution by one or more processors of a computer system, the one or more programs including instructions for:

comparing a current scene state with a previous scene state, wherein the current scene state includes a first plurality of objects having respective properties, and wherein the previous scene state includes a second plurality of objects having respective properties;

detecting a difference between the respective properties of the first plurality of objects and the respective properties of the second plurality of objects;

in accordance with the difference between the respective properties being detected, retrieving one or more pre-encoded first video fragments based on the detected difference, wherein each of the one or more pre-encoded first video fragments is a portion of a full frame of video; and

compositing the video frame sequence, wherein the video frame sequence includes at least one of the one or more pre-encoded first video fragments.