System and method for exploiting scene graph information in construction of an encoded video sequence

• US 9,219,922 B2
• Filed: 06/06/2013
• Issued: 12/22/2015
• Est. Priority Date: 06/06/2013
• Status: Active Grant

• Current Assignee: ACTIVEVIDEO NETWORKS, LLC
• Original Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Inventors: Brockmann, Ronald Alexander, Dev, Anuj, Hoeben, Maarten
• Primary Examiner(s): COUSO, JOSE L
• Application Number:
  US13/911,948
• Publication Number:
  US 20140362086A1
• Time in Patent Office: 929 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search

Abstract

A system method and computer program product for creating a composited video frame sequence for an application. A current scene graph state for the application is compared to a previous scene graph state wherein each scene graph state includes a plurality of hierarchical nodes that represent one or more objects at each node. A video construction engine determines if one or more objects have moved based upon a comparison of the scene graph states. If one or more objects have moved based upon the scene graph comparison, motion information about the objects is determined and the motion information is forwarded to a stitcher module. The motion information is used to define motion vectors for use by the stitcher module in construction of the composited video frame sequence.

Claims

1. A method for creating a composited video frame sequence, the method comprising:

at a system including one or more processors and memory storing instructions for execution by the processor;

comparing a current scene graph state with a previous scene graph state, wherein the current scene graph state and the previous scene graph state are respective states of a tree data structure that includes a plurality of hierarchical nodes associated with one or more objects within a graphical scene and that includes spatial information for the one or more objects representing respective positions of the one or more objects in the graphical scene,

wherein the comparing comprises comparing, for at least some of the one or more objects, the spatial information of the current scene graph state with the spatial information of the previous scene graph state;

determining if at least one object of the one or more objects has moved based upon the comparison of the scene graph states;

in accordance with a determination that at least one object of the one or more objects has moved, determining motion information about the at least one object based on the comparison of the scene graph states; and

defining motion vectors for the composited video frame sequence based on the motion information.

2. The method according to claim 1 wherein determining motion information also includes determining graphical information about the objects and the graphical information is used in an encoding process.

3. The method according to claim 1 wherein the motion vectors are two-dimensional motion vectors.

4. The method according to claim 1, further comprising:

converting the current scene graph state and the previous scene graph state from a three dimensional representation to a two dimensional representation.

5. The method according to claim 1, further comprising:

constructing an MPEG encoded video sequence using the motion vectors.

6. The method according to claim 1, wherein the current scene graph state and the previous scene graph state are created by an application engine.

7. The method according to claim 1, wherein the current scene graph state includes OpenGL function calls.

8. The method according to claim 1 further comprising:

creating a hash of parameters for objects that have changed between the current and previous scene graph states.

9. The method according to claim 8, further comprising:

comparing the hash to a table of hashes representative of previous scene graph states;

if the current hash matches a hash within the table of hashes, retrieving MPEG encoded elements associated with the hash within the table of hashes; and

providing the MPEG encoded elements to a stitcher for stitching the MPEG encoded elements with other MPEG encoded elements to form one or more MPEG encoded video frames in a series of MPEG encoded video frames.

10. The method according to claim 9 further comprising:

providing two dimension screen representation data for objects that have changed between the current and previous scene graph states to an encoder wherein the encoder compresses the data using at least spatial techniques to produce one or more encoded elements.

11. The method according to claim 10, wherein the encoded elements are passed to a stitching block forming an encoded MPEG frame from the encoded elements where the encoded MPEG frame is part of an MPEG video sequence.

20. The method according to claim 1, wherein the plurality of hierarchical nodes comprises one or more nodes having one or more respective sub-nodes, wherein the one or more respective sub-nodes of the one or more nodes correspond to the one or more objects within the graphical scene and include the respective spatial information for the one or more objects.

12. A non-transitory computer readable storage medium storing one or more programs for creating an encoded video frame sequence, the one or more programs comprising instructions for:

comparing a current scene graph state with a previous scene graph state, wherein the current scene graph state and the previous scene graph state are respective states of a tree data structure that includes a plurality of hierarchical nodes associated with one or more objects within a graphical scene and that includes spatial information for the one or more objects representing respective positions of the one or more objects in the graphical scene,

wherein the comparing comprises comparing, for at least some of the one or more objects, the spatial information of the current scene graph state with the spatial information of the previous scene graph state;

determining if at least one object of the one or more objects has moved based upon the comparison of the scene graph state;

in accordance with a determination that at least one object of the one or more objects has moved, determining motion information about the at least one object based on the comparison of the scene graph states; and

defining motion vectors for the composited video frame sequence based on the motion information.

13. The non-transitory computer readable storage medium according to claim 12, the one or more programs further comprising instructions for:

constructing an MPEG encoded video sequence using the motion vectors.

14. The non-transitory computer readable storage medium according to claim 12, wherein the current scene graph state and the previous scene graph state are determined by an application engine.

15. The non-transitory computer readable storage medium according to claim 14, wherein the current scene graph state includes OpenGL function calls.

16. The non-transitory computer readable storage medium according to claim 12, the one or more programs further comprising instructions for:

creating a hash of parameters for objects that have changed between the current screen graph state and the previous screen graph state.

17. The non-transitory computer readable storage medium according to claim 16, the one or more programs further comprising instructions for:

comparing the hash to a table of hashes of parameters for objects from previous scene graph states;

retrieving MPEG encoded elements associated with the hash from the table of hashes if the hash for parameters for elements that have changed in the current scene graph state matches a hash within the table of hashes; and

providing the MPEG encoded elements to a stitcher for stitching the MPEG encoded elements associated with the matched hash with other MPEG encoded elements to form one or more MPEG encoded video frames in a series of MPEG encoded video frames.

18. The non-transitory computer readable storage medium according to claim 12, the one or more programs further comprising instructions for:

providing data representative of a two dimension screen representation of elements that have changed between the current scene graph state and the previous scene graph state to an encoder wherein the encoder compresses the data using at least spatial techniques to produce one or more MPEG elements.

19. The non-transitory computer readable storage medium according to claim 18, the one or more programs further comprising instructions for passing the encoded elements to the stitcher to form an encoded MPEG frame from the encoded MPEG elements where the encoded MPEG frame is part of an MPEG video sequence.

21. The non-transitory computer readable storage medium according to claim 12, wherein the plurality of hierarchical nodes comprises one or more nodes having one or more respective sub-nodes, wherein the one or more respective sub-nodes of the one or more nodes correspond to the one or more objects within the graphical scene and include the respective spatial information for the one or more objects.