Virtual decals for precision alignment and stabilization of motion graphics on mobile video

• US 9,215,368 B2
• Filed: 12/02/2013
• Issued: 12/15/2015
• Est. Priority Date: 12/02/2012
• Status: Active Grant

• Current Assignee: Bachir Babale, Michael Benjamin Karp
• Original Assignee: Bachir Babale, Michael Benjamin Karp
• Inventors: Babale, Bachir, Karp, Michael Benjamin
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): Segura, Cynthia
• Application Number:
  US14/094,423
• Publication Number:
  US 20140160320A1
• Time in Patent Office: 743 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

The virtual decals for precision alignment of motion graphics of the present invention provides a method by which a person using a mobile video capturing device may apply a special effect or motion graphic to the video being captured using a virtual decal present on the screen of the mobile device, providing a reference to the user allowing the user to compensate for camera movement and properly frame the video to which the motion graphic will be applied using post-processing. The present invention further provides a method for measuring the optic flow of a given video scene and applying that data in a predictive manner so as to stabilize a motion graphic against the scene. Post processing places such a motion graphic within the processed video in such a way that it appears to be part of the original scene.

Claims

1. A method for video processing using a mobile computer, comprising:

selecting a motion graphic for displaying at a location in a video scene;

employing a view finder provided by the mobile computer to view the video scene while positioning a virtual decal at the location in the video scene, wherein the virtual decal represents the selected motion graphic;

employing the mobile computer to capture a sequence of video frames of the video scene while at least a portion of the virtual decal is viewable in the video scene with the view finder; and

iteratively processing each recorded video frame from the sequence of captured video frames using the mobile computer, wherein the processing includes;

determining a plurality of tracking points within each video frame based on at least one characteristic of an image included in the video frame;

determining a quality score for each of the tracking points;

if the quality score corresponding to an individual tracking point included in the plurality of tracking points is less than a defined threshold, discarding the individual tracking point from the plurality of tracking points;

determining at least one reference point within the video frame based on at least the location of a portion of the plurality of determined tracking points; and

embedding the motion graphic in the video frame at the location of the virtual decal, wherein the motion graphic is further positioned in the video frame based at least on the reference point.

2. The method of claim 1, further comprising, if a quantity of tracking points included in the plurality of tracking points drops below a threshold, determine another plurality of tracking points from a next video frame.

3. The method of claim 1, wherein determining the quality score for each of the tracking points, further comprises, determining an age of the tracking point and the variance between its predicted location and its actual location within each video frame.

4. The method of claim 1, wherein capturing the sequence of video frames of the video scene using the mobile computer, further comprises, displaying a visual timing cue in the mobile computer'"'"'s view finder that indicates a duration for playing back the motion graphic.

5. The method of claim 1, wherein displaying the virtual decal in the video scene, further comprises, modifying a location of the virtual decal within the view finder based on at least one dynamic property of the motion graphic.

6. The method of claim 1, further comprising, enabling a user to adjust a size of the virtual decal for determining a size of the motion graphic for embedding in each of the video frames.

7. The method claim 1, wherein determining a plurality of tracking points within the video frame, further comprises, computing a score based on at least one of contrast, brightness, darkness, pattern matching, edge detection, or corner-recognition.

8. The method of claim 1, further comprising, plotting a position of the motion graphic using a coordinate system that is separate from another coordinate system that corresponds to each video frame.

9. A mobile computer for video processing, comprising:

a transceiver for communicating over the network;

a memory for storing at least instructions;

a processor device that is operative to execute instructions that enable actions, including;

selecting a motion graphic for displaying at a location in a video scene;

employing a view finder provided by the mobile computer to view the video scene while positioning a virtual decal at the location in the video scene, wherein the virtual decal represents the selected motion graphic;

employing the mobile computer to capture a sequence of video frames of the video scene while at least a portion of the virtual decal is viewable in the video scene with the view finder; and

iteratively processing each recorded video frame from the sequence of captured video frames using the mobile computer, wherein the processing includes;

determining a plurality of tracking points within each video frame based on at least one characteristic of an image included in the video frame;

determining a quality score for each of the tracking points;

if the quality score corresponding to an individual tracking point included in the plurality of tracking points is less than a defined threshold, discarding the individual tracking point from the plurality of tracking points;

determining at least one reference point within the video frame based on at least the location of a portion of the plurality of determined tracking points; and

embedding the motion graphic in the video frame at the location of the virtual decal, wherein the motion graphic is further positioned in the video frame based at least on the reference point.

10. The mobile computer of claim 9, further comprising, if a quantity of tracking points included in the plurality of tracking points drops below a threshold, determine another plurality of tracking points from a next video frame.

11. The mobile computer of claim 9, wherein determining the quality score for each of the tracking points, further comprises, determining an age of the tracking point and the variance between its predicted location and its actual location within each video frame.

12. The mobile computer of claim 9, wherein capturing the sequence of video frames of the video scene using the mobile computer, further comprises, displaying a visual timing cue in the mobile computer'"'"'s view finder that indicates a duration for playing back the motion graphic.

13. The mobile computer of claim 9, wherein displaying the virtual decal in the video scene, further comprises, modifying a location of the virtual decal within the view finder based on at least one dynamic property of the motion graphic.

14. The mobile computer of claim 9, further comprising, enabling a user to adjust a size of the virtual decal for determining a size of the motion graphic for embedding in each of the video frames.

15. The mobile computer of claim 9, wherein determining a plurality of tracking points within the video frame, further comprises, computing a score based on at least one of contrast, brightness, darkness, pattern matching, edge detection, or corner-recognition.

16. The mobile computer of claim 9, further comprising, plotting a position of the motion graphic using a coordinate system that is separate from another coordinate system that corresponds to each video frame.

17. A processor readable non-transitory storage media that includes instructions for video processing using a mobile computer, wherein the mobile computer that executes at least a portion of the instructions enables actions, comprising:

selecting a motion graphic for displaying at a location in a video scene;

employing a view finder provided by the mobile computer to view the video scene while positioning a virtual decal at the location in the video scene, wherein the virtual decal represents the selected motion graphic;

employing the mobile computer to capture a sequence of video frames of the video scene while at least a portion of the virtual decal is viewable in the video scene with the view finder; and

iteratively processing each recorded video frame from the sequence of captured video frames using the mobile computer, wherein the processing includes;

determining a plurality of tracking points within each video frame based on at least one characteristic of an image included in the video frame;

determining a quality score for each of the tracking points;

if the quality score corresponding to an individual tracking point included in the plurality of tracking points is less than a defined threshold, discarding the individual tracking point from the plurality of tracking points;

determining at least one reference point within the video frame based on at least the location of a portion of the plurality of determined tracking points; and

embedding the motion graphic in the video frame at the location of the virtual decal, wherein the motion graphic is further positioned in the video frame based at least on the reference point.

18. The media of claim 17, further comprising, if a quantity of tracking points included in the plurality of tracking points drops below a threshold, determine another plurality of tracking points from a next video frame.

19. The media of claim 17, wherein determining the quality score for each of the tracking points, further comprises, determining an age of the tracking point and the variance between its predicted location and its actual location within each video frame.

20. The media of claim 17, wherein capturing the sequence of video frames of the video scene using the mobile computer, further comprises, displaying a visual timing cue in the mobile computer'"'"'s view finder that indicates a duration for playing back the motion graphic.

21. The media of claim 17, wherein displaying the virtual decal in the video scene, further comprises, modifying a location of the virtual decal within the view finder based on at least one dynamic property of the motion graphic.

22. The media of claim 17, further comprising, enabling a user to adjust a size of the virtual decal for determining a size of the motion graphic for embedding in each of the video frames.

23. The media of claim 17, wherein determining a plurality of tracking points within the video frame, further comprises, computing a score based on at least one of contrast, brightness, darkness, pattern matching, edge detection, or corner-recognition.

24. The media of claim 17, further comprising, plotting a position of the motion graphic using a coordinate system that is separate from another coordinate system that corresponds to each video frame.