3D TRACKED POINT VISUALIZATION USING COLOR AND PERSPECTIVE SIZE

• US 20140285624A1
• Filed: 06/25/2012
• Published: 09/25/2014
• Est. Priority Date: 06/25/2012
• Status: Active Grant

• Current Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Acquavella, James, Wilk, Daniel M., Simons, David
• Granted Patent: US 9,208,577 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2219/2012 Colour editing, changing, or manipulating; Use of colour codes
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 13/257 Colour aspects

Abstract

One exemplary embodiment involves receiving a plurality of three-dimensional (3D) track points for a plurality of frames of a video, wherein the 3D track points are extracted from a plurality of two-dimensional source points. The embodiment further involves rendering the 3D track points across a plurality of frames of the video on a two-dimensional (2D) display. Additionally, the embodiment involves coloring each of the 3D track points wherein the color of each 3D track point visually distinguishes the 3D track point from a plurality of surrounding 3D track points, and wherein the color of each 3D track point is consistent across the frames of the video. The embodiment also involves sizing each of the 3D track points based on a distance between a camera that captured the video and a location of the 2D source points referenced by the respective one of the 3D track points.

Claims

1. A computer-implemented method comprising:

receiving a plurality of three-dimensional (3D) track points for a plurality of frames of a video;

rendering the 3D track points for a frame of the video on a two-dimensional (2D) display;

coloring each of the 3D track points with a color selected from a group of colors, wherein the color of each 3D track point visually distinguishes the 3D track point from a plurality of surrounding 3D track points, and wherein the color of each 3D track point is consistent across the frames of the video; and

sizing each of the 3D track points based on a distance between a camera that captured the video and a location referenced by the respective one of the 3D track points.

2. The method of claim 1, wherein the 3D track points are generated by:

tracking a plurality of 2D source points from the video;

solving for 3D scene data from the 2D source points; and

correlating the 3D scene data with the 2D source points to extract the 3D track points.

3. The method of claim 2, wherein the 3D scene data is represented by a plurality of position values of stationary objects in the video.

4. The method of claim 2, wherein the 3D scene data is represented by the camera location, an orientation of the camera when capturing the video, and a viewing angle of the camera over a span of the video.

5. The method of claim 1, wherein a subset of the 3D track points exist only during a subset of the frames comprising the video.

6. The method of claim 1, wherein a plurality of video objects occlude a subset of the 3D track points for a period of time during playback.

7. The method of claim 1, wherein the 3D track points that correspond to each video frame are rendered on the 2D display simultaneously with the respective video frame during playback.

8. The method of claim 1, wherein the colors of the 3D track points are selected at random.

9. The method of claim 1, wherein the colors are selected to increase temporal coherence of the 3D track points.

10. The method of claim 1, wherein the 3D track points located farther away from the camera are smaller than the 3D track points located closer to the camera.

11. The method of claim 1, wherein the colors for the 3D track points are selected from a group of unique colors.

12. The method of claim 1, wherein each 3D track point is represented by a hollow character to provide contrast for visibility.

13. A system comprising:

a processor for executing instructions stored in computer-readable medium on one or more devices providing an application for specifying modifications to electronic content,

the application comprising one or more modules configured to perform the steps comprising;

extracting a plurality of three-dimensional (3D) track points from a plurality of two-dimensional (2D) source points associated with plurality of frames of a video;

rendering the 3D track points on a 2D display across the frames of the video, wherein each one of the 3D track points appear in close proximity to the corresponding 2D source point; and

coloring the 3D track points with a color scheme to increase temporal coherence of each one of the 3D track points.

14. The system of claim 13, further comprising:

sizing each one of the 3D track points based on a distance between a camera that captured the video and a location referenced by the respective one of the 3D track points;

15. The system of claim 13, wherein the color scheme comprises selecting a unique and distinguishable color for each 3D track point to differentiate the 3D track point from a plurality of surrounding 3D track points.

16. The system of claim 14, wherein the 3D track points located closer to the camera are bigger than the 3D track points located farther away from the camera.

17. The system of claim 13, wherein the 3D track points are not visible when the 2D source point associated with the respective 3D track point is not visible.

18. A non-transitory computer-readable medium on which is encoded program code, the program code comprising:

program code for receiving a plurality of three-dimensional (3D) track points associated with a plurality of frames of a video, wherein the 3D track points are extracted from a plurality of two-dimensional (2D) source points;

program code for rendering the 3D track points on a 2D display across the frames of the video; and

program code for sizing the 3D track points based on a distance between a camera capturing the video and the location of the 2D source point associated with the respective 3D track point.

19. The non-transitory computer-readable medium of claim 18 further comprising program code for coloring the 3D track points with a unique and distinguishable color to differentiate each of the 3D track points from a plurality of surrounding 3D track points.

20. The non-transitory computer-readable medium of claim 18, wherein the 3D track points are not rendered when the 2D source points associated with each of the 3D track points are occluded by an object in the video.