Method of capturing, processing, and rendering images

• US 7,889,197 B2
• Filed: 01/26/2007
• Issued: 02/15/2011
• Est. Priority Date: 01/26/2007
• Status: Expired due to Fees

• Current Assignee: Captivemotion, Inc.
• Original Assignee: Captivemotion, Inc.
• Inventors: Kraver, Adam
• Primary Examiner(s): Wu; Xiao M
• Assistant Examiner(s): McDowell, Jr.; Maurice
• Application Number:
  US11/627,513
• Publication Number:
  US 20080180436A1
• Time in Patent Office: 1,481 Days
• Field of Search: 345/420, 345/423
• US Class Current: 345/420
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A computer-implemented method for capturing and processing a series of images captures a first image incorporating a surface having a plurality of markers varying in texture or color to obtain an image representation. A plurality of alignment tracking points, obtained from the plurality of markers, is designated. The plurality of alignment tracking points is utilized in a first image frame to locate a position of a first marker in a second, succeeding image frame. The first marker is reused in the second, succeeding image frame if located. Otherwise, a defined region surrounding the position of the first marker is searched for a second marker matching the first marker in texture or color. The second marker is used in the second, succeeding image if located. Otherwise, a best guess position of the first marker is interpolated by processing translation information of a third marker geometrically interconnected to the first marker.

Claims

1. A computer-implemented method of capturing and processing a series of images incorporating a plurality of digitized markers, comprising:

capturing a first image frame with a first video camera having a first optical position to generate a first image representation;

capturing the first image frame with a second video camera having a second optical position to generate a second image representation;

identifying a plurality of alignment tracking points from the plurality of digitized markers using each of the first and second image representations; and

utilizing the plurality of alignment tracking points in the first image frame to locate a position of a first marker in a second, succeeding image frame, wherein if the first marker is identified, the first marker is reused in the second image frame, and if the first marker is not identified;

searching a defined region surrounding the position of the first marker for a second, matching marker, the second, matching marker used in the second, succeeding image frame if the second, matching marker is identified within the defined region, and if the matching marker is not identified within the defined region;

interpolating a best guess position of the first marker by processing translation information of a third marker geometrically interconnected to the first marker, the first and third marker being geometrically interconnected using a three-dimensional mesh.

2. The method of claim 1, wherein the plurality of interconnected digitized markers are generated in a three-dimensional mesh representation.

3. The method of claim 1, further including synchronizing the first and second image representations to generate a composite image representation representing correlated first and second image representations.

4. The method of claim 1, further including processing the first image representation through a marker segmentation algorithm to obtain a first image segmentation representation.

5. The method of claim 1, wherein the plurality of digitized markers vary in texture or color.

6. The method of claim 1, wherein the first marker and the second, matching marker match in color or texture.

7. The method of claim 1, further including transforming the plurality of alignment tracking points from a world space representation to a local space representation of the first image.

8. The method of claim 1, further including:

dragging a projected tracking point in the first image representation until a third, matching marker is overlapped;

locking the first image representation to prevent further use of the third, matching marker; and

projecting a line from the third, matching marker in the first image representation into view of the second image representation, wherein if a fourth marker is identified along the line, the fourth marker is used in the second, succeeding image frame of the second image representation, and if the fourth marker is not identified;

utilizing the closest, fifth marker from the projected tracking point in the first image representation to the projected line in the second, succeeding image frame.

9. A computer-implemented method for capturing and processing a series of images, comprising:

capturing a first image frame incorporating a surface having a plurality of markers varying in texture or color using a plurality of cameras to obtain a plurality of image representations;

designating a plurality of alignment tracking points obtained from the plurality of markers using one of the plurality of image representations; and

utilizing the plurality of alignment tracking points in a first image frame to locate a position of a first marker in a second, succeeding image frame, the first marker reused in the second, succeeding image frame if located, wherein if the first marker is not located;

searching a defined region surrounding the position of the first marker for a second marker matching the first marker in texture or color, the second marker used in the second, succeeding image frame if located within the defined region, and, if the second marker is not located within the defined region;

interpolating a best guess position of the first marker by processing translation information of a third marker geometrically interconnected to the first marker.

10. The method of claim 9, further including synchronizing each of the plurality of image representations to generate a composite image representation.

11. The method of claim 9, further including processing the image representation through a segmentation algorithm to obtain an image segmentation representation.

12. The method of claim 9, further including transforming the plurality of alignment tracking points from a world space representation to a local space representation of the first image.

13. The method of claim 9, further including, subsequent to the step of capturing a first image, generating a plurality of two-dimensional tracking points which have been triangulated into a two-dimensional mesh representation.

14. The method of claim 13, wherein the two-dimensional mesh representation is combined with a plurality of optical flow tracking points to generate a three-dimensional mesh representation having three-dimensional tracking points.

15. The method of claim 14, wherein the plurality of alignment tracking points is selected from the three-dimensional tracking points of the three-dimensional mesh representation.

16. The method of claim 9, further including:

dragging a projected tracking point in the image representation until a third marker, matching the projected tracking point in texture or color, is overlapped;

locking the image representation to prevent further use of the third marker; and

projecting a line from the third marker in the image representation into view of a second image representation, wherein if a fourth marker is identified along the line, the fourth marker is used in the second, succeeding image frame of the second image representation, and if the fourth marker is not identified;

utilizing the closest, fifth marker from the projected tracking point in the first image representation to the projected line in the second, succeeding image frame.

17. A computer program product:

including a non-transitory computer usable medium including a computer usable program code for capturing and processing a series of images, the computer program product comprising;

computer usable program code for capturing a first image incorporating a surface having a plurality of markers varying in texture or color to obtain an image representation;

computer usable program code for designating a plurality of alignment tracking points obtained from the plurality of markers; and

computer usable program code for utilizing the plurality of alignment tracking points in a first image frame to locate a position of a first marker in a second, succeeding image frame, the first marker reused in the second, succeeding image frame if located, wherein if the first marker is not located;

computer usable program code for searching a defined region surrounding the position of the first marker for a second marker matching the first marker in texture or color, the second marker used in the second, succeeding image if located within the defined region, and, if the second marker is not located within the defined region;

computer usable program code for interpolating a best guess position of the first marker by processing translation information of a third marker geometrically interconnected to the first marker.

18. The computer program product of claim 17, wherein the step of capturing the first image is performed with a plurality of camera to obtain a plurality of image representations.

19. The computer program product of claim 18, wherein the step of designating a plurality of alignment tracking points is performed on each of the plurality of image representations.

20. The computer program product of claim 19, further including synchronizing each of the plurality of image representations to generate a composite image representation.

21. The computer program product of claim 17, further including processing the image representation through a segmentation algorithm to obtain an image segmentation representation.

22. The computer program product of claim 17, further including transforming the plurality of alignment tracking points from a world space representation to a local space representation of the first image.

23. A computer system for capturing and processing a series of images, comprising:

a memory device; and

a processing device electrically connected to the memory device, wherein the processing device;

captures a first image incorporating a surface having a plurality of markers varying in texture or color, generating an image representation stored on the memory device,

designates a plurality of alignment tracking points obtained from the plurality of markers, and

utilizes the plurality of alignment tracking points in a first image frame to locate a position of a first marker in a second, succeeding image frame, the first marker reused in the second, succeeding image frame if located, wherein if the first marker is not located, the processing device;

searches a defined region surrounding the position of the first marker for a second marker matching the first marker in texture or color, the second marker used in the second, succeeding image if located within the defined region, and, if the second marker is not located within the defined region, the processing device;

interpolates a best guess position of the first marker by processing translation information of a third marker geometrically interconnected to the first marker.

24. The computer system of claim 23, wherein the processing device captures the first image using a plurality of cameras to obtain a plurality of image representations.

25. The computer system of claim 23, wherein the processing device designates a plurality of alignment tracking points upon each of the plurality of image representations.

26. The computer system of claim 23, wherein the processing device synchronizes each of the plurality of image representations to generate a composite image representation.

27. The computer system of claim 23, wherein the processing device processes the image representation through a segmentation algorithm to obtain an image segmentation representation.

28. The computer system of claim 23, wherein the processing device transforms the plurality of alignment tracking points from a world space representation to a local space representation of the first image.