METHOD AND SYSTEM FOR GENERATING A 3D MODEL FROM IMAGES

• US 20100284607A1
• Filed: 06/27/2008
• Published: 11/11/2010
• Est. Priority Date: 06/29/2007
• Status: Active Grant

• Current Assignee: Three Pixels Wide Pty Ltd.
• Original Assignee: Three Pixels Wide Pty Ltd.
• Inventors: Dick, Anthony Robert, Torr, Philip Hilaire Sean, Van Den Hengel, Anton John, Thormaehlen, Thorsten, Ward, Benjamin William Stephen
• Granted Patent: US 8,699,787 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20096 Interactive definition of curve of interest
• CPC Class Codes: G06T 7/564 from contours

Abstract

A method for generating a three dimensional (3D) model of an object from a series of two dimensional (2D) images is described. The series of 2D images depict varying views of the object and have associated camera parameter information. The method includes the steps of tracing the object in a first 2D image selected from the series of 2D images to provide a first set of tracing information, then tracing the object in a second 2D image selected from the series of 2D images to provide a second set of tracing information. The 3D model of the object is then generated based on the camera parameter information and the first and second sets of tracing information.

Claims

1. A method for generating a three dimensional (3D) model of an object from a series of two dimensional (2D) images, wherein the series of 2D images depict varying views of the object and have associated camera parameter information, the method including the steps of:

tracing the object in a first 2D image selected from the series of 2D images to provide a first set of tracing information;

tracing the object in a second 2D image selected from the series of 2D images to provide a second set of tracing information; and

generating the 3D model of the object based on the camera parameter information and the first and second sets of tracing information.

2. The method of claim 1, further including the step of projecting an estimated representation of the object derived from the first set of tracing information onto the second 2D image.

3. The method of claim 2, wherein the step of tracing the object in the second 2D image includes modifying or refining the projected representation of the object derived from the first set of tracing information to trace the object in the second 2D image, thereby providing the second set of tracing information.

4. The method of claim 1, wherein the step of generating the 3D model further includes basing the generation on additional 3D point information related to the object.

5. The method of claim 1, wherein the series of 2D images is a sequence of 2D images.

6. The method of claim 5, wherein the sequence of 2D images is a video image sequence.

7. The method of claim 6, wherein the camera parameter information is derived from a structure from motion analysis of the video image sequence.

8. The method of claim 1, wherein the steps of tracing are facilitated by snap fitting to pixel boundaries.

9. A method for generating a 3D model of an object from a plurality of two dimensional (2D) images, wherein the plurality of 2D images depict varying views of the object, the method including the steps of:

deriving a first set of 3D information related to the object from a first 2D image and associated camera parameter information, the first 2D image selected from the plurality of 2D images;

deriving a second set of 3D information related to the object from a second 2D image and associated camera parameter information, the second 2D image selected from the plurality of 2D images; and

generating the 3D model based on the first and second sets of 3D information.

10. The method of claim 9, wherein the steps of deriving first and second sets of 3D information related to the object include forming first and second 3D representations of the object for the each 2D image.

11. The method of claim 10, wherein the step of generating the 3D model includes forming an optimal 3D representation of the object based on the first and second 3D representations of the object.

12. A method for generating a 3D model of an object from a series of two dimensional (2D) images, wherein the plurality of 2D images depict varying views of the object and have associated camera parameter information, the method including the steps of:

selecting a geometric primitive and lining up the geometric primitive with the shape characteristics of the object to trace the object in a first 2D image;

modifying the geometric primitive after being projected onto a second 2D image to trace the object in the second 2D image; and

generating the 3D model based on the geometric primitive and the camera parameter information associated with the first and second 2D images.

13. The method of claim 12, wherein the geometric primitive is a 3D geometric primitive.

14. A method for generating a 3D model of an object from a series of 2D images, wherein the series of 2D images depict varying views of the object and have associated camera parameter information, the method including the steps of:

tracing the object in a first 2D image selected from the series of 2D images to provide a first set of tracing information;

processing a second 2D image selected from the series of 2D images to determine feature information of the object; and

generating the 3D model of the object based on the camera parameter information, the first set of tracing information and the feature information.

15. The method of claim 14, wherein the step of processing includes forming a 2D image similarity measure to determine similarities of the appearance of the object in the second 2D image to the appearance of the object in the first 2D image to determine feature information.

16. The method of claim 15, wherein the 2D image similarity measure is a cross-correlation measure.

17. The method of claim 14, wherein the series of 2D images is a sequence of 2D images and the first and second 2D images are neighboring images in the sequence of 2D images.

18. A method for inserting a 3D model of an object derived from a first series of 2D images (“

cutting/copying”

) into a second series of 2D images (“

pasting”

), the method including the steps of;

generating a 3D model of the object based on camera parameter information associated with the first series of 2D images and by tracing the object in at least two images from the first series of images; and

inserting the 3D model into the second series of 2D images.

19. The method of claim 18, wherein the step of inserting includes orienting the 3D model according to camera parameter information associated with the second series of 2D images.

20. The method of claim 19, further including the step of generating 3D models of a selection of objects in the second series of 2D images.

21. The method of claim 18, wherein the first and second series of 2D images are video image sequences respectively.

22. A method for generating a 3D model of a complex object, the method including the steps of:

generating a component 3D model of a component part of the complex object; and

exploiting the shape of the component 3D model to develop a 3D model of the entire complex object.

23. The method of claim 22, wherein the step of exploiting the shape of the component 3D model includes extruding the component 3D model in a predetermined direction.

24. The method of claim 22, wherein the step of exploiting the shape of the component 3D model includes reflecting the component 3D model in a mirror plane.