System and method for determining structure and motion using multiples sets of images from different projection models for object modeling

• US 6,661,913 B1
• Filed: 06/19/1999
• Issued: 12/09/2003
• Est. Priority Date: 05/05/1999
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Anandan, Padmanabhan, Zhang, Zhengyou, Shum, Heung-Yeung
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): Bali, Vikkram
• Application Number:
  US09/336,218
• Time in Patent Office: 1,634 Days
• Field of Search: 382/103, 382/107, 382/154, 382/276, 345/419, 345/420, 345/584, 345/587, 345/619, 345/664, 345/418, 345/473
• US Class Current: 382/154
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

The present invention is embodied in systems and methods for determining structure and motion of a three-dimensional (3D) object using two-dimensional (2D) images of the object obtained from multiple sets of views with different projection models, such as from a full perspective view and a weak perspective views. A novel fundamental matrix is derived that embodies the epipolar geometry between a full perspective view and a weak perspective view. The systems and methods of the present invention preferably uses the derived fundamental matrix together with the 2D image information of the full and weak perspective views to digitally reconstruct the 3D object and produce results with multi-resolution processing techniques. These techniques include recovering and refining motion parameters and recovering and refining structure parameters of the fundamental matrix. The results can include, for example, 3D positions of points, camera position between different views, texture maps, and the like.

Claims

1. A method for performing multi-resolution modeling of a three-dimensional object comprising:

obtaining full and weak perspective views of the object as two-dimensional images;

digitally combining the images; and

producing combination data defined by epipolar geometrical representations of the object and digitally reconstructing the object from the combination data.

2. The method of claim 1, wherein the full and weak perspective views are obtained as a proximal camera view modeled as the full perspective view of the object and a distant camera view approximated as the weak perspective view of the object.

3. The method of claim 2, further comprising producing three-dimensional position points and camera positions between the views from the combination data.

4. The method of claim 2, further comprising producing texture maps of the three dimensional object from the combination data.

5. A method for modeling a three-dimensional object comprising:

obtaining a first view of the object from a first projection model as two-dimensional first image;

obtaining a second view of the object from a second projection model different from the first projection model as two-dimensional second image;

digitally combining the first and second images;

producing epipolar geometrical representations of the images with a matrix defined by motion and structure parameters; and

digitally reconstructing the object from the parameters of the matrix.

6. The method of claim 5, wherein obtaining the first view of the object includes obtaining a close camera field of view of the object and wherein obtaining the second view of the object includes obtaining a wide camera field of view of the object.

7. The method of claim 6, wherein the close camera field of view is approximated as a weak perspective view and the wide camera field of view is a full perspective view.

8. The method of claim 5, wherein obtaining the first view of the object includes obtaining a proximal view modeled as a full perspective view of the object and obtaining the second view of the object includes obtaining a distant view approximated as a weak perspective view of the object.

9. The method of claim 5, wherein obtaining the first view of the object includes obtaining a map orthophoto of object modeled as a weak-perspective view and obtaining the second view of the object includes obtaining a normal camera photograph modeled as a full perspective view.

10. A multi-resolution modeling system for digitally modeling three-dimensional objects comprising:

a camera that obtains a first view of the object from a first projection model as a first two-dimensional image and a second view of the object from a second projection model different than the first projection model as a second two-dimensional image;

a digital processor that combines the images and produces epipolar geometrical representations of the object with a matrix defined by motion and structure parameters; and

a reconstruction processor that reconstructs the object from the parameters of the matrix.

11. The multi-resolution modeling system of claim 10, wherein the first view is obtained as a proximal camera view modeled as a full perspective view of the object and the second view is a distant camera view approximated as a weak perspective view of the object.

12. The multi-resolution modeling system of claim 10, wherein the reconstruction processor produces three-dimensional position points and camera positions between the views from the parameters.

13. The multi-resolution modeling system of claim 10, wherein the reconstruction processor produces texture maps of the three dimensional object from the parameters.

14. The multi-resolution modeling system of claim 10, wherein the first view is obtained as a close camera field of view of the object and the second view is obtained as a wide camera field of view of the object.

15. The multi-resolution modeling system of claim 14, wherein the close camera field of view is approximated as a weak perspective view and the wide camera field of view is a full perspective view.

16. The multi-resolution modeling system of claim 10, wherein the first view is obtained as a proximal view modeled as a full perspective view of the object and the second view is obtained as a distant view approximated as a weak perspective view of the object.

17. The multi-resolution modeling system of claim 10, wherein the first view is obtained as a map orthophoto of object modeled as a weak-perspective view and the second view is obtained as a normal camera photograph modeled as a full perspective view.

18. A computer-readable storage device encoded with a set of instructions for digitally modeling three-dimensional objects, the set of instructions comprising:

capturing multiple views of the object from differing projection models as two-dimensional images;

a digital processor that combines the images and produces epipolar geometrical representations of the object with a matrix defined by motion and structure parameters; and

a reconstruction processor that reconstructs the object from the parameters of the matrix.

19. The set of instructions of claim 18, wherein capturing multiple views of the object includes capturing a close camera field of view of the object from a first projection model and capturing a wide camera field of view of the object from a second projection model different from the fist projection model.

20. The set of instructions of claim 19, wherein the close camera field of view is approximated as a weak perspective view and the wide camera field of view is a full perspective view.

21. The set of instructions of claim 18, wherein capturing multiple views of the object includes capturing a proximal view modeled as a full perspective view of the object from a first projection model and capturing a distant view approximated as a weak perspective view of the object from a second projection model different from the fist projection model.

22. The set of instructions of claim 18, wherein capturing multiple views of the object includes capturing a map orthophoto of object modeled as a weak-perspective view and capturing a normal camera photograph modeled as a full perspective view.

23. The set of instructions of claim 18, wherein the multiple views are captured as at least one proximal camera view modeled as a full perspective view of the object and at least one distant camera view approximated as a weak perspective view of the object.

24. The set of instructions of claim 18, further comprising producing three-dimensional position points-and camera positions between the multiple views from the parameters.

25. The set of instructions of claim 18, further comprising producing texture maps of the three dimensional object from the parameters.