Method for reconstructing a three-dimensional object from a closed-loop sequence of images taken by an uncalibrated camera

• US 6,061,468 A
• Filed: 07/28/1997
• Issued: 05/09/2000
• Est. Priority Date: 07/28/1997
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Compaq Computer Corporation (HP Inc.)
• Inventors: Kang, Sing Bing
• Primary Examiner(s): MEHTA, BHAVESH M
• Application Number:
  US08/901,391
• Time in Patent Office: 1,016 Days
• Field of Search: 382/154, 382/284, 382/285, 382/294, 382/296, 348/42-48, 348/578, 348/584, 348/36, 348/37, 356/3, 356/8, 356/12, 345/419, 345/425
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/579 from motion

Abstract

In a computerized method, the three-dimensional structure of an object is recovered from a closed-loop sequence of two-dimensional images taken by a camera undergoing some arbitrary motion. In one type of motion, the camera is held fixed, while the object completes a full 360° rotation about an arbitrary axis. Alternatively, the camera can make a complete rotation about the object. In the sequence of images, feature tracking points are selected using pair-wise image registration. Ellipses are fitted to the feature tracking points to estimate the tilt of the axis of rotation. A set of variables are set to fixed values while minimizing an image-based objective function to extract a set of first structure and motion parameters. Then the set of variables freed while minimizing of the objective function continues to extract a second set of structure and motion parameters that are substantially the same as the first set of structure and motion parameters.

Claims

1. A computerized method for performing a Euclidean reconstruction of an unknown three-dimensional (3-D) structure from a sequence of two-dimensional (2-D) images of the structure, the 3-D structure undergoing closed loop motion relative to the camera, the method comprising the steps of:

generating a closed-loop sequence of 2-D images of the unknown 3-D structure with an uncalibrated camera;

forming at least one set of consecutive 2-D images in the closed-loop sequence;

selecting feature tracking points captured from the unknown 3-D structure in the at least one set of 2-D images and determining a progression of one or more feature tracking points in the images of the at least one set;

estimating the relative orientation between the unknown 3-D structure and the camera with the progression of the feature tracking points;

fixing a set of parameters while minimizing an objective function to extract a set of first structure and motion parameters of the unknown 3-D structure; and

varying the set of parameters while minimizing the objective function to extract a set of second structure and motion parameters of the unknown 3-D structure until the set of second structure and motion parameters are substantially identical to the set of first structure and motion parameters.

2. The method of claim 1 wherein the closed-loop sequence of images is sparse.

3. The method of claim 2 wherein the closed-loop sequence of images includes less than fifty images.

4. The method of claim 1 wherein intrinsic parameters of the camera are unknown.

5. The method of claim 1 wherein a relative displacement between successive images of the closed-loop sequence is unknown.

6. The method of claim 1 further including the step of estimating a local rotation angle from the number of images in the closed-loop sequence.

7. The method of claim 1 further including the step of masking static features in the closed-loop sequence of images.

8. The method of claim 1 further including the steps of:

registering adjacent pairs of images in the closed-loop sequence of images; and

selecting the feature tracking points from the registered images.

9. The method of claim 8 wherein the image registration is spline-based.

10. The method of claim 1 wherein the selected feature tracking points are locally tracked until a termination condition is met.

11. The method of claim 1 wherein minimizing the objective function includes applying iterative Levenberg-Marquardt algorithm.

12. The method of claim 1 wherein the closed-loop sequence of images is measured while the structure undergoes a complete orbital motion.

13. The method of claim 12 wherein the axis of rotation with respect to the camera is unknown.

16. The method of claim 1 wherein a first and last images measured in the sequence are positionally adjacent to form the closed-loop.

14. The method of 13 wherein ellipses are fitted to the selected feature tracking points.

15. The method of claim 14 wherein the angle of tilt of the axis of rotation is estimated from the orientation and shape of the fitted ellipses.