Method for extracting a three-dimensional model using appearance-based constrained structure from motion

• US 6,047,078 A
• Filed: 10/03/1997
• Issued: 04/04/2000
• Est. Priority Date: 10/03/1997
• Status: Expired due to Term

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Digital Equipment Corporation (HP Inc.)
• Inventors: Kang, Sing Bing
• Primary Examiner(s): Johns, Andrew W.
• Application Number:
  US08/943,774
• Time in Patent Office: 914 Days
• Field of Search: 382/107, 382/118, 382/154, 382/236, 382/294, 382/238, 345/419, 348/409, 348/415
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

In a computerized method, a three-dimensional model is extracted from a sequence of images that includes a reference image. Each image in the sequence is registered with the reference image to determine image features. The image features are used to recover structure and motion parameters using geometric constraints in the form of a wireframe mode. A predicted appearance is generated for each image using the recovered structure and motion parameters, and each predicted appearance is registered with the corresponding image. The recovering, generating, and registering steps are repeated until the average pixel value difference (color or intensity) between the predicted appearances and the corresponding images is less than a predetermined threshold.

Claims

1. A computerized method for extracting a three-dimensional model from a sequence of images of an object, comprising the steps of:

(a) registering each image in the sequence with a reference image to determine image features;

(b) recovering structure and motion parameters from the image features using geometric constraints;

(c) generating a predicted appearance for each image using the recovered structure and motion parameters;

(d) registering each predicted appearance with the corresponding image; and

repeating the recovering (b), generating (c) and registering (d) steps until a termination condition is reached.

2. The method of claim 1 wherein the registering is done using spline-based image registration.

3. The method of claim 1 wherein the geometric constraints are imposed directly on a 3-D wireframe model of the object.

4. The method of claim 1 wherein the object is a face.

5. The method of claim 1 wherein the sequence of images includes at least two images.

6. The method of claim 1 wherein the repeating terminates when an average pixel value difference between each predicted appearance and each corresponding image is less than a predetermined threshold.

7. The method of claim 1 wherein the sequence of images are provided by a camera and parameters of the camera are generally unknown.

8. The method of claim 3 wherein the wireframe model is composed of triangular facets.