Hand-held 3D vision system

• US 20030025788A1
• Filed: 08/06/2001
• Published: 02/06/2003
• Est. Priority Date: 08/06/2001
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Beardsley, Paul
• Granted Patent: US 6,781,618 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 5/2224 related to virtual studio applications

Abstract

A 3D model of a scene is constructed by acquiring first images of a scene having unknown characteristics with a first camera. Corresponding second images of a another scene having known characteristics are acquired by a second camera. The first and second cameras have a fixed physical relationship to each other. Only the second images are analyzed to determine corresponding positions of the second camera while acquiring the first images. Then, the first images are assembled into the 3D model using the corresponding positions and the fixed physical relationship of the first and second camera.

Claims

1. A method for constructing a 3D model of a scene, comprising:

acquiring first images of a first scene having unknown characteristics by a first camera;

acquiring corresponding second images of a second scene having known characteristics by a second camera, the first and second cameras having a fixed physical relationship to each other;

analyzing only the second images to determine corresponding positions of the second camera while acquiring the first images; and

assembling only the first images into a 3D model using the determined corresponding positions and the fixed physical relationship of the first and second camera.

3. The method of claim 1 wherein the known characteristics of the second scene include a rigid structure.

4. The method of claim 1 wherein the known characteristics include a position-registration pattern.

5. The method of claim 1 wherein the known characteristics include a fixed set of visual beacons, each with a unique distinguishable identity.

6. The method of claim 1 wherein the known characteristics include a moving object with known motion.

7. The method of claim 1 wherein the first scene includes a 3D object, and the 3D model assembled from the first images corresponds to the 3D object.

8. The method of claim 1 wherein pixels of a particular first image and a corresponding particular second image are disjoint.

9. The method of claim 1 wherein a particular first image and a corresponding particular second image are acquired concurrently.

10. The method of claim 1 wherein the first and second cameras are hand-held while acquiring the first and second images without constraining motions of the first and second cameras.

11. The method of claim 1 wherein the first camera is oriented at right angles with respect to the second camera, and the first and second cameras each have a field of view less than 90°

.

12. The method of claim 11 wherein the first camera is rotated in a plane while the second camera is rotated about an axis substantially perpendicular to the plane to acquire a panoramic view of the first scene.

13. The method of claim 1 wherein the first camera is oriented at an oblique angle with respect to the second camera, and the second camera is rotated in a circle while acquiring the first and second images to a view of all sides of an object in the first scene.

14. The method of claim 1 further comprising:

placing a position-registration pattern in the second scene, the position-registration pattern having a central blank central area to form the first scene;

placing an object in the central blank area; and

moving the first camera around the object, and aiming the second camera at the position-registration pattern while acquiring the first and second images.

15. The method of claim 1 wherein the first scene is an interior of a building and the second scene includes rigid architectural details of the building.

16. The method of claim 15 wherein the rigid architectural details are ceiling tiles.

17. The method of claim 1 further comprising:

acquiring corresponding third images of the second scene by a third camera, the third cameras having a fixed physical relationship to the first and second cameras, and the third images overlapping with the second images.

18. The method of claim 1 wherein the first and second cameras are mounted within a single camera body.

19. The method of claim 1 wherein the second camera is removable mounted on the first camera.

20. The method of claim 1 wherein the first camera has a high-resolution color sensor and the second camera has a low-resolution monochrome sensor.

21. The method of claim 1 wherein a field of view of the second camera is substantially larger than a field of view of the first camera.

22. The method of claim 1 further comprising:

calibrating the first and second cameras using a planar calibration pattern having identification information.

23. The method of claim 22 further comprising:

acquiring first calibration images of the calibration pattern with the first camera;

acquiring a corresponding second calibration image for each first image, the first image and the corresponding second image being non-overlapping;

determining a first planar homography between each first image and the calibration pattern to generate a first synthetic calibration pattern;

determining a second planar homography between each second image and the calibration pattern to generate a second synthetic calibration pattern; and

deriving a relative pose of the first and second camera from the first and second synthetic calibration patterns.

2. The method of claim 2 further comprising:

rendering the 3D model on a display device.

24. An apparatus method for constructing a 3D model of a scene, comprising:

a first camera configured to acquire first images of a first scene having unknown characteristics;

a second camera having a fixed physical relationship to the first camera, the second camera configured to acquire corresponding second images of a second scene having known characteristics;

means for analyzing only the second images to determine corresponding positions of the second camera while acquiring the first images; and

means for assembling only the first images into a 3D model using the determined corresponding positions and the fixed physical relationship of the first and second camera.