Method and apparatus for producing an enhanced 3D model of an environment or an object
First Claim
1. A method of producing an enhanced three-dimensional (3D) model of an environment or an object, comprising the steps of:
- a) acquiring 3D information of an environment or an object from at least one range sensing device;
b) producing a 3D model of the environment or object from the acquired 3D information suitable for visualization, analysis or post-processing;
c) projecting a light pattern onto a region of interest of the environment or object;
d) acquiring data from an additional sensing device trained on the region of interest of the environment or object onto which the light pattern is projected, the additional sensing device being positioned independently of the at least one range sensing device;
e) registering the acquired data from the additional sensing device within the region of interest with the 3D information from the at least one range sensing device by detecting said light pattern observed by the additional sensing device and establishing correspondence between the data from the additional sensing device and the data from the at least one range sensing device; and
f) combining the data from the additional sensing device with the 3D model to give an enhanced 3D model.
8 Assignments
0 Petitions
Accused Products
Abstract
The present invention provides a system (method and apparatus) for creating photorealistic 3D models of environments and/or objects from a plurality of stereo images obtained from a mobile stereo camera and optional monocular cameras. The cameras may be handheld, mounted on a mobile platform, manipulator or a positioning device. The system automatically detects and tracks features in image sequences and self-references the stereo camera in 6 degrees of freedom by matching the features to a database to track the camera motion, while building the database simultaneously. A motion estimate may be also provided from external sensors and fused with the motion computed from the images. Individual stereo pairs are processed to compute dense 3D data representing the scene and are transformed, using the estimated camera motion, into a common reference and fused together. The resulting 3D data is represented as point clouds, surfaces, or volumes. The present invention also provides a system (method and apparatus) for enhancing 3D models of environments or objects by registering information from additional sensors to improve model fidelity or to augment it with supplementary information by using a light pattern projector. The present invention also provides a system (method and apparatus) for generating photo-realistic 3D models of underground environments such as tunnels, mines, voids and caves, including automatic registration of the 3D models with pre-existing underground maps.
76 Citations
26 Claims
-
1. A method of producing an enhanced three-dimensional (3D) model of an environment or an object, comprising the steps of:
-
a) acquiring 3D information of an environment or an object from at least one range sensing device; b) producing a 3D model of the environment or object from the acquired 3D information suitable for visualization, analysis or post-processing; c) projecting a light pattern onto a region of interest of the environment or object; d) acquiring data from an additional sensing device trained on the region of interest of the environment or object onto which the light pattern is projected, the additional sensing device being positioned independently of the at least one range sensing device; e) registering the acquired data from the additional sensing device within the region of interest with the 3D information from the at least one range sensing device by detecting said light pattern observed by the additional sensing device and establishing correspondence between the data from the additional sensing device and the data from the at least one range sensing device; and f) combining the data from the additional sensing device with the 3D model to give an enhanced 3D model. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 22, 23, 24, 25, 26)
-
-
9. An apparatus for producing an enhanced three-dimensional (3D) model of an environment or an object, comprising the steps of:
-
a) at least one range sensing device for acquiring 3D information of an environment or an object; b) processing means for producing a 3D model of the environment or object from the acquired 3D information suitable for visualization, analysis or post-processing; c) light pattern projection means for projecting a pattern of light onto a region of interest of the environment or object; d) an additional sensing device trained on the region of interest for acquiring data from the region of interest onto which the light pattern is projected, said additional sensing device being positioned independent of said at least one range sensing device; and e) processing means configured for registering the acquired data from the additional sensing device trained on the region of interest containing the projected light pattern with the 3D information from the range sensing device by detecting the light pattern observed by the additional sensing device and establishing correspondence between the data from the additional sensing device and the data from the at least one range sensing device, the processing means being configured for combining the data from the additional sensing device with the 3D model to give an enhanced 3D model. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21)
-
Specification