System and a method of three-dimensional modeling and restitution of an object

• US 20030202089A1
• Filed: 02/21/2003
• Published: 10/30/2003
• Est. Priority Date: 02/21/2002
• Status: Active Grant

• Current Assignee: BASF Corporation (BASF SE), Yodea
• Original Assignee: BASF Corporation (BASF SE), Yodea
• Inventors: Alhadef, Laurent, Alhadef, Bernard
• Granted Patent: US 7,187,401 B2
• Time in Patent Office: Days
• US Class Current: 348/42
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 21/188 for accumulated errors, e.g. by coupling inertial systems with absolute positioning systems
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

The invention relates to a system for three-dimensionally modeling and restitution a subject, the system comprising a housing suitable for being carried in the hand by an operator, the housing having an optical unit with stereoscopic digital video cameras co-operating with an inertial navigation unit having a trihedron of optical fiber gyros and a trihedron of accelerometers for generating position and displacement signals in a frame of reference. The housing may also include, in particular, a telemeter laser, an infrared video camera, a GPS unit, inclinometers, and tri-flux magnetometers. A second subassembly connected to the housing by an optical and/or electrical connection cable contains an electrical power supply, electronic circuits, and a microcomputer for processing signals delivered by the housing and for generating a three-dimensional model of the subject, implementing a photogrammetric technique.

Claims

1. / A system for three-dimensional modeling and restitution of a subject of a determined scene in a three-dimensional space, the system comprising:

a first subsystem comprising, placed in a single housing that is portable for an operator, at least one video camera for acquiring sequences of digital images of said subject, co-operating with first and second series of measurement means generating measurement signals measuring the attitude, location, and displacement of said video camera relative to three axes in a three-dimensional frame of reference, said first and second series of measurement means each comprising devices generating measurements relative to each of said axis in said frame of reference;

a second subsystem comprising at least digital data processor means receiving the data signals characterizing said sequences of acquired digital images and said measurement signals measuring the attitude, location, and displacement of said video camera in said frame of reference, and responding to said data signals of acquired digital image sequences and said location and displacement signals to generate a three-dimensional model of said subject; and

connection means between said first and second subsystems, said connection means conveying at least said data signals of acquired digital image sequences and said location and displacement signals.

2. / A system according to claim 1, wherein said video camera comprises a semiconductor charge-coupled device that is sensitive to polychromatic light together with interchangeable objective lenses of different focal lengths.

3. / A system according to claim 1, wherein said first series of measurement means is constituted by a trihedron of three gyros, each gyro being in alignment with one of said axes of said frame of reference.

4. / A system according to claim 1, wherein said second series of measurement means is constituted by a trihedron of three accelerometers, each accelerometer being in alignment with one of said axes of said frame of reference.

5. / A system according to claim 1, wherein said measurement means generating signals measuring attitude, location, and displacement of said video camera relative to said frame of reference comprise locating means operating on electromagnetic signals transmitted by “

GPS”

type artificial satellites generating signals for locating said acquired digital image sequences.

6. / A system according to claim 1, wherein said measurement means generating the signals measuring attitude, location, and displacement of said video camera relative to said frame of reference comprise at least one magnetic sensor responsive to coded signals transmitted by magnetic beacons disposed at determined locations of known coordinates relative to said frame of reference.

7. / A system according to claim 1, wherein said first subsystem includes at least one inclinometer generating signals for correlating with at least the location signals generated by said measurement means.

8. / A system according to claim 1, wherein said first subsystem includes at least one mono- or multi-flux magnetometer generating signals for correlating with at least the location signals generated by said measurement means.

9. / A system according to claim 1, wherein a third subsystem is provided comprising at least one coordinate-initializing device comprising a support, said device being placed at a determined location in said scene of coordinates known relative to said frame of reference, and wherein said housing for transporting said first subsystem includes an initialization terminal for coupling in temporary manner with said support so as to reset said measurement means generating said signals measuring the attitude, the location, and the displacement on said initialization coordinates.

10. / A system according to claim 1, wherein said first subsystem further comprises a second video camera identical to said first video camera and servo-controlled thereto on an articulated arm in such a manner as to acquire first and second digital image sequences of said subject using a stereoscopic method, and wherein data signals characterizing said first and second acquired digital image sequences are transmitted to said digital data processor means of said second subsystem and are used to facilitate so-called “

outlining”

operations on the images in which the points constituting the outlines of determined objects in said images are detected.

11. / A system according to claim 1, wherein said first subsystem further comprises an infrared video camera for acquiring digitized thermal images of said subject, and wherein data signals characterizing said acquired digital thermal images are transmitted to said digital data processor means of said second subsystem and used to facilitate so-called “

outlining”

operations on the images in which the points constituting the outlines of determined objects in said images are detected.

12. / A system according to claim 1, wherein said first subsystem further comprises a telemeter.

13. / A system according to claim 1, wherein:

said second subsystem is contained in a second housing for placing in a bag of the backpack type suitable for being transported by said operator;

said system includes electrical power supply means for powering said digital data processing means and said first subsystem via said connection means; and

said digital data processor means comprise at least a microcomputer co-operating at least with means for storing said signals received from said first subsystem together with determined stored programs for processing purposes, interface and matching members for use with said received signals, data input means, and display means for displaying data processed by said microcomputer and for displaying said three-dimensional model.

14. / A method of modeling and restitution in three dimensions a subject of a determined scene in a three-dimensional space implementing the system according to claim 1, the method comprising at least the following steps:

taking a video recording to acquire digital image sequences of said subject by moving, in said space, said housing that includes at least one video camera, and generating the corresponding signals, and using said measurement means to generate at least two series of signals characterizing attitude, location, and displacement data of said housing while taking the video recording;

using said connection means to transfer said signals characterizing said digital image sequences and said signals characterizing the attitude, position, and displacement of said housing to said associated digital data processor means;

storing said transmitted data in said storage means associated with said digital data processing means; and

processing said signals by determined software means under the control of said digital data processor means in such a manner as to generate a three-dimensional model of said subject by combining said data associated with said signals characterizing said acquired digital image sequences and said signals acquiring the attitude, position, and displacement of said housing while taking said video recording.

15. / A method according to claim 14, including an additional step performed after said step of storing said transmitted data, the additional step consisting in a so-called “

outlining”

operation using determined software means under the control of said digital data processor means so as to increase the accuracy with which points are determined that characterize the outlines of determined objects contained in said acquired image sequences.

16. / A method according to claim 14, including an additional step consisting in resetting said measurement means that generate said signals measuring attitude, location, and displacement, resetting being performed by a determined software processing operation on the images acquired during said acquisition step so as to compensate for drift of said measurement means.

17. / A method according to claim 14, including an additional step consisting in measuring the angular speed of said subject so as to acquire signals for adjusting parameters associated with taking said video recording.

18. / A method according to claim 14, including an additional step consisting in making photographs of a so-called “

orthophotographic”

type on the basis of data associated with said three-dimensional models obtained by said digital data processing.

19. / A method according to claim 14, including an iterative additional step for navigating in a three-dimensional universe on the basis of a three-dimensional model so as to verify the parameters that have been determined for said model and so as to undertake conditional modification of said three-dimensional model as a function of the result of said verification, by acquiring new image signals and new attitude, location, and displacement measurements.

20. / A method according to claim 14, including an additional step consisting, while taking said video, in acquiring redundant data signals characterizing said digital images and/or said attitude, location, and displacement measurements of said housing in such a manner as to improve the accuracy of said measurements by determined signal processing.