System and a method of three-dimensional modeling and restitution of an object

• US 7,187,401 B2
• Filed: 02/21/2003
• Issued: 03/06/2007
• Est. Priority Date: 02/21/2002
• Status: Active Grant

• Current Assignee: BASF Corporation (BASF SE), Yodea
• Original Assignee: BASF Corporation (BASF SE), Yodea
• Inventors: Alhadef, Laurent, Alhadef, Bernard
• Primary Examiner(s): VO, TUNG T
• Application Number:
  US10/369,530
• Publication Number:
  US 20030202089A1
• Time in Patent Office: 1,474 Days
• Field of Search: 348 42- 46, 348 96- 97, 348/157, 348/74, 700/90, 396/324, 702/150, 382/154
• US Class Current: 348/42
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 21/188 for accumulated errors, e.g. by coupling inertial systems with absolute positioning systems
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A system for three-dimensionally modeling and restitution a subject, the system comprising a housing suitable for being carried in the hand by an operator, the housing having an optical unit with stereoscopic digital video cameras co-operating with an inertial navigation unit having a trihedron of optical fiber gyros and a trihedron of accelerometers for generating position and displacement signals in a frame of reference. The housing may also include, in particular, a telemeter laser, an infrared video camera, a GPS unit, inclinometers, and tri-flux magnetometers. A second subassembly connected to the housing by an optical and/or electrical connection cable contains an electrical power supply, electronic circuits, and a microcomputer for processing signals delivered by the housing and for generating a three-dimensional model of the subject, implementing a photogrammetric technique.

Claims

1. A system for three-dimensional modeling and restitution of a subject of a determined scene in a three-dimensional space having three axes in a three-dimensional frame of reference, the system comprising:

a first subsystem comprising, at least one portable video camera configured to acquire sequences of digital images of said subject, first and second series of measurement devices cooperating with said at least one portable video camera and generating data measurement signals relative to each of said axes in said three-dimensional frame of reference, so as to characterize the attitude, location, and displacement of said at least one portable video camera relative to said three axes in said three-dimensional frame of reference;

a second subsystem comprising at least a digital data processor configured to receive signals characterizing said sequences of acquired digital images and said data measurement signals, and responding to said received signals characterizing said acquired sequences of digital image and said data measurement signals to generate a three-dimensional model of said subject; and

connection means coupled between said first and second subsystems, said connection means configured to convey at least said signals characterizing said acquired sequences of digital images and said data measurement signals, and wherein

said first series of measurement devices further comprises a trihedron of three gyros, each gyro being in alignment with one of said axes of said frame of reference; and

said second series of measurement devices further comprises a trihedron of three accelerometers, each accelerometer being in alignment with one of said axes of said frame of reference.

2. A system according to claim 1, wherein said at least one portable video camera comprises a semiconductor charge-coupled device that is sensitive to polychromatic light together with interchangeable objective lenses of different focal lengths.

3. A system according to claim 1, wherein said first and second series of measurement devices comprise locating means configured to operate on electromagnetic signals transmitted by global positioning system (“

GPS”

)-type artificial satellites configured to generate signals for locating said acquired sequences of digital images.

4. A system according to claim 1, wherein said first and second series of measurement devices comprise at least one magnetic sensor responsive to coded signals transmitted by magnetic beacons, said magnetic beacons disposed at determined locations of known coordinates relative to said frame of reference.

5. A system according to claim 1, wherein said first subsystem further includes at least one inclinometers configured to generate signals for correlating with at least the data measurement signals corresponding to the location of said at least one portable video camera generated by said first and second series of measurement devices.

6. A system according to claim 1, wherein said first subsystem further includes at least one mono- or multi-flux magnetometer configured to generate signals for correlating with at least the data signals corresponding to the location of said at least one portable video camera generated by said first and second series of measurement devices.

7. A system according to claim 1, further comprising a third subsystem, said third subsystem including:

at least one coordinate-initializing device comprising a support, and configured to determine initialization coordinates, said at least one coordinate-initializing device being placed at a determined location in said scene of coordinates known relative to said frame of reference, and wherein

said system further includes a housing for transporting said first subsystem and, said housing including an initialization terminal for coupling in temporary manner with said support so as to reset said measurement devices on said initialization coordinates.

8. A system according to claim 1, wherein said first subsystem further comprises:

a second portable video camera, identical to said first portable video camera, and servo-controlled thereto on an articulated arm so as to acquire said sequences of digital images of said subject using a stereoscopic method, and wherein

data signals characterizing said acquired sequences of digital images are transmitted to said digital data processor, and are used to facilitate “

outlining”

operations on the images in which the points constituting the outlines of determined objects in said images are detected.

9. A system according to claim 1, wherein said first subsystem further comprises an infrared video camera configured to acquire digitized thermal images of said subject, and wherein data signals characterizing said acquired digital thermal images are transmitted to said digital data processor, and used to facilitate “

outlining”

operations on the images in which the points constituting the outlines of determined objects in said images are detected.

10. A system according to claim 1, wherein said first subsystem further comprises a telemeter.

11. A system according to claim 1, further comprising:

a first housing configured to house said first subsystem and a second housing configured to house said second subsystem, said second housing being of a type suitable for placing in a bag of the backpack type for transport by an operator;

an electrical power supply configured to power said digital data processor and said first subsystem via said connection means; and

said digital data processor comprises;

at least a microcomputer co-operating at least with a storage device configured to store said signals received from said first subsystem together with determined stored programs for processing purposes;

interface and matching members configured for use with said received signals;

a data input; and

a display for displaying data processed by said microcomputer and for displaying said three-dimensional model.

12. A method of modeling and restitution in three dimensions, a subject of a determined scene in a three-dimensional space having three axes in a three-dimensional frame of reference, the method comprising:

recording a video using at least one portable video camera to acquire digital image sequences of said subject, and generating data measurement signals, comprising at least two series of signals characterizing attitude, location, and displacement data of said at least one portable video camera;

transmitting said signals characterizing said acquired digital image sequences and said data measurement signals characterizing the attitude, position, and displacement of said at least one portable video camera to an associated digital data processor;

storing data from said transmitted data signals in a store associated with said digital data processor; and

processing said signals by determined software under the control of said digital data processor so as to generate a three-dimensional model of said subject by combining said data associated with said signals characterizing said acquired digital image sequences and said data signals characterizing the attitude, location, and displacement of said at least one portable video camera while recording said video, and wherein the generating data measurement signals is performed by;

a first series of measurement devices including a trihedron of three gyros, where each gyro is aligned with one of said axes of said frame of reference; and

a second series of measurement devices including a trihedron of three accelerometers, where each accelerometer is aligned with one of said axes of said frame of reference.

13. A method according to claim 12, further comprising, after said step of storing said transmittal data, using determined software under the control of said digital data processor to perform an “

outlining”

operation so as to increase the accuracy with which points are determined that characterize the outlines of determined objects contained in said acquired digital image sequences.

14. A method according to claim 12, further comprising resetting said data measurement signals, said resetting being performed by a determined software processing operation on the images acquired by the video recording so as to compensate for drift of said data measurement signals.

15. A method according to claim 12, further comprising measuring angular speed of said subject so as to acquire signals for adjusting parameters associated with said recording a video.

16. A method according to claim 12, further comprising making photographs of an “

orthophotographic”

type on the basis of data associated with said three-dimensional models obtained by said digital data processor.

17. A method according to claim 12, further comprising iteratively navigating in a three-dimensional universe on the basis of said three-dimensional model so as to verify parameters that have been determined for said three-dimensional model, and so as to undertake conditional modification of said three-dimensional model as a function of the result of said verification, by acquiring new digital image sequences and new data measurement signals.

18. A method according to claim 12, further comprising, while recording said video, acquiring redundant signals characterizing said digital image sequences and/or said data measurement signals so as to improve the accuracy of said measurements by determined signal processing.