Method and apparatus for high resolution 3D imaging as a function of camera position, camera trajectory and range

• US 7,991,222 B2
• Filed: 02/05/2010
• Issued: 08/02/2011
• Est. Priority Date: 07/06/2004
• Status: Active Grant

• Current Assignee: Topcon Positioning Systems Incorporated (Topcon Corporation)
• Original Assignee: Topcon Positioning Systems Incorporated (Topcon Corporation)
• Inventors: Chen, William, Dimsdale, Jerry, Lewis, Andrew
• Primary Examiner(s): ZARKA, DAVID PETER
• Application Number:
  US12/701,371
• Publication Number:
  US 20100188504A1
• Time in Patent Office: 543 Days
• Field of Search: 382/154
• US Class Current: 382/154
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 15/002 Active optical surveying means optical plumbing G01C15/105
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/183 Compensation of inertial measurements, e.g. for temperature effects
• CPC Class Codes: G01S 17/89 for mapping or imaging

Abstract

A system and method for imaging a three-dimensional scene having one or more objects. The system includes a light source, a detector array, a timing circuit, an inertial guidance system and a processor connected to the timing circuit and the inertial guidance system. The light source generates an optical pulse and projects the optical pulse on an object so that it is reflected as a reflected pulse. The detector array includes a plurality of detectors, wherein the detectors are oriented to receive the reflected pulse. The timing circuit determines when the reflected pulse reached detectors on the detector array. The inertial guidance system measures angular velocity and acceleration. The processor forms a composite image of the three-dimensional scene as a function of camera position and range to objects in the three-dimensional scene.

Claims

1. A method of calculating trajectory, comprising:

providing a device wherein the device includes an inertial guidance system and an array of optical detectors;

transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse;

detecting arrival of the first reflected pulse at the array of detectors;

forming a first range image from data corresponding to the first reflected pulse arriving at the array of detectors;

moving the camera relative to objects in the scene;

monitoring angular velocity and acceleration via the inertial guidance system;

transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse;

detecting arrival of the second reflected pulse at the array of detectors;

forming a second range image from data corresponding to the second reflected pulse arriving at the array of detectors;

determining registration-derived camera motion as a function of registration of the first and second range images; and

determining an estimated camera trajectory as a function of the registration-derived camera motion, the angular velocity measured by the inertial guidance system and the acceleration measured by the inertial guidance system using a process that minimizes sensor measurement errors and image alignment errors.

2. The method of claim 1, wherein the inertial guidance system includes gyroscopes and accelerometers.

3. The method of claim 1, wherein the inertial guidance system includes a global positioning system, gyroscopes and accelerometers.

4. The method of claim 1, wherein determining includes applying a Kalman filter to each of the registration-derived camera motion, the angular velocity measured by the inertial guidance system, and the acceleration measured by the inertial guidance system to determine the estimated camera trajectory.

5. The method of claim 4, wherein the inertial guidance system includes a gyroscope and an accelerometer.

6. The method of claim 1, wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein determining includes applying a Kalman filter to each of the registration-derived camera motion, measurements derived from the global positioning system, the angular velocity and the acceleration to arrive at the estimated camera trajectory.

7. A device, comprising:

a light source, wherein the light source generates an optical pulse and projects the optical pulse on an object so that it is reflected as a reflected pulse;

a detector array, wherein the detector array may includes a plurality of detectors, wherein the detectors are oriented to receive the reflected pulse;

a timing circuit connected to the detector array, wherein the timing circuit determines when the reflected pulse reached detectors on the detector array;

an inertial guidance system which measures angular velocity and acceleration; and

a processor connected to the timing circuit and the inertial guidance system, wherein the processor calculates registration-derived camera motion as a function of registration of range images from different pulses and wherein the processor determines estimated camera trajectory by combining information corresponding to the registration-derived camera motion with information corresponding to the angular velocity and acceleration;

wherein combining includes calculating estimated camera trajectory using a process that minimizes sensor measurement errors and image alignment errors.

8. The camera according to claim 7, wherein the processor combines information corresponding to the registration-derived camera motion with information corresponding to the angular velocity and acceleration received from the inertial guidance system by applying a Kalman filter to the registration-derived camera motion and to the angular velocity and acceleration measured by the inertial guidance system.

9. The device of claim 7, wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein determining an estimated camera trajectory includes applying a Kalman filter to each of the registration-derived camera motion, satellite range measurements generated by the global positioning system, the angular velocity and the acceleration to arrive at the estimated camera trajectory.

10. The device of claim 7, wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein determining an estimated camera trajectory includes applying a Kalman filter to each of the registration-derived camera motion, position estimates derived from the global positioning system, the angular velocity and the acceleration to arrive at the estimated camera trajectory.

11. The device of claim 7, wherein the processor forms a composite image of a three-dimensional scene as a function of the estimated camera trajectory and range measurements to objects in the three-dimensional scene.

12. The device according to claim 7, wherein the light source is a laser and wherein the detectors are avalanche photodiodes operating in non-linear Geiger mode.

13. A method of imaging a three-dimensional scene having one or more objects, comprising:

providing a camera, wherein the camera includes an inertial guidance system and an array of optical detectors;

transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse;

detecting arrival of the first reflected pulse at the array of detectors;

forming a first range image from data corresponding to the first reflected pulse arriving at the array of detectors;

moving the camera relative to objects in the scene;

monitoring angular velocity and acceleration via the inertial guidance system;

transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse;

detecting arrival of the second reflected pulse at the array of detectors;

forming a second range image from data corresponding to the second reflected pulse arriving at the array of detectors;

determining registration-derived camera motion as a function of registration of the first and second range images;

determining an estimated camera trajectory as a function of the registration-derived camera motion, the angular velocity measured by the inertial guidance system and the acceleration measured by the inertial guidance system using a process that minimizes sensor measurement errors and image alignment errors; and

forming a composite image of the three-dimensional scene as a function of the first and second range images and the estimated camera trajectory.

14. The method of claim 13, wherein the inertial guidance system includes gyroscopes and accelerometers.

15. The method of claim 13, wherein the inertial guidance system includes a global positioning system, gyroscopes and accelerometers.

16. The method of claim 13, wherein determining includes applying a Kalman filter to each of the registration-derived camera motion, the angular velocity measured by the inertial guidance system, and the acceleration measured by the inertial guidance system to determine the estimated camera trajectory.

17. The method of claim 16, wherein the inertial guidance system includes a gyroscope and an accelerometer.

18. The method of claim 13, wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein determining includes applying a Kalman filter to each of the registration-derived camera motion, position estimates derived from the global positioning system, the angular velocity and the acceleration to arrive at the estimated camera trajectory.

19. A device, including:

a pulsed light source;

means for transmitting light toward objects in a three-dimensional scene;

optics for collecting light during the time for light to transmit from said pulsed light source, reflect from said objects, be collected and focused by said optics;

a plurality of detectors oriented for receiving the collected light and converting the collected light into an electrical signal;

timing means, connected to the detectors, for determining a transit time for a pulse to leave the pulsed light source, reflect off the objects and return to one or more of the plurality of deflectors;

an inertial guidance system which measures angular velocity and acceleration; and

a processor connected to the timing means and the inertial guidance system, wherein the processor calculates registration-derived camera motion as a function of registration of range images from different pulses and wherein the processor determines estimated camera trajectory by combining information corresponding to the registration-derived camera motion with information corresponding to the angular velocity and acceleration;

wherein the processor combines information corresponding to the registration-derived camera motion with information corresponding to the angular velocity and acceleration received from the inertial guidance system so as to minimize sensor measurement errors and image alignment errors.

20. The device according to claim 19, wherein the light source is a laser.

21. The device according to claim 20, wherein the detectors are avalanche photodiodes operating in non-linear Geiger mode.

22. The device according to claim 21, wherein the processor forms a composite image of the scene as a function of range measurements to objects in the scene and the estimated camera trajectory.

23. The device according to claim 19, wherein the processor combines information corresponding to the registration-derived camera motion with information corresponding to the angular velocity and acceleration received from the inertial guidance system by applying a Kalman filter to the registration-derived camera motion and to the angular velocity and acceleration measured by the inertial guidance system.

24. The device according to claim 23, wherein the light source is a laser.

25. The device according to claim 24, wherein the detectors are avalanche photodiodes operating in non-linear Geiger mode.

26. A method, comprising:

providing a camera, wherein the camera includes an inertial guidance system and one or more optical detector arrays;

transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse;

detecting arrival of the first reflected pulse at the one or more optical detector arrays;

recording first range information as a function of the arrival of the first reflected pulse;

forming a first range image from the first range information;

moving the camera relative to objects in the scene;

recording angular velocity and acceleration information;

transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse;

detecting arrival of the second reflected pulse at the one or more optical detector arrays;

recording second range information as a function of the arrival of the second reflected pulse;

forming a second range image from the second range information;

determining registration-derived camera motion as a function of registration of the first and second range images;

estimating camera trajectory as a function of the registration-derived camera motion and of the recorded angular velocity information and the recorded acceleration information; and

wherein estimating camera trajectory includes applying a process to each of the registration-derived camera motion, the angular velocity information and the recorded acceleration information to arrive at the estimated camera trajectory that minimizes sensor measurement errors and image alignment errors.

27. The method according to claim 26, wherein the method further comprises forming a composite image of the three-dimensional scene as a function of the estimated camera trajectory, the recorded first range information and the recorded second range information.

28. The method according to claim 26, wherein the inertial guidance system includes a gyroscope and an accelerometer and wherein estimating camera trajectory includes applying a Kalman filter to each of the registration-derived camera motion, the angular velocity information and the recorded acceleration information to arrive at the estimated camera trajectory.

29. The method according to claim 28, wherein the method further comprises forming a composite image of the three-dimensional scene as a function of the estimated camera trajectory, the recorded first range information and the recorded second range information.

30. The method according to claim 26, wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein estimating camera trajectory includes applying a Kalman filter to each of the registration-derived camera motion, position estimates derived from the global positioning system, the angular velocity information and the recorded acceleration information to arrive at the estimated camera trajectory.

31. The method according to claim 30, wherein the method further comprises forming a composite image of the three-dimensional scene as a function of the estimated camera trajectory, the recorded first range information and the recorded second range information.

32. A system, comprising:

a focal plane array range imaging optical pulsing camera; and

an inertial guidance system, connected to the camera, wherein the inertial guidance system provides angular velocity and acceleration information corresponding to movements of the camera; and

a processor, connected to the camera and to the inertial guidance system, wherein the processor calculates registration-derived camera motion as a function of registration of range images from different pulses, wherein the processor estimates camera trajectory by combining information corresponding to the registration-derived camera motion with information corresponding to the angular velocity and acceleration measured by the inertial guidance system and wherein the processor forms a composite image of the three-dimensional scene as a function of the estimated camera trajectory and range information measured by the camera;

wherein the processor combines information corresponding to the registration-derived camera motion with information corresponding to the angular velocity and acceleration received from the inertial guidance system so as to minimize sensor measurement errors and image alignment errors.

33. The system of claim 32, wherein the processor combines information corresponding to the registration-derived camera motion with information corresponding to the angular velocity and acceleration received from the inertial guidance system by applying a Kalman filter to the registration-derived camera motion and to the angular velocity and acceleration measured by the inertial guidance system.

34. The system according to claim 32, wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein the processor estimates camera trajectory by applying a Kalman filter to each of the registration-derived camera motion, position estimates derived from the global positioning system, the angular velocity information and the recorded acceleration information to arrive at the estimated camera trajectory.

35. A method for collecting range images from a focal plane array imaging optical pulsing camera and placing them on a common coordinate system, the method comprising:

attaching an inertial guidance system to the camera, wherein the inertial guidance system includes a set of gyroscopes and accelerometers;

continuously measuring the set of gyroscopes and accelerometers;

recording first range information at a first camera position;

recording second range information at a second camera position;

determining an estimated camera motion in moving the camera from the first to the second camera position consistent with the first and second range information and the measurements from the set of gyroscopes and accelerometers attached to the camera, wherein determining an estimated camera motion includes combining registration-derived camera motion information reflecting movement of the camera between the first and second range images with inertial guidance information reflecting movement of the camera as measured by the set of gyroscopes and accelerometers attached to the camera to arrive at an estimated camera trajectory; and

forming a composite image from the first and second range information;

wherein forming a composite image includes forming first and second range images from the first and second range information and determining alignment of the first and second range images as a function of the estimated camera motion;

wherein combining includes applying a process to each of the registration-derived camera motion information and the inertial guidance information to arrive at the estimated camera trajectory that minimizes sensor measurement errors and image alignment errors.

36. The method of claim 35, wherein combining includes applying a Kalman filter to the registration-derived camera motion information and the inertial guidance information to arrive at the estimated camera trajectory.

37. The method of claim 35, wherein the inertial guidance system further includes a global positioning system and wherein combining includes applying a Kalman filter to the registration-derived camera motion information, to global positioning system information and to the inertial guidance information to arrive at the estimated camera trajectory.

38. A method, comprising:

receiving information reflective of a sequence of scenes captured by an imaging device;

receiving angular velocity and acceleration information via an inertial guidance system, wherein the angular velocity information and the acceleration information reflect movement of the imaging device between each of the scenes;

determining changes in scene registration between two or more scenes from the sequence of scenes; and

determining an estimated trajectory of the imaging device by combining the angular velocity and acceleration measured by the inertial guidance system with the movement of the imaging device estimated from the changes in scene registration to arrive at an estimated trajectory for the imaging device;

wherein combining the angular velocity and acceleration measured by the inertial guidance system with the movement of the imaging device reflected by changes in the scene registration includes passing the angular velocity information and the acceleration information received from the inertial guidance system and data representing the movement of the imaging device reflected by changes in scene registration through a process that minimizes sensor measurement errors and image alignment errors.

39. The method of claim 38, wherein the method further comprises forming a composite image as a function of the scenes captured by an imaging device and the estimated trajectory of the imaging device.

40. The method of claim 38, wherein combining the angular velocity and acceleration measured by the inertial guidance system with the movement of the imaging device reflected by changes in scene registration includes passing data representing the angular velocity and acceleration measured by the inertial guidance system and data representing the movement of the imaging device reflected by changes in scene registration through a Kalman filter.

41. The method of claim 38, wherein combining the angular velocity and acceleration measured by the inertial guidance system with the movement of the imaging device reflected by changes in scene registration includes passing data representing the angular velocity and acceleration measured by the inertial guidance system, data corresponding to global positioning measurements and data representing the movement of the imaging device reflected by changes in scene registration through a Kalman filter.

42. The method according to claim 38, wherein the information reflective of a sequence of scenes captured by the imaging device includes range images representative of each scene.

43. A device for imaging a three-dimensional scene having one or more objects, including:

a pulsed light source;

means for transmitting a first and second light pulse toward said objects;

optics for collecting light during the time for light to transit from said pulsed light source, reflect from said objects, be collected and focused by said optics;

an array of optical detectors oriented for receiving the collected light and converting the collected light into an electrical signal;

timing means, connected to the optical detectors, for determining a transit time for a pulse to leave the pulsed light source, reflect off the objects and return to one or more of the plurality of detectors;

an inertial guidance system which measures angular velocity and acceleration for the device; and

a processor connected to the timing means and the inertial guidance system, wherein the processor calculates registration-derived camera motion as a function of a first range image formed from the first light pulse and a second range image formed from the second light pulse and determines an estimated device trajectory as a function of both the registration-derived camera motion and the angular velocity and acceleration of the device and forms a composite image of the three-dimensional scene as a function of the estimated device trajectory and the range measured to objects in the three-dimensional scene;

wherein the processor combines information corresponding to the registration-derived camera motion with information corresponding to the angular velocity and acceleration of the device so as to minimize sensor measurement errors and image alignment errors.

44. The device according to claim 43, wherein the processor applies a Kalman filter to each of the registration-derived camera motion, the angular velocity measured by the inertial guidance system and the acceleration measured by the inertial guidance system to determine the estimated device trajectory.

45. The device according to claim 44, wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein the Kalman filter corrects the estimated camera trajectory as a function of a global positioning system measurement.

46. The device according to claim 43, wherein the processor uses registrations between non-consecutive scans and registrations between consecutive scans to determine an estimate of camera location for each scan.

47. The device according to claim 43, wherein the inertial guidance system includes a global positioning system, gyroscopes and accelerometers and wherein the processor applies a Kalman filter to each of the registration-derived camera motion, changes in position measured by the global positioning system, the angular velocity measured by the inertial guidance system and the acceleration measured by the inertial guidance system to determine the estimated device trajectory.

48. The device according to claim 47, wherein the changes in position measured by the global positioning system include satellite range measurements.

49. The device according to claim 47, wherein the changes in position measured by the global positioning system include changes in earth position as calculated by the global positioning system.

50. A method of imaging a three-dimensional scene having one or more objects, comprising:

providing a camera, wherein the camera includes an inertial guidance system and one or more optical detector arrays;

transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse;

detecting arrival of the first reflected pulse at the one or more optical detector arrays;

recording first range information as a function of the arrival of the first reflected pulse;

forming a first range image from the first range information;

moving the camera relative to objects in the scene;

recording angular velocity and acceleration information;

transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse;

detecting arrival of the second reflected pulse at the one or more optical detector arrays;

recording second range information as a function of the arrival of the second reflected pulse;

forming a second range image from the second range information;

determining a registration-derived camera motion as a function of registration of the first and second range images;

estimating camera trajectory, wherein estimating camera trajectory includes determining an estimated camera trajectory as a function of the registration-derived camera motion and of the recorded angular velocity information and the recorded acceleration information using a process that minimizes sensor measurement error and image alignment errors; and

forming a composite image of the three-dimensional scene as a function of the estimated camera trajectory, the recorded first range information and the recorded second range information.

51. The method according to claim 50, wherein determining an estimated camera trajectory includes applying a Kalman filter to the registration-derived camera motion and to the angular recorded angular velocity information and the recorded acceleration information.

52. The method according to claim 50, wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein determining an estimated camera trajectory includes applying a Kalman filter to each of the registration-derived camera motion, satellite range measurements generated by the global positioning system, and the angular recorded angular velocity information and the recorded acceleration information.

53. The method according to claim 50, wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein determining an estimated camera trajectory includes applying a Kalman filter to each of the registration-derived camera motion, position estimates derived from the global positioning system, and the angular recorded angular velocity information and the recorded acceleration information.