Method and apparatus for high resolution 3D imaging as a function of camera position, camera trajectory and range
First Claim
1. A method of imaging a three-dimensional scene having one or more objects, comprising:
- providing a camera, wherein the camera includes an inertial guidance system and an array of optical detectors;
transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse;
detecting arrival of the first reflected pulse at the array of detectors;
forming a first scene image from data corresponding to the first reflected pulse arriving at the array of detectors;
moving the camera relative to objects in the scene;
monitoring angular velocity and acceleration via the inertial guidance system;
transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse;
detecting arrival of the second reflected pulse at the array of detectors;
forming a second scene image from data corresponding to the second reflected pulse arriving at the array of detectors;
determining a range estimate of camera position as a function of registration of the first and second range images;
determining camera position and trajectory as a function of the range estimate, the angular velocity measured by the inertial guidance system and the acceleration measured by the inertial guidance system, wherein determining includes combining the range estimate with the angular velocity measured by the inertial guidance system and the acceleration measured by the inertial guidance system to arrive at a composite camera position and a composite camera trajectory that minimizes sensor measurement errors and image alignment errors; and
forming a composite image of the three-dimensional scene as a function of the first and second scene images, the composite camera position, and the composite camera trajectory.
3 Assignments
0 Petitions
Accused Products
Abstract
A system and method for imaging a three-dimensional scene having one or more objects. The system includes a light source, a detector array, a timing circuit, an inertial guidance system and a processor connected to the timing circuit and the inertial guidance system. The light source generates an optical pulse and projects the optical pulse on an object so that it is reflected as a reflected pulse. The detector array includes a plurality of detectors, wherein the detectors are oriented to receive the reflected pulse. The timing circuit determines when the reflected pulse reached detectors on the detector array. The inertial guidance system measures angular velocity and acceleration. The processor forms a composite image of the three-dimensional scene as a function of camera position and range to objects in the three-dimensional scene.
-
Citations
49 Claims
-
1. A method of imaging a three-dimensional scene having one or more objects, comprising:
-
providing a camera, wherein the camera includes an inertial guidance system and an array of optical detectors; transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse; detecting arrival of the first reflected pulse at the array of detectors; forming a first scene image from data corresponding to the first reflected pulse arriving at the array of detectors; moving the camera relative to objects in the scene; monitoring angular velocity and acceleration via the inertial guidance system; transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse; detecting arrival of the second reflected pulse at the array of detectors; forming a second scene image from data corresponding to the second reflected pulse arriving at the array of detectors; determining a range estimate of camera position as a function of registration of the first and second range images; determining camera position and trajectory as a function of the range estimate, the angular velocity measured by the inertial guidance system and the acceleration measured by the inertial guidance system, wherein determining includes combining the range estimate with the angular velocity measured by the inertial guidance system and the acceleration measured by the inertial guidance system to arrive at a composite camera position and a composite camera trajectory that minimizes sensor measurement errors and image alignment errors; and forming a composite image of the three-dimensional scene as a function of the first and second scene images, the composite camera position, and the composite camera trajectory. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A method of imaging a three-dimensional scene having one or more objects, comprising:
-
providing a camera, wherein the camera includes an inertial guidance system and an array of optical detectors; transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse; detecting arrival of the first reflected pulse at the array of detectors; forming a first scene image from data corresponding to the first reflected pulse arriving at the array of detectors; moving the camera relative to objects in the scene; monitoring angular velocity and acceleration via the inertial guidance system; transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse; detecting arrival of the second reflected pulse at the array of detectors; forming a second scene image from data corresponding to the second reflected pulse arriving at the array of detectors; determining a range estimate of camera position as a function of registration of the first and second range images; determining camera position and trajectory as a function of the range estimate and of the angular velocity and acceleration measured by the inertial guidance system, wherein determining includes combining the range estimate with the angular velocity and acceleration measured by the inertial guidance system to arrive at a composite camera position and a composite camera trajectory, wherein combining the range estimate with the angular velocity and acceleration measured by the inertial guidance system includes applying a Kalman filter to each of the range estimate, the angular velocity measured by the inertial guidance system, and the acceleration measured by the inertial guidance system to determine camera position and trajectory; and forming a composite image of the three-dimensional scene as a function of the first and second scene images, the composite camera position, and the composite camera trajectory. - View Dependent Claims (7, 8)
-
-
9. A camera for imaging a three-dimensional scene having one or more objects, comprising:
-
a light source, wherein the light source generates an optical pulse and projects the optical pulse on an object so that it is reflected as a reflected pulse; a detector array, wherein the detector array may includes a plurality of detectors, wherein the detectors are oriented to receive the reflected pulse; a timing circuit connected to the detector array, wherein the timing circuit determines when the reflected pulse reached detectors on the detector array; an inertial guidance system which measures angular velocity and acceleration; and a processor connected to the timing circuit and the inertial guidance system, wherein the processor calculates a range estimate of camera position as a function of registration of range images from consecutive pulses and wherein the processor determines camera position and trajectory by combining information corresponding to the range estimate with information corresponding to the angular velocity and acceleration and forms a composite image of the three-dimensional scene as a function of camera position and range to objects in the three-dimensional scene; wherein combining includes calculating camera position and trajectory in a way that minimizes sensor measurement errors and image alignment errors. - View Dependent Claims (10)
-
-
11. A camera for imaging a three-dimensional scene having one or more objects, comprising:
-
a light source, wherein the light source generates an optical pulse and projects the optical pulse on an object so that it is reflected as a reflected pulse; a detector array, wherein the detector array may includes a plurality of detectors, wherein the detectors are oriented to receive the reflected pulse; a timing circuit connected to the detector array, wherein the timing circuit determines when the reflected pulse reached detectors on the detector array; an inertial guidance system which measures angular velocity and acceleration; and a processor connected to the timing circuit and the inertial guidance system, wherein the processor calculates a range estimate of camera position as a function of registration of range images from consecutive pulses and wherein the processor determines camera position and trajectory by combining information corresponding to the range estimate with information corresponding to the angular velocity and acceleration and forms a composite image of the three-dimensional scene as a function of camera position and range to objects in the three-dimensional scene; wherein the processor combines information corresponding to the range estimate with information corresponding to the angular velocity and acceleration received from the inertial guidance system by applying a Kalman filter to the range estimate and to the angular velocity and acceleration measured by the inertial guidance system. - View Dependent Claims (12, 13)
-
-
14. A device for imaging a three-dimensional scene having one or more objects, including:
-
a pulsed light source; means for transmitting light toward said objects; optics for collecting light during the time for light to transit from said pulsed light source, reflect from said objects, be collected and focused by said optics; a plurality of detectors oriented for receiving the collected light and converting the collected light into an electrical signal; timing means, connected to the detectors, for determining a transit time for a pulse to leave the pulsed light source, reflect off the objects and return to one or more of the plurality of deflectors; an inertial guidance system which measures angular velocity and acceleration; and a processor connected to the timing means and the inertial guidance system, wherein the processor calculates a range estimate of camera position as a function of registration of range images from consecutive pulses and wherein the processor determines camera position and trajectory by combining information corresponding to the range estimate with information corresponding to the angular velocity and acceleration and forms a composite image of the three-dimensional scene as a function of range to objects in the three-dimensional scene and camera position and trajectory; wherein the processor combines information corresponding to the range estimate with information corresponding to the angular velocity and acceleration received from the inertial guidance system so as to minimize sensor measurement errors and image alignment errors. - View Dependent Claims (15, 16)
-
-
17. A device for imaging a three-dimensional scene having one or more objects, including:
-
a pulsed light source; means for transmitting light toward said objects; optics for collecting light during the time for light to transit from said pulsed light source, reflect from said objects, be collected and focused by said optics; a plurality of detectors oriented for receiving the collected light and converting the collected light into an electrical signal; timing means, connected to the detectors, for determining a transit time for a pulse to leave the pulsed light source, reflect off the objects and return to one or more of the plurality of deflectors; an inertial guidance system which measures angular velocity and acceleration; and a processor connected to the timing means and the inertial guidance system, wherein the processor calculates a range estimate of camera position as a function of registration of range images from consecutive pulses and wherein the processor determines camera position and trajectory by combining information corresponding to the range estimate with information corresponding to the angular velocity and acceleration and forms a composite image of the three-dimensional scene as a function of range to objects in the three-dimensional scene and camera position and trajectory; wherein the processor combines information corresponding to the range estimate with information corresponding to the angular velocity and acceleration received from the inertial guidance system by applying a Kalman filter to the range estimate and to the angular velocity and acceleration measured by the inertial guidance system. - View Dependent Claims (18, 19)
-
-
20. A method of imaging a three-dimensional scene having one or more objects, comprising:
-
providing a camera, wherein the camera includes an inertial guidance system and one or more optical detector arrays; transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse; detecting arrival of the first reflected pulse at the one or more optical detector arrays; recording first range information as a function of the arrival of the first reflected pulse; forming a first scene image from the first range information; moving the camera relative to objects in the scene; recording angular velocity and acceleration information; transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse; detecting arrival of the second reflected pulse at the one or more optical detector arrays; recording second range information as a function of the arrival of the second reflected pulse; forming a second scene image from the second range information; determining a range estimate of camera position as a function of registration of the first and second range images; estimating camera position and camera trajectory as a function of the range estimate and of the recorded angular velocity information and the recorded acceleration information, wherein estimating includes combining the range estimate with the angular velocity information and the recorded acceleration information to arrive at a composite camera position and a composite camera trajectory; and forming a composite image of the three-dimensional scene as a function of the composite camera position, the composite camera trajectory, the recorded first range information and the recorded second range information; wherein combining includes applying a process to each of the range estimate, the angular velocity information and the recorded acceleration information to arrive at the composite camera position and the composite camera trajectory that minimizes sensor measurement errors and image alignment errors.
-
-
21. A method of imaging a three-dimensional scene haying one or more objects, comprising:
-
providing a camera, wherein the camera includes an inertial guidance system and one or more optical detector arrays; transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse; detecting arrival of the first reflected pulse at the one or more optical detector arrays; recording first range information as a function of the arrival of the first reflected pulse; forming a first scene image from the first range information; moving the camera relative to objects in the scene; recording angular velocity and acceleration information; transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse; detecting arrival of the second reflected pulse at the one or more optical detector arrays; recording second range information as a function of the arrival of the second reflected pulse; forming a second scene image from the second range information; determining a range estimate of camera position as a function of registration of the first and second range images; estimating camera position and camera trajectory as a function of the range estimate and of the recorded angular velocity information and the recorded acceleration information, wherein estimating includes combining the range estimate with the angular velocity information and the recorded acceleration information to arrive at a composite camera position and a composite camera trajectory; and forming a composite image of the three-dimensional scene as a function of the composite camera position, the composite camera trajectory, the recorded first range information and the recorded second range information; wherein the inertial guidance system includes a gyroscope and an accelerometer and wherein combining includes applying a Kalman filter to each of the range estimate, the angular velocity information and the recorded acceleration information to arrive at the composite camera position and the composite camera trajectory.
-
-
22. A method of imaging a three-dimensional scene having one or more objects, comprising:
-
providing a camera, wherein the camera includes an inertial guidance system and one or more optical detector arrays; transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse; detecting arrival of the first reflected pulse at the one or more optical detector arrays; recording first range information as a function of the arrival of the first reflected pulse; forming a first scene image from the first range information; moving the camera relative to objects in the scene; recording angular velocity and acceleration information; transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse; detecting arrival of the second reflected pulse at the one or more optical detector arrays; recording second range information as a function of the arrival of the second reflected pulse; forming a second scene image from the second range information; determining a range estimate of camera position as a function of registration of the first and second range images; estimating camera position and camera trajectory as a function of the range estimate and of the recorded angular velocity information and the recorded acceleration information, wherein estimating includes combining the range estimate with the angular velocity information and the recorded acceleration information to arrive at a composite camera position and a composite camera trajectory; and forming a composite image of the three-dimensional scene as a function of the composite camera position, the composite camera trajectory, the recorded first range information and the recorded second range information; wherein the inertial guidance system includes a global positioning system, a gyroscope and an accelerometer and wherein combining includes applying a Kalman filter to each of the range estimate, the recorded angular velocity information and the recorded acceleration information to arrive at the composite camera position and the composite camera trajectory and then correcting the composite camera position and composite camera trajectory as a function of a global positioning measurement.
-
-
23. A system for imaging a three dimensional scene, comprising:
-
a focal plane array range imaging optical pulsing camera; and an inertial guidance system, connected to the camera, wherein the inertial guidance system provides angular velocity and acceleration information corresponding to movements of the camera; and a processor, connected to the camera and to the inertial guidance system, wherein the processor calculates a range estimate of camera position as a function of registration of range images from consecutive pulses, wherein the processor estimates camera position and trajectory by combining information corresponding to the range estimate with information corresponding to the angular velocity and acceleration measured by the inertial guidance system and wherein the processor forms a composite image of the three-dimensional scene as a function of camera position, camera trajectory and range information measured by the camera; wherein the processor combines information corresponding to the range estimate with information corresponding to the angular velocity and acceleration received from the inertial guidance system so as to minimize sensor measurement errors and image alignment errors. - View Dependent Claims (24, 25, 26)
-
-
27. A system for imaging a three dimensional scene, comprising:
-
a focal plane array range imaging optical pulsing camera; and an inertial guidance system, connected to the camera, wherein the inertial guidance system provides angular velocity and acceleration information corresponding to movements of the camera; and a processor, connected to the camera and to the inertial guidance system, wherein the processor calculates a range estimate of camera position as a function of registration of range images from consecutive pulses, wherein the processor estimates camera position and trajectory by combining information corresponding to the range estimate with information corresponding to the angular velocity and acceleration measured by the inertial guidance system and wherein the processor forms a composite image of the three-dimensional scene as a function of camera position, camera trajectory and range information measured by the camera; wherein the inertial guidance system includes a global positioning system, gyroscopes and accelerometers and wherein combining includes applying a Kalman filter to each of the range estimate, the angular velocity and the acceleration to arrive at the camera position and the camera trajectory and then correcting the camera position and the camera trajectory as a function of a global positioning measurement.
-
-
28. A system for imaging a three dimensional scene, comprising:
-
a focal plane array range imaging optical pulsing camera; and an inertial guidance system, connected to the camera, wherein the inertial guidance system provides angular velocity and acceleration information corresponding to movements of the camera; and a processor, connected to the camera and to the inertial guidance system, wherein the processor calculates a range estimate of camera position as a function of registration of range images from consecutive pulses, wherein the processor estimates camera position and trajectory by combining information corresponding to the range estimate with information corresponding to the angular velocity and acceleration measured by the inertial guidance system and wherein the processor forms a composite image of the three-dimensional scene as a function of camera position, camera trajectory and range information measured by the camera; wherein combining includes applying a Kalman filter to each of the range estimate, the angular velocity and the acceleration to arrive at the camera position and the camera trajectory. - View Dependent Claims (29, 30, 31, 32)
-
-
33. A method for collecting range images from a focal plane array imaging optical pulsing camera and placing them on a common coordinate system, the method comprising:
-
attaching an inertial guidance system to the camera, wherein the inertial guidance system includes a set of gyroscopes and accelerometers; continuously measuring the set of gyroscopes and accelerometers; recording first range information at a first camera position; recording second range information at a second camera position; determining an estimated camera motion in moving the camera from the first to the second camera position consistent with the first and second range information and the measurements from the set of gyroscopes and accelerometers attached to the camera, wherein determining an estimated camera motion includes combining range estimate information reflecting movement of the camera between the first and second range images with inertial guidance information reflecting movement of the camera as measured by the set of gyroscopes and accelerometers attached to the camera to arrive at a camera position and trajectory; and forming a composite image from the first and second range images;
wherein forming a composite image includes determining alignment of the first and second range images as a function of the estimated camera motion;wherein combining includes applying a process to each of the range estimate information and the inertial guidance information to arrive at the camera position and the camera trajectory that minimizes sensor measurement errors and image alignment errors. - View Dependent Claims (34, 35)
-
-
36. A method of imaging a three-dimensional scene having one or more objects, comprising:
-
receiving two-dimensional information reflective of a first scene captured by an imaging device; receiving two-dimensional information reflective of a second scene captured by the imaging device; calculate a range estimate as a function of the first and second scenes; receiving angular velocity and acceleration via an inertial guidance system, wherein the angular velocity and the acceleration reflect movement of the imaging device between the first and the second scenes; determining position and trajectory of the imaging device as a function of the range estimate and of the angular velocity and acceleration measured by the inertial guidance system and movement of the imaging device reflected by changes in scene registration, wherein determining includes combining the range estimate with the angular velocity and acceleration measured by the inertial guidance system to arrive at a composite position and a composite trajectory for the imaging device; and forming a composite image of the three-dimensional scene as a function of the first and second scenes and the composite position and the composite trajectory of the imaging device; wherein combining the angular velocity and acceleration measured by the inertial guidance system and the movement of the imaging device reflected by changes in scene registration includes passing data representing the angular velocity and acceleration measured by the inertial guidance system and data representing the movement of the imaging device reflected by changes in scene registration through a process that minimizes sensor measurement errors and image alignment errors.
-
-
37. A method of imaging a three-dimensional scene having one or more objects, comprising:
-
receiving two-dimensional information reflective of a first scene captured by an imaging device; receiving two-dimensional information reflective of a second scene captured by the imaging device; calculate a range estimate as a function of the first and second scenes; receiving angular velocity and acceleration via an inertial guidance system, wherein the angular velocity and the acceleration reflect movement of the imaging device between the first and the second scenes; determining position and trajectory of the imaging device as a function of the range estimate and of the angular velocity and acceleration measured by the inertial guidance system and movement of the imaging device reflected by changes in scene registration, wherein determining includes combining the range estimate with the angular velocity and acceleration measured by the inertial guidance system to arrive at a composite position and a composite trajectory for the imaging device; and forming a composite image of the three-dimensional scene as a function of the first and second scenes and the composite position and the composite trajectory of the imaging device; wherein combining the angular velocity and acceleration measured by the inertial guidance system and the movement of the imaging device reflected by changes in scene registration includes passing data representing the angular velocity and acceleration measured by the inertial guidance system and data representing the movement of the imaging device reflected by changes in scene registration through a Kalman filter.
-
-
38. A method of imaging a three-dimensional scene having one or more objects, comprising:
-
receiving two-dimensional information reflective of a first scene captured by an imaging device; receiving two-dimensional information reflective of a second scene captured by the imaging device; calculate a range estimate as a function of the first and second scenes; receiving angular velocity and acceleration via an inertial guidance system, wherein the angular velocity and the acceleration reflect movement of the imaging device between the first and the second scenes; determining position and trajectory of the imaging device as a function of the range estimate and of the angular velocity and acceleration measured by the inertial guidance system and movement of the imaging device reflected by changes in scene registration, wherein determining includes combining the range estimate with the angular velocity and acceleration measured by the inertial guidance system to arrive at a composite position and a composite trajectory for the imaging device; and forming a composite image of the three-dimensional scene as a function of the first and second scenes and the composite position and the composite trajectory of the imaging device; wherein combining the angular velocity and acceleration measured by the inertial guidance system and the movement of the imaging device reflected by changes in scene registration includes passing data representing the angular velocity and acceleration measured by the inertial guidance system, data corresponding to global positioning measurements and data representing the movement of the imaging device reflected by changes in scene registration though a Kalman filter.
-
-
39. A method of imaging a three-dimensional scene having one or more objects, comprising:
-
providing a camera, wherein the camera includes an inertial guidance system and an array of optical detectors; transmitting a first optical pulse so that a portion of the first optical pulse is reflected from the scene as a first reflected pulse; detecting arrival of the first reflected pulse at the array of detectors so as to form a first range image; moving the camera relative to objects in the scene; monitoring angular velocity and acceleration said the inertial guidance system; transmitting a second optical pulse so that a portion of the second optical pulse is reflected from the scene as a second reflected pulse; detecting arrival of the second reflected pulse at the array of detectors so as to form a second range image; determining a range estimate of camera position as a function of registration of the first and second range images; determining the camera position and the camera trajectory by integrating, using a Kalman filter the registration estimates with the angular velocity and acceleration measured by the inertial guidance system; and forming a composite image of the three-dimensional scene as a function of camera position, camera trajectory and range to objects in the three-dimensional scene. - View Dependent Claims (40)
-
-
41. A device for imaging a three-dimensional scene having one or more objects, including:
-
a pulsed light source; means for transmitting a first and second light pulse toward said objects; optics for collecting light during the time for light to transit from said pulsed light source, reflect from said objects, be collected and focused by said optics; an array of optical detectors oriented for receiving the collected light and converting the collected light into an electrical signal; timing means, connected to the optical detectors, for determining a transit time for a pulse to leave the pulsed light source, reflect off the objects and return to one or more of the plurality of deflectors; an inertial guidance system which measures angular velocity and acceleration for the device; and a processor, connected to the timing means and the inertial guidance system, wherein the processor calculates a range estimate as a function of a first range image formed from the first light pulse and a second range image formed from the second light pulse and determines device position and trajectory as a function of both the range estimate and the angular velocity and acceleration of the device and forms a composite image of the three-dimensional scene as a function of camera position and trajectory and the range to objects in the three-dimensional scene; wherein the processor combines information corresponding to the range estimate with information corresponding to the angular velocity and acceleration of the device so as to minimize sensor measurement errors and image alignment errors. - View Dependent Claims (42, 43, 44)
-
-
45. A device for imaging a three-dimensional scene having one or more objects, comprising:
-
a pulsed light source; means for transmitting a first and second light pulse toward said objects; optics for collecting light during the time for light to transit from said pulsed light source, reflect from said objects, be collected and focused by said optics; an array of optical detectors oriented for receiving the collected light and converting the collected light into an electrical signal; timing means, connected to the optical detectors, for determining a transit time for a pulse to leave the pulsed light source, reflect off the objects and return to one or more of the plurality of deflectors; an inertial guidance system which measures angular velocity and acceleration for the device; and a processor connected to the timing means and the inertial guidance system, wherein the processor calculates a range estimate as a function of a first range image formed from the first light pulse and a second range image formed from the second light pulse and determines device position and trajectory as a function of both the range estimate and the angular velocity and acceleration of the device and forms a composite image of the three-dimensional scene as a function of camera position and trajectory and the range to objects in the three-dimensional scene; wherein the processor applies a Kalman filter to each of the range estimate, the angular velocity measured by the inertial guidance system and the acceleration measured by the inertial guidance system to determine device position and trajectory. - View Dependent Claims (46, 47, 48, 49)
-
Specification