TRACKING A MOVING OBJECT FROM A CAMERA ON A MOVING PLATFORM

• US 20080054158A1
• Filed: 09/05/2006
• Published: 03/06/2008
• Est. Priority Date: 09/05/2006
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Bedros, Saad J., Morellas, Vassilios, Ariyur, Kartik B.
• Granted Patent: US 7,411,167 B2
• Time in Patent Office: Days
• US Class Current: 250/203.1
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30181 Earth observation
• CPC Class Codes: G06T 2207/30212 Military
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

A method to dynamically stabilize a target image formed on an image plane of an imaging device located in a moving vehicle. The method includes setting an origin in the image plane of the imaging device at an intersection of a first axis, a second axis and a third axis, imaging a target so that an image centroid of the target image is at the origin of the image plane, monitoring sensor data indicative of a motion of the vehicle, and generating pan and tilt output to stabilize the image centroid at the origin in the image plane to compensate for vehicle motion and target motion. The pan and tilt output are generated by implementing exponentially stabilizing control laws. The implementation of the exponentially stabilizing control laws is based at least in part on the sensor data.

Claims

1. A method to dynamically stabilize a target image formed on an image plane of an imaging device located in a moving vehicle, the method comprising:

setting an origin in the image plane of the imaging device at an intersection of a first axis, a second axis and a third axis, wherein the third axis is an optical axis of the imaging device;

imaging a target so that an image centroid of the target image is at the origin of the image plane;

monitoring sensor data indicative of a motion of the vehicle; and

generating pan and tilt output to stabilize the image centroid at the origin in the image plane to compensate for vehicle motion and target motion, wherein the pan and tilt output are generated by implementing exponentially stabilizing control laws, wherein implementing the exponentially stabilizing control laws is based at least in part on the sensor data including data indicative of a distance to the target.

2. The method of claim 1, wherein the vehicle motion comprises a translation and a rotation, and wherein generating pan and tilt output comprises:

generating rotation output to stabilize the image centroid within a selected distance from the origin; and

maintaining the image centroid within the selected distance from the origin as the vehicle moves with respect to the target based on the rotation output,

the method further comprising generating zoom output to stabilize the target-image size within a selected size range to compensate for the vehicle motion and the target motion.

3. The method of claim 2, further comprising:

maintaining a target-image size within the selected size range as the vehicle moves with respect to the target, based on the zoom output.

4. The method of claim 3, wherein maintaining the target-image size within the selected size range comprises:

adjusting a lens system of the imaging device along the third axis to offset the translation the moving vehicle, wherein adjusting the lens system is based on the generated zoom output to stabilize the target-image size to a selected size, and wherein maintaining the image centroid within the selected distance from the origin comprises;

adjusting a tilt of the imaging device about the first axis to offset a pitch of the moving vehicle, wherein adjusting the tilt is based on the generated rotation output; and

adjusting a pan of the imaging device about the second axis to offset a yaw of the moving vehicle, wherein adjusting the pan is based on the generated rotation output.

5. The method of claim 1, wherein the motion of the vehicle comprises a translation and a rotation, and wherein implementing the exponentially stabilizing control laws comprises:

generating rotation output to stabilize the image centroid within a selected distance from the origin; and

maintaining the image centroid within the selected distance from the origin as the vehicle moves with respect to the target based on the rotation output.

6. The method of claim 5, wherein maintaining the image centroid within the selected distance from the origin comprises:

adjusting a tilt of the imaging device about the first axis to offset a pitch of the moving vehicle, wherein the adjusting the tilt is based on the generated rotation output; and

adjusting a pan of the imaging device about the second axis to offset a yaw of the moving vehicle, wherein the adjusting the pan is based on the generated rotation output.

7. The method of claim 1, wherein monitoring sensor data indicative of a motion of the vehicle comprises:

monitoring a pitch of the moving vehicle about a fourth axis;

monitoring a yaw of the moving vehicle about a fifth axis; and

monitoring a roll of the moving vehicle about a sixth axis.

8. The method of claim 1, wherein setting the origin of the image plane of the imaging device comprises:

setting the origin at about a center of a field of view of the imaging device.

9. The method of claim 1, the method further comprising:

positioning the imaging device at a center of gravity of the vehicle.

10. A system comprising:

an imaging device positioned within a moving vehicle, the imaging device having an image plane including an origin at an intersection of a first axis, a second axis and a third axis;

sensors to sense translation and rotations of the vehicle about a fourth axis, a fifth axis and a sixth axis and to sense a distance to a target; and

a programmable processor adapted to execute software to implement exponentially stabilizing control laws to maintain an image centroid of the target image at the origin of the image plane, the programmable processor communicatively coupled to receive sensor data from the sensors and to generate rotation output to stabilize the target image formed on the image plane of the imaging device when the vehicle moves with respect to a target.

11. The system of claim 10, wherein the programmable processor is further adapted to implement the exponentially stabilizing control laws to maintain a target-image size within a selected size range as the vehicle moves with respect to the target wherein the programmable processor generates zoom output to stabilize the target-image size within the selected size range when the vehicle moves with respect to the target.

12. The system of claim 10, further comprising:

a housing fixedly attached to the vehicle, wherein the imaging device is located in the housing.

13. The system of claim 12, further comprising:

at least one motor mechanically coupled between the housing and the imaging device, wherein the rotation output generated by the programmable processor initiates a rotation operation by the motor to rotate the imaging device within the housing, wherein the third axis is an optical axis of the imaging device and the optical axis intersects the target.

14. The system of claim 13, wherein the sensors are located in the housing.

15. The system of claim 10, wherein the sensors includes at least one of a first gyrocompass aligned to the fourth axis, a second gyrocompass aligned to the fifth axis, a third gyrocompass aligned to the sixth axis, a first accelerometer aligned to the fourth axis, a second accelerometer aligned to the fifth axis, a third accelerometer aligned to the sixth axis, an inertial measurement unit, an inertial navigation system, a range sensor, and a global positioning system.

16. A computer-readable medium having computer-executable instructions for performing a method comprising:

determining the transformation of an origin of an imaging device positioned in a vehicle based on received first translation data and first rotation data;

implementing exponentially stabilizing control laws based on the determined transformation and a distance to an image target;

generating a rotation output from the exponentially stabilizing control laws adapted to redirect an optical axis of the imaging device to maintain an image centroid within a selected distance from the origin of the imaging device;

generating a zoom output from the exponentially stabilizing control laws adapted to modify a lens system of the imaging device, wherein an apparent distance between the imaged target and the imaging device is maintained;

determining a system latency in redirecting the optical axis and modifying the lens system along the optical axis; and

determining the transformation of the origin of the imaging device with respect to global coordinates based on second translation data and second rotation data monitored during at least one of the pitch, the yaw, and the roll of the vehicle, the redirecting of the optical axis, the modifying of the lens system, and the system latency, wherein the centroid of the target image is maintained within a selected distance from the origin of the imaging device and the apparent distance between the imaged target and the imaging device is maintained.

17. The computer-readable of claim 16, the medium further comprising instructions for:

periodically implementing the exponentially stabilizing control laws based on the determined transformations to generate rotation output and zoom output;

periodically rotating the imaging device responsive to the generated rotation output to continuously maintain the image centroid to within the selected distance from the origin of the imaging device; and

periodically modifying the lens system responsive to the generated zoom output wherein an apparent distance between the imaged target and the imaging device is continuously maintained in order to compensate for the vehicle motion.

18. The computer-readable of claim 17, wherein the period is less than ten millisecond.

19. The computer-readable of claim 16, the medium further comprising instructions for:

continuously implementing the exponentially stabilizing control laws based on the determined transformations to compensate for the vehicle motion;

continuously rotating the imaging device responsive to the generated rotation output to continuously maintain the image centroid to within the selected distance from the origin of the imaging device; and

continuously modifying of the lens system responsive to the generated zoom output wherein an apparent distance between the imaged target and the imaging device is continuously maintained in order to compensate for the vehicle motion

20. The computer-readable of claim 19, the medium further comprising instructions for:

continuously outputting information indicative of a rotation operation to motors controlling the imaging device.