Gimbaled camera object tracking system

• US 9,936,133 B2
• Filed: 08/19/2015
• Issued: 04/03/2018
• Est. Priority Date: 08/19/2015
• Status: Active Grant

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: Bosscher, Paul M., Summer, Matthew D., Bowman, William S., Pollard, Jeffrey M.
• Primary Examiner(s): Li, Tracy Y
• Application Number:
  US14/830,023
• Publication Number:
  US 20170050563A1
• Time in Patent Office: 958 Days
• CPC Class Codes: G01S 3/00 Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received position-fixing by co-ordinating a plurality of determinations of direction or position lines G01S5/00
• CPC Class Codes: G01S 3/781 Details
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

A system for automatically controlling a gimbaled camera system of a vehicle. The system includes a camera positioned relative to a body of the vehicle and one or more sensors configured to sense the pointing direction of the camera. One or more sensors are configured to monitor movement of the vehicle relative to a surface. A processor is configured to receive the sensed camera pointing direction data and vehicle movement data. The processor establishes and stores a target position representative of the position of a target object relative to the vehicle body based on an object independent association and automatically adjusts the camera pointing direction in response to the vehicle movement data such that the camera remains aimed on the target position. A method for automatically controlling the gimbaled camera system is also provided.

Claims

1. A system for automatically controlling a gimbaled camera system of a vehicle, comprising:

a camera positioned relative to a body of the vehicle;

one or more sensors configured to sense the pointing direction of the camera;

one or more sensors configured to monitor movement of the vehicle relative to a surface; and

a processor configured to

receive first sensed pan and tilt orientation data acquired when the camera was aimed at a first point on or adjacent to a target object while the vehicle was at a first stationary position, receive second sensed pan and tilt orientation data acquire when the camera was aimed at the first point while the vehicle was at a second stationary position, and vehicle movement data representative of the vehicle'"'"'s movement from the first stationary position to the second stationary position,

use vehicle movement data and the first and second sensed pan and tilt orientation data to establish a target position representative of the position of the target object relative to the vehicle body based on an object independent association, and

automatically adjust the pointing direction of the camera in response to the vehicle'"'"'s movement such that the camera remains aimed on the target position.

2. The system of claim 1, wherein

the first point is a point which the target object intersects the surface, and

the processor is further configured to determine a first target location representative of the first point based on the pointing direction data when the camera is aimed at the first point.

3. The system of claim 2, wherein the first target location is established as the target position.

4. The system of claim 2, wherein

the object independent association further includes aiming the camera at a second point on the target object which is substantially co-planar with the first point in a plane perpendicular to the surface, and

the processor is further configured to establish the second point as the target position based on an offset in the camera pointing direction data from the second point compared to the first point.

5. The system of claim 1, wherein the processor:

stores camera pointing direction data representative of the first stationary position, while a beam of a LIDAR unit substantially co-linear with the camera is directed at the first point such that the processor determines a distance between the camera and the first point based on a reflected beam received by the LIDAR unit; and

establishes and stores the first point as the target position based on the camera pointing direction data representative of the first stationary position and the determined distance.

6. The system of claim 1, wherein the one or more sensors configured to monitor movement of the vehicle relative to the surface includes an Inertial Measurement Unit (“

IMUD”

).

7. The system of claim 1, wherein the vehicle includes one or more drive elements and the one or more sensors configured to monitor movement of the vehicle relative to the surface includes a sensor configured to monitor movement of the one or more drive elements.

8. The system of claim 1, wherein the processor is further configured to override the automatic adjustment of the camera based on a received override signal and to return to automatic adjustment of the camera to aim on the target position based on a current position of the vehicle based on receipt of a reactivation signal.

9. The system of claim 1, wherein the processor is further configured to automatically adjust a zoom of the camera based on a current position of the vehicle such that the object remains at a generally constant size in a field of view of the camera.

10. A system for automatically controlling a gimbaled camera system of a vehicle comprising:

a camera positioned relative to a body of the vehicle;

one or more sensors configured to sense the pointing direction of the camera;

one or more sensors configured to monitor movement of the vehicle relative to a surface; and

a processor configured to receive the sensed pan and tilt orientation data and vehicle movement data, wherein the processor establishes and stores a target position representative of the position of a target object relative to the vehicle body based on an object independent association;

and automatically adjusts the pointing direction of the camera in response to the vehicle movement data such that the camera remains aimed on the target position;

wherein the object independent association includes

aiming the camera at a first point on the target object when the vehicle is in a first stationary position such that the processor stores the camera pointing direction data representative of the first stationary position, and

moving the vehicle to a second stationary position and aiming the camera at the first point on the target object such that the processor stores the camera pointing direction data representative of the secondary stationary position and the movement data representative of the movement from the first stationary position to the second stationary position; and

wherein the processor is configured to establish and store the first point as the target position based on a comparison of the camera pointing direction data representative of the first stationary position, the camera pointing direction data representative of the second stationary position, and the movement data representative of the movement from the first stationary position to the second stationary position.

11. The system of claim 10, wherein the processor is further configured such that if a first line representative of the camera pointing direction data representative of the first stationary position and a second line representative of the camera pointing direction data representative of the second stationary position do not intersect, the target position is estimated as the midpoint of a line segment representing the shortest distance between the first and second lines.

12. A method for automatically controlling a gimbaled camera system of a vehicle, comprising:

manually aiming a camera of the gimbaled camera system at a first point on or adjacent to a target object while the vehicle is at a first stationary position;

generating first camera pointing direction data while the camera is aimed at the first point and the vehicle is at the first stationary position;

manually aiming the camera at the first point while the vehicle is at a second stationary position;

generating second camera pointing direction data while the camera is aimed at the first point and the vehicle is at the second stationary position;

obtaining vehicle movement data representative of the vehicle'"'"'s movement from the first stationary position to the second stationary position;

activating the auto-tracking mode of the camera;

using the first camera pointing direction data, the second camera pointing direction data, and the vehicle movement data to establish a target position through an object independent association between a location of the camera and the target object;

storing the target position;

monitoring movement of the vehicle relative to the position of the vehicle when the target position was established; and

automatically adjusting the pointing direction of the camera in response to movement of the vehicle such that the aim of the camera remains on the target position.

13. The method of claim 12, further comprising automatically adjusting the zoom level of the camera in response to movement of the vehicle such that a substantially constant zoom on the target object is maintained.

14. The method of claim 12, further comprising:

adjusting the pointing direction of the camera in response to a manual override signal;

recalling the stored target position upon receipt of an automatic tracking reactivation signal; and

automatically adjusting the pan and tilt of the camera based on a current position of the vehicle such that the aim of the camera remains on the target position.

15. The method of claim 12, wherein:

the first point is a point at which the target object intersects the surface; and

a first target location is determined which is representative of the first point based on the first camera pointing direction data when the camera is aimed at the first point.

16. The method of claim 15, further comprising:

aiming the camera at a second point on or adjacent to the target object which is substantially co-planar with the first point in a plane perpendicular to the surface; and

establishing the second point as the target position based on an offset in the camera pointing direction data from the second point compared to the first point.

17. The method of claim 12, further comprising:

directing a beam of a LIDAR unit substantially co-linear with the camera at the first point;

determining a distance between the camera and the first point based on a reflected beam received by the LIDAR unit; and

establishing the first point as the target position based on the first camera pointing direction data.

18. A method for automatically controlling a gimbaled camera system of a vehicle comprising:

manually aiming a camera of the gimbaled camera system at a target object;

activating the auto-tracking mode of the camera;

establishing a target position through an object independent association between a location of the camera and the target object;

storing the target position;

monitoring movement of the vehicle relative to the position of the vehicle when the target position was established; and

automatically adjusting the pointing direction of the camera in response to movement of the vehicle such that the aim of the camera remains on the target position;

wherein the step of establishing a target position through an object independent association between a location of the camera and the target object comprises;

aiming the camera at a first point on the target object when the vehicle is in a first stationary position and storing the camera pointing direction data representative of the first stationary position;

moving the vehicle to a second stationary position and aiming the camera at the first point on the target object and storing camera pointing direction data representative of the secondary stationary position and movement data representative of the movement from the first stationary position to the second stationary position; and

establishing the first point as the target position based on a comparison of the camera pointing direction data representative of the first stationary position, the camera pointing direction data representative of the second stationary position and the movement data representative of the movement from the first stationary position to the second stationary position.