Optical imaging system for unmanned aerial vehicle

• US 8,581,981 B2
• Filed: 04/26/2007
• Issued: 11/12/2013
• Est. Priority Date: 04/28/2006
• Status: Active Grant

• Current Assignee: Southwest Research Institute
• Original Assignee: Southwest Research Institute
• Inventors: Alley, Kevin, Lopez, Roger, Somers, Richard
• Primary Examiner(s): Nguyen, Phuoc
• Assistant Examiner(s): John, Clarence
• Application Number:
  US11/740,743
• Publication Number:
  US 20090015674A1
• Time in Patent Office: 2,392 Days
• Field of Search: 3484001-4041, 348144-147, 37524001-24016
• US Class Current: 348/144
• CPC Class Codes: G01C 11/025 by scanning the object
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

An optical imaging system and associated methods for capturing images from an aircraft, such as a UAV. A camera unit on-board the aircraft is remotely controlled from an image control station. The image control station receives image data from the camera unit, and also delivers control signals for determining a viewing mode of the image.

Claims

1. An optical imaging system, for mounting on an aircraft for airborne image capture of the ground and for delivery of pictorial images to a remote image control station, comprising:

a camera unit having at least four image sensors, all sensors being identical, mounted on the underside of the aircraft;

wherein each sensor has an associated lens and is mounted on the underside of the aircraft with its lens to provide a fixed field of view for each sensor;

wherein each sensor is oriented on a surface of the underside that is non-planar to the ground, such that images from the sensors appear tilted with respect to one another in the absence of transformation processing;

wherein the optical axis of each sensor has a predetermined and non-perpendicular angle of incidence with the ground different from that of one or more of the other sensors;

wherein the sensors are arranged in an and operable to simultaneously acquire different but adjacent images such that they may be stitched together to provide a full field image having a combined larger field of view;

a processing unit for processing pixel data from the sensors and for interpreting command signals from the image control station;

wherein the processing unit is programmed to perform image processing tasks including at least a transformation process to map the simultaneously acquired images from each sensor to a common plane having the combined larger field of view;

wherein the transformation process is a projection transformation process applied to each image from the sensors to compensate for the different angles of incidence of the sensors, and is unique for each sensor orientation;

an encoder for formatting the pixel data into a National Television System Committee (NTSC) signal having a series of frames of pixel data;

a receiver for receiving control signals from the image control station and a transmitter for delivering the NTSC signal to the image control station;

wherein the control signals comprise at least a set of viewing mode control signals that determine which mode from a set of viewing modes is to be a viewing mode of the image to be transmitted to the control station;

wherein the set of viewing modes comprise at least;

a full field of view mode, a region of interest mode, which provides an image within the full field of view, and a fixed area zoom mode, which provides an image of the same physical area on the ground as the aircraft changes altitude;

wherein the set of viewing modes further comprise a tracking mode for tracking a region of interest, and wherein the tracking mode is performed by repeatedly changing the region of interest by shifting the region of interest across the full field of view in the direction of the flight of the aircraft; and

wherein the processing unit interprets a viewing mode control signal to determine which pixels of the full field of view to transmit to the image control station.

2. The system of claim 1, wherein the control unit is implemented with at least a universal asynchronous receiver/transmitter (UART).

3. The system of claim 1, wherein the processing unit is programmed to poll the radio unit to determine if control signals have been received.

4. The system of claim 1, wherein the control signals represent commands to zoom, pan, or tilt the image.

5. The system of claim 1, wherein the camera unit has multiple sensors, and the control signal represents a command to pan, tilt, or zoom the image from any of the sensors.

6. The system of claim 1, wherein the viewing modes include at least a tracking mode for tracking a region of interest, and wherein the tracking mode accommodates the flight of the aircraft.

7. The system of claim 1, wherein at least one of the modes is a route tracking mode, which provides images of a pre-determined route on the ground.

8. The system of claim 1, wherein the camera unit has a field of view based on an expected flight altitude and a desired pixel density per foot on the ground.

9. The system of claim 1, wherein the set of pixels for each frame for each mode are the same in number, such that the same number of pixels is transmitted from the processing unit regardless of mode.

10. The system of claim 1, wherein the region of interest mode provides an image with pixel data from at least two different sensors.

11. The system of claim 1, wherein the combined field of view covers at least 120 degrees horizontally.

12. A method of capturing pictorial images from an unmanned aircraft for airborne image capture of the ground and for delivery of the images to a remote image control station, comprising:

mounting at least four image sensors, all sensors being identical, mounted on the underside of the aircraft;

wherein each sensor has an associated lens and is mounted on the underside of the aircraft with its lens to provide a fixed field of view for each sensor;

wherein each sensor is oriented on a surface of the underside that is non-planar to the ground, such that images from the sensors appear tilted with respect to one another in the absence of transformation processing;

wherein the optical axis of each sensor has a predetermined and non-perpendicular angle of incidence with the ground different from that of one or more of the other sensors;

wherein the sensors are arranged and operable to simultaneously acquire different but adjacent images such that the images may be stitched together to provide a full field image having a combined larger full field of view;

programming a processing unit on board the aircraft to process pixel data from the sensors and to interpret command signals from the image control station;

wherein the processing unit is programmed to perform image processing tasks including at least a transformation process to map the simultaneously acquired images from each sensor to a common plane having the combined larger full field of view;

wherein the transformation process is a projection transformation process applied to each image from the sensors to compensate for the different angles of incidence of the sensors, and is unique for each sensor orientation;

using an on-board encoder to format the pixel data into a National Television System Committee (NTSC) signal having a series of frames of pixel data;

using an on-board receiver to receive control signals from the image control station and a transmitter for delivering the NTSC signal to the image control station;

wherein the control signals comprise at least a set of viewing mode control signals that determine which mode from a set of viewing modes is to be a viewing mode of the image to be transmitted to the control station;

wherein the set of viewing modes comprise at least;

a full field of view mode, a region of interest mode, which provides an image within the full field of view, and a fixed area zoom mode, which provides an image of the same physical area on the ground as the aircraft changes altitude;

wherein the set of viewing modes further comprises a tracking mode for tracking a region of interest, and wherein the tracking mode is performed by repeatedly changing the region of interest by shifting the region of interest across the full field of view in the direction of the flight of the aircraft; and

wherein the processing unit interprets a viewing mode control signal to determine which pixels of the full field of view to transmit to the image control station.