UNMANNED AERIAL VEHICLE IMAGE PROCESSING SYSTEM AND METHOD

• US 20120320203A1
• Filed: 06/15/2012
• Published: 12/20/2012
• Est. Priority Date: 06/17/2011
• Status: Active Grant

• Current Assignee: National Cheng KUNG University (Government of The Republic of China)
• Original Assignee: National Cheng KUNG University (Government of The Republic of China)
• Inventors: Liu, Cheng Chien
• Granted Patent: US 9,336,568 B2
• Time in Patent Office: Days
• US Class Current: 348/144
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 11/04 Interpretation of pictures
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An unmanned aerial vehicle image processing system includes a digital camera mounted on an unmanned aerial vehicle, an autopilot controller installed in the unmanned aerial vehicle and pre-loaded with a base image, and a computer device. The autopilot controller programs a flight path as a series of mission lags to cover a target polygon on the base image in a lawnmower pattern for controlling the unmanned aerial vehicle to perform a flight mission in which the autopilot controller records plural GPS/INS data and triggers the digital camera to take plural photos which are further calibrated to generate plural calibrated photos corresponding to the plural GPS/INS data, respectively. The computer device processes the plural calibrated photos to generate plural projected photos by using the corresponding plural GPS/INS data, respectively, and stitches the plural projected photos to a mosaic image by a photo-stitching process.

Claims

1. An unmanned aerial vehicle image processing system, comprising:

a digital camera mounted on an unmanned aerial vehicle;

an autopilot controller installed in the unmanned aerial vehicle acrd pre-loaded with a fly file generated from a base image drawn thereon a target polygon, wherein the autopilot controller is configured to program a flight path as a series of mission lags to cover the target polygon on the base image in a lawnmower pattern for controlling the unmanned aerial vehicle to perform a flight mission by traveling along the flight path in which the autopilot controller records plural GPS/INS data and triggers the digital camera plural times to take plural photos which are further calibrated based on calibration parameters of the digital camera to generate plural calibrated photos corresponding to the plural UPS/INS data, respectively; and

a computer device for receiving the plural calibrated photos and the plural GPS/INS data, processing the plural calibrated photos to generate plural projected photos by using the corresponding plural GPS/INS data, respectively, and stitching the plural projected photos to a mosaic image by a photo-stitching process.

2. The unmanned aerial vehicle image processing system as claimed in claim 1, wherein the computer device further geo-referencing the mosaic image by specifying coordinates of three points referenced from the base image.

3. The unmanned aerial vehicle image processing system as claimed in claim 1, wherein the flight path is programmed to specify flight courses to be parallel wind direction.

4. The unmanned aerial vehicle image processing system as claimed in claim 3, wherein the flight path is programmed to add additional waypoints on the flight path to ensure that a full U-turn can be made by the unmanned aerial vehicle.

5. The unmanned aerial vehicle image processing system as claimed in claim 1, wherein the plural photos maintain certain fraction of overlapping.

6. The unmanned aerial vehicle image processing system as claimed in claim 1, wherein each photo taken in the flight mission is calibrated by calibration parameters of lens distortion.

7. The unmanned aerial vehicle image processing system as claimed in claim 1, wherein the base image is cut into plural reference images corresponding to the plural photos taken in the flight mission, respectively and the computer device compares each projected photo to its corresponding reference image to generate an RST-corrected GPS/INS data, applies RST transformation to geo-reference the corresponding calibrated photo with the RST-corrected GPS/INS data, so as to generate an RST photo, and stitches all RST photos to a mosaic image by a photo-stitching process.

8. The unmanned aerial vehicle image processing system as claimed in claim 7, wherein each RST-corrected GPS/INS data is generated by selecting three pairs of conjugate points to conduct an iterated calculation.

9. The unmanned aerial vehicle image processing system as claimed in claim 1, wherein a DTM for the base image is cut into plural reference DTMs corresponding to the plural photos taken in the flight mission, respectively, and the computer device determines detailed relief displacement for each pixel in each photo taken in the flight mission with a corresponding reference DTM and three pairs of conjugate points, orthorectifies each calibrated photo to an ortho photo based on the detailed relief displacements of the corresponding photo, and stitches all ortho photos to a mosaic image by a photo-stitching process.

12. The image processing method as claimed in claim 1, wherein the flight path is programmed to specify flight courses to be parallel wind direction.

13. The image processing method as claimed in claim 12, wherein the flight path is programmed to add additional waypoints on the flight path to ensure that a full U-turn can be made by the unmanned aerial vehicle.

10. An image processing method for use in are unmanned aerial vehicle image processing system including a digital camera mounted on an unmanned aerial vehicle, an autopilot controller installed in the unmanned aerial vehicle, and a computer device, the method comprising the steps of:

(A) configuring the autopilot controller, which is pre-loaded with a fly file generated from a base image drawn thereon a target polygon, to program a flight path as a series of mission lags to cover the target polygon on the base image in a lawnmower pattern for controlling the unmanned aerial vehicle to perform a flight mission by traveling along the flight path in which the autopilot controller records plural GPS/INS data and triggers the digital camera plural times to take plural photos which are further calibrated based on calibration parameters of the digital camera to generate plural calibrated photos corresponding to the plural GPS/INS data, respectively;

(B) downloading the plural calibrated photos and the plural GPS/INS data into the computer device for processing the plural calibrated photos to generate plural projected photos by using the corresponding plural GPS/INS data respectively; and

(C) stitching the plural projected photos to a mosaic image by a photo-stitching process.

11. The image processing method as claimed in claim 10, further comprises:

geo-referencing the mosaic mage by specifying coordinates of three points referenced from the base image.

14. The image processing method as claimed in claim 10, wherein the plural photos maintain certain fraction of overlapping.

15. The image processing method as claimed in claim 10, wherein each photo taken in the flight mission is calibrated by calibration parameters of lens distort.

16. The image processing method as claimed in claim 10, wherein the base image is cut into plural reference images corresponding to the plural photos taken in the flight mission, respectively, and the computer device compares each projected photo to its corresponding reference image to generate an RST-corrected GPS/INS data, applies RST transformation to geo-reference the corresponding calibrated photo with the RST-corrected GPS/INS data, so as to generate an RST photo, and stitches all RST photos to a mosaic image by a photo-stitching process.

17. The image processing method as claimed in claim 16, wherein each RST-corrected GPS/INS data is generated by selecting three pairs of conjugate points to conduct an iterated calculation.

18. The image processing method as claimed in claim 10, wherein a DTM for the base image is cut into plural reference DTMs corresponding to the plural photos taken in the flight mission, respectively, and the computer device determines detailed relief displacement for each pixel in each photo taken in the flight mission with a corresponding reference DTM and three pairs of conjugate points, orthorectifies each calibrated photo to an ortho photo based on the detailed relief displacements of the corresponding photo, and stitches all ortho photos to a mosaic image by a photo-stitching process.