Methods and apparatus for image processing

• US 10,389,949 B2
• Filed: 10/14/2016
• Issued: 08/20/2019
• Est. Priority Date: 06/08/2015
• Status: Expired due to Fees

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: Yang, Kang, Ren, Wei
• Primary Examiner(s): Young, Patricia I
• Application Number:
  US15/293,761
• Publication Number:
  US 20170302838A1
• Time in Patent Office: 1,040 Days
• Field of Search: 348117
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 7/49 based on structural texture description, e.g. using primitives or placement rules
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 23/72 Combination of two or more compensation controls
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

Apparatus and methods for image processing are provided. Imaging devices may be coupled to movable objects, such as unmanned aerial vehicles (UAVs). One or more images may be captured by an imaging device, such as a camera. The one or more images may be captured under the same or different parameters. The one or more captured images may be processed for use in computer vision applications. Processing the one or more captured images may involve analyzing a texture of the images. Subsequent images may be captured under the same or different parameters.

Claims

1. A system comprising:

a non-transitory computer readable medium storing instructions; and

a processor coupled to the non-transitory computer readable medium and configured to execute the instructions to;

receive a plurality of images, wherein each of the plurality of images are captured using an image capture device under conditions that vary at least one parameter of the image capture device;

perform an analysis of texture of each of the plurality of images;

select one or more images from the plurality of images based on the analyzed texture of each of the plurality of images; and

generate a signal to vary or maintain the parameter of the image capture device based on the analysis of the texture to yield a desired texture quality of a subsequent image;

wherein the parameter is exposure time and/or gain of the image capture device, and the plurality of images are captured using different exposure times and/or different gains.

2. The system of claim 1, wherein performing an analysis of texture of each of the plurality of images comprises performing an analysis of feature points of each of the plurality of images.

3. The system of claim 1, wherein the plurality of images are captured by the image capture device within 0.5 seconds.

4. The system of claim 1, wherein the at least one parameter of the image capture device is varied a) by applying a positive or negative predetermined offset to the at least one parameter of the image capture device, or b) by multiplying or dividing the at least one parameter of the image capture device by a predetermined amount.

5. The system of claim 1, wherein the selected image(s) yields a desired texture quality relative to other images of the plurality of images.

6. The system of claim 5, wherein the image capture device is configured to capture a subsequent image using the same parameter of the image capture device that was used in capturing the selected image(s), and wherein the subsequent image is used in a computer vision process.

7. The system of claim 1, wherein the selected image(s) meets or exceeds a predetermined threshold condition, and wherein the predetermined threshold condition is determined based on a planned use of one or more of the plurality of images in a computer vision application.

8. The system of claim 1, wherein the selected image(s) meets or exceeds a predetermined threshold condition, and wherein the predetermined threshold condition includes a threshold number, distribution, or quality of feature points.

9. The system of claim 1, wherein the processor is further configured to execute the instructions to determine how the parameter of the image capture device is to be varied.

10. The system of claim 1, wherein the texture of the image is analyzed using a FAST or Harris corner detection algorithm.

11. The system of claim 1, wherein the image capture device is located on board an unmanned aerial vehicle (UAV).

12. A system comprising:

a non-transitory computer readable medium storing instructions; and

a processor coupled to the non-transitory computer readable medium and configured to execute the instructions to;

receive an image captured by an image capture device;

perform an analysis of texture of the image; and

generate a signal to vary or maintain a parameter of the image capture device based on the analysis of the texture to yield a desired texture quality of a subsequent image;

wherein the parameter is exposure time and/or gain of the image capture device.

13. The system of claim 12, wherein performing the analysis of texture of the image comprises performing an analysis of feature points of the image.

14. The system of claim 12, wherein the subsequent image is used in a computer vision process comprising navigation.

15. The system of claim 12, wherein the texture of the image is analyzed using a FAST or Harris corner detect algorithm.

16. The system of claim 12, wherein the image capture device is located on board an unmanned aerial vehicle (UAV).

17. The system of claim 12, wherein the processor is further configured to execute the instructions to determine how the parameter of the image capture device is to be varied or maintained using an AEC/AGC algorithm.

18. The system of claim 12, wherein the desired texture quality includes meeting and/or exceeding a predetermined threshold condition, and wherein the predetermined threshold condition is determined based on a planned use of the image or the subsequent image in a computer vision application.

19. The system of claim 12, wherein the desired texture quality includes meeting and/or exceeding a predetermined threshold condition, and wherein the predetermined threshold condition includes a threshold number, distribution, or quality of feature points.