Camera configuration on movable objects

• US 9,778,662 B2
• Filed: 07/18/2016
• Issued: 10/03/2017
• Est. Priority Date: 07/08/2015
• Status: Active Grant

• Current Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Original Assignee: SZ DJI Technology Co., Ltd. (d/b/a DJI) (iFlight Technology Company Limited)
• Inventors: Tang, Ketan, Zhang, Honghui, Zhao, Cong
• Primary Examiner(s): WONG, ALLEN C
• Application Number:
  US15/213,118
• Publication Number:
  US 20170010623A1
• Time in Patent Office: 442 Days
• Field of Search: 348144
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: G01C 3/00 Measuring distances in line of sight; Optical rangefinders tapes, chains or wheels for measuring length G01B3/00; active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves, G01S17/48
• CPC Class Codes: G01C 3/14 with binocular observation at a single point, e.g. stereoscopic type G01C3/20 takes precedence
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Systems and methods for obstacle detection and state information determination are provided. In some embodiments, a movable object may carry one or more imaging devices. The imaging devices may be arranged on the movable object so as to have a field of view oriented vertically relative to the movable object. The arrangement of the imaging device may complement or supplant existing arrangement schemes and provide efficient, multi-functional and cost-effective means of arranging imaging devices on movable objects.

Claims

1. A method of detecting obstacles using an imaging device carried by a movable object, the method comprising:

receiving, at one or more processors, first image data of an obstacle from the imaging device carried by the movable object, wherein the imaging device is arranged on the movable object so as to have a field of view oriented upward or downward relative to the movable object;

generating, at the one or more processors, a control signal to control one or more propulsion units of the movable object to cause the movable object to move;

receiving, at the one or more processors, second image data of the obstacle from the imaging device after the movable object has moved;

calculating, with aid of the one or more processors, a distance between the movable object and the obstacle based on the first image data and the second image data; and

determining a speed moved by the movable object between collection of the first image data and the second image data, based on the first image data and the second image data.

2. The method of claim 1, further comprising determining a height of the obstacle above the movable object, based on the first image data and the second image data.

3. The method of claim 1, wherein the movable object is an unmanned aerial vehicle (UAV).

4. The method of claim 1, wherein the calculating step comprises assessing the first and second image data in order to detect a change in a portion of the field of view of the imaging device occupied by the obstacle in the first image data collected prior to movement by the movable object and in the second image data collected after the movable object has moved.

5. The method of claim 1, further comprising determining a horizontal distance or a vertical distance moved by the movable object between the collection of the first image data and the second image data, based on the first image data and the second image data.

6. The method of claim 1, wherein the imaging device is a monocular camera.

7. The method of claim 1, wherein the movable object carries one or more imaging devices that have a field of view oriented upward and a field of view oriented downward relative to the movable object.

8. The method of claim 1, wherein the obstacle is positioned at a side of the movable object, and wherein the distance from the movable object to the obstacle is based on an angle of view of the obstacle within the first image data and the second image data.

9. The method of claim 1, wherein the calculating step comprises:

(1) detecting feature points in the first image data and the second image data; and

(2) determining changes in the feature points between the first image data and the second image data.

10. A system for detecting obstacles using a movable object, the system comprising:

an imaging device carried by the movable object and configured to obtain image data of an obstacle, wherein the imaging device is arranged on the movable object so as to have a field of view oriented upward or downward relative to the movable object;

one or more propulsion units configured to effect movement of the movable object; and

one or more processors individually or collectively configured to;

receive first image data of the obstacle from the imaging device;

generate control signals to control the one or more propulsion units to cause the movable object to move;

receive second image data of the obstacle from the imaging device after the movable object has moved;

calculate a distance between the movable object and the obstacle based on the first image data and the second image data; and

determine a speed moved by the movable object between collection of the first image data and the second image data, based on the first image data and the second image data.

11. The system of claim 10, wherein the one or more processors are individually or collectively configured to determine a height of the obstacle above the movable object, based on the first image data and the second image data.

12. The system of claim 10, wherein the movable object is an unmanned aerial vehicle (UAV).

13. The system of claim 10, wherein the control signals cause the movable object to move a horizontal distance or a vertical distance.

14. The system of claim 10, wherein the one or more processors are individually or collectively configured to assess the first and second image data in order to detect changes in a portion of the field of view of the imaging device occupied by the obstacle prior to and after the movable object has moved.

15. The system of claim 10, wherein the one or more processors are individually or collectively configured to determine a horizontal distance or a vertical distance moved.

16. A method of detecting obstacles using a movable object, the method comprising:

receiving image data of an obstacle from an imaging device carried by the movable object, wherein the imaging device is configured to have a field of view oriented upward or downward relative to the movable object;

receiving attitude information for the movable object;

calculating, via one or more processors, a distance between the movable object and the obstacle based on the image data of the obstacle and the attitude information for the movable object; and

determining a speed moved by the movable object between collection of a first image and a second image from the image data, based on the first image and the second image.

17. The method of claim 16, wherein the calculating step comprises determining a portion of the field of view of the imaging device that is occupied by the obstacle, based on the image data.

18. The method of claim 16, wherein the movable object is an unmanned aerial vehicle (UAV).

19. The method of claim 16, further comprising operating the movable object within an indoor environment, wherein the image data comprises images of a ceiling within the indoor environment.

20. The method of claim 16, further comprising operating the movable object within an outdoor environment, wherein the image data comprises images of the sky.

21. The method of claim 16, wherein the field of view of the imaging device is oriented downward relative to the movable object.

22. The method of claim 21, wherein the movable object is operated within an indoor environment and the image data comprises images of a floor within the indoor environment.

23. A system for detecting obstacles using a movable object, the system comprising:

an imaging device carried by the movable object and configured to obtain image data of an obstacle, wherein the imaging device is configured to have a field of view oriented upward or downward relative to the movable object; and

one or more processors individually or collectively configured to;

receive the image data of the obstacle from the imaging device;

receive attitude information for the movable object; and

calculate a distance between the movable object and the obstacle based on the image data of the obstacle and the attitude information for the movable object; and

determine a speed moved by the movable object between collection of a first image and a second image from the image data, based on the first image and the second image.

24. The system of claim 23, wherein the one or more processors are individually or collectively configured to determine a portion of the field of view of the imaging device that is occupied by the obstacle, based on the image data.

25. The system of claim 23, wherein the movable object is an unmanned aerial vehicle (UAV).

26. The system of claim 23, wherein the movable object is operated within an indoor environment and the image data comprises images of a ceiling within the indoor environment.

27. The system of claim 23, wherein the movable object is operated within an outdoor environment and the image data comprises images of the sky.

28. The system of claim 23, wherein the field of view of the imaging device is oriented downward relative to the movable object.

29. The system of claim 28, wherein the movable object is operated within an indoor environment and the image data comprises images of a floor within the indoor environment.