Systems and methods for vehicle guidance

• US 10,269,257 B1
• Filed: 07/07/2016
• Issued: 04/23/2019
• Est. Priority Date: 08/11/2015
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Gohl, Pascal, Omari, Sammy
• Primary Examiner(s): Elahi, Shan E
• Application Number:
  US15/204,375
• Time in Patent Office: 1,020 Days
• Field of Search: 348 47
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/17 taken from planes or by drones
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 5/0069 : specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/045 : Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04N 13/128 : Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/271 : wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals

Abstract

This disclosure relates to systems and methods for vehicle guidance. Stereo images may be obtained at different times using a stereo image sensor. A depth image may be determined based on an earlier obtained pair of stereo images. The depth image may be refined based on predictions of an earlier stereo image and a later obtained stereo image. Depth information for an environment around a vehicle may be obtained. The depth information may characterize distances between the vehicle and the environment around the vehicle. A spherical depth map may be generated from the depth information. Maneuver controls for the vehicle may be provided based on the spherical depth map.

Claims

1. An unmanned aerial vehicle, the unmanned aerial vehicle comprising:

a stereo image sensor generating visual information from an environment around the unmanned aerial vehicle; and

a processing apparatus coupled to a memory storing instructions that when executed causes the processing apparatus to;

determine depth information for the environment around the unmanned aerial vehicle, the depth information characterizing distances between the unmanned aerial vehicle and the environment around the unmanned aerial vehicle;

generate a spherical depth map from the depth information, a center of the spherical depth map coinciding with a location of the unmanned aerial vehicle and the spherical depth map representing distances to closest surfaces of the environment around the unmanned aerial vehicle as a function of longitude angles and latitude angles, the spherical depth map being divided into map cells corresponding to the longitude angles and the latitude angles, the map cells including a first map cell corresponding to a first longitude angle and a first latitude angle; and

provide maneuver controls for the unmanned aerial vehicle based on the spherical depth map.

2. The unmanned aerial vehicle of claim 1,

wherein the stereo image sensor comprises:

a first image sensor configured to generate first visual output signals conveying first visual information within a first field of view of the first image sensor; and

a second image sensor configured to generate second visual output signals conveying second visual information within a second field of view of the second image sensor;

wherein the depth information is determined by comparing the first visual information with the second visual information.

3. The unmanned aerial vehicle of claim 1, wherein:

generating the spherical depth map includes determining distance values for the map cells, the distance values determined based on the distances to the closest surfaces of the environment around the unmanned aerial vehicle, wherein a first distance value for the first map cell is determined based on a first distance to a first closest surface of the environment around the unmanned aerial vehicle at the first longitude angle and the first latitude angle.

4. The unmanned aerial vehicle of claim 3, wherein the determination of the distance values disregards distances to surfaces of the environment around the unmanned aerial vehicle at individual longitude angles and individual latitude angles that are greater than the distances to the closest surfaces of the environment around the unmanned aerial vehicle at the individual longitude angles and the individual latitude angles.

5. The unmanned aerial vehicle of claim 3, wherein the distance values correspond only to the closest surfaces of the environment around the unmanned aerial vehicle at individual longitude angles and individual latitude angles.

6. The unmanned aerial vehicle of claim 3, wherein the spherical depth map includes a dead zone in a polar region, and the distance values are not determined for the map cells in the dead zone.

7. The unmanned aerial vehicle of claim 1, wherein the processing apparatus comprises a hardware-implemented processor and a software-implemented processor.

8. The unmanned aerial vehicle of claim 7, wherein the hardware-implemented processor is located remotely from the software-implemented processor.

9. The unmanned aerial vehicle of claim 1, further including instructions that when executed causes the processing apparatus to:

responsive to detecting the unmanned aerial vehicle traveling a threshold distance, transform the spherical depth map such that the center of the spherical depth map coincides with the location of the unmanned aerial vehicle.

10. A computer-implemented method for collision avoidance of an unmanned aerial vehicle, the method comprising:

generating visual information from an environment around the unmanned aerial vehicle;

determining depth information for the environment around the unmanned aerial vehicle, the depth information characterizing distances between the unmanned aerial vehicle and the environment around the unmanned aerial vehicle;

generating a spherical depth map from the depth information, a center of the spherical depth map coinciding with a location of the unmanned aerial vehicle and the spherical depth map representing distances to closest surfaces of the environment around the unmanned aerial vehicle as a function of longitude angles and latitude angles, the spherical depth map being divided into map cells corresponding to the longitude angles and the latitude angles, the map cells including a first map cell corresponding to a first longitude angle and a first latitude angle; and

providing maneuver controls for the unmanned aerial vehicle based on the spherical depth map.

11. The method of claim 10, wherein a stereo image sensor of the unmanned aerial vehicle generates the visual information from the environment around the unmanned aerial vehicle, further wherein the stereo image sensor comprises:

a first image sensor configured to generate first visual output signals conveying first visual information within a first field of view of the first image sensor; and

a second image sensor configured to generate second visual output signals conveying second visual information within a second field of view of the second image sensor.

12. The method of claim 11, further comprising:

determining the depth information by comparing the first visual information with the second visual information.

13. The method of claim 10, further comprising:

generating the spherical depth map by determining distance values for the map cells, the distance values determined based on the distances to the closest surfaces of the environment around the unmanned aerial vehicle, wherein a first distance value for the first map cell is determined based on a first distance to a first closest surface of the environment around the unmanned aerial vehicle at the first longitude angle and the first latitude angle.

14. The method of claim 13, wherein the determination of the distance values disregards distances to surfaces of the environment around the unmanned aerial vehicle at individual longitude angles and individual latitude angles that are greater than the distances to the closest surfaces of the environment around the unmanned aerial vehicle at the individual longitude angles and the individual latitude angles.

15. The method of claim 13, wherein the distance values correspond only to the closest surfaces of the environment around the unmanned aerial vehicle at individual longitude angles and individual latitude angles.

16. The method of claim 13, wherein the spherical depth map includes a dead zone in a polar region, and the distance values are not determined for the map cells in the dead zone.

17. The method of claim 10, wherein the unmanned aerial vehicle includes a processing apparatus that comprises a hardware-implemented processor and a software-implemented processor.

18. The method of claim 17, wherein the hardware-implemented processor is located remotely from the software-implemented processor.

19. The method of claim 10, further comprising:

responsive to detecting the unmanned aerial vehicle traveling a threshold distance, transform the spherical depth map such that the center of the spherical depth map coincides with the location of the unmanned aerial vehicle.

20. The method of claim 10, wherein dividing the spherical depth map into map cells enables mapping of the depth information to a fixed sized grid of discrete bearings.