Navigation based on radar-cued visual imaging

• US 10,690,770 B2
• Filed: 04/04/2019
• Issued: 06/23/2020
• Est. Priority Date: 02/20/2014
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Rosenblum, Kevin, Dagan, Erez
• Primary Examiner(s): Bythrow, Peter M
• Application Number:
  US16/375,554
• Publication Number:
  US 20190235073A1
• Time in Patent Office: 446 Days
• CPC Class Codes: G01S 13/426 Scanning radar, e.g. 3D radar G01S13/66 takes precedence
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/9318 Controlling the steering
• CPC Class Codes: G01S 2013/93185 Controlling the brakes
• CPC Class Codes: G01S 2013/9319 Controlling the accelerator
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/93273 on the top of the vehicles
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A navigation system for a vehicle may include at least one image capture device configured to acquire a plurality of images of an environment of a vehicle and a radar sensor to detect an object in the environment of the vehicle and to provide and output including range information indicative of at least one of a range or range rate between the vehicle and the object. The system may also include at least one processing device programmed to: receive the plurality of images from the at least one image capture device; receive the output from the radar sensor; determine, for each of a plurality of image segments in a first image, from among the plurality of images, and corresponding image segments in a second image, from among the plurality of images, an indicator of optical flow; use range information determined based on the output of the radar sensor together with the indicators of optical flow determined for each of the plurality of image segments in the first image and the corresponding image segments in the second image to calculate for each of a plurality of imaged regions at least one value indicative of a focus of expansion; identify a target object region, including at least a subset of the plurality of imaged regions that share a substantially similar focus of expansion; and cause a system response based on the identified target object region.

Claims

1. A navigation system for a vehicle, the system comprising:

at least one image capture device configured to acquire a plurality of images of an environment of a vehicle;

a radar sensor to detect an object in the environment; and

at least one processing device programmed to;

receive the plurality of images from the at least one image capture device;

receive output from the radar sensor;

determine, from the plurality of images, an indicator of optical flow;

determine a value indicative of an expected optical inflation associated with the object in the plurality of images;

determine that the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow; and

suppress, when the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow, a system response.

2. The navigation system of claim 1, wherein the at least one processing device is further programmed to identify a target object within a region of interest based on the expected optical inflation value and the indicator of optical flow.

3. The navigation system of claim 1, wherein the at least one processing device is further programmed to determine that the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow based on a target object range and a range rate.

4. The navigation system of claim 1, wherein the at least one processing device is further programmed to determine that the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow for image portions associated with a road or background objects.

5. The navigation system of claim 1, wherein the at least one processing device is further programmed to derive an estimated focus of expansion (FOE) associated with the object based upon the optical flow.

6. The navigation system of claim 1, wherein the at least one processing device is further programmed to derive an estimated focus of expansion associated with the object based upon a time-to-collision (TTC).

7. The navigation system of claim 6, wherein the estimated focus of expansion (FOE) corresponds to a direction that a vehicle is traveling while capturing the plurality of images.

8. The navigation system of claim 1, wherein the at least one processing device is further programmed to determine the value indicative of the expected optical inflation based on the radar sensor.

9. The navigation system of claim 1, wherein the system response includes at least one of braking, steering, and deceleration.

10. A vehicle including a navigation system, the vehicle comprising:

at least one image capture device positioned on the vehicle and configured to acquire a plurality of images of an environment of the vehicle;

a radar sensor to detect an object in the environment; and

at least one processing device programmed to;

receive the plurality of images from the at least one image capture device;

receive an output from the radar sensor;

determine, from the plurality of images, an indicator of optical flow;

determine a value indicative of an expected optical inflation associated with the object in the plurality of images;

determine that the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow; and

suppress, when the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow, a system response.

11. The vehicle including navigation system of claim 10, wherein the at least one processing device is further programmed to identify a target object within a region of interest based on the expected optical inflation value and the indicator of optical flow.

12. The vehicle including navigation system of claim 11, wherein the at least one processing device is further programmed to determine that the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow based on a target object range and a range rate.

13. The vehicle including navigation system of claim 11, wherein the at least one processing device is further programmed to determine that the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow for image portions associated with a road or background objects.

14. The vehicle including navigation system of claim 10, wherein the at least one processing device is further programmed to derive an estimated focus of expansion (FOE) associated with the object based upon the optical flow.

15. The vehicle including navigation system of claim 10, wherein the at least one processing device is further programmed to derive an estimated focus of expansion associated with the object based upon a time-to-collision (TTC).

16. The vehicle including navigation system of claim 14, wherein the estimated focus of expansion (FOE) corresponds to a direction that the vehicle is traveling while capturing the plurality of images.

17. The vehicle including navigation system of claim 10, wherein the at least one processing device is further programmed to determine the value indicative of the expected optical inflation based on the radar sensor.

18. The vehicle included navigation system of claim 10, wherein the system response includes at least one of braking, steering, and deceleration.

19. A navigation method for a vehicle, the method comprising:

receiving a plurality of images from the at least one image capture device;

receiving an output from the radar sensor;

determining, from the plurality of images, an indicator of optical flow;

determining a value indicative of an expected optical inflation associated with the object in the plurality of images;

determining that the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow; and

suppressing, when the value indicative of the expected optical inflation associated with the object does not match the indicator of optical flow, a system response.

20. The navigation method of claim 19, further comprising identifying a target object within a region of interest based on the expected optical inflation value and the indicator of optical flow.