Suboptimal immediate navigational response based on long term planning

• US 10,627,830 B2
• Filed: 06/06/2019
• Issued: 04/21/2020
• Est. Priority Date: 01/05/2016
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon, Shalev-Shwartz, Shai, Shammah, Shaked, Shashua, Amnon
• Primary Examiner(s): Dager, Jonathan M
• Application Number:
  US16/433,926
• Publication Number:
  US 20190286155A1
• Time in Patent Office: 320 Days
• Field of Search: None
• CPC Class Codes: B60W 2050/0088 Adaptive recalibration
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2520/16 Pitch
• CPC Class Codes: B60W 2520/18 Roll
• CPC Class Codes: B60W 2552/15 Road slope, i.e. the inclination of a road segment in the longitudinal direction
• CPC Class Codes: B60W 2552/30 Road curve radius
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4046 Behavior, e.g. aggressive or erratic
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 2554/804 Relative longitudinal speed
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0953 the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 30/18163 Lane change; Overtaking manoeuvres
• CPC Class Codes: B60W 50/0097 Predicting future conditions
• CPC Class Codes: B60W 50/045 Monitoring control system parameters
• CPC Class Codes: B60W 60/0011 involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles
• CPC Class Codes: B60W 60/00276 : for two or more other traffic participants
• CPC Class Codes: G01C 21/34 : Route searching; Route guidance
• CPC Class Codes: G01C 21/3453 : Special cost functions, i.e. other than distance or default speed limit of road segments
• CPC Class Codes: G01C 21/3602 : Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0055 : with safety arrangements
• CPC Class Codes: G05D 1/0231 : using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0253 : extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G06N 20/00 : Machine learning
• CPC Class Codes: G06N 3/00 : Computing arrangements based on biological models
• CPC Class Codes: G06N 3/006 : based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 3/044 : Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/08 : Learning methods
• CPC Class Codes: G06N 5/046 : Forward inferencing; Production systems
• CPC Class Codes: G06N 7/01 : Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 : of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A system for navigating an autonomous vehicle using reinforcement learning techniques is provided. The system includes at least one processing device programmed to: receive, from a camera, a plurality of images representative of an environment of the host vehicle; analyze the plurality of images to identify a present navigational state associated with the host vehicle; select a second potential navigational action based on a determination that an expected reward associated with a fourth indicator is greater than an expected reward associated with a second indicator.

Claims

1. A navigation system for a host vehicle, the system comprising:

at least one processing device programmed to;

receive, from a camera, a plurality of images representative of an environment of the host vehicle;

analyze the plurality of images to identify a present navigational state associated with the host vehicle;

determine a first potential navigational action for the host vehicle based on the identified present navigational state;

determine a first indicator of an expected reward based on the first potential navigational action and the identified present navigational state;

predict a first future navigational state based on the first potential navigational action;

determine a second indicator of an expected reward associated with at least one future action determined to be available to the host vehicle in response to the first future navigational state;

determine a second potential navigational action for the host vehicle based on the identified present navigational state;

determine a third indicator of an expected reward based on the second potential navigational action and the identified present navigational state;

predict a second future navigational state based on the second potential navigational action;

determine a fourth indicator of an expected reward associated with at least one future action determined to be available to the host vehicle in response to the second future navigational state;

select the second potential navigational action based on a determination that, while the expected reward associated with the first indicator is greater than the expected reward associated with the third indicator, the expected reward associated with the fourth indicator is greater than the expected reward associated with the first indicator; and

cause at least one adjustment of a navigational actuator of the host vehicle in response to the selected second potential navigational action.

2. The navigation system of claim 1, wherein the navigational actuator includes at least one of a steering mechanism, a brake, or an accelerator.

3. The navigation system of claim 1, wherein the first potential navigational action includes a merge in front of a target vehicle and wherein the second potential navigational action includes a merge behind the target vehicle.

4. The navigation system of claim 1, wherein a difference between the expected reward associated with the fourth indicator and the expected reward associated with the first indicator is greater than a difference between the expected reward associated with the third indicator and the expected reward associated with the first indicator.

5. The navigation system of claim 1, wherein a difference between the expected reward associated with the fourth indicator and the expected reward associated with the second indicator is greater than a difference between the expected reward associated with the third indicator and the expected reward associated with the first indicator.

6. An autonomous vehicle, the autonomous vehicle comprising:

a frame;

a body attached to the frame;

a camera; and

at least one processing device programmed to;

receive, from the camera, a plurality of images representative of an environment of the autonomous vehicle;

analyze the plurality of images to identify a present navigational state associated with the autonomous vehicle;

determine a first potential navigational action for the autonomous vehicle based on the identified present navigational state;

determine a first indicator of an expected reward based on the first potential navigational action and the identified present navigational state;

predict a first future navigational state based on the first potential navigational action;

determine a second indicator of an expected reward associated with at least one future action determined to be available to the autonomous vehicle in response to the first future navigational state;

determine a second potential navigational action for the autonomous vehicle based on the identified present navigational state;

determine a third indicator of an expected reward based on the second potential navigational action and the identified present navigational state;

predict a second future navigational state based on the second potential navigational action;

determine a fourth indicator of an expected reward associated with at least one future action determined to be available to the autonomous vehicle in response to the second future navigational state;

select the second potential navigational action based on a determination that, while the expected reward associated with the first indicator is greater than the expected reward associated with the third indicator, the expected reward associated with the fourth indicator is greater than the expected reward associated with the first indicator; and

cause at least one adjustment of a navigational actuator of the autonomous vehicle in response to the selected second potential navigational action.

7. The autonomous vehicle of claim 6, wherein the navigational actuator includes at least one of a steering mechanism, a brake, or an accelerator.

8. The autonomous vehicle of claim 6, wherein the first potential navigational action includes a merge in front of a target vehicle and wherein the second potential navigational action includes a merge behind the target vehicle.

9. The autonomous vehicle of claim 6, wherein a difference between the expected reward associated with the fourth indicator and the expected reward associated with the first indicator is greater than a difference between the expected reward associated with the third indicator and the expected reward associated with the first indicator.

10. The autonomous vehicle of claim 6, wherein a difference between the expected reward associated with the fourth indicator and the expected reward associated with the second indicator is greater than a difference between the expected reward associated with the third indicator and the expected reward associated with the first indicator.

11. A method for navigating an autonomous vehicle, the method comprising:

receiving, from a camera, a plurality of images representative of an environment of the autonomous vehicle;

analyzing the plurality of images to identify a present navigational state associated with the autonomous vehicle;

determining a first potential navigational action for the autonomous vehicle based on the identified present navigational state;

determining a first indicator of an expected reward based on the first potential navigational action and the identified present navigational state;

predicting a first future navigational state based on the first potential navigational action;

determining a second indicator of an expected reward associated with at least one future action determined to be available to the autonomous vehicle in response to the first future navigational state;

determining a second potential navigational action for the autonomous vehicle based on the identified present navigational state;

determining a third indicator of an expected reward based on the second potential navigational action and the identified present navigational state;

predicting a second future navigational state based on the second potential navigational action;

determining a fourth indicator of an expected reward associated with at least one future action determined to be available to the autonomous vehicle in response to the second future navigational state;

selecting the second potential navigational action based on a determination that, while the expected reward associated with the first indicator is greater than the expected reward associated with the third indicator, the expected reward associated with the fourth indicator is greater than the expected reward associated with the first indicator; and

causing at least one adjustment of a navigational actuator of the autonomous vehicle in response to the selected second potential navigational action.