Methods for autonomously navigating across uncontrolled and controlled intersections

• US 10,338,591 B2
• Filed: 11/21/2017
• Issued: 07/02/2019
• Est. Priority Date: 11/22/2016
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Baalke, Uriah, Braun, Stav, Jin, Sonia
• Primary Examiner(s): Nguyen, Tan Q
• Application Number:
  US15/819,860
• Publication Number:
  US 20180329418A1
• Time in Patent Office: 588 Days
• Field of Search: 701 27
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 50/0097 Predicting future conditions
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0214 in accordance with safety or protection criteria, e.g. avoiding hazardous areas monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0244 using reflecting strips
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0276 using signals provided by a source external to the vehicle involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 50/40 Business processes related to the transportation industry shipping G06Q10/083
• CPC Class Codes: G08G 1/09623 Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle G08G1/0967 takes precedence
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

One variation of a method for autonomously navigating along a crosswalk includes: at a first time, navigating autonomously along a sidewalk toward a crosswalk coinciding with a navigation route assigned to the autonomous vehicle; recording optical data of a scene proximal the autonomous vehicle via an optical sensor integrated into the autonomous vehicle; aligning an anteroposterior axis of the autonomous vehicle to the crosswalk detected in the optical data; identifying a pedestrian proximal the crosswalk in the optical data; in response to the pedestrian entering the crosswalk at a second time succeeding the first time, predicting right of way of the autonomous vehicle to enter the crosswalk; and, in response to predicting right of the autonomous vehicle to enter the crosswalk, autonomously navigating from the sidewalk into the crosswalk and autonomously navigating along the crosswalk to an opposing sidewalk according to the navigation route.

Claims

1. A method for autonomously navigating along a first crosswalk at an intersection by an autonomous vehicle, wherein the method comprises:

at a first time, autonomously navigating along a first sidewalk toward the first crosswalk at the intersection, wherein the first crosswalk coincides with a navigation route assigned to the autonomous vehicle;

recording optical data of a scene proximate the autonomous vehicle via an optical sensor integrated into the autonomous vehicle;

detecting the first crosswalk in the optical data;

aligning an anteroposterior axis of the autonomous vehicle to the first crosswalk detected in the optical data;

detecting a first pedestrian proximate the intersection in the optical data;

determining that the first pedestrian has entered the first crosswalk at a second time based at least in part on the optical data, wherein the second time follows the first time;

in response to determining that the first pedestrian has entered the first crosswalk at the second time,

predicting right of way of the autonomous vehicle to enter the intersection; and

in response to predicting the right of way of the autonomous vehicle to enter the intersection,

autonomously navigating from the first sidewalk into the first crosswalk; and

autonomously navigating along the first crosswalk to a second sidewalk according to the navigation route.

2. The method of claim 1, further comprising:

detecting an unprotected crosswalk label associated with the intersection in a navigation map; and

determining that the first crosswalk is an unprotected crosswalk based at least in part on the unprotected crosswalk label, wherein detecting the first pedestrian proximate the intersection in the optical data comprises;

detecting the first pedestrian within the first sidewalk proximate the first crosswalk in optical data recorded over a first period of time, wherein the first period of time precedes the second time,

wherein the method further comprises, prior to the second time;

in response to detecting the first pedestrian within the first sidewalk proximate the first crosswalk in the optical data recorded over the first period of time,

calculating a first confidence score for the right of way of the autonomous vehicle to enter the intersection;

determining that the first confidence score is less than a first threshold score; and

in response to determining that the first confidence score is less than the first threshold score, halting on the first sidewalk proximate the first crosswalk,

wherein predicting the right of way of the autonomous vehicle to enter the intersection comprises;

in response to determining that the first pedestrian has entered the first crosswalk at the second time,

calculating a second confidence score for the right of way of the autonomous vehicle to enter the intersection; and

determining that the second confidence score exceeds the first confidence score, and

wherein autonomously navigating from the first sidewalk into the first crosswalk comprises;

in response to determining that the second confidence score exceeds the first confidence score,

autonomously navigating from the first sidewalk into the first crosswalk.

3. The method of claim 2, further comprising:

detecting a vehicle approaching the intersection in the optical data recorded over the first period of time;

determining a first velocity of the vehicle based at least in part on the optical data recorded over the first period of time, wherein the first confidence score is calculated as an inverse function of the first velocity of the vehicle;

determining a second velocity of the vehicle based at least in part on optical data recorded over a second period of time, wherein the second period of time follows the first period of time; and

determining that the second velocity is less than the first velocity,

wherein the second confidence score for the right of way of the autonomous vehicle is calculated based at least in part on the second velocity in response to determining that the second velocity is less than the first velocity.

4. The method of claim 2, wherein autonomously navigating along the first crosswalk to the second sidewalk comprises:

autonomously navigating from the first sidewalk into the first crosswalk via a near ramp at the intersection, wherein the near ramp extends between at least a portion of the first sidewalk and at least a first portion of the first crosswalk;

autonomously navigating within the first crosswalk to a far ramp, wherein the far ramp extends between at least a second portion of the first crosswalk and at least a portion of the second sidewalk, and wherein autonomously navigating within the first crosswalk to the far ramp comprises autonomously maintaining at least a threshold distance between the autonomous vehicle and the first pedestrian as the first pedestrian moves through the first crosswalk; and

autonomously navigating from the first crosswalk to the second sidewalk via the far ramp.

5. The method of claim 1, wherein detecting the first pedestrian proximate the intersection in the optical data comprises:

detecting the first pedestrian within the second sidewalk proximate the first crosswalk in optical data recorded over a first period of time preceding the second time,

wherein the method further comprises;

prior to the second time;

determining that the first pedestrian is stopped within the second sidewalk based at least in part on the optical data recorded over the first period of time preceding the second time;

in response to determining that the first pedestrian is stopped within the second sidewalk,

calculating a first confidence score for the right of way of the autonomous vehicle to enter the intersection;

determining that the first confidence score is less than a first threshold score; and

in response to determining that first confidence score is less than the first threshold score, halting on the first sidewalk proximate the first crosswalk,

wherein predicting the right of way of the autonomous vehicle to enter the intersection comprises;

in response to determining that the first pedestrian has entered the first crosswalk at the second time,

determining a speed of the first pedestrian based at least in part on the optical data;

calculating a second confidence score for the right of way of the autonomous vehicle to enter the intersection as an inverse function of a difference between the speed of the first pedestrian and a predefined nominal pedestrian speed; and

wherein autonomously navigating from the first sidewalk into the first crosswalk comprises;

in response to determining that the second confidence score exceeds the first confidence score,

autonomously navigating from the first sidewalk into the first crosswalk and past the first pedestrian.

6. The method of claim 1, wherein detecting the first pedestrian proximate the intersection in the optical data comprises:

detecting a group of pedestrians within the first sidewalk adjacent the first crosswalk, wherein the first pedestrian is one of the group of pedestrians, wherein predicting the right of way of the autonomous vehicle to enter the intersection comprises;

determining that a subset of the group of pedestrians has entered the first crosswalk at approximately the second time based at least in part on the optical data;

determining that the subset of the group of pedestrians exceeds a threshold proportion of the group of pedestrians; and

in response to determining that the subset of the group of pedestrians exceeds the threshold proportion, predicting the right of way of the autonomous vehicle to enter the intersection, and

wherein autonomously navigating along the first crosswalk to the second sidewalk comprises;

autonomously following at least some of the subset of pedestrians along the first crosswalk to the second sidewalk.

7. The method of claim 1, further comprising:

detecting a second pedestrian within a second crosswalk perpendicular to the first crosswalk at approximately the second time; and

in response to detecting the second pedestrian within the second crosswalk;

withdrawing a prediction of the right of way of the autonomous vehicle to enter the intersection; and

transmitting a query to a remote operator portal for manual confirmation to enter the intersection.

8. The method of claim 1, further comprising:

detecting a traffic signal label associated with the intersection in a navigation map;

determining that the intersection is populated with a lighted traffic signal based on the traffic signal label associated with the intersection in the navigation map; and

scanning the optical data for the lighted traffic signal to determine right of way at the intersection;

wherein identifying the pedestrian proximate the intersection in the optical data comprises;

determining that the lighted traffic signal is obstructed based at least in part on the optical data;

in response to determining that the lighted traffic signal is obstructed,

scanning the optical data for pedestrians proximate a location of the first crosswalk indicated in the navigation map; and

detecting the first pedestrian on the first sidewalk adjacent the first crosswalk prior to the second time,

wherein predicting the right of way of the autonomous vehicle to enter the intersection comprises;

in response to detecting the first pedestrian on the first sidewalk adjacent the first crosswalk prior to the second time,

determining a lack of right of way of the autonomous vehicle to enter the intersection based at least in part on presence of the first pedestrian on the first sidewalk adjacent the first crosswalk prior to the second time; and

in response to determining that the first pedestrian has entered the first crosswalk at the second time,

predicting possession of right of way of the autonomous vehicle to enter the intersection based at least in part on presence of the first pedestrian in the first crosswalk at the second time.

9. The method of claim 1, wherein detecting the first pedestrian proximate the intersection in the optical data comprises:

scanning the scene proximate the autonomous vehicle for pedestrians;

detecting the first pedestrian within the first sidewalk proximate the first crosswalk in optical data recorded via the optical sensor over a period of time preceding the second time; and

calculating an angular yaw offset between the first pedestrian and the first crosswalk;

wherein predicting the right of way of the autonomous vehicle to enter the intersection comprises;

predicting an intent of the first pedestrian to enter the first crosswalk as a function of a duration of the period of time and as an inverse function of the angular yaw offset; and

in response to predicting the intent of the first pedestrian to enter the first crosswalk,

determining that the intent of the first pedestrian to enter the first crosswalk exceeds a threshold intent; and

associating the right of way of the autonomous vehicle to enter the intersection with entry of the first pedestrian into the first crosswalk in response to determining that the intent of the first pedestrian to enter the first crosswalk exceeds the threshold intent.

10. The method of claim 1, wherein autonomously navigating along the first sidewalk comprises:

autonomously navigating toward a portion of the first sidewalk adjacent the first crosswalk at a first velocity, and

wherein autonomously navigating from the first sidewalk into the first crosswalk comprises;

autonomously navigating from the portion of the first sidewalk into the first crosswalk at a second velocity, wherein the second velocity is less than the first velocity.

11. The method of claim 1, wherein detecting the first pedestrian proximate the intersection in the optical data comprises:

detecting the first pedestrian in a second crosswalk at the intersection, wherein the second crosswalk intersects the navigation route,

wherein predicting right of way of the autonomous vehicle to enter the intersection comprises;

calculating a confidence score for the right of way of the autonomous vehicle to enter the intersection based at least in part on presence of the first pedestrian in the second crosswalk that intersects the navigation route;

determining that the confidence score exceeds a threshold score; and

in response to determining that the confidence score exceeds the threshold score,

transmitting a query to a remote operator portal for confirmation to enter the intersection;

transmitting a portion of the optical data to the remote operator portal;

holding at the first sidewalk adjacent the first crosswalk; and

receiving the confirmation to enter the intersection from the remote operator portal in response to the query, and

wherein autonomously navigating from the first sidewalk into the first crosswalk comprises;

autonomously navigating from the first sidewalk into the first crosswalk in response to receiving confirmation to enter the intersection from the remote operator portal.

12. The method of claim 1, wherein detecting the first pedestrian proximate the intersection in the optical data comprises:

detecting the first pedestrian in a second crosswalk at the intersection, wherein the second crosswalk intersects the navigation route,

wherein predicting right of way of the autonomous vehicle to enter the intersection comprises;

calculating a confidence score for the right of way of the autonomous vehicle to enter the intersection based at least in part on presence of the first pedestrian in the second crosswalk that intersects the navigation route;

determining that the confidence score exceeds a threshold score;

in response to determining that the confidence score exceeds the threshold score,

transmitting a first query to a remote operator portal for confirmation to enter the intersection;

transmitting a portion of the optical data to the remote operator portal;

holding at the first sidewalk adjacent the first crosswalk; and

not receiving confirmation to enter the intersection from the remote operator portal in response to the first query, and

wherein autonomously navigating from the first sidewalk into the first crosswalk comprises;

in response to not receiving confirmation to enter the intersection from the remote operator portal in response to the first query;

remaining on the first sidewalk adjacent the first crosswalk;

recording optical data of the scene proximate the autonomous vehicle via the optical sensor at a third time, wherein the third time succeeds the second time;

calculating a revised confidence score for the right of way of the autonomous vehicle to enter the intersection based at least in part on the optical data recorded at the third time succeeding the second time;

determining that the revised confidence score exceeds a second threshold score; and

in response to determining that the revised confidence score exceeds the second threshold score,

transmitting a second query to the remote operator portal for confirmation to enter the intersection; and

receiving the confirmation to enter the intersection from the remote operator portal in response to the second query,

wherein autonomously navigating from the first sidewalk into the first crosswalk comprises;

autonomously navigating from the first sidewalk into the first crosswalk in response to receiving the confirmation to enter the intersection from the remote operator portal in response to the second query.

13. The method of claim 11,

wherein recording optical data of the scene proximate the autonomous vehicle via the optical sensor comprises:

recording a color video feed of the scene proximate the autonomous vehicle via the optical sensor, wherein the optical sensor comprises a color camera integrated into the autonomous vehicle; and

wherein transmitting the portion of the optical data to the remote operator portal comprises;

querying a navigation map for a georeferenced area of interest associated with the intersection;

cropping frames in the color video feed around the georeferenced region of interest; and

transmitting the cropped frames in the color video feed to a remote computer system via wireless communication protocol for distribution to the remote operator portal.

14. A method for autonomously navigating along a crosswalk comprising, by an autonomous vehicle:

autonomously navigating along a first sidewalk toward the crosswalk at an intersection, wherein the crosswalk coincides with a navigation route assigned to the autonomous vehicle;

recording optical data of a scene proximate the autonomous vehicle via an optical sensor integrated into the autonomous vehicle;

aligning an anteroposterior axis of the autonomous vehicle to the crosswalk detected in the optical data;

at a first time, calculating a first confidence score for right of way of the autonomous vehicle to enter the crosswalk based at least in part on proximity of a vehicle detected proximate the intersection in the optical data;

at a second time succeeding the first time, detecting a pedestrian proximate the intersection in the optical data;

determining whether the first confidence score exceeds a first threshold score;

in response to determining that the first confidence score does not exceed the first threshold score,

determining that the pedestrian is entering the crosswalk;

calculating a second confidence score for the right of way of the autonomous vehicle to enter the crosswalk based at least in part on presence of the pedestrian in the crosswalk, wherein the second confidence score is greater than the first confidence score; and

determining whether the second confidence score exceeds a first threshold score;

in response to determining that the second confidence score exceeds the first threshold score,

transmitting a query to a remote operator portal for confirmation for the autonomous vehicle to enter the crosswalk;

receiving the confirmation for the autonomous vehicle to enter the crosswalk from the remote operator portal; and

in response to receiving the confirmation for the autonomous vehicle to enter the crosswalk from the remote operator portal, autonomously navigating from the first sidewalk into the crosswalk; and

autonomously navigating along the crosswalk to a second sidewalk according to the navigation route.

15. The method of claim 14, further comprising:

in response to determining that the first confidence score exceeds the first threshold score,

autonomously navigating from the first sidewalk into the crosswalk; and

autonomously navigating along the crosswalk to the second sidewalk according to the navigation route.

16. The method of claim 14, wherein recording the optical data of the scene comprises:

recording a color video feed of the scene proximate the autonomous vehicle via the optical sensor, wherein the optical sensor is a color camera integrated into the autonomous vehicle;

querying a navigation map for a georeferenced area of interest associated with the intersection;

cropping frames in the color video feed around the georeferenced region of interest; and

transmitting cropped frames in the color video feed to a remote computer system via wireless communication protocol for distribution to the remote operator portal.

17. The method of claim 14, wherein the second confidence score is at least one of:

proportional to a size and a density of a group of pedestrians detected in the crosswalk at approximately the second time;

inversely proportional to a detected speed of the group of pedestrians moving across the crosswalk at approximately the second time;

inversely proportional to a number of pedestrians moving across a second crosswalk perpendicular to the crosswalk at approximately the second time; and

inversely proportional to speeds of vehicles approaching the intersection perpendicular to the crosswalk at approximately the second time.

18. The method of claim 14, wherein detecting the pedestrian proximate the intersection in the optical data comprises:

detecting the pedestrian within the first sidewalk proximate the crosswalk; and

wherein autonomously navigating along the crosswalk to the second sidewalk comprises;

autonomously navigating from the first sidewalk into the crosswalk via a near ramp at the intersection, wherein the near ramp extends between at least a portion of the first sidewalk and at least a first portion of the crosswalk;

autonomously navigating within the crosswalk to a far ramp, wherein the far ramp extends between at least a second portion of the crosswalk and at least a portion of the second sidewalk, and wherein autonomously navigating within the crosswalk to the far ramp comprises autonomously maintaining at least a threshold distance between the autonomous vehicle and the pedestrian as the autonomous vehicle navigates within the crosswalk; and

autonomously navigating from the crosswalk to the second sidewalk via the far ramp.

19. A method for autonomously navigating along a crosswalk at an intersection by an autonomous vehicle, wherein the method comprises:

autonomously navigating along a first sidewalk toward a crosswalk at the intersection, wherein the crosswalk coincides with a navigation route assigned to the autonomous vehicle;

recording optical data of a scene proximate the autonomous vehicle via an optical sensor integrated into the autonomous vehicle;

detecting the crosswalk in the optical data;

aligning an anteroposterior axis of the autonomous vehicle to the crosswalk detected in the optical data;

at a first time, determining states of lighted traffic signals, proximate the intersection based at least in part on the optical data;

at approximately the first time, calculating a velocity of a vehicle approaching the intersection;

calculating a confidence score for right of way of the autonomous vehicle to enter the intersection based at least in part on the states of the lighted traffic signals proximate the intersection and the velocity of the vehicle;

determining that the confidence score exceeds a threshold score;

in response to determining that the confidence score exceeds the threshold score, transmitting a query to a remote operator portal to confirm entry of the autonomous vehicle into the crosswalk;

receiving confirmation of the entry of the autonomous vehicle into the crosswalk from the remote operator portal in response to the query; and

in response to receiving the confirmation from the remote operator portal;

autonomously navigating from the first sidewalk into the crosswalk; and

autonomously navigating along the crosswalk to a second sidewalk according to the navigation route.

20. The method of claim 19, further comprising:

detecting a pedestrian proximate the intersection at approximately the first time based at least in part on the optical data;

determining that the pedestrian entered the crosswalk at a second time based at least in part on the optical data, wherein the second time follows the first time; and

in response to determining that the pedestrian entered the crosswalk at the second time,

increasing the confidence score for the right of way of the autonomous vehicle to enter the intersection.