Recognizing assigned passengers for autonomous vehicles

• US 10,579,788 B2
• Filed: 08/17/2017
• Issued: 03/03/2020
• Est. Priority Date: 08/17/2017
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Dyer, John Wesley, Torres, Luis, Epstein, Michael, Chen, Yu-Hsin
• Primary Examiner(s): Thiaw, Catherine
• Application Number:
  US15/679,485
• Publication Number:
  US 20190057209A1
• Time in Patent Office: 929 Days
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: G06F 21/35 communicating wirelessly
• CPC Class Codes: G06F 21/44 Program or device authentication
• CPC Class Codes: G06F 2221/2111 Location-sensitive, e.g. geographical location, GPS
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G08B 13/19647 Systems specially adapted for intrusion detection in or around a vehicle
• CPC Class Codes: G08G 1/202 Dispatching vehicles on the basis of a location, e.g. taxi dispatching
• CPC Class Codes: H04L 2209/84 Vehicles
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 9/3228 One-time or temporary data, i.e. information which is sent for every authentication or authorization, e.g. one-time-password, one-time-token or one-time-key
• CPC Class Codes: H04L 9/3234 involving additional secure or trusted devices, e.g. TPM, smartcard, USB or software token network architectures or network communication protocols for supporting authentication of entities using an additional device in a packet data network H04L63/0853

Abstract

Aspects of the disclosure provide systems and methods for recognizing an assigned passenger. For instance, dispatching instructions to pick up a passenger at a pickup location are received. The instructions include authentication information for authenticating a client computing device associated with the passenger. A vehicle is maneuvered in an autonomous driving mode towards the pickup location. The client device is then authenticated. After authentication, a set of pedestrians within a predetermined distance of the vehicle are identified from sensor information generated by a sensor of the vehicle and location information is received over a period of time from the client device. The received location information is used to estimate a velocity of the passenger. This estimated velocity is used to identify a subset of set of pedestrians that is likely to be the passenger. The vehicle is stopped to allow the passenger to enter the vehicle based on the subset.

Claims

1. A method of recognizing an assigned passenger, the method comprising:

receiving, by one or more processors of a vehicle, dispatching instructions to pick up the assigned passenger at a pickup location, the dispatching instructions including authentication information for a client computing device associated with the assigned passenger;

maneuvering, by the one or more processors, the vehicle towards the pickup location in an autonomous driving mode;

authenticating, by the one or more processors, the client computing device using the authentication information;

after authenticating the client computing device, identifying, by the one or more processors, from sensor information generated by a sensor of the vehicle a set of pedestrians within a predetermined distance of the vehicle;

after authenticating the client computing device, receiving, by the one or more processors, information from the client computing device identifying location of the client computing device over a period of time;

using, by the one or more processors, the received information to estimate a velocity of the client computing device;

receiving, by the one or more processors, information indicative of the velocities of pedestrians within a set of pedestrians within a predetermined distance of the vehicle;

comparing, by the one or more processors, the estimated velocity of the client computing device to the information indicative of the velocities of the pedestrians and thereby identify a subset of the set of pedestrians that is likely to be the assigned passenger; and

stopping, by the one or more processors, the vehicle to allow the assigned passenger to enter the vehicle based on the identified subset.

2. The method of claim 1, wherein the received information includes orientation information generated by a sensor of the client computing device, and the method further includes:

determining an orientation of each pedestrian of the set of pedestrians; and

comparing the orientation information with the determined orientations, and wherein the comparison is further used to identify the subset.

3. The method of claim 1, further comprising using the sensor information to detect a gaze direction for each pedestrian of the set of pedestrians, and wherein the gaze detection for each pedestrian is further used to identify the subset.

4. The method of claim 1, further comprising using the sensor information determine a number of other pedestrians corresponding to pedestrians within a predetermined distance of each pedestrian of the set of pedestrians, and wherein the determined number of other pedestrians within the predetermined distance of each pedestrian is further used to identify the subset.

5. The method of claim 4, wherein the dispatching instructions further identify a number of passengers, and wherein the identified number of passengers is further used to identify the subset.

6. The method of claim 1, wherein the set of pedestrians is updated as additional location information is received from the client computing device.

7. The method of claim 1, wherein stopping the vehicle includes stopping the vehicle closer to a pedestrian of the subset than to the pickup location.

8. The method of claim 1, wherein stopping the vehicle includes stopping the vehicle before the vehicle reaches the pickup location.

9. The method of claim 1, further comprising:

using the sensor information to identify a characteristic that is different between two or more pedestrians of the set of pedestrians;

sending a request to the client device, the request including a question regarding the characteristic; and

receiving a response from the client computing device, and wherein the response is further used to identify the subset.

10. The method of claim 1, wherein using the estimated velocity to identify a subset of the set of pedestrians that is likely to be the assigned passenger includes inputting the estimated velocity into a model in order to identify a likelihood that each pedestrian of the set of pedestrians is the assigned passenger, and wherein the likelihood that each pedestrian of the set of pedestrians is the passenger is further used to identify the subset.

11. A system for recognizing an assigned passenger, the system comprising one or more hardware processors configured to:

receive dispatching instructions to pick up a passenger at a pickup location, the dispatching instructions including authentication information for a client computing device associated with the assigned passenger;

maneuver a vehicle towards the pickup location in an autonomous driving mode;

authenticate the client computing device using the authentication information;

after authenticating the client computing device, identify from sensor information generated by a sensor of the vehicle a set of pedestrians corresponding to pedestrians within a predetermined distance of the vehicle;

after authenticating the client computing device, receive location information from the client computing device over a period of time;

receive information from the client computing device identifying locations of the client computing device over a period of time;

use the received information to estimate a velocity of the client computing device;

receive information indicative of the velocities of pedestrians within a set of pedestrians within a predetermined distance of the vehicle;

compare the estimated velocity of the client computing device to the information indicative of the velocities of the pedestrians and thereby identify a subset of the set of pedestrians that is likely to be the passenger;

stop the vehicle to allow the passenger to enter the vehicle based on the identified subset.

12. The system of claim 11, wherein the received information includes orientation information generated by a sensor of the client computing device, and the one or more processors are further configured to:

determine an orientation of the passenger; and

compare the orientation information with the determined orientation, and wherein the comparison is further used to identify the subset.

13. The system of claim 11, wherein the one or more processors are further configured to use the sensor information to detect a gaze direction for each pedestrian of the set of pedestrians, and wherein the gaze detection for each pedestrian is further used by the one or more processors to identify the subset.

14. The system of claim 11, wherein the one or more processors are further configured to use the sensor information determine a number of other pedestrians corresponding to pedestrians within a predetermined distance of each pedestrians of the set of pedestrians, and wherein the determined number of other pedestrians within the predetermined distance of each pedestrian is further used by the one or more processors to identify the subset.

15. The system of claim 14, wherein the dispatching instructions further identify a number of passengers, and wherein the identified number of passengers is further used by the one or more processors to identify the subset.

16. The system of claim 11, wherein the set of pedestrians is updated as additional location information is received from the client computing device.

17. The system of claim 11, wherein stopping the vehicle includes stopping the vehicle closer to a pedestrian of the subset than to the pickup location.

18. The system of claim 11, wherein stopping the vehicle includes stopping the vehicle before the vehicle reaches the pickup location.

19. The system of claim 11, wherein the one or more processors are further configured to:

use the sensor information to identify a characteristic that is different between two or more pedestrians of the set of pedestrians;

send a request to the client device, the request including a question regarding the characteristic; and

receive a response from the client computing device, and wherein the response is further used to identify the subset.

20. The system of claim 11, further comprising the vehicle.