SYSTEM AND METHOD FOR INTENDED PASSENGER DETECTION

• US 20170153714A1
• Filed: 02/14/2017
• Published: 06/01/2017
• Est. Priority Date: 03/03/2016
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: Cruise Automation, Inc. (General Motors Company)
• Inventors: Gao, Peter, Wisniowski, Matthias
• Granted Patent: US 10,088,846 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/047 Optimisation of routes or paths, e.g. travelling salesman problem
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08G 1/202 Dispatching vehicles on the basis of a location, e.g. taxi dispatching
• CPC Class Codes: H04W 4/02 Services making use of location information

Abstract

A system and method for locating and identifying a potential passenger by an autonomous vehicle includes: receiving a pickup request for the potential passenger, where the pickup request includes a geographic location for locating and picking up the potential passenger by the autonomous vehicle; locating the potential passenger based on the geographic location and using data from one or more sensors of the autonomous vehicle; and in response to locating the potential passenger, controlling the autonomous vehicle to a position to enable pickup of the potential passenger.

Claims

1. A system for use by an autonomous vehicle to locate an intended passenger, the system comprising:

a user interface configured to interact with and provide information to a vehicle routing coordinator or an autonomous vehicle;

the vehicle routing coordinator being configured to provide a pickup request to the autonomous vehicle based on information from the user interface or based on projected ridership in a specific geographic area;

the autonomous vehicle comprising a sensor suite for sensing circumstances surrounding the autonomous vehicle, the autonomous vehicle being configured to locate and identify an intended passenger desiring to be picked up;

at the autonomous vehicle;

receiving or inferring a pickup request, wherein a pickup request relates to a desire by the intended passenger to be picked up by the autonomous vehicle, wherein the pickup request comprises a general location area of the intended passenger;

locating the intended passenger using one or more of passenger location data, passenger identity data, vehicle map data, and autonomous vehicle sensor data, wherein locating the intended passenger includes using a general location area of the intended passenger and one or more additional location data points of the intended passenger to interpolate a refined location area having a more precisely defined location area of the intended passenger; and

picking up the intended passenger based on the refined location area of the intended passenger.

2. The system of claim 1, wherein inferring the pickup request including using the sensor suite of the autonomous vehicle to identify one or more behaviors of the intended passenger indicating the desire to be picked up the autonomous vehicle.

3. The system of claim 2, wherein identifying the one or more user behaviors includes identifying a signaling by the intended passenger toward the autonomous vehicle, wherein the signaling comprises performing one or more movements or gestures by the intended passenger, and

wherein inferring the pickup requests includes interpreting the performance of the one or more movements or gestures by the intended passenger as a request for a ride or pickup.

4. A method for locating and/or identifying a potential passenger for pickup by an autonomous vehicle, the method comprising:

receiving a pickup request for a potential passenger, the pickup request comprising a geographic location for locating and picking up the potential passenger by the autonomous vehicle;

locating the potential passenger based on the geographic location and using data from one or more sensors of the autonomous vehicle; and

in response to locating the potential passenger, controlling the autonomous vehicle to a position to enable pickup of the potential passenger.

5. The method of claim 4, wherein the received pickup request is based on an express request for a ride submitted via a user interface by the potential passenger, wherein the express request includes the geographic location of the potential passenger and one or more additional locating data points that is used by the autonomous vehicle to determine a position within the area of the geographic location for picking up the intended passenger.

6. The method of claim 4, wherein the received pickup request is based on an inference by the autonomous vehicle that the potential passenger desires a ride from the autonomous vehicle.

7. The method of claim 6, wherein the inference by the autonomous vehicle is based on identifying one or more behaviors of the potential passenger, wherein the identified one or more behaviors of the potential passenger includes a performance of one or more signals by the potential passenger that are directed toward the autonomous vehicle, wherein when the autonomous vehicle determines that the one or more signals match one or more predetermined ride requesting signals, the autonomous vehicle infers that the potential passenger desires to be picked up by the autonomous vehicle.

8. The method of claim 4, wherein locating the potential passenger includes:

receiving image data from a computing device of the potential passenger, wherein the image data comprises images or video of circumstances surrounding the potential passenger;

comparing the image data to predetermined map data or sensor data of the autonomous vehicle; and

determining the location of the potential passenger based on the comparison.

9. The method of claim 8, wherein the determining the location is performed by one or more of:

(i) calculating an image transformation between the image data from the potential passenger and real-time image data sensed by the autonomous vehicle; and

(ii) performing feature recognition and/or edge detection on the image data and matching resulting parameterized data from the feature recognition and/or edge detection to a predetermined map accessible to the autonomous vehicle.

10. The method of claim 4, wherein locating the passenger includes receiving a selection or indication by the potential passenger of a location using map data presented to the potential passenger.

11. The method of claim 4, the received pickup request comprising inferring the pickup request by the autonomous vehicle, wherein inferring the pickup request comprises interpreting light signals from a mobile computing device of the potential passenger as a desire of the potential passenger to be picked up by the autonomous vehicle.

12. The method of claim 4, the received pickup request comprising inferring the pickup request by the autonomous vehicle, wherein inferring the pickup request comprises:

identifying a body posture of the potential passenger; and

interpreting the body posture of the potential passenger as a desire of the potential passenger to be picked up by the autonomous vehicle.

13. The method of claim 4, the received pickup request comprising inferring the pickup request by the autonomous vehicle, wherein inferring the pickup request comprises:

capturing a facial image or video of the potential passenger; and

interpreting the facial image of the potential passenger as a desire of the potential passenger to be picked up by the autonomous vehicle.

14. The method of claim 4, the received pickup request comprising inferring the pickup request by the autonomous vehicle, wherein inferring the pickup request comprises:

tracking a movement of a mobile computing device of the potential passenger; and

interpreting the tracked movements of the mobile computing device as a desire of the potential passenger to be picked up by the autonomous vehicle.

15. The method of claim 4, wherein receiving the pickup request comprises:

receiving, via an external input interface of the autonomous vehicle, input from the potential passenger indicating that a pickup by the autonomous vehicle is desired.

16. The method of claim 4, wherein locating the potential passenger includes:

receiving a livestream from a mobile computing device of the potential passenger, wherein the livestream includes an augmented reality overlay, wherein the augmented reality overlay generates data and/or objects overlaid the livestream that demonstrates the autonomous vehicle'"'"'s interpretation of the circumstances in the livestream; and

using, by the autonomous vehicle, data of the livestream and augmented reality overlay to determine the location of the intended passenger.

17. The method of claim 16, wherein the autonomous vehicle uses the data of the livestream and augmented reality overlay to instruct the potential passenger of a modified location for pickup.