Environmental condition identification assistance for autonomous vehicles

• US 9,958,870 B1
• Filed: 09/29/2015
• Issued: 05/01/2018
• Est. Priority Date: 09/29/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Graybill, Jules Cook, Parker, Erik Resch, Tappen, Marshall Friend, Siegel, Hilliard Bruce, Norwood, Jonathan Blair
• Primary Examiner(s): Mott, Genna M
• Application Number:
  US14/868,883
• Time in Patent Office: 945 Days
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 40/02 related to ambient conditions
• CPC Class Codes: B60W 50/08 Interaction between the driver and the control system
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G05D 1/00 Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots drive control systems specially adapted for autonomous road vehicles B60W60/00
• CPC Class Codes: G05D 1/02 Control of position or course in two dimensions
• CPC Class Codes: G05D 1/021 specially adapted to land vehicles

Abstract

Disclosed are various embodiments for non-mechanical assistance for autonomous vehicles. An unknown environmental condition may be detected that affects a navigation of the autonomous vehicle. A passenger and/or other source may be presented an inquiry requesting identification of the unknown environmental condition. A response to the inquiry may be used as an input by the vehicle control system in determining appropriate navigation of the autonomous vehicle.

Claims

1. A system, comprising:

an autonomous vehicle; and

a controller configured to control operation of the autonomous vehicle, the controller being further configured to perform a method comprising;

identifying a plurality of passengers of the autonomous vehicle based at least in part on at least one of an image, an audio signal, a fingerprint, a handprint, a user login, a personal identification number (PIN), or a radio frequency identification (RFID) tag;

generating a respective score for individual passengers of the plurality of passengers based at least in part on a sum of a plurality of weights assigned to a plurality of factors, the plurality of factors including at least one of an age, a level of impairment, a passenger confirmation received from at least one passenger of the plurality of passengers, or an accuracy of prior responses, individual factors being weighted with a different weight of the plurality of weights;

selecting a master passenger from the plurality of passengers based at least in part on the respective score of the master passenger, wherein information provided by the master passenger is considered accurate by the controller;

detecting an unknown environmental condition affecting navigation of the autonomous vehicle;

determining a plurality of potential options for an identification of the unknown environmental condition;

generating a user interface including an inquiry related to an identification of the unknown environmental condition, wherein the inquiry includes the plurality of potential options;

presenting the user interface via a display of a windshield of the autonomous vehicle;

receiving a response to the inquiry via an input to the user interface, the response including a selection of a particular option of the plurality of potential options;

verifying that the master passenger provided the response based at least in part on a combination of a plurality of indicators comprising at least one of a biometric, the RFID tag, a voice analysis, or a seat location; and

determining a specific navigation for the autonomous vehicle based at least in part on the identification of the unknown environmental condition and a verification that the master passenger provided the response.

2. The system of claim 1, wherein the unknown environmental condition comprises at least one of a passenger preference, an unknown intention of a pedestrian, an unrecognizable action associated with an individual positioned within an intended navigation route, an unknown order for proceeding through an intersection, or an unknown merging order.

3. The system of claim 1, wherein the controller is further configured to perform the method further comprising communicating with at least one other autonomous vehicle to determine whether the at least one other autonomous vehicle has previously identified the unknown environmental condition.

4. A system, comprising:

an autonomous vehicle; and

a controller configured to control navigation of the autonomous vehicle, the controller further being configured to perform a method comprising;

detecting an unknown environmental condition limiting an ability to navigate the autonomous vehicle based at least in part on at least one of a visual stimulus or an audio stimulus, the at least one of the visual stimulus or audio stimulus occurring from within the autonomous vehicle;

generating an inquiry requesting assistance in identifying the unknown environmental condition;

presenting the inquiry to a plurality of sources;

receiving a plurality of responses to the inquiry, the plurality of responses including an identification of the unknown environmental condition;

verifying that the plurality of responses were received from a plurality of reliable sources, individual reliable sources of the plurality of reliable sources being identified based at least in part on a combination of a plurality of indicators comprising at least one of a biometric, a radio frequency identification (RFID) tag, a voice analysis, or a seat location;

verifying the identification based at least in part on at least one of a unanimity of the plurality of responses, a weighted average associated with the plurality of responses, or a majority of the plurality of responses; and

determining a specific navigation for the autonomous vehicle based at least in part on the identification in response to verifying the identification.

5. The system of claim 4, wherein the unknown environmental condition comprises at least one of an unknown object, an unknown obstacle, a passenger preference, construction, an unknown intention of a pedestrian, an unrecognizable action associated with an individual compromising an intended navigation route, an unknown stopping location, an unknown order for proceeding through an intersection, an unknown merging order, or an unknown terrain.

6. The system of claim 4, wherein the controller is further configured to perform the method further comprising communicating with another autonomous vehicle to determine whether the other autonomous vehicle has identified the unknown environmental condition.

7. The system of claim 4, wherein the plurality of reliable sources comprise a plurality of selected passengers of the autonomous vehicle, the plurality of selected passengers being selected from a plurality of passengers in the autonomous vehicle to be the reliable sources based at least in part on a respective score generated by a sum of weights assigned to a plurality of factors comprising at least one of an age, a competence, an accuracy of prior responses, a passenger confirmation, or a level of impairment.

8. The system of claim 4, wherein at least one reliable source of the plurality of reliable sources is a third party entity, and wherein presenting the inquiry to the plurality of sources further comprises transmitting the inquiry over a network to a client computing device associated with the third party entity, the inquiry to be presented via the client computing device.

9. The system of claim 4, wherein presenting the inquiry to the plurality of sources comprises rendering the inquiry via a display device of the autonomous vehicle.

10. The system of claim 9, wherein the display device is embedded within a windshield of the autonomous vehicle.

11. The system of claim 9, wherein the display device is a projection device configured to project the inquiry onto a surface of the autonomous vehicle.

12. The system of claim 4, wherein the autonomous vehicle comprises a plurality of touch sensors embedded within a windshield of the autonomous vehicle, and at least one response is received via a particular touch sensor of the plurality of touch sensors.

13. A method, comprising:

detecting, via an autonomous vehicle controller, an unknown environmental condition based at least in part on sensor data received from a plurality of sensors associated with an autonomous vehicle controlled by the autonomous vehicle controller, the unknown environmental condition limiting an ability to navigate the autonomous vehicle;

generating, via the autonomous vehicle controller, an inquiry related to determining identification of the unknown environmental condition, the inquiry including a plurality of potential identifications of the unknown environmental condition;

rendering, via the autonomous vehicle controller, the inquiry via a display of the autonomous vehicle;

receiving, via the autonomous vehicle controller, a plurality of responses to the inquiry, individual responses of the plurality of responses including a respective identification of the plurality of potential identifications of the unknown environmental condition;

verifying, via the autonomous vehicle controller, that the plurality of responses were received from a plurality of reliable sources based at least in part on a combination of a plurality of indicators comprising at least one of a biometric, a personal identification number (PIN), a radio frequency identification (RFID) tag, a voice analysis, or a seat location;

determining a particular identification of the unknown environmental condition based at least in part on at least one of a unanimity of the plurality of responses, a weighted average of the plurality of responses, or a majority of the plurality of responses; and

determining, via the autonomous vehicle controller, a specific navigation for the autonomous vehicle based at least in part on the determination of the particular identification.

14. The method of claim 13, wherein the unknown environmental condition comprises at least one of an unknown object, an unknown obstacle, a passenger preference, construction, an unknown intention of a pedestrian, an unrecognizable action associated with an individual positioned relative to an intended navigation route, an unknown stopping location, an unknown order for proceeding through an intersection, an unknown merging order, or an unknown terrain.

15. The method of claim 13, further comprising identifying, via the autonomous vehicle controller, a passenger of the autonomous vehicle based at least in part on at least one of an image, an audio signal, a fingerprint, a handprint, a user login, the personal identification number (PIN), or the radio frequency identification (RFID) tag.

16. The method of claim 15, further comprising determining, via the autonomous vehicle controller, that the passenger is considered a reliable source based at least in part on a sum of a plurality of weights assigned to a plurality of factors comprising at least one of an age, an accuracy of prior responses, a passenger confirmation, or a level of impairment, individual factors being weighted with a different weight of the plurality of weights, and the autonomous vehicle controller configured to consider information provided by the reliable source as accurate.

17. The method of claim 15, wherein at least one response is a gesture response, the gesture response being detected via an imaging device of the autonomous vehicle.

18. The method of claim 15, wherein at least one response is a verbal response, the verbal response being received via an audio input device of the autonomous vehicle.

19. The method of claim 13, further comprising:

generating an audio alert associated with the inquiry; and

broadcasting the audio alert via an audio output device of the autonomous vehicle in addition to rendering the inquiry via the display to notify a passenger of the autonomous vehicle of the inquiry, the audio alert being separate from the inquiry.

20. The method of claim 13, wherein the display further comprises a plurality of touch sensors embedded within a windshield, at least one response being received via a particular touch sensor.