Sensor-based object-detection optimization for autonomous vehicles

• US 9,720,415 B2
• Filed: 11/04/2015
• Issued: 08/01/2017
• Est. Priority Date: 11/04/2015
• Status: Active Grant

• Current Assignee: Zoox Inc. (Amazon.com, Inc.)
• Original Assignee: Zoox Inc. (Amazon.com, Inc.)
• Inventors: Levinson, Jesse Sol, Kentley, Timothy David, Douillard, Bertrand Robert
• Primary Examiner(s): Mawari, Redhwan K
• Assistant Examiner(s): Torchinsky, Edward
• Application Number:
  US14/756,991
• Publication Number:
  US 20170123428A1
• Time in Patent Office: 636 Days
• Field of Search: None
• CPC Class Codes: B60W 2050/0064 using a remote, e.g. cordless, transmitter or receiver unit, e.g. remote keypad or mobile phone
• CPC Class Codes: B60W 2050/0215 Sensor drifts or sensor failures
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4026 Cycles
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/406 Traffic density
• CPC Class Codes: B60W 2554/60 Traversable objects, e.g. speed bumps or curbs
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 50/00 Details of control systems for road vehicle drive control not related to the control of a particular sub-unit , e.g. process diagnostic or vehicle driver interfaces
• CPC Class Codes: B60W 50/02 Ensuring safety in case of control system failures, e.g. by diagnosing, circumventing or fixing failures
• CPC Class Codes: B60W 50/0205 Diagnosing or detecting failures; Failure detection models
• CPC Class Codes: B60W 60/0011 involving control alternatives for a single driving scenario, e.g. planning several paths to avoid obstacles
• CPC Class Codes: G01S 13/06 Systems determining position data of a target
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G05D 1/0214 : in accordance with safety or protection criteria, e.g. avoiding hazardous areas monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13
• CPC Class Codes: G05D 1/0231 : using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/024 : in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0248 : in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0257 : using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0274 : using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0276 : using signals provided by a source external to the vehicle involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058
• CPC Class Codes: G05D 1/0291 : Fleet control monitoring fleets in traffic control systems for road vehicles G08G1/127, G08G1/127
• CPC Class Codes: G07C 5/08 : Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
• CPC Class Codes: G07C 5/0816 : Indicating performance data, e.g. occurrence of a malfunction

Abstract

Various embodiments relate generally to autonomous vehicles and associated mechanical, electrical and electronic hardware, computer software and systems, and wired and wireless network communications to provide an autonomous vehicle fleet as a service. In particular, a method may include receiving an indication of a sensor anomaly, determining one or more sensor recovery strategies based on the sensor anomaly, and executing a course of action that ensures the autonomous vehicle system operates within accepted parameters. Alternative sensors may be relied upon to cover for the sensor anomaly, which may include a failed sensor while the autonomous vehicle is in operation.

Claims

1. A method comprising:

identifying a sensor anomaly at an autonomous vehicle system, the autonomous vehicle system comprising an autonomous vehicle having a plurality of sensors;

determining a type of the sensor anomaly;

retrieving, from a data store and based on the sensor anomaly, data representing at least one sensor recovery strategy, the at least one sensor recovery strategy including one or more courses of action to modify operation of the autonomous vehicle in response to the sensor anomaly;

selecting a course of action from the one or more courses of action based at least in part on the type of the sensor anomaly; and

executing the course of action at the autonomous vehicle system, wherein executing the course of action causes the autonomous vehicle to operate within a range of operational parameters defined by the course of action, wherein

the range of operational parameters comprises a particular trajectory, and

executing the course of action causes the autonomous vehicle system to perform a maneuver to orient the autonomous vehicle such that the autonomous vehicle is substantially aligned with the particular trajectory and the particular trajectory is within respective fields of view of a majority of the plurality of sensors.

2. The method of claim 1, wherein the type of the sensor anomaly comprises a sensor failure.

3. The method of claim 1, further comprising:

receiving data representing the sensor anomaly;

determining that the data is within a specified range of laser return intensities; and

determining, based at least in part on the data, that the sensor anomaly is caused by a reflectivity phenomena.

4. The method of claim 3, wherein:

the reflectivity phenomena comprises receiving direct sunlight,

selecting the course of action includes;

accessing a map database, and

determining, using at least one map stored in the map database, alternative trajectory that substantially avoids direct sunlight, and

executing the course of action includes causing the autonomous vehicle system to maneuver substantially along at least part of the alternative trajectory.

5. The method of claim 3, wherein:

the reflectivity phenomena comprises receiving indirect sunlight reflected from a surface,

selecting the course of action includes;

accessing a map database, and

determining, using at least one map stored in the map database, an alternative trajectory that substantially avoids indirect sunlight, and

executing the course of action includes causing the autonomous vehicle system to maneuver substantially along at least part of the alternative trajectory.

6. The method of claim 3, wherein:

the reflectivity phenomena comprises receiving light emitted by headlights of other vehicles,

selecting the course of action includes;

accessing a map database, and

determining, using at least one map stored in the map database, an alternative trajectory that includes less traffic relative to traffic associated with a current trajectory of the autonomous vehicle system, and

executing the course of action includes causing the autonomous vehicle system to maneuver substantially along at least part of the alternative trajectory.

7. The method of claim 3, wherein:

the reflectivity phenomena comprises receiving radiation, emitted by external lights, that interferes with LIDAR sensors of the autonomous vehicle system,

selecting the course of action includes;

accessing a map database, and

determining, using at least one map stored in the map database, an alternative trajectory, different from a current trajectory of the autonomous vehicle system, that includes fewer external lights relative to a number of external lights associated with the current trajectory, and

executing the course of action includes causing the autonomous vehicle system to maneuver substantially along at least part of the alternative trajectory.

8. The method of claim 1, wherein the type of the sensor anomaly comprises a degraded functionality of a first sensor of the plurality of sensors, and the method further comprises:

prioritizing signals received from a second sensor of the plurality of sensors over those of the first sensor based at least in part on determining that the type of the sensor anomaly is the degraded functionality of the first sensor.

9. The method of claim 1, further comprising identifying a first sensor of the plurality of sensors corresponding to the sensor anomaly, wherein executing the course of action includes prioritizing computational processing of one or more sensors of the plurality of sensors other than the first sensor based on the sensor anomaly.

10. A method comprising:

receiving, at an autonomous vehicle system comprising an autonomous vehicle having a plurality of sensors, an indication of a sensor anomaly;

determining at least one sensor recovery strategy based at least in part on the sensor anomaly; and

executing, at the autonomous vehicle system, a course of action included in the at least one sensor recovery strategy, wherein

the course of action defines a range of operational parameters including a particular trajectory, and

executing the course of action causes the autonomous vehicle system to perform a maneuver to orient the autonomous vehicle such that the autonomous vehicle is substantially aligned with the particular trajectory and the particular trajectory is within respective fields of view of a majority of the plurality of sensors.

11. The method of claim 10, wherein the plurality of sensors includes at least two LIDAR sensors arranged at one end of the autonomous vehicle system, and wherein the indication of the sensor anomaly comprises an indication that one LIDAR sensor of the at least two LIDAR sensors has failed.

12. The method of claim 11, wherein determining at least one sensor recovery strategy further comprises:

retrieving the at least one sensor recovery strategy based on a physical location of the one LIDAR sensor that has failed relative to the autonomous vehicle system.

13. The method of claim 10, wherein determining at least one sensor recovery strategy further comprises:

retrieving one or more log files based on the sensor failure; and

selecting the course of action based on the retrieved one or more log files.

14. The method of claim 10, wherein the course of action comprises:

causing the autonomous vehicle to stop.

15. The method of claim 10, further comprising:

generating a guidance data request based on the at least one sensor recovery strategy;

sending the guidance data request to a guidance system;

receiving data representing vehicle trajectory guidance based on the request; and

executing the course of action in accordance with the data representing vehicle trajectory guidance.

16. The method of claim 15, wherein the data representing vehicle trajectory guidance is based on input received from a teleoperator system.

17. The method of claim 15, wherein the data representing vehicle trajectory guidance is based on a sensor recovery scenario retrieved from a fleet management system.

18. A method comprising:

receiving, at an autonomous vehicle system comprising an autonomous vehicle having a plurality of sensors, an indication of a sensor anomaly, wherein the indication identifies at least one sensor of the plurality of sensors corresponding to the sensor anomaly;

determining a field of view of a remainder of sensors of the plurality of sensors, wherein the remainder of sensors is exclusive of the at least one sensor;

based at least in part on the field of view, determining respective confidence levels associated with each path of a plurality of potential trajectory paths;

selecting a trajectory path of the plurality of potential trajectory paths based at least in part on the respective confidence levels;

causing the autonomous vehicle system to perform a maneuver to orient the autonomous vehicle such that the autonomous vehicle is substantially aligned with the selected trajectory path and the selected trajectory path is within respective fields of view of a majority of the plurality of sensors; and

causing the autonomous vehicle system to travel along the selected trajectory path.

19. The method of claim 18, wherein the sensor anomaly comprises at least one of sensor failure, degradation of functionality, reflectivity phenomena, or misalignment.

20. The method of claim 18, wherein the respective confidence levels are further based at least in part on map data.

21. The method of claim 18, wherein the selected trajectory path comprises at least one of:

a trajectory bringing the autonomous vehicle to a stop,

a trajectory changing a directionality of the autonomous vehicle, or

a trajectory that minimizes a reflectivity phenomenon.