System and method for evaluating the perception system of an autonomous vehicle
First Claim
1. A method for optimizing object detection performed by autonomous vehicles, the method comprising:
- receiving, by one or more computing devices having one or more processors, sensor data captured by a sensor of an autonomous vehicle, wherein the sensor data includes one or more first object labels applied by the autonomous vehicle, each of the one or more first object labels corresponding to an object of the sensor data;
receiving, by the one or more computing devices, one or more second object labels for the sensor data;
determining, by the one or more computing devices, a difference between a first value indicating a quantity of the one or more first object labels applied by the autonomous vehicle and a second value indicating a quantity of the one or more second object labels;
determining, by the one or more computing devices, whether the autonomous vehicle has met a predetermined missed object threshold corresponding to a maximum allowable quantity of object labels not applied by the autonomous vehicle by comparing the difference to the threshold; and
when the autonomous vehicle has met the threshold, generating a recommendation for an adjustment of at least one object detection parameter of at least one of a plurality of sensors in order to optimize object detection.
5 Assignments
0 Petitions
Accused Products
Abstract
A method and apparatus are provided for optimizing one or more object detection parameters used by an autonomous vehicle to detect objects in images. The autonomous vehicle may capture the images using one or more sensors. The autonomous vehicle may then determine object labels and their corresponding object label parameters for the detected objects. The captured images and the object label parameters may be communicated to an object identification server. The object identification server may request that one or more reviewers identify objects in the captured images. The object identification server may then compare the identification of objects by reviewers with the identification of objects by the autonomous vehicle. Depending on the results of the comparison, the object identification server may recommend or perform the optimization of one or more of the object detection parameters.
250 Citations
20 Claims
-
1. A method for optimizing object detection performed by autonomous vehicles, the method comprising:
-
receiving, by one or more computing devices having one or more processors, sensor data captured by a sensor of an autonomous vehicle, wherein the sensor data includes one or more first object labels applied by the autonomous vehicle, each of the one or more first object labels corresponding to an object of the sensor data; receiving, by the one or more computing devices, one or more second object labels for the sensor data; determining, by the one or more computing devices, a difference between a first value indicating a quantity of the one or more first object labels applied by the autonomous vehicle and a second value indicating a quantity of the one or more second object labels; determining, by the one or more computing devices, whether the autonomous vehicle has met a predetermined missed object threshold corresponding to a maximum allowable quantity of object labels not applied by the autonomous vehicle by comparing the difference to the threshold; and when the autonomous vehicle has met the threshold, generating a recommendation for an adjustment of at least one object detection parameter of at least one of a plurality of sensors in order to optimize object detection. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A system for optimizing object detection performed by autonomous vehicles, the system comprising one or more computing devices having one or more processors configured to:
-
receive sensor data captured by an autonomous vehicle, wherein the sensor data includes one or more first object labels applied by the autonomous vehicle, each of the one or more first object labels corresponding to an object of the sensor data; receive one or more second object labels for the sensor data; determine a difference between a first value indicating a quantity of the one or more first object labels applied by the autonomous vehicle and a second value indicating a quantity of the one or more second object labels; determine whether the autonomous vehicle has met a predetermined missed object threshold corresponding to a maximum allowable quantity of object labels not applied by the autonomous vehicle by comparing the difference to the threshold; and when the autonomous vehicle has met the threshold, generate a recommendation for an adjustment of at least one object detection parameter of at least one of a plurality of sensors in order to optimize object detection. - View Dependent Claims (12, 13, 14, 15, 17, 18)
-
-
16. A non-transitory, tangible recording medium on which instructions are stored, the instructions, when executed by one or more processors of one or more computing devices, cause the one or more processors to perform a method for optimizing object detection performed by autonomous vehicles, the method comprising:
-
receiving sensor data captured by an autonomous vehicle, wherein the sensor data includes one or more first object labels applied by the autonomous vehicle, each of the one or more first object labels corresponding to an object of the sensor data; receiving one or more second object labels for the sensor data; determining a difference between a first value indicating a quantity of the one or more first object labels applied by the autonomous vehicle and a second value indicating a quantity of the one or more second object labels; determining whether the autonomous vehicle has met a predetermined missed object threshold corresponding to a maximum allowable quantity of object labels not applied by the autonomous vehicle by comparing the difference to the threshold; and when the autonomous vehicle has met the threshold, generating a recommendation for an adjustment of at least one object detection parameter of at least one of a plurality of sensors in order to optimize object detection. - View Dependent Claims (19, 20)
-
Specification