Methods and systems for vehicle perception feedback to classify data representative of types of objects and to request feedback regarding such classifications

• US 9,489,635 B1
• Filed: 11/01/2012
• Issued: 11/08/2016
• Est. Priority Date: 11/01/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhu, Jiajun
• Primary Examiner(s): Prince, Jessica M
• Application Number:
  US13/666,043
• Time in Patent Office: 1,468 Days
• Field of Search: 348/148, 701/23, 701/504
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/025 Extracting rules from data
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/7784 based on feedback from supervisors
• CPC Class Codes: G06V 10/987 with the intervention of an operator
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Aspects of the disclosure relate generally to methods and systems for improving object detection and classification. An example system may include a perception system and a feedback system. The perception system may be configured to receive data indicative of a surrounding environment of a vehicle, and to classify one or more portions of the data as representative of a type of object based on parameters associated with a machine learning classifier. The feedback system may be configured to request feedback regarding a classification of an object by the perception system based on a confidence level associated with the classification being below a threshold, and to cause the parameters associated with the machine classifier to be modified based on information provided in response to the request.

Claims

1. A system in a vehicle, the system comprising:

a display interface;

a perception system configured to receive data indicative of a surrounding environment of the vehicle, and to classify one or more portions of the data as being representative of a particular type of object based on parameters associated with a machine learning classifier, wherein the data indicative of the surrounding environment comprises point cloud data that is representative of the object; and

a feedback system configured to determine a confidence level associated with the classification by the perception system based on a relationship between features extracted from the point cloud data and expected characteristics of features for the particular type of object, to request via the display interface feedback from a user of the vehicle regarding the classification by the perception system responsive to determining that the confidence level is below a threshold, and to cause the parameters associated with the machine learning classifier to be modified based on information provided by the user in response to the request.

2. The system of claim 1, wherein the information provided in response to the request comprises an indication of a type of the object.

3. The system of claim 2, wherein based on the indication of the type of the object corresponding to the classification of the object by the perception system, the feedback system is configured to add the one or more portions of data to training data that is used to determine the parameters associated with the machine learning classifier.

4. The system of claim 1, wherein the vehicle is configured to operate in an autonomous mode.

5. The system of claim 1, further comprising one or more sensors configured to collect data about a surrounding environment of the vehicle.

6. The system of claim 5, wherein the one or more sensors comprise a laser scanner configured to determine the point cloud data that is representative of the object.

7. The system of claim 6, wherein the one or more sensors further comprise one or more of a camera and a radar.

8. The system of claim 1, wherein the machine learning classifier comprises a decision tree classifier.

9. A method comprising:

receiving data about a surrounding environment of a vehicle from one or more sensors of the vehicle, wherein the data comprises point cloud data;

determining, by a processor, one or more portions of the point cloud data that are representative of an object;

determining, by the processor, a particular type of the object based at least on features extracted from the one or more portions of the point cloud data and parameters associated with a machine learning classifier;

determining, by the processor, a confidence level associated with the determination of the type of the object based on a relationship between the features extracted from the one or more portions of the point cloud data and expected characteristics of features for the particular type of object; and

responsive to determining that the confidence level is below a threshold, requesting via a display interface of the vehicle feedback from a user of the vehicle regarding the determination of the particular type of the object.

10. The method of claim 9, further comprising:

receiving information from the user in response to the request, wherein the information comprises an indication of a type of the object; and

based on the indication of the type of the object corresponding to the determination of the type of the object by the processor, adding the one or more portions of the point cloud data to training data that is used to determine the parameters associated with the machine learning classifier.

11. The method of claim 9, wherein the vehicle is configured to operate in an autonomous mode.

12. The method of claim 9, wherein the one or more sensors comprise a laser scanner configured to determine the point cloud data.

13. The method of claim 12, wherein the one or more sensors further comprise one or more of a camera and a radar.

14. The method of claim 9, wherein the machine learning classifier comprises a decision tree classifier.

15. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

receiving data about a surrounding environment of a vehicle from one or more sensors of the vehicle, wherein the data comprises point cloud data;

determining one or more portions of the point cloud data that are representative of an object;

determining a particular type of the object based at least on features extracted from the one or more portions of the point cloud data and parameters associated with a machine learning classifier;

determining a confidence level associated with the determination of the particular type of the object based on a relationship between the features extracted from the one or more portions of the point cloud data and expected characteristics of features for the particular type of object; and

responsive to determining that the confidence level is below a threshold, requesting via a display interface of the vehicle feedback from a user of the vehicle regarding the determination of the particular type of the object.

16. The non-transitory computer-readable medium of claim 15, wherein the functions further comprise receiving information in response to the request, wherein the information comprises an indication of a type of the object.

17. The non-transitory computer-readable medium of claim 16, wherein the functions further comprise based on the indication of the type of the object corresponding to the determined type of the object, adding the one or more portions of the point cloud data to training data that is used to determine the parameters associated with the machine learning classifier.

18. The non-transitory computer-readable medium of claim 15, wherein the machine learning classifier comprises a decision tree classifier.