Face and license plate detection in street level images with 3-D road width features estimated from laser data

• US 8,761,498 B1
• Filed: 01/26/2012
• Issued: 06/24/2014
• Est. Priority Date: 01/26/2012
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wu, Bo
• Primary Examiner(s): Bhatnagar, Anand
• Assistant Examiner(s): PARK, SOO JIN
• Application Number:
  US13/359,235
• Time in Patent Office: 880 Days
• US Class Current: 382/159
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names

Abstract

A computer implemented system for identifying license plates and faces in street-level images is disclosed. The system includes an object detector configured to determine a set of candidate objects in the image, a feature vector module configured to generate a set of feature vectors using the object detector to generate a feature vector for each candidate object in the set of candidate objects, a composite feature vector module to generate a set of composite feature vectors by combining each generated feature vector with a corresponding road or street description of the object in question, and an identifier module configured to identify objects of a particular type using a classifier that takes a set of composite feature vectors as input and returns a list of candidate objects that are classified as being of the particular type as output.

Claims

1. A computer-implemented method for generating a classifier for detecting objects in a digital image, the method comprising:

(a) providing a training data set of objects in one or more digital images, wherein the objects appear near or in a street or roadway, and wherein the set of objects includes some objects that are labeled as being of a particular type along with objects that are labeled as not being of the particular type;

(b) generating a set of feature vectors using the results of an object detector to generate a feature vector for each object in the set of objects, wherein each feature vector includes a detection score;

(c) providing, for each object, a description of the road or street in which or near which each object appears, the description including a corresponding road or street width estimate;

(d) generating a set of composite feature vectors by combining each generated feature vector with the corresponding road or street description; and

(e) generating an object classifier using a machine learning algorithm that takes the set of composite feature vectors as input and returns the object classifier as output.

2. The method of claim 1, wherein the generating (b) comprises using an object detector configured to detect license plates or faces.

3. The method of claim 2, wherein the generating (b) comprises using an object detector including a primary high-recall sliding-window detector and a secondary high-precision low-sensitivity detector.

4. The method of claim 1, wherein the generating (e) comprises using a Support Vector Machine (SVD) algorithm to generate the classifier.

5. A computer-implemented method for identifying objects in a digital image, wherein the objects appear near or in a street or roadway, the method comprising:

(a) determining a set of candidate objects in the image using an object detector;

(b) generating a set of feature vectors using an object detector to generate a feature vector for each candidate object in the set of candidate objects, wherein each feature vector includes a detection score;

(c) providing, for each candidate object in the set of candidate objects, a description of the road or street in which or near which the candidate object appears, the description including a corresponding road or street width estimate;

(d) generating a set of composite feature vectors by combining each generated feature vector with the corresponding road or street description; and

(e) identifying objects of a particular type using a classifier that takes the set of composite feature vectors as input and returns a list of candidate objects that are classified as being of the particular type.

6. The method of claim 5, wherein the generating (b) comprises using an object detector configured to detect license plates or faces.

7. The method of claim 6, wherein the generating (b) comprises using an object detector including a primary high-recall sliding-window detector and a secondary high-precision low-sensitivity detector.

8. A non-transitory computer readable storage medium having program instructions stored thereon that, when executed by a processor, cause the processor to generate a classifier for detecting objects in a digital image, the program instructions comprising computer readable code that causes a processor to:

(a) receive a training data set of objects in one or more digital images, wherein the objects appear near or in a street or roadway, and wherein the set of objects include some objects that are labeled as being of a particular type along with objects that are labeled as not being of the particular type, and wherein the data set further comprises, for each object, a description of the road or street in which or near which each object appears, the description including a corresponding road or street width estimate;

(b) generate a set of feature vectors using the results of an object detector to generate a feature vector for each object in the set of objects, wherein each feature vector includes a detection score;

(c) generate a set of composite feature vectors by combining each generated feature vector with the corresponding road or street description; and

(d) generate an object classifier using a machine learning algorithm that takes the set of composite feature vectors as input and returns the object classifier as output.

9. The computer readable storage medium of claim 8, wherein the program instructions further comprise computer readable code that causes the processor to use in stage (b) an object detector configured to detect license plates or faces.

10. The computer readable storage medium of claim 9, wherein the program instructions further comprise computer readable code that causes the processor to use in stage (b) an object detector that includes a primary high-recall sliding-window detector and a secondary high-precision low-sensitivity detector.

11. The computer readable storage medium of claim 8, wherein the program instructions further comprise computer readable code that causes the processor to use in stage (d) a Support Vector Machine (SVD) algorithm to generate the classifier.

12. A non-transitory computer readable storage medium having program instructions stored thereon that, when executed by a processor, cause the processor to identifying objects in a digital image, wherein the objects appear near or in a street or roadway, the program instructions comprising computer readable code that causes a processor to:

(a) determine a set of candidate objects in the image using an object detector;

(b) generate a set of feature vectors using an object detector to generate a feature vector for each candidate object in the set of candidate objects, wherein each feature vector includes a detection score;

(c) receive, for each candidate object in the set of candidate objects, a description of the road or street in which or near which the candidate object appears, the description including a corresponding road or street width estimate;

(d) generate a set of composite feature vectors by combining each generated feature vector with the corresponding road or street description; and

(e) identify objects of a particular type using a classifier that takes the set of composite feature vectors as input and returns a list of candidate objects that are classified as being of the particular type.

13. The computer readable storage medium of claim 12, wherein the program instructions further comprise computer readable code that causes the processor to use in stage (b) an object detector configured to detect license plates or faces.

14. The computer readable storage medium of claim 13, wherein the program instructions further comprise computer readable code that causes the processor to use in stage (b) an object detector including a primary high-recall sliding-window detector and a secondary high-precision low-sensitivity detector.

15. A computer-implemented system for generating a classifier for detecting objects in a digital image, the system comprising:

(a) a data receiver module configured to receive a training data set of objects in one or more digital images, wherein the objects appear near or in a street or roadway, and wherein the set of objects include some objects that are labeled as being of a particular type along with objects that are labeled as not being of the particular type, and wherein the data set further comprises, for each object, a description of the road or street in which or near which each object appears, the description including a corresponding road or street width estimate;

(b) a feature vector module configured to generate a set of feature vectors using an object detector to generate a feature vector for each object in the set of objects, wherein each feature vector includes a detection score;

(c) a composite feature vector module configured to generate a set of composite feature vectors by combining each generated feature vector with the corresponding road or street description; and

(d) an object classifier module configured to generate an object classifier using a machine learning algorithm that takes the set of composite feature vectors as input and returns the object classifier as output.

16. The system of claim 15, wherein the feature vector module further comprises an object detector configured to detect license plates or faces.

17. The system of claim 16, wherein the feature vector module further comprises an object detector including a primary high-recall sliding-window detector and a secondary high-precision low-sensitivity detector.

18. A computer-implemented system for identifying objects in a digital image, wherein the objects appear near or in a street or roadway, the system comprising:

(a) a data receiver module configured to receive a data set including a digital image containing objects appearing near or in a street or roadway, and wherein the data set further comprises, for each object, a description of the road or street in which or near which each object appears, the description including a corresponding road or street width estimate;

(b) an object detector configured to determine a set of candidate objects in the image;

(c) a feature vector module configured to generate a set of feature vectors using an object detector to generate a feature vector for each candidate object in the set of candidate objects, wherein each feature vector includes a detection score;

(d) a composite feature vector module configured to generate a set of composite feature vectors by combining each generated feature vector with the corresponding road or street description; and

(e) an identification module configured to identify objects of a particular type using a classifier that takes the set of composite feature vectors as input and returns a list of candidate objects that are classified as being of the particular type.

19. The system of claim 18, wherein the object detector is configured to detect license plates or faces.

20. The system of claim 19, wherein the object detector further comprises a primary high recall sliding-window detector and a secondary high-precision low-sensitivity detector.