Object detection with false positive filtering

• US 8,345,921 B1
• Filed: 05/11/2009
• Issued: 01/01/2013
• Est. Priority Date: 03/10/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Frome, Andrea, Abdulkader, Ahmad, Wu, Bo, Bissacco, Alessandro, Neven, Hartmut, Vincent, Luc, Adam, Hartwig, Cheung, German, Zennaro, Marco
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US12/453,432
• Time in Patent Office: 1,331 Days
• Field of Search: 382/105, 382/118, 382/103
• US Class Current: 382/103
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

Embodiments of this invention relate to detecting and blurring images. In an embodiment, a system detects objects in a photographic image. The system includes an object detector module configured to detect regions of the photographic image that include objects of a particular type at least based on the content of the photographic image. The system further includes a false positive detector module configured to determine whether each region detected by the object detector module includes an object of the particular type at least based on information about the context in which the photographic image was taken.

Claims

1. A system for detecting objects in a image, comprising:

an object detector module configured to detect regions of the image based on at least the content of the image, wherein at least some of the regions include objects of a particular type; and

a false positive detector module configured to determine whether each region detected by the object detector module includes an object of the particular type, based on at least information about the context in which the image was taken, wherein the false positive detector module comprises;

a simple false positive detector module configured to determine whether each re ion detected by the object detector module includes an object of the particular type; and

a composite false positive detector module, that requires more computing resources than the simple false positive detector module, configured to determine based on at least one image feature not used by the simple false positive detector module, whether each region detected by the simple false positive detector module includes the object of the particular type,

wherein the object detector module and false positive detector module are implemented on at least one processor and memory.

2. The system of claim 1, wherein the object detector module is configured to detect regions that include faces.

3. The system of claim 1, wherein the object detector module is configured to detect regions that include license plates.

4. The system of claim 1, wherein the false positive detector module is configured to determine whether each region detected by the object detector module includes an object of the particular type at least based on information describing the shape of the region.

5. The system of claim 1, wherein the false positive detector module is configured to determine whether each region detected by the object detector module includes an object of the particular type at least based on information describing the appearance of the object in the region.

6. The system of claim 1, wherein the false positive detector module is configured to determine whether each region detected by the object detector module includes an object of the particular type at least based on a degree to which the region overlaps with other regions in the image detected by the object detector module.

7. The system of claim 1, wherein the image is taken from a camera rosette that takes panoramic images, and

the information about the context in which the image was taken comprises information indicating which camera in the camera rosette took the image.

8. The system of claim 7, wherein the image is at least a portion of a street-level panoramic image.

9. The system of claim 1, wherein the false positive detector module comprises an artificial neural network.

10. The system of claim 9, wherein the artificial neural network is trained using regions detected by the object detector module in other images.

11. The system of claim 1, wherein the simple false positive detector module comprises a first artificial neural network, and

the composite false positive detector module comprises a second artificial neural network with more nodes than the first neural network.

12. The system of claim 11, wherein the first artificial neural network is configured to determine whether regions detected by the simple false positive detector module include the object of the particular type based on at least:

(a) information describing the shape of the region generated by the object detector module,

(b) information describing the appearance of the object in the region generated by the object detector module,

(c) information indicating which camera in the camera rosette took the image, or

(d) a score, determined by the object detector module, that corresponds to a confidence that the region includes an object of the particular type;

and the second artificial neural network is configured to determine whether regions detected by the simple detector module include the object of the particular type at least based on one or more of the information (a)-(d), and one or more of;

(e) information describing an overlap between the region and other regions in the image detected by the object detector module,

(f) a face color probability determined using a histogram model trained on human labeled faces,

(g) an aggregate color statistic determined based on the color hue, saturation, and value within the region, or

(h) a value determined using a third artificial neural network configured to determine a likelihood that the region includes an object of the particular type based on a grayscale version of the region of the image.

13. The system of claim 1, further comprising:

an image blurring module configured to blur the region of the image when the false positive detector module determines that the region includes an object of the particular type.

14. A method for detecting objects in an image, comprising:

(a) detecting, on a computing device, regions of the image based on at least the content of the image, wherein at least some of the regions include objects of a particular type; and

(b) determining, on a computing device, whether each region detected in (a) includes an object of the particular type, based on at least information about the context in which the image was taken, the determining (b) comprising;

(i) determining, based on at least one image feature, whether each region detected in (a) includes an object of the particular type; and

(ii) determining, based on at least one image feature not used in the determining (i), whether each region detected in (i) includes the object of the particular type, wherein the determining (ii) uses more computing resources than the determining (i).

15. The method of claim 14, wherein the detecting (a) comprises detecting regions that include faces.

16. The method of claim 14, wherein the detecting (a) comprises detecting regions that include license plates.

17. The method of claim 14, wherein the determining (b) comprises determining whether each region detected in (a) includes an object of the particular type at least based on information describing the shape of the region.

18. The method of claim 14, wherein the determining (b) comprises determining whether each region detected in (a) includes an object of the particular type at least based on information describing the appearance of the object in the region.

19. The method of claim 14, wherein the determining (b) comprises determining whether each region detected in (a) includes an object of the particular type at least based on a degree to which the region overlaps with other regions in the image detected by the object detector module.

20. The method of claim 14, wherein the determining (b) comprises determining whether each region includes an object using an artificial neural network, further comprising:

(c) training the artificial neural network using regions detected by an object detector module in other images.

21. The method of claim 14, wherein the detecting (a) comprises detecting the regions using an object detector module, and the method further comprising:

(c) training an object detector module to increase a recall rate at which the regions are detected.

22. A computer-implemented method for detecting objects in an image, comprising:

(a) detecting regions of the image based on at least a content of the image, wherein at least some of the regions include objects of a particular type;

for each region detected in (a);

(b) determining, using a first detector module, a score indicating a degree of likelihood that the region includes an object of the particular type;

(c) when the score determined in (b) does not exceed a threshold, determining that the region does not include any objects of the particular type; and

(d) when the score determined in (b) exceeds the threshold, using a second detector module requiring more computing resources than the first detector module to determine, based on at least one image feature not used by the first detector module, whether the region includes the object of the particular type.

23. The method of claim 22, wherein the determining (b) includes determining the score based at least in part on information describing a camera that took the image.

25. The program storage device of claim 23, wherein the determining (b) includes determining the score based at least in part on information describing a camera that took the image.

24. A program storage device readable by a processor, tangibly embodying a program of instructions executable by the processor to perform method steps for detecting objects in an image, said method steps comprising:

(a) detecting regions of the image based on at least a content of the image, wherein at least some of the regions include objects of a particular type;

for each region detected in (a);

(b) determining, using a first detector module, a score indicating a degree of likelihood that the region includes an object of the particular type;

(c) when the score determined in (b) does not exceed a threshold, determining that the region does not include any objects of the particular type; and

(d) when the score determined in (b) exceeds the threshold, using a second detector module requiring more computing resources than the first detector module to determine, based on at least one image feature not used by the first detector module, whether the region includes the object of the particular type.