Detecting objects in images

• US 9,076,065 B1
• Filed: 01/26/2012
• Issued: 07/07/2015
• Est. Priority Date: 01/26/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vijayanarasimhan, Sudheendra
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): VANCHY JR, MICHAEL J
• Application Number:
  US13/359,468
• Time in Patent Office: 1,258 Days
• Field of Search: 382/103, 382/159, 382/170, 382/171, 382/225, 382/281
• US Class Current: 1/1
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

Techniques for detecting the location of an object of interest in a visual image are presented. A detector component extracts Histogram of Gradient (HOG) features from grid regions associated with the visual image. A trained linear filter model uses a classifier to facilitate differentiating between positive and negative instances of the object in grid regions based on HOG features. A classifier component detects the K top-scoring activations of filters associated with the visual image. The classifier component detects the location of the object in the visual image based on a generalized Hough transform, given filter locations associated with the visual image. The classifier component projects the object location given filter activations and clusters the filter activations into respective clusters. The classifier component classifies whether a cluster is associated with the object based on the weighted sum of the activation scores of filters within the cluster and object detection criteria.

Claims

1. A system, comprising:

at least one memory that stores computer executable components; and

at least one processor that executes the following computer executable components stored in the at least one memory;

a detector component that extracts Histogram of Gradient (HOG) features from grid regions associated with a visual image to facilitate detection of a location of an object of interest in the visual image; and

a classifier component that uses a trained linear filter model to determine whether the visual image potentially contains the object of interest based at least in part on the HOG features, wherein the classifier component clusters a subset of filter activations associated with the trained filter model to generate a cluster of filter activations that identifies a potential location of the object of interest in the visual image, and wherein the classifier component determines whether the cluster of filter activations is associated with the object of interest in the visual image based at least in part on a Hough transform and a weighted sum of filter activation scores of the subset of filter activations within the cluster of filter activations.

2. The system of claim 1, wherein the detector component divides a bounding box into a grid, and extracts the HOG features from the grid regions defined by the grid, wherein the bounding box is at least potentially associated with the object of interest in the visual image.

3. The system of claim 1, wherein the detector component comprises a trainer component, wherein the trainer component uses a support vector machine to train a linear filter model to produce the trained linear filter model that differentiates between a subset of example positive visual images that contain the object of interest and a subset of example negative visual images that do not contain the object of interest, and wherein the subset of example positive visual images includes a first subset of HOG features indicative of the example positive visual images containing the object of interest and the subset of example negative visual images includes a second subset of HOG features indicative of the example negative images not containing the object of interest.

4. The system of claim 3, wherein the detector component divides bounding boxes associated with a subset of example visual images into grid regions, and extracts respective HOG features from each of the grid regions, wherein the respective bounding boxes are associated with one or more objects in example visual images of the subset of example visual images, and wherein the subset of example visual images comprises the subset of example positive visual images and the subset of example negative visual images.

5. The system of claim 1, wherein the classifier component detects, in the visual image, a defined number of higher-scoring filter activations of each filter associated with the subset of filter activations associated with the trained linear filter model, based at least in part on the HOG features, to facilitate detection of object instances in the visual image, wherein the higher-scoring filter activations have higher filter activation scores as compared to other filter activation scores of other filter activations associated with the subset of filter activations.

6. The system of claim 5, wherein the classifier component projects the location of the object of interest, based at least in part on the higher-scoring filter activations using mean shift in relation to space of a bounding box associated with the object of interest, and wherein the classifier component obtains the location of the object of interest based at least in part on the Hough transform and the respective filter locations associated with the subset of filter activations associated with the trained linear filter model, wherein the Hough transform is a generalized Hough transform.

7. The system of claim 6, wherein the classifier component classifies whether respective clusters of filter activations of a subset of clusters of filter activations, comprising the cluster of filter activations, relate to the object of interest, and identifies the cluster of filter activations as relating to the object of interest based at least in part on the classification of the respective clusters of filter activations.

8. The system of claim 7, wherein the classifier component obtains a weighted sum of respective filter activation scores associated with respective filters within a particular cluster of filter activations of the subset of clusters of filter activations to facilitate classification of the particular cluster of filter activations.

9. The system of claim 8, wherein the classifier component analyzes a subset of training visual images to generate respective weighted sums of the respective filter activation scores, and trains a linear classifier to differentiate between instances of accurate detection of a location of the object of interest in visual images and detected false positives relating to incorrect instances of detection of a location of the object of interest in visual images.

10. The system of claim 1, wherein the detector component is further configured to comprise a trainer component, wherein the trainer component is configured to use a support vector machine to train the classifier component to differentiate between a subset of example positive instances of accurate detection of a location the object of interest in a visual image and a subset of example negative instances of accurate indication that a visual image does not contain the object of interest.

11. A method, comprising:

employing at least one processor to facilitate execution of code instructions retained in at least one memory device, the at least one processor, in response to execution of the code instructions, performs operations comprising;

determining, using a trained linear filter model, whether a video frame potentially contains an object of interest based at least in part on Histogram of Gradient (HOG) features extracted from grid regions associated with the video frame;

clustering a subset of filter activations associated with the video frame to generate a cluster of filter activations that potentially identifies a location of the object of interest in the video frame, wherein the subset of filter activations is derived at least in part from the trained linear filter model; and

classifying whether the cluster of filter activations is associated with the object of interest in the video frame, based at least in part on a Hough transform and a weighted sum of filter activation scores of the subset of filter activations within the cluster of filter activations.

12. The method of claim 11, further comprising:

detecting the location of the object of interest in the video frame based at least in part on a weighted Hough transform.

13. The method of claim 11, further comprising:

dividing a bounding box associated with the video frame into a grid; and

extracting respective subsets of the HOG features from the grid regions defined by the grid.

14. The method of claim 11, further comprising:

identifying a subset of example video frames relating to the object of interest and comprising a subset of positive example video frames and a subset of negative example video frames;

dividing bounding boxes of the subset of example video frames into grids to form respective subsets of grid regions defined by the grids, wherein respective bounding boxes depict an extent of the object of interest in respective example video frames in the subset of example video frames;

extracting respective HOG features from each grid region in the subset of example video frames; and

training a linear filter model to generate the trained linear filter model that differentiates between the subset of example positive video frames and the subset of example negative video frames.

15. The method of claim 11, further comprising:

detecting a defined number of higher-scoring filter activations of each filter associated with the subset of filter activations associated with the trained linear filter model, wherein the higher-scoring filter activations have higher filter activation scores relative to other filter activation scores of other filter activations associated with the subset of filter activations;

projecting the location of the object of interest, based at least in part on a subset of the higher-scoring filter activations associated with the trained linear filter model in relation to the video frame; and

generating a subset of clusters of filter activations, comprising respective subsets of filter activations associated with the higher-scoring filter activations, using mean shift in relation to space of a bounding box associated with the object of interest.

16. The method of claim 15, further comprising:

identifying the location of the object of interest based at least in part on the Hough transform and the respective filter locations associated with the respective filter activations in the subset of filter activations associated with the trained linear filter model, wherein the Hough transform is a generalized Hough transform.

17. The method of claim 15, further comprising:

calculating a weighted sum of respective filter activation scores associated with respective filter activations within a particular cluster of filter activations of the subset of clusters of filter activations to facilitate classification of the particular cluster of filter activations;

classifying whether respective clusters of filter activations of the subset of clusters of filter activations relate to the object of interest; and

identifying the cluster of filter activations as relating to the object of interest based at least in part on the classifying of the respective clusters of filter activations, wherein the cluster of filter activations is indicative of the location of the object of interest within the video frame.

18. The method of claim 17, further comprising:

training a classifier to differentiate between instances of accurate detection of a location of the object of interest in video frames, and instances of accurate detection of the object of interest not being located in video frames or false positive indications of location of the object of interest in video frames, based at least in part on a subset of training video frames; and

identifying respective weights to be used to obtain respective weighted sums of the respective filter activation scores, based at least in part on the analysis results and the training of the classifier.

19. The method of claim 18, further comprising:

updating the training of the classifier based at least in part on a subset of video frames classified by the classifier.

20. A system, comprising:

means for identifying, using a trained linear filter model, whether a visual image potentially contains an object of interest based at least in part on Histogram of Gradient (HOG) features extracted from grid regions associated with the visual image;

means for clustering a subset of filter activations associated with the visual image to generate a cluster of filter activations that potentially identifies a location of the object of interest in the visual image, wherein the subset of filter activations is derived at least in part from the trained linear filter model; and

means for classifying whether the cluster of filter activations is associated with the object of interest in the visual image, based at least in part on a Hough transform and a weighted sum of filter activation scores of the subset of filter activations within the cluster of filter activations.