Detection of objects in an image using self similarities

• US 9,569,694 B2
• Filed: 01/06/2016
• Issued: 02/14/2017
• Est. Priority Date: 06/11/2010
• Status: Expired due to Fees

• Current Assignee: Technische Universitã¤T Darmstadt, Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Technische Universitã¤T Darmstadt, Toyota Motor Europe SA (Toyota Motor Corporation)
• Inventors: Othmezouri, Gabriel, Sakata, Ichiro, Schiele, Bernt, Walk, Stefan, Majer, Nikodem, Schindler, Konrad
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): BEG, SAMAH A
• Application Number:
  US14/989,504
• Publication Number:
  US 20160117571A1
• Time in Patent Office: 405 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

An image processor (10) has a window selector for choosing a detection window within the image, and a self similarity computation part (40) for determining self-similarity information for a group of the pixels in any part of the detection window, to represent an amount of self-similarity of that group to other groups in any other part of the detector window, and for repeating the determination for groups in all parts of the detection window, to generate a global self similarity descriptor for the detection window. A classifier (50) is used for classifying whether an object is present based on the global self-similarity descriptor. By using global self-similarity rather than local similarities more information is captured which can lead to better classification. In particular, it helps enable recognition of more distant self-similarities inherent in the object, and self-similarities present at any scale.

Claims

1. An image processor for detection of an object in an image or sequence of images, each image being formed of pixels, and the image processor comprising:

a window selector for choosing a detection window within the image,

a self-similarity computation part configured to determine similarity information for a group of the pixels in any part of the detection window, to represent an amount of self-similarity of that group to other groups in every other part of the detection window, and for repeating the determination for groups in all parts of the detection window, to generate a global self-similarity descriptor for the chosen detection window, wherein the self-similarity computation part is configured to determine a plurality of similarity values, each similarity value being based on a comparison of self-similarity information for a first group of the pixels with a second group of the pixels, and wherein the plurality of similarity values are comprised in the global self-similarity descriptor;

a classifier configured to classify whether the object is present in the detection window of the image from the global self-similarity descriptor for that detection window; and

a color distribution part arranged to determine a distribution of colors of the pixels of the groups,

wherein the self-similarity information comprises an amount of self-similarity of the color distributions, and

wherein the self-similarity computation part comprises;

a histogram generator arranged to determine a histogram of values for a feature of pixels in the group, by using interpolation; and

a histogram similarity part arranged to determine similarities between histograms for different groups of pixels in the detection window by a histogram intersection.

2. The image processor of claim 1, wherein the self-similarity information further comprises an amount of self-similarity of colors of pixels of the group.

3. The image processor of claim 1, further comprising a gradient information determination part for determining gradient information by determining a distribution of intensity gradients in a cell of pixels, and inputting such gradient information for cells over all parts of the detection window to the classifier,

wherein the classifier is further configured to use the gradient information to classify whether the object is present.

4. The image processor of claim 3, further comprising a combiner part for combining the self-similarity information and the distribution of intensity gradients before input to the classifier.

5. The image processor of claim 1, further comprising a flow determination part arranged to determine a flow of the groups in terms of motion vectors of the pixels of the groups over successive images in a sequence of images, and

wherein the self-similarity information further comprises an amount of self-similarity of the flow.

6. The image processor of claim 1, further comprising a motion detection part for detecting motion vectors for parts of the image,

wherein the classifier is further configured to classify based on the motion vectors of parts in the detection window.

11. An integrated circuit having the image processor of claim 1.

7. A method of using an image processor for detection of an object in an image or sequence of images, each image being formed of pixels, and the method having the steps of:

choosing a detection window within the image,

determining self-similarity information for a group of the pixels in every part of the detection window, to represent an amount of self-similarity of that group to other groups in every other part of the detection window;

comparing self-similarity information for a first group of pixels to self-similarity information of a second group of pixels to determine a similarity value based on a result of the comparing;

repeating the determination and comparing for groups in all parts of the detection window, to generate a global self-similarity descriptor for the chosen detection window including the determined similarity values;

classifying whether the object is present in the detection window of the image from the global self-similarity descriptor for that detection window; and

determining a distribution of colors of the pixels of the groups,

wherein the self-similarity information comprises an amount of self-similarity of the color distributions, and

wherein the determining comprises;

determining a histogram of values for a feature of pixels in the group, by using interpolation; and

determining similarities between histograms for different groups of pixels in the detection window by a histogram intersection.

8. The method of claim 7, further comprising determining gradient information by determining a distribution of intensity gradients in a cell of pixels, and determining such gradient information for cells over all parts of the detection window,

wherein the classifying comprises using the gradient information to classify whether the object is present.

9. The method of claim 7, further comprising determining a flow of the groups in terms of motion vectors of the pixels of the groups over successive images in a sequence of images,

wherein the self-similarity information further comprises an amount of self-similarity of the flow.

10. A non-transitory computer-readable medium having embodied thereon computer-executable instructions which when executed by a computer cause the computer to carry out the method of claim 7.