Rapid object detection by combining structural information from image segmentation with bio-inspired attentional mechanisms

• US 9,147,255 B1
• Filed: 08/14/2013
• Issued: 09/29/2015
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Zhang, Lei, Cheng, Shinko Y., Chen, Yang, Honda, Alexander L., Kim, Kyungnam, Khosla, Deepak, Jeong, Changsoo S.
• Primary Examiner(s): Mehta, Bhavesh
• Assistant Examiner(s): Moyer, Andrew
• Application Number:
  US13/967,227
• Time in Patent Office: 776 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

Described is a system for rapid object detection combining structural information with bio-inspired attentional mechanisms. The system oversegments an input image into a set of superpixels, where each superpixel comprises a plurality of pixels. For each superpixel, a bounding box defining a region of the input image representing a detection hypothesis is determined. An average residual saliency (ARS) is calculated for the plurality of pixels belonging to each superpixel. Each detection hypothesis that is out of a range of a predetermined threshold value for object size is eliminated. Next, each remaining detection hypothesis having an ARS below a predetermined threshold value is eliminated. Then, color contrast is calculated for the region defined by the bounding box for each remaining detection hypothesis. Each detection hypothesis having a color contrast below a predetermined threshold is eliminated. Finally, the remaining detection hypotheses are output to a classifier for object recognition.

Claims

1. A system for rapid object detection, the system comprising:

one or more processors and a non-transitory memory having instructions encoded thereon such that when the instructions are executed, the one or more processors perform operations of;

oversegmenting an input image into a set of superpixels, each superpixel comprising a plurality of pixels;

determining, for each superpixel, a bounding box defining a region of the input image representing a detection hypothesis;

calculating, for each superpixel, an average residual saliency (ARS) for the plurality of pixels belonging to the superpixel;

eliminating each detection hypothesis that is out of a range of a predetermined threshold value for object size, resulting in a first set of remaining detection hypotheses;

eliminating each remaining detection hypothesis in the first set of remaining detection hypotheses having an ARS below a predetermined threshold value, resulting in a second set of remaining detection hypotheses;

calculating, for each remaining detection hypothesis in the second set of remaining detection hypotheses, color contrast for the region defined by the bounding box, and eliminating each detection hypothesis in the second set of remaining detection hypotheses having a color contrast below a predetermined threshold value, resulting in a third set of remaining detection hypotheses; and

outputting the third set of remaining detection hypotheses to a classifier for object recognition.

2. The system as set forth in claim 1, wherein the one or more processors further perform an operation of using quick-shift image segmentation for oversegmenting the input image.

3. The system as set forth in claim 1, wherein the one or more processors further perform an operation of calculating a spectral residual saliency map for the input image, wherein the spectral residual saliency map is used to calculate the ARS for the plurality of pixels within each superpixel.

4. The system as set forth in claim 1, wherein the one or more processors further perform an operation of calculating color contrast as follows:

CC(w,S)=χ

²(h(w),h(S)),

where CC denotes color contrast, w denotes a bounding box in a detection hypothesis, S is a rectangular ring surrounding w, and h( ) denotes a histogram of color.

5. A computer-implemented method for rapid object detection, comprising:

an act of causing a data processor to execute instructions stored on a non-transitory memory such that upon execution, the data processor performs operations of;

oversegmenting an input image into a set of superpixels, each superpixel comprising a plurality of pixels;

determining, for each superpixel, a bounding box defining a region of the input image representing a detection hypothesis;

calculating, for each superpixel, an average residual saliency (ARS) for the plurality of pixels belonging to the superpixel;

eliminating each detection hypothesis that is out of a range of a predetermined threshold value for object size, resulting in a first set of remaining detection hypotheses;

eliminating each remaining detection hypothesis in the first set of remaining detection hypotheses having an ARS below a predetermined threshold value, resulting in a second set of remaining detection hypotheses;

calculating, for each remaining detection hypothesis in the second set of remaining detection hypotheses, color contrast for the region defined by the bounding box, and eliminating each detection hypothesis in the second set of remaining detection hypotheses having a color contrast below a predetermined threshold value, resulting in a third set of remaining detection hypotheses; and

outputting the third set of remaining detection hypotheses to a classifier for object recognition.

6. The method as set forth in claim 5, wherein the data processor further performs an operation of using quick-shift image segmentation for oversegmenting the input image.

7. The method as set forth in claim 5, wherein the data processor further performs an operation of calculating a spectral residual saliency map for the input image, wherein the spectral residual saliency map is used to calculate the ARS for the plurality of pixels within each superpixel.

8. The method as set forth in claim 5, wherein the data processor further performs an operation of calculating color contrast as follows:

CC(w,S)=χ

²(h(w),h(S)),

where CC denotes color contrast, w denotes a bounding box in a detection hypothesis, S is a rectangular ring surrounding w, and h( ) denotes a histogram of color.

9. A computer program product for rapid object detection, the computer program product comprising computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of:

oversegmenting an input image into a set of superpixels, each superpixel comprising a plurality of pixels;

determining, for each superpixel, a bounding box defining a region of the input image representing a detection hypothesis;

calculating, for each superpixel, an average residual saliency (ARS) for the plurality of pixels belonging to the superpixel;

eliminating each detection hypothesis that is out of a range of a predetermined threshold value for object size, resulting in a first set of remaining detection hypotheses;

eliminating each remaining detection hypothesis in the first set of remaining detection hypotheses having an ARS below a predetermined threshold value, resulting in a second set of remaining detection hypotheses;

calculating, for each remaining detection hypothesis in the second set of remaining detection hypotheses, color contrast for the region defined by the bounding box, and eliminating each detection hypothesis in the second set of remaining detection hypotheses having a color contrast below a predetermined threshold value, resulting in a third set of remaining detection hypotheses; and

outputting the third set of remaining detection hypotheses to a classifier for object recognition.

10. The computer program product as set forth in claim 9, further comprising instructions for causing the processor to perform an operation of using quick-shift image segmentation for oversegmenting the input image.

11. The computer program product as set forth in claim 9, further comprising instructions for causing the processor to perform an operation of calculating a spectral residual saliency map for the input image, wherein the spectral residual saliency map is used to calculate the ARS for the plurality of pixels within each superpixel.

12. The computer program product as set forth in claim 9, further comprising instructions for causing the processor to perform an operation of calculating color contrast as follows:

CC(w,S)=χ

²(h(w),h(S)),

where CC denotes color contrast, w denotes a bounding box in a detection hypothesis, S is a rectangular ring surrounding w, and h( ) denotes a histogram of color.