Regionlets with Shift Invariant Neural Patterns for Object Detection

• US 20150117760A1
• Filed: 10/17/2014
• Published: 04/30/2015
• Est. Priority Date: 10/30/2013
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Wang, Xiaoyu, Lin, Yuanqing, Zou, Will, Sun, Miao
• Granted Patent: US 9,202,144 B2
• Time in Patent Office: Days
• US Class Current: 382/157
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

Systems and methods are disclosed for detecting an object in an image by determining convolutional neural network responses on the image; mapping the responses back to their spatial locations in the image; and constructing features densely extract shift invariant activations of a convolutional neural network to produce dense features for the image.

Claims

1. A method for detecting an object in an image, comprising:

determining convolutional neural network responses on the image;

mapping the responses back to their spatial locations in the image; and

constructing features densely extract shift invariant activations of a convolutional neural network to produce dense features for the image.

2. The method of claim 1, comprising sharing feature extraction computation for object candidates.

3. The method of claim 1, comprising augmenting the features with hard-coded features.

4. The method of claim 1, comprising densely extract shift invariant activations of a convolutional neural network trained on an image database.

5. The method of claim 1, comprising feeding the features to a Regionlets framework.

6. The method of claim 5, comprising using a boosting procedure to choose effective features.

7. The method of claim 1, comprising determining a horizontal location x of a neural pattern feature as:

8. The method of claim 7, comprising determining:

S_i=S_i-1×

s_i.

where s_i is a current stride using neural patterns output by the previous layers as pixel; and

determining pixel locations of neural patterns in different layers recursively by going up the hierarchy.

9. The method of claim 1, comprising determining a fixed length visual representation for a regionlet of arbitrary resolution.

10. The method of claim 9, comprising generating a local Dense Neural Patterns (DNPs) histogram, or average pooling of DNPs, inside each regionlet.

11. The method of claim 10, comprising generating DNPs in a regionlet r as {x_i|iε

(1, . . . N_r)}, where i indicates the index of the feature point, N_r is the total number of feature points in regionlet r and determining a final feature for r as;

12. The method of claim 10, wherein each dimension of deep neural patterns corresponds to a histogram bin and their values from different spatial locations are accumulated inside a regionlet, comprising normalizing histograms using L-0 norm.

13. The method of claim 10, comprising incorporating DNP into a Regionlets detector learning framework.

14. The method of claim 10, comprising applying a weak learner based on a 1-D feature, and uniformly sample DNP Regionlets configuration space to construct a weak classifier pool.

15. The method of claim 10, wherein each configuration specifies a spatial configuration of Regionlets and feature dimension of the DNP.

16. The method of claim 14, wherein the generated feature pool is easily augmented to the pool of other features including HOG, LBP or Covariance.

17. A system for detecting an object in an image, comprising:

means for determining convolutional neural network responses on the image;

means for mapping the responses back to their spatial locations in the image; and

means for constructing features densely extract shift invariant activations of a convolutional neural network to produce dense features for the image.

18. The system of claim 17, comprising means for sharing feature extraction computation for object candidates.

19. The system of claim 17, comprising means for augmenting the features with hard-coded features.

20. The system of claim 17, comprising means for densely extract shift invariant activations of a convolutional neural network trained on an image database.