Multi-layer aggregation for object detection

• US 9,767,385 B2
• Filed: 08/12/2014
• Issued: 09/19/2017
• Est. Priority Date: 08/12/2014
• Status: Active Grant

• Current Assignee: Siemens Healthineers AG (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Nguyen, Hien, Singh, Vivek Kumar, Zheng, Yefeng, Georgescu, Bogdan, Comaniciu, Dorin, Zhou, Shaohua Kevin
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): RICE, ELISA M
• Application Number:
  US14/457,381
• Publication Number:
  US 20160048741A1
• Time in Patent Office: 1,134 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Object detection uses a deep or multiple layer network to learn features for detecting the object in the image. Multiple features from different layers are aggregated to train a classifier for the object. In addition or as an alternative to feature aggregation from different layers, an initial layer may have separate learnt nodes for different regions of the image to reduce the number of free parameters. The object detection is learned or a learned object detector is applied.

Claims

1. A method for object detection, the method comprising:

obtaining images of an object;

defining an input layer and a plurality of sequential feature layers subsequent to the input layer of a multi-layer feature learning network, features from the input layer provided directly to a first of the sequential feature layers, features from each of the sequential feature layers provided directly to a next of the sequential feature layers, the sequential feature layers comprising hidden layers;

providing an aggregator layer receiving the features directly from multiple layers of the sequential feature layers of the multi-layer feature learning network, the features from different ones of the sequential feature layers provided to subsequent ones of the sequential feature layers and also provided directly to the aggregator layer without processing by the subsequent ones, the aggregator layer aggregating the received features;

optimizing, jointly and by a processor, the multi-layer feature learning network and the aggregator layer using the images of the object; and

outputting, by the processor, a set of learned features represented by the optimized multi-layer feature learning network and a detector that makes use of the generated features by the optimized aggregator layer, the set of learned features being for distinguishing the object and the detector being for classifying the object.

2. The method of claim 1 wherein obtaining the images comprises obtaining medical images with the object comprising an organ or medical anomaly.

3. The method of claim 1 wherein obtaining the images comprises obtaining security images with the object comprising a person.

4. The method of claim 1 wherein defining comprises defining the multi-layer feature learning network as a deep architecture.

5. The method of claim 1 wherein defining comprises defining the sequential feature layers as auto-encoder layers.

6. The method of claim 1 wherein defining comprises defining the sequential feature layers as restricted Boltzmann machine layers.

7. The method of claim 1 wherein defining comprises defining the multi-layer feature learning network as having at least four of the sequential feature layers, outputs from a first of the sequential feature layers being first features fed to a second of the sequential feature layers, outputs from the second of the sequential feature layers being second features fed to a third of the sequential feature layers, outputs of the third of the sequential feature layers being third features fed to a fourth of the sequential feature layers, and outputs of the fourth, third, and second of the sequential feature layers being fed directly to the aggregator layer.

8. The method of claim 1 wherein optimizing comprises machine learning.

9. The method of claim 1 wherein optimizing jointly comprises using back projection between adjacent ones of the sequential feature layers and from the aggregator layer to the multiple ones of the sequential feature layers.

10. The method of claim 1 wherein optimizing comprises learning a mapping function of the aggregator layer with a probabilistic boosting tree or support vector machine.

11. The method of claim 1 wherein outputting the detector comprises outputting the detector as a classifier of a plurality of attributes based on the features of the multiple feature layers.

12. The method of claim 1 wherein outputting the learned feature comprises outputting the features of the separate feature layers as different levels of abstraction based on reconstruction of the object.

13. The method of claim 1 further comprising dividing each of the images into sub-regions, wherein defining comprises defining a first of the sequential feature layers into separate local feature learners for respective sub-regions and defining a second of the sequential feature layers as a global feature learner for the entirety of the images from outputs of the separate local feature learners.

14. The method of claim 1 further comprising eliminating candidates for the object with pre-programmed features, and wherein optimizing comprises optimizing using remaining candidates after the eliminating.

15. A method for object detection, the method comprising:

obtaining images of an object;

defining a plurality of sequential feature layers of a multi-layer feature learning network;

providing an aggregator layer receiving features from multiple layers of the multi-layer feature learning network;

optimizing, jointly and by a processor, the multi-layer feature learning network and the aggregator layer using the images of the object, the optimizing jointly using back projection between adjacent ones of the sequential feature layers and from the aggregator layer to the multiple ones of the sequential feature layers; and

outputting, by the processor, a set of learned features represented by the optimized multi-layer feature learning network and a detector that makes use of the generated features by the optimized aggregator layer, the set of learned features being for distinguishing the object and the detector being for classifying the object.