MULTI-LAYER AGGREGATION FOR OBJECT DETECTION

• US 20160048741A1
• Filed: 08/12/2014
• Published: 02/18/2016
• Est. Priority Date: 08/12/2014
• Status: Active Grant

• Current Assignee: Siemens Healthineers AG (Siemens AG)
• Original Assignee: Siemens AG
• Inventors: Singh, Vivek Kumar, Zheng, Yefeng, Georgescu, Bogdan, Comaniciu, Dorin, Zhou, Shaohua Kevin, Nguyen, Hien
• Granted Patent: US 9,767,385 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Object detection uses a deep or multiple layer network to learn features for detecting the object in the image. Multiple features from different layers are aggregated to train a classifier for the object. In addition or as an alternative to feature aggregation from different layers, an initial layer may have separate learnt nodes for different regions of the image to reduce the number of free parameters. The object detection is learned or a learned object detector is applied.

Claims

1. A method for object detection, the method comprising:

obtaining images of an object;

defining a plurality of sequential feature layers of a multi-layer feature learning network;

providing an aggregator layer receiving features from multiple layers of the multi-layer feature learning network;

optimizing, jointly and by a processor, the multi-layer feature learning network and the aggregator layer using the images of the object; and

outputting, by the processor, a set of learned features represented by the optimized multi-layer feature learning network and a detector that makes use of the generated features by the optimized aggregator layer, the set of learned features being for distinguishing the object and the detector being for classifying the object.

2. The method of claim 1 wherein obtaining the images comprises obtaining medical images with the object comprising an organ or medical anomaly.

3. The method of claim 1 wherein obtaining the images comprises obtaining security images with the object comprising a person.

4. The method of claim 1 wherein defining comprises defining the multi-layer feature learning network as a deep architecture.

5. The method of claim 1 wherein defining comprises defining the sequential feature layers as auto-encoder layers.

6. The method of claim 1 wherein defining comprises defining the sequential feature layers as restricted Boltzmann machine layers.

7. The method of claim 1 wherein defining comprises defining the multi-layer feature learning network as having at least four of the sequential feature layers, outputs from a first of the sequential feature layers being first features fed to a second of the sequential feature layers, outputs from the second of the sequential feature layers being second features fed to a third of the sequential feature layers, outputs of the third of the sequential feature layers being third features fed to a fourth of the sequential feature layers, and outputs of the fourth, third, and second of the sequential feature layers being fed to the aggregator layer.

8. The method of claim 1 wherein optimizing comprises machine learning.

9. The method of claim 1 wherein optimizing jointly comprises using back projection from the aggregator layer to the multiple ones of the feature layers.

10. The method of claim 1 wherein optimizing comprises learning a mapping function of the aggregator layer with a probabilistic boosting tree or support vector machine.

11. The method of claim 1 wherein outputting the detector comprises outputting the detector as a classifier of a plurality of attributes based on the features of the multiple feature layers.

12. The method of claim 1 wherein outputting the learned feature comprises outputting the features of the separate feature layers as different levels of abstraction based on reconstruction of the object.

13. The method of claim 1 further comprising dividing each of the images into sub-regions, wherein defining comprises defining a first of the sequential feature layers into separate local feature learners for respective sub-regions and defining a second of the sequential feature layers as a global feature learner for the entirety of the images from outputs of the separate local feature learners.

14. The method of claim 1 further comprising eliminating candidates for the object with pre-programmed features, and wherein optimizing comprises optimizing using remaining candidates after the eliminating.

15. In a non-transitory computer readable storage medium having stored therein data representing instructions executable by a programmed processor for object detection, the storage medium comprising instructions for:

receiving an image of an object;

detecting, by the processor, the object in the image with hidden features generated from hidden layers of a deep network, the hidden features learned from the hidden layers comprising different layers of abstraction, where the detecting uses an aggregation in a machine-learnt classifier, by a machine, of the hidden features from the different layers input as a feature vector; and

outputting detection of the object.

16. The non-transitory computer readable storage medium of claim 15 wherein receiving the image comprises receiving the image as visible information, and wherein detecting comprises detecting with the hidden features of the deep network comprising node outputs of a multi-layer network where the nodes are trained auto-encoder or restricted Boltzmann machine blocks.

17. The non-transitory computer readable storage medium of claim 15 wherein detecting comprises detecting the hidden features as first features in a first layer, second attributes in a second layer with the first features as input, and third attributes in a third layer with the second attributes as input, the feature vector of the machine-learnt classifier being at least the second and third attributes.

18. The non-transitory computer readable storage medium of claim 15 wherein detecting comprises detecting with a first set of the hidden features being for respective sub-parts of the image and with a second set of the hidden features being for the image with inputs from all of the first set.

19. The non-transitory computer readable storage medium of claim 15 wherein detecting further comprises limiting candidates with pre-programmed features prior to detecting with the hidden features of the deep network.

20. A method for object detection, the method comprising:

dividing images of an object into local parts;

training, by a machine, first blocks of feature nodes to reconstruct the local parts of the images;

training, by the machine, second blocks of second attribute nodes to reconstruct the feature nodes;

training, by the machine, third blocks of third attribute nodes to reconstruct the second attribute nodes; and

training, by the machine, a feature aggregator to classify the object, the feature aggregator trained with the second and third attribute nodes as inputs.