Object detection with neural network

• US 10,275,688 B2
• Filed: 12/17/2014
• Issued: 04/30/2019
• Est. Priority Date: 12/17/2014
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Jiang, Xiaoheng
• Primary Examiner(s): Liew, Alex Kok S
• Application Number:
  US15/535,280
• Publication Number:
  US 20170351936A1
• Time in Patent Office: 1,595 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: Y04S 10/50 Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Abstract

According to an example aspect of the present invention, there is provided an apparatus comprising at least one processing core and at least one memory including computer program code, the at least one memory and the computer program code being configured to, with the at least one processing core, cause the apparatus at least to nm a convolutional neural network comprising an input layer arranged to provide signals to a first convolutional layer and a last convolutional layer, run a first intermediate classifier, the first intermediate classifier operating on a set of feature maps of the first convolutional layer, and decide to abort or to continue processing of a signal set based on a decision of the first intermediate classifier.

Claims

1. An apparatus comprising at least one processor and at least one memory including computer program code for one or more programs, the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to perform at least the following:

run a convolutional neural network comprising an input layer arranged to provide signals to a first convolutional layer and a last convolutional layer, wherein the first convolutional layer has a plurality of feature maps;

determine one support vector machine for a respective one of the feature maps;

run training data via the support vector machine to determine respective classification error performance of the respective feature map;

rank the feature maps based on the respective classification error performance;

reconstruct the first intermediate classifier using a number of the feature maps of the first convolutional layer with top ranking, wherein the number is bigger than one and smaller than a total number of the feature maps of the first convolutional layer; and

decide to abort or to continue processing of a signal set based on a decision run via the reconstructed first intermediate classifier.

2. The apparatus according to claim 1, wherein the convolutional neural network comprises a second convolutional layer between the first convolutional layer and the last convolutional layer, and a second intermediate classifier which operates on a set of feature maps of the second convolutional layer, and the apparatus is configured to decide to abort or to continue processing of the signal set based on a decision of the second intermediate classifier.

3. The apparatus according to claim 2, wherein the convolutional neural network comprises a third convolutional layer between the second convolutional layer and the last convolutional layer, and a third intermediate classifier which operates on a set of feature maps of the third convolutional layer, and the apparatus is configured to decide to abort or to continue processing of the signal set based on a decision of the third intermediate classifier.

4. The apparatus according to claim 1, wherein the apparatus is further configured to:

concatenate elements of the respective feature map into one vector as the support vector machine.

5. The apparatus according to claim 2, wherein the set of feature maps of the second convolutional layer comprises a subset of all feature maps of the second convolutional layer.

6. The apparatus according to claim 3, wherein the set of feature maps of the third convolutional layer comprises a subset of all feature maps of the third convolutional layer.

7. The apparatus according to claim 1, wherein the signal set comprises a processed sub-image of an image from the input layer.

8. The apparatus according to claim 1, wherein the signal set comprises a processed version of an image from the input layer.

9. The apparatus according to claim 1, wherein the convolutional neural network is configured to detect an image of a person.

10. The apparatus according to claim 9, wherein the convolutional neural network is configured to detect an image of a pedestrian.

11. The apparatus according to claim 3, wherein the apparatus is configured to select at least one of the set of feature maps of the first convolutional layer, the set of feature maps of the second convolutional layer and the set of feature maps of the third convolutional layer based on at least one criterion.

20. The apparatus according to claim 1, wherein the apparatus is further configured to:

cascade the number of the feature maps of the first convolutional layer into a decision tree as the reconstructed first intermediate classifier.

12. A method comprising:

running a convolutional neural network comprising an input layer arranged to provide signals to a first convolutional layer and a last convolutional layer, wherein the first convolutional layer has a plurality of feature maps;

determining one support vector machine for a respective one of the feature maps;

running training data via the support vector machine to determine respective classification error performance of the respective feature map;

ranking the feature maps based on the respective classification error performance;

reconstructing the first intermediate classifier using a number of the feature maps of the first convolutional layer with top ranking, wherein the number is bigger than one and smaller than a total number of the feature maps of the first convolutional layer; and

deciding to abort or to continue processing of a signal set based on a decision run via the reconstructed first intermediate classifier.

13. The method according to claim 12, wherein the convolutional neural network comprises a second convolutional layer between the first convolutional layer and the last convolutional layer, and a second intermediate classifier which operates on a set of feature maps of the second convolutional layer, and the method comprises deciding to abort or to continue processing of the signal set based on a decision of the second intermediate classifier.

14. The method according to claim 13, wherein the convolutional neural network comprises a third convolutional layer between the second convolutional layer and the last convolutional layer, and a third intermediate classifier which operates on a set of feature maps of the third convolutional layer, and the method comprises deciding to abort or to continue processing of the signal set based on a decision of the third intermediate classifier.

15. The method according to claim 12, wherein the apparatus is further configured to:

concatenate elements of the respective feature map into one vector as the support vector machine.

16. The method according to claim 13, wherein the set of feature maps of the second convolutional layer comprises a subset of all feature maps of the second convolutional layer.

17. The method according to claim 14, wherein the set of feature maps of the third convolutional layer comprises a subset of all feature maps of the third convolutional layer.

18. The method according to claim 12, wherein the signal set comprises a processed sub-image of an image from the input layer.

19. A non-transitory computer readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to at least perform the following steps:

running a convolutional neural network comprising an input layer arranged to provide signals to a first convolutional layer and a last convolutional layer, wherein the first convolutional layer has a plurality of feature maps;

determining one support vector machine for a respective one of the feature maps;

running training data via the support vector machine to determine respective classification error performance of the respective feature map;

ranking the feature maps based on the respective classification error performance;

reconstructing the first intermediate classifier using a number of the feature maps of the first convolutional layer with top ranking, wherein the number is bigger than one and smaller than a total number of the feature maps of the first convolutional layer; and

deciding to abort or to continue processing of a signal set based on a decision run via the reconstructed first intermediate classifier.