SIMILARITY-BASED DETECTION OF PROMINENT OBJECTS USING DEEP CNN POOLING LAYERS AS FEATURES
First Claim
1. A method for object localization in a query image, comprising:
- for each of a set of annotated images, providing an annotated image representation based on activations output by a layer of a model derived from part of a trained neural network, the annotated images each being annotated with object location information;
for a query image, generating a query image representation based on activations output by the layer of the model;
identifying a subset of the annotated images comprising computing a similarity between the query image representation and each of the annotated image representations;
transferring object location information from at least one of the subset of annotated images to the query image; and
outputting information based on the transferred object location information,wherein at least one of the generating a query image representation, identifying a subset of the annotated images, transferring object location information, and outputting information is performed with a processor.
7 Assignments
0 Petitions
Accused Products
Abstract
A system and method provide object localization in a query image based on a global representation of the image generated with a model derived from a convolutional neural network. Representations of annotated images and a query image are each generated based on activations output by a layer of the model which precedes the fully-connected layers of the neural network. A similarity is computed between the query image representation and each of the annotated image representations to identify a subset of the annotated images having the highest computed similarity. Object location information from at least one of the subset of annotated images is transferred to the query image and information is output, based on the transferred object location information.
-
Citations
20 Claims
-
1. A method for object localization in a query image, comprising:
-
for each of a set of annotated images, providing an annotated image representation based on activations output by a layer of a model derived from part of a trained neural network, the annotated images each being annotated with object location information; for a query image, generating a query image representation based on activations output by the layer of the model; identifying a subset of the annotated images comprising computing a similarity between the query image representation and each of the annotated image representations; transferring object location information from at least one of the subset of annotated images to the query image; and outputting information based on the transferred object location information, wherein at least one of the generating a query image representation, identifying a subset of the annotated images, transferring object location information, and outputting information is performed with a processor. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A system for object localization in a query image, comprising:
-
memory which stores a model derived from a trained neural network, the model comprising a sequence of convolutional layers which receive as input output activations of a prior convolutional layer or an image in the case of a first convolutional layer; a representation generator which generates a representation of a query image based on the activations of a selected one of the convolutional layers for the query image and an annotated image representation for each of a set of annotated images based on activations output by the selected layer of the model for the annotated image, the annotated images each being annotated with object location information; a retrieval component which retrieves a subset of similar images from the set of annotated images, based on a similarity between respective representations; a segmentation component which transfers the object location information from at least one of the subset of annotated images to the query image; an output component which outputs information based on the transferred object location information; and a processor which implements the representation generator, retrieval component, segmentation component, and output component. - View Dependent Claims (19)
-
-
20. A method for object localization in a query image, comprising:
-
adding a new layer to a pre-trained neural network to generate a model comprising a sequence of convolutional layers which each act on the output of a respective previous layer, the new layer being positioned after the last of the convolutional layers; updating weights of the new layer and of the convolutional layers of the model by backpropagation; for each of a set of annotated images, generating an annotated image representation based on activations output by the new layer of the model, the annotated images each being annotated with object location information; for a query image, generating a query image representation based on activations output by the new layer of the model; identifying a subset of the annotated images comprising computing a similarity between the query image representation and each of the annotated image representations; transferring object location information from at least one of the subset of annotated images to the query image; and outputting information based on the transferred object location information, wherein at least one of the updating weights, generating the image representations, identifying a subset of the annotated images, transferring object location information, and outputting information is performed with a processor.
-
Specification