Visual object recognition

• US 10,049,307 B2
• Filed: 04/04/2016
• Issued: 08/14/2018
• Est. Priority Date: 04/04/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Pankanti, Sharathchandra U., Peng, Xi, Ratha, Nalini K.
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US15/089,707
• Publication Number:
  US 20170286809A1
• Time in Patent Office: 862 Days
• Field of Search: None
• CPC Class Codes: G06F 18/2414 Smoothing the distance, e.g. radial basis function networks [RBFN]
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

Technical solutions are described for training an object-recognition neural network that identifies an object in a computer-readable image. An example method includes assigning a first neural network for determining a visual alignment model of the images for determining a normalized alignment of the object. The method further includes assigning a second neural network for determining a visual representation model of the images for recognizing the object. The method further includes determining the visual alignment model by training the first neural network and determining the visual representation model by training the second neural network independent of the first. The method further includes determining a combined object recognition model by training a combination of the first neural network and the second neural network. The method further includes recognizing the object in the image based on the combined object recognition model by passing the image through each of the neural networks.

Claims

1. A computer-implemented method for training an object-recognition neural network to identify an object in a computer-readable image, the method comprising:

assigning, using a processor system, a first neural network for determining a visual alignment model of a plurality of input images, wherein the visual alignment model is used to determine a normalized alignment of an object in the input images;

assigning, using the processor system, a second neural network for determining a visual representation model of the input images, wherein the visual representation model is used to recognize the object in the input images;

determining the visual alignment model by training the first neural network;

determining the visual representation model by training the second neural network;

determining a combined object recognition model by training a combination of neural networks comprising the first neural network and the second neural network; and

recognizing the object in the computer-readable image based on the combined object recognition model by passing the computer-readable image through the first neural network and the second neural network of the combination of neural networks.

2. The computer-implemented method of claim 1, further comprising:

splitting an object-recognition neural network into a plurality of independent networks including the first neural network and the second neural network, wherein a first subset of the independent networks operates on content-based attributes of the images and a second subset of the independent networks operates on content-unaware attributes of the images.

3. The computer-implemented method of claim 1, wherein the visual alignment model is determined by training the first neural network using a first set of images and the visual representation model is determined by training the second neural network using a second set of images.

4. The computer-implemented method of claim 3, wherein the combined object recognition model is determined by training a combination of the first neural network and the second neural network using a third set of images, distinct from the first set of images and the second set of images.

5. The computer-implemented method of claim 1, wherein the first neural network is trained in parallel to the second neural network.

6. The computer-implemented method of claim 5, wherein the first neural network is trained by a first computer system and the second neural network is trained by a second computer system, which is distinct from the first computer system.

7. The computer-implemented method of claim 1, wherein the first neural network is trained by a first processor in parallel to the second neural network being trained by a second processor.

8. The computer-implemented method of claim 1, further comprising:

assigning a third neural network for determining a contrast normalization model of the images, wherein the contrast normalization model is used to determine a normalized contrast value of the input images; and

determining the combined object recognition model by further training a combination of the first neural network, the second neural network, and the third neural network.

9. The computer-implemented method of claim 8, wherein recognizing the object in the computer-readable image based on the combined object recognition model comprises:

passing the computer-readable image to the first neural network and to the third neural network in parallel; and

passing outputs of the first neural network and the third neural network to the second neural network, wherein the second neural network recognizes the object in the computer-readable image based on the respective outputs.

10. The computer-implemented method of claim 1, wherein determining the normalized alignment by training the first neural network comprises determining transformations in 3-dimensional space to align the object in the computer-readable image in a normalized position.

11. A system for training an object-recognition neural network to identify an object in a computer-readable image, the system comprising:

a memory; and

a processor communicatively coupled to the memory, wherein the processor is configured to;

assign a first neural network for determining a visual alignment model of a plurality of input images, wherein the visual alignment model is used to determine a normalized alignment of an object in the input images;

assign a second neural network for determining a visual representation model of the input images, wherein the visual representation model is used to recognize the object in the input images;

determine the visual alignment model by training the first neural network;

determine the visual representation model by training the second neural network;

determine a combined object recognition model by training a combination of neural networks comprising the first neural network and the second neural network; and

recognize the object in the computer-readable image based on the combined object recognition model by passing the computer-readable image through the first neural network and the second neural network of the combination of neural networks.

12. The system of claim 11, wherein the processor is further configured to split an object-recognition neural network into a plurality of independent networks including the first neural network and the second neural network, wherein a first subset of the independent networks operates on content-based attributes of the images and a second subset of the independent networks operates on content-unaware attributes of the images.

13. The system of claim 11, wherein the visual alignment model is determined by training the first neural network using a first set of images and the visual representation model is determined by training the second neural network using a second set of images.

14. The system of claim 13, wherein the combined object recognition model is determined by training a combination of the first neural network and the second neural network using a third set of images, distinct from the first set of images and the second set of images.

15. The system of claim 11, wherein the first neural network is trained in parallel to the second neural network.

16. The system of claim 11, wherein the first neural network is trained by a first computer system and the second neural network is trained by a second computer system, which is distinct from the first computer system.

17. A computer program product for training an object-recognition neural network to identify an object in a computer-readable image, the computer program product comprising a non-transitory computer readable storage medium, the computer readable storage medium comprising computer executable instructions, wherein the computer readable storage medium comprises instructions to:

assign a first neural network for determining a visual alignment model of a plurality of input images, wherein the visual alignment model is used to determine a normalized alignment of an object in the input images;

assign a second neural network for determining a visual representation model of the input images, wherein the visual representation model is used to recognize the object in the input images;

determine the visual alignment model by training the first neural network;

determine the visual representation model by training the second neural network;

determine a combined object recognition model by training a combination of neural networks comprising the first neural network and the second neural network; and

recognize the object in the computer-readable image based on the combined object recognition model by passing the computer-readable image through the first neural network and the second neural network of the combination of neural networks.

18. The computer program product of claim 17, wherein the visual alignment model is determined by training the first neural network using a first set of images and the visual representation model is determined by training the second neural network using a second set of images.

19. The computer program product of claim 18, wherein the combined object recognition model is determined by training a combination of the first neural network and the second neural network using a third set of images, distinct from the first set of images and the second set of images.

20. The computer program product of claim 17, wherein the first neural network is trained in parallel to the second neural network, and wherein the first neural network is trained by a first computer system and the second neural network is trained by a second computer system, which is distinct from the first computer system.