×

Learning method and learning device for object detector based on CNN using 1×1 convolution to be used for hardware optimization, and testing method and testing device using the same

  • US 10,395,140 B1
  • Filed: 01/23/2019
  • Issued: 08/27/2019
  • Est. Priority Date: 01/23/2019
  • Status: Active Grant
First Claim
Patent Images

1. A method for learning parameters of an object detector based on a CNN, comprising steps of:

  • (a) a learning device, if at least one training image is acquired, (i) instructing one or more convolutional layers to generate at least one initial feature map by applying one or more convolution operations to the training image, (ii) instructing an RPN to generate one or more proposals corresponding to each of one or more objects in the training image by using the initial feature map, and (iii) (iii-1) instructing a pooling layer to apply one or more pooling operations to each region, corresponding to each of the proposals, on the initial feature map, to thereby generate pooled feature maps per each of the proposals, and instructing a first transposing layer to concatenate each of pixels, per each of the proposals, in each of corresponding same locations on the pooled feature maps per each of the proposals, to thereby generate an integrated feature map, or (iii-2) instructing the pooling layer to apply the pooling operations to each region, corresponding to each of the proposals, on the initial feature map, to thereby generate the pooled feature maps per each of the proposals, and instructing the pooling layer to concatenate each of the pixels, per each of the proposals, in each of the corresponding same locations on the pooled feature maps per each of the proposals, to thereby generate the integrated feature map;

    (b) the learning device instructing a first 1×

    1 convolutional layer to apply a 1×

    1 convolution operation to the integrated feature map, to thereby generate a first adjusted feature map whose volume is adjusted, and instructing a second 1×

    1 convolutional layer to apply the 1×

    1 convolution operation to the first adjusted feature map, to thereby generate a second adjusted feature map whose volume is adjusted; and

    (c) the learning device (c1) (i) instructing a second transposing layer to divide the second adjusted feature map by each of the pixels, to thereby generate pixel-wise feature maps per each of the proposals, and instructing a classifying layer to generate object class information on each of the proposals by using the pixel-wise feature maps per each of the proposals, or (ii) instructing the classifying layer to divide the second adjusted feature map by each of the pixels, to thereby generate the pixel-wise feature maps per each of the proposals, and instructing the classifying layer to generate the object class information on each of the proposals by using the pixel-wise feature maps per each of the proposals, (c2) instructing a detecting layer to generate object detection information corresponding to the objects in the training image by referring to the object class information and the pixel-wise feature maps per each of the proposals, and (c3) instructing a detection loss layer to calculate one or more object detection losses by referring to the object detection information and its corresponding GT, to thereby learn at least part of parameters of the second 1×

    1 convolutional layer, the first 1×

    1 convolutional layer, and the convolutional layers by backpropagating the object detection losses.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×