Neural network training system

• US 10,592,780 B2
• Filed: 03/30/2018
• Issued: 03/17/2020
• Est. Priority Date: 03/30/2018
• Status: Active Grant

• Current Assignee: White Raven Ltd.
• Original Assignee: White Raven Ltd.
• Inventors: Gurvich, Roni, Ilan, Idan, Avni, Ofer, Yagev, Stav
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US15/942,226
• Publication Number:
  US 20190303725A1
• Time in Patent Office: 718 Days
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/776 Validation; Performance evaluation
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

In order for the feature extractors to operate with sufficient accuracy, a high degree of training is required. In this situation, a neural network implementing the feature extractor may be trained by providing it with images having known correspondence. A 3D model of a city may be utilized in order to train a neural network for location detection. 3D models are sophisticated and allow manipulation of viewer perspective and ambient features such as day/night sky variations, weather variations, and occlusion placement. Various manipulations may be executed in order to generate vast numbers of image pairs having known correspondence despite having variations. These image pairs with known correspondence may be utilized to train the neural network to be able to generate feature maps from query images and identify correspondence between query image feature maps and reference feature maps. This training can be accomplished without requiring the capture of real images with known correspondence. Capture of real images with known correspondence is cumbersome, time and resource-intensive, and difficult to manage.

Claims

1. A method for training a neural network implementation of a feature extractor and a correspondence matcher comprising the steps of:

set rendering parameters in a 3D model wherein said rendering parameters include at least position and orientation;

rendering a frame according to said rendering parameters to establish a current image;

processing ray-tracing information of said frame to solve a ground-truth correspondence map of said current image with a neighbor image;

incrementing said rendering parameters; and

providing said ground-truth correspondence map as a first training input to said neural network.

2. The method of claim 1 further comprising the steps of:

processing said current image through a feature extractor to obtain a current feature map;

processing said current feature map through a correspondence matcher against a feature map derived from said neighbor image to obtain a feature extraction correspondence map;

providing said feature extraction correspondence map as a second training input to said neural network;

applying a loss function to said feature extraction correspondence map and said ground-truth correspondence map; and

updating said feature extractor on the basis of a result of the application of said loss function.

3. The method of claim 2 further comprising the step of updating said correspondence matcher consistent with the step of updating said feature extractor.

4. The method of claim 3 wherein the step of updating said correspondence matcher comprises the step of adjusting weights of said correspondence matcher.

5. The method of claim 4 wherein the step of updating weights of said correspondence matcher by back propagation.

6. The method of claim 2 wherein the step of updating said feature extractor further comprises the step of updating weights of said feature extractor.

7. The method of claim 6 wherein the step of updating weights of said feature extractor is by back propagation.

8. The method of claim 1 wherein said step of incrementing further comprises the step of modifying position.

9. The method according to claim 1 wherein said rendering parameters further comprise light level.

10. The method according to claim 1 wherein said rendering parameters further comprise orientation.

11. The method according to claim 1 wherein said rendering parameters further comprise occlusions.

12. The method according to claim 11 wherein said occlusions further comprise one or more of simulated vegetation, simulated pedestrians, and simulated vehicles.

13. The method according to claim 1 wherein said rendering parameters further comprise a position and orientation corresponding to the position and orientation consistent with the position and orientation of a forward-looking vehicle camera located on a virtual street of said 3D model.

14. The method according to 13 wherein said step of incrementing comprises changing said position and orientation to that [which is] consistent with a forward-looking vehicle camera moved a fixed distance along said virtual street of said 3D model.