System and method for procedurally synthesizing datasets of objects of interest for training machine-learning models

• US 10,643,106 B2
• Filed: 05/15/2018
• Issued: 05/05/2020
• Est. Priority Date: 02/15/2016
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Clayton, Jesse, Glavtchev, Vladimir
• Primary Examiner(s): Seth, Manav
• Application Number:
  US15/979,547
• Publication Number:
  US 20180260662A1
• Time in Patent Office: 721 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/046 Forward inferencing; Production systems
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A system and method for procedurally synthesizing a training dataset for training a machine-learning model. In one embodiment, the system includes: (1) a training designer configured to describe variations in content of training images to be included in the training dataset and (2) an image definer coupled to the training designer, configured to generate training image definitions in accordance with the variations and transmit the training image definitions: to a 3D graphics engine for rendering into corresponding training images, and further to a ground truth generator for generating associated ground truth corresponding to the training images, the training images and the associated ground truth comprising the training dataset.

Claims

1. A system for procedurally synthesizing a training dataset for training a machine-learning model, comprising:

a training designer configured to describe variations in content of training images to be included in said training dataset; and

an image definer coupled to said training designer, configured to generate training image definitions in accordance with said variations and transmit said training image definitions;

to a 3D graphics engine for rendering into corresponding training images, and

to a ground truth generator for generating, at least partially in parallel with the rendering by the 3D graphics engine, associated ground truth corresponding to said training images, said training images and said associated ground truth comprising said training dataset.

2. The system as recited in claim 1 wherein said training designer is further configured to accept input from a user establishing limits of said variations.

3. The system as recited in claim 1 wherein said training designer is further configured to refine said variations to eliminate variations that are physically impossible.

4. The system as recited in claim 1 wherein said training image definitions are expressed in a standard graphics language.

5. The system as recited in claim 1 wherein said 3D graphics engine is a conventional graphics engine.

6. The system as recited in claim 1 wherein said ground truth generator is further configured to generate said ground truth from said training image definitions expressed in a graphics language.

7. The system as recited in claim 1 wherein said ground truth includes coordinates locating an object of interest in said training images.

8. A system for procedurally synthesizing a training dataset for training a machine-learning model, comprising:

a first processor configured to generate training image definitions in accordance with variations in content of training images to be included in said training dataset;

a second processor configured to render corresponding training images from said training image definitions; and

a third processor configured to generate, from said training image definitions and at least partially in parallel with said render of said second processor, associated ground truth corresponding to said training images, said training images and said associated ground truth comprising said training dataset.

9. The system as recited in claim 8 further comprising a computer configured to accept input from a user establishing limits of said variations, describe said variations, and transmit said training image definitions to said first processor.

10. The system as recited in claim 9 wherein said computer is further configured to refine said variations to eliminate variations that are physically impossible.

11. The system as recited in claim 8 wherein said first processor generates said training image definitions in a 3D graphics language employable by 3D graphics engine.

12. The system as recited in claim 8 wherein said second processor employs a 3D graphics engine to render said corresponding training images.

13. The system as recited in claim 8 wherein said third processor is further configured to generate said ground truth from said training image definitions expressed in a graphics language.

14. The system as recited in claim 8 wherein said ground truth includes coordinates locating an object of interest in said training images.

15. The system as recited in claim 8 wherein said second and third processors employ parallel processing.

16. The system as recited in claim 8 wherein said second and third processors are cloud servers.

17. The system as recited in claim 8 wherein said first processor is a different processor than said second processor or said third processor.

18. The system as recited in claim 8 wherein said variations are in characteristics of said content.

19. The system as recited in claim 18 wherein said characteristics are selected from the group consisting of:

ambient lighting level,

ambient lighting source location,

size of an object of interest,

location of an object of interest,

colors of an object of interest,

numbers of colors of an object of interest,

numbers and types of objects, and

types of background.

20. The system as recited in claim 18 wherein said variations are limited to prevent unrealistic training images.