Method for accommodating missing descriptor and property data while training neural network models

• US 20040186815A1
• Filed: 12/19/2003
• Published: 09/23/2004
• Est. Priority Date: 12/20/2002
• Status: Abandoned Application

• Current Assignee: Accelrys Incorporated
• Original Assignee: Accelrys Incorporated
• Inventors: Stockfisch, Thomas P.
• Application Number:
  US10/742,626
• Publication Number:
  US 20040186815A1
• Time in Patent Office: Days
• US Class Current: 706/16
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Systems and methods are described for training a neural network with a set of training items that contains items missing one or more X descriptors and/or one or more Y property values. Missing property values are accommodated by not back propagating error due to predictions of the missing property values. Missing descriptors are accommodated by first providing initial estimates for the missing descriptors. The missing descriptors are predicted in replicated output nodes along with properties of interest. Error is then back propagated all the way to the missing descriptor input nodes in order to adjust the estimates. Iteration can provide optimized estimates of the missing descriptors along with an optimized neural network. Missing descriptors in new items whose properties of interest are to be predicted can similarly be accommodated.

Claims

1. A method of constructing a neural network comprising a set of property prediction outputs and a set of descriptor inputs, said method comprising updating connection weights of said neural network using a training set of physical items, wherein at least some of said physical items of said training set have one or more undefined properties corresponding to at least one output of said neural network.

2. The method of claim 1, wherein at least some of said physical items of said training set have one or more undefined descriptors corresponding to at least one input of said neural network.

3. The method of claim 2, additionally comprising optimizing estimates for undefined descriptors of said training set of physical items.

4. A method of training a neural network, the neural network operating to provide predictions of at least one attribute of interest of a physical item, wherein the physical item has one or more measured or computed descriptors representative of one or more physical characteristics of the physical item, and wherein the physical item has one or more unknown properties of interest which have not been physically measured or otherwise previously determined, the neural network comprising a plurality of layers, each layer comprising one or more nodes, wherein the first layer comprises one or more input nodes that are configured to receive as input the one or more descriptors, wherein the last layer comprises one or more output nodes that are configured to output predictions of values for the one or more unknown properties of interest, and wherein one or more layers between the first and last layers comprise one or more hidden nodes, each hidden node in a layer receiving as input the output of nodes in the immediately preceding layer and producing as output the result of a function of the output of nodes in the immediately preceding layer and one or more connection weights, the method comprising:

providing a set of training items, said items comprising one or more physically measured or previously determined descriptors representing physical characteristics of said items, and one or more physically measured or previously determined values for one or more properties of interest;

applying said one or more physically measured or previously determined descriptors of said set of items to said input nodes of said neural network;

receiving as output from said output nodes a set of predicted values for properties of said set of training items;

comparing only a subset of said set of predicted values with corresponding values in said physically measured or previously determined values; and

adjusting said connection weights based at least in part on said comparing.

5. The method of claim 4, wherein said compared subset consists of each predicted value that has a corresponding physically measured or previously determined value.

6. A method of training a neural network, the neural network operating to provide predictions of at least one attribute of interest of a physical item, wherein the physical item has one or more measured or computed descriptors representative of one or more physical characteristics of the physical item, and wherein the physical item has one or more unknown properties of interest which have not been physically measured or otherwise previously determined, the neural network comprising a plurality of layers, each layer comprising one or more nodes, wherein the first layer comprises one or more input nodes that are configured to receive as input the one or more descriptors, wherein the last layer comprises one or more output nodes that are configured to output predictions of values for the one or more unknown properties of interest, and wherein one or more layers between the first and last layers comprise one or more hidden nodes, each hidden node in a layer receiving as input the output of nodes in the immediately preceding layer and producing as output the result of a function of the output of nodes in the immediately preceding layer and one or more connection weights, the method comprising:

providing a set of training items, said items comprising one or more physically measured or previously determined descriptors representing physical characteristics of said items, and one or more physically measured or previously determined values for one or more properties of interest;

applying said one or more physically measured or previously determined descriptors of said set of items to said input nodes of said neural network;

applying an initial estimate for a descriptor corresponding to a characteristic of an item in said set of training items to one of said input nodes;

receiving as output from said output nodes a set of predicted values for properties of the set of training items;

comparing said set of predicted values with corresponding values in said one or more physically measured or previously determined values; and

adjusting said initial estimate based at least in part on said comparing.

7. A method of predicting values for one or more properties of interest of a physical item, the properties of interest representing physical properties of the item, the item comprising a plurality of descriptors representing one or more physical characteristics of the item, the method comprising:

providing a neural network configured to receive as input values for said plurality of descriptors and provide as output values for said one or more properties of interest;

providing to said neural network measured or computed values for at least one of said plurality of descriptors,

estimating values for one or more other of said plurality of descriptors have not been physically measured or otherwise previously determined and providing said estimates to said neural network;

receiving as output from said neural network predicted values for said one or more properties of interest and said plurality of descriptors; and

adjusting said estimates for said plurality of descriptors that have not been physically measured or otherwise previously determined using one or more of said outputs.

8. The method of claim 7 further comprising iteratively performing the receiving and adjusting acts to optimize said estimates.

9. A computer implemented system for training a neural network for predicting one or more properties of interest of a physical item, said system comprising:

a memory storing a set of descriptors and a set of values for said properties of interest for a subset of a set of training physical items, wherein said set of descriptors and said set of values for said properties were physically measured or previously determined, said memory also storing at least one descriptor and values for at least one property of interest for at least one of said set of training items not part of said subset, wherein at least one property of said at least one training item is not stored in said memory;

a neural network calculation module operative to receive as input said set of descriptors and calculate predicted values for said properties of interest;

a comparison module operative to compare said predicted values with corresponding ones of said set of physically measured or previously determined values for at least one property; and

a connection weight adjustment module operative to adjust connection weights of said neural network based on output from said comparison module.

10. The system of claim 9 further comprising an initial estimate module operative to provide initial estimates for at least one descriptor not stored in said memory as input to said neural network calculation module and wherein said connection weight adjustment module is operative to adjust said initial estimate based on output from said comparison module.

11. A method of predicting a set of pre-defined characteristics of formulations that are likely to exhibit a desired property comprising:

providing a neural network trained to predict the existence of said desired property given a value for each of said set of pre-defined characteristics;

providing value estimates for each of said set of pre-defined characteristics so as to produce a predicted property from said neural network,

back propagating an error between said predicted property and said desired property to produce a correction value for said value estimates;

adjusting said value estimates based on said correction value.

12. The method of claim 11, comprising iterating said providing value estimates, back propagating, and adjusting steps to produce an optimized set of pre-defined characteristics.