Convexification method of training neural networks and estimating regression models
First Claim
1. A method of training a neural system that comprises a neural network, said method comprising the steps ofevaluating a risk-averting error criterion;
- adjusting at least one weight of said neural network to reduce a value of said risk-averting error criterion; and
adjusting a risk-sensitivity index of said risk-averting error criterion,wherein said risk-averting error criterion comprises an exponential function of an output of said neural system.
0 Assignments
0 Petitions
Accused Products
Abstract
A method of training neural systems and estimating regression coefficients of regression models with respect to an error criterion is disclosed. If the error criterion is a risk-averting error criterion, the invented method performs the training/estimation by starting with a small value of the risk-sensitivity index of the risk-averting error criterion and gradually increasing it to ensure numerical feasibility. If the error criterion is a risk-neutral error criterion such as a standard sum-of-squares error criterion, the invented method performs the training/estimation first with respect to a risk-averting error criterion associated with the risk-neutral error criterion. If the result is not satisfactory for the risk-neutral error criterion, further training/estimation is performed either by continuing risk-averting training/estimation with decreasing values of the associated risk-averting error criterion or by training/estimation with respect to the given risk-neutral error criterion or by both.
12 Citations
20 Claims
-
1. A method of training a neural system that comprises a neural network, said method comprising the steps of
evaluating a risk-averting error criterion; -
adjusting at least one weight of said neural network to reduce a value of said risk-averting error criterion; and adjusting a risk-sensitivity index of said risk-averting error criterion, wherein said risk-averting error criterion comprises an exponential function of an output of said neural system. - View Dependent Claims (2, 3, 4)
-
-
5. A method of training a neural system with respect to a risk-neutral error criterion, said neural system comprising a neural network, said method comprising the steps of
training said neural system with respect to a risk-averting error criterion; - and
further training said neural system with respect to said risk-neutral error criterion, wherein said risk-averting error criterion comprises an exponential function of an output of said neural system, and whereby said neural system will have a small value of said risk-neutral error criterion. - View Dependent Claims (6, 7, 8, 9, 10)
adjusting at least one weight of said neural system to reduce said risk-averting error criterion; and adjusting a risk-sensitivity index of said risk-averting error criterion.
- and
-
9. The method of claim 8, wherein said step of adjusting a risk-sensitivity index is performed through starting with a small risk-sensitivity index, gradually increasing said risk-sensitivity index.
-
10. The method of claim 8, wherein said step of adjusting a risk-sensitivity index is performed by a method comprising a step of centering and bounding a plurality of exponents in said risk-averting error criterion.
-
11. A method of estimating at least one regression coefficient of a regression model, said method comprising the steps of
evaluating a risk-averting error criterion; -
adjusting said at least one regression coefficient to reduce a value of said risk-averting error criterion; and adjusting a risk-sensitivity index of said risk-averting error criterion, wherein said risk-averting error criterion comprises an exponential function of an output of said regression model. - View Dependent Claims (12, 13, 14)
-
-
15. A method of estimating at least one regression coefficient of a regression model with respect to a risk-neutral error criterion, said method comprising the steps of
estimating said at least one regression coefficient with respect to a risk-averting error criterion; - and
further estimating said at least one regression coefficient with respect to said risk-neutral error criterion, wherein said risk-averting error criterion comprises an exponential function of an output of said regression model, whereby said regression model will have a small value of said risk-neutral error criterion. - View Dependent Claims (16, 17, 18, 19, 20)
adjusting said at least one regression coefficient to reduce said risk-averting error criterion; and adjusting a risk-sensitivity index of said risk-averting error criterion.
- and
-
19. The method of claim 18, wherein said step of adjusting said risk-sensitivity index is performed through starting with a small risk-sensitivity index and gradually increasing said risk-sensitivity index.
-
20. The method of claim 18, wherein said step of adjusting a risk-sensitivity index is performed by a method comprising a step of centering and bounding a plurality of exponents in said risk-averting error criterion.
Specification