Main Content

BFGS quasi-Newton backpropagation

`net.trainFcn = 'trainbfg'`

sets the network
`trainFcn`

property.

`[`

trains the network with `trainedNet`

,`tr`

] = train(`net`

,...)`trainbfg`

.

`trainbfg`

is a network training function that updates weight and bias
values according to the BFGS quasi-Newton method.

Training occurs according to `trainbfg`

training parameters, shown here
with their default values:

`net.trainParam.epochs`

— Maximum number of epochs to train. The default value is 1000.`net.trainParam.showWindow`

— Show training GUI. The default value is`true`

.`net.trainParam.show`

— Epochs between displays (`NaN`

for no displays). The default value is 25.`net.trainParam.showCommandLine`

— Generate command-line output. The default value is`false`

.`net.trainParam.goal`

— Performance goal. The default value is 0.`net.trainParam.time`

— Maximum time to train in seconds. The default value is`inf`

.`net.trainParam.min_grad`

— Minimum performance gradient. The default value is`1e-6`

.`net.trainParam.max_fail`

— Maximum validation failures. The default value is`6`

.`net.trainParam.searchFcn`

— Name of line search routine to use. The default value is`'srchbac'`

.

Parameters related to line search methods (not all used for all methods):

`net.trainParam.scal_tol`

— Divide into delta to determine tolerance for linear search. The default value is 20.`net.trainParam.alpha`

— Scale factor that determines sufficient reduction in perf. The default value is`0.001`

.`net.trainParam.beta`

— Scale factor that determines sufficiently large step size. The default value is`0.1`

.`net.trainParam.delta`

— Initial step size in interval location step. The default value is`0.01`

.`net.trainParam.gamma`

— Parameter to avoid small reductions in performance, usually set to 0.1 (see`srch_cha`

). The default value is`0.1`

.`net.trainParam.low_lim`

— Lower limit on change in step size. The default value is`0.1`

.`net.trainParam.up_lim`

— Upper limit on change in step size. The default value is`0.5`

.`net.trainParam.maxstep`

— Maximum step length. The default value is`100`

.`net.trainParam.minstep`

— Minimum step length. The default value is`1.0e-6`

.`net.trainParam.bmax`

— Maximum step size. The default value is`26`

.`net.trainParam.batch_frag`

— In case of multiple batches, they are considered independent. Any nonzero value implies a fragmented batch, so the final layer’s conditions of a previous trained epoch are used as initial conditions for the next epoch. The default value is`0`

.

`trainbfg`

can train any network as long as its weight, net input, and
transfer functions have derivative functions.

Backpropagation is used to calculate derivatives of performance `perf`

with respect to the weight and bias variables `X`

. Each variable is adjusted
according to the following:

X = X + a*dX;

where `dX`

is the search direction. The parameter `a`

is
selected to minimize the performance along the search direction. The line search function
`searchFcn`

is used to locate the minimum point. The first search direction
is the negative of the gradient of performance. In succeeding iterations the search direction
is computed according to the following formula:

dX = -H\gX;

where `gX`

is the gradient and `H`

is a approximate
Hessian matrix. See page 119 of Gill, Murray, and Wright (*Practical
Optimization*, 1981) for a more detailed discussion of the BFGS quasi-Newton
method.

Training stops when any of these conditions occurs:

The maximum number of

`epochs`

(repetitions) is reached.The maximum amount of

`time`

is exceeded.Performance is minimized to the

`goal`

.The performance gradient falls below

`min_grad`

.Validation performance has increased more than

`max_fail`

times since the last time it decreased (when using validation).

[1] Gill, Murray, & Wright,
*Practical Optimization*, 1981

`cascadeforwardnet`

| `feedforwardnet`

| `traingdm`

| `traingda`

| `traingdx`

| `trainlm`

| `trainrp`

| `traincgf`

| `traincgb`

| `trainscg`

| `traincgp`

| `trainoss`