Adaptive learning enhancement to automated model maintenance

• US 7,092,922 B2
• Filed: 05/21/2004
• Issued: 08/15/2006
• Est. Priority Date: 05/23/2003
• Status: Active Grant

• Current Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Original Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Inventors: Duan, Baofu, Pao, Yoh-Han, Meng, Zhuo
• Primary Examiner(s): Patel, Ramesh
• Application Number:
  US10/851,630
• Publication Number:
  US 20050033709A1
• Time in Patent Office: 816 Days
• Field of Search: 706/12, 706/14, 706/15, 706/16, 706/25, 706/26, 706/31, 706/33, 706/42, 703/4, 703/6, 700/47, 700/48, 700/50, 365/200
• US Class Current: 706/16
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

An adaptive learning method for automated maintenance of a neural net model is provided. The neural net model is trained with an initial set of training data. Partial products of the trained model are stored. When new training data are available, the trained model is updated by using the stored partial products and the new training data to compute weights for the updated model.

Claims

1. An adaptive learning method for automated maintenance of a neural net model, comprising:

training a neural net model with an initial set of training data, the neural net model having one or more original weights associated with the neural net model;

storing partial products of the trained model, the partial products comprising a portion of the initial training set of training data and a new set of training data; and

updating the trained model by using the stored partial products and new training data to compute adjusted weights for the updated model.

2. The method of claim 1, wherein the adjusted weights of the updated model are a least-squares solution for training the neural net model with a combined set consisting of (i) the new training data and (ii) the initial set of training data.

3. The method of claim 1, wherein an amount of information corresponding to the partial products of the trained model depends on the size of the neural net model but not on the size of the initial set of training data.

4. The method of claim 1, wherein the trained model is updated by using the stored partial products along with a forgetting factor α

.

5. The method of claim 1, wherein the neural net model includes a functional link net.

6. The method of claim 5, wherein the weights of the updated model are computed using an orthogonal least squares technique.

7. The method of claim 5, wherein the updated model has more functional link nodes than the trained model.

8. The method of claim 5, further comprising computing a least-squares error of the updated model.

9. The method of claim 5, wherein the updated model has less functional link nodes than the trained model.

10. The method of claim 5, further comprising:

determining a plurality of candidate functions,

wherein selected ones of the candidate functions are used to create the functional link net model.

11. The method of claim 10, further comprising generating reserve candidate functions after the neural net model is trained, until a number of unused candidate functions reaches a predetermined threshold number.

12. The method of claim 11, wherein selected ones of the unused candidate functions are used to expand the functional line net model.

13. The method of claim 5, further comprising:

determining whether the new training data falls in a range of the initial set of training data; and

creating one or more local nets by using the new training data, if the new training data does not fall in the range of the initial set of training data.

14. The method of claim 1, further comprising:

determining additional partial products by using the new training data;

determining updated partial products for the updated model by using the stored partial products and the additional partial products; and

storing the updated partial products.

15. The method of claim 14, further comprising updating further the updated model, when additional new training data become available, by using the additional new training data and the updated partial products.

16. The method of claim 1, wherein the new training data includes streaming data, and the method is used to update the trained neural net model in real time with the streaming new training data.

17. The method of claim 1, further comprising:

receiving streaming new training data;

computing additional partial products corresponding to the new training data; and

computing the weights for the updated model by using the additional partial products corresponding to the new training data.

18. A computer system for automated maintenance of a neural net model, comprising:

a memory operable to store partial products of a neural net model, the partial products comprising a portion of an initial training set of training data and a new set of training data; and

a processor coupled to the memory and operable to;

train the neural net model with an initial set of training data, the neural net model having one or more original weights associated with the neural net model; and

update the trained neural net model by using the stored partial products and new training data to compute adjusted weights for the updated model.

19. The system of claim 18, wherein the adjusted weights of the updated model are a least-squares solution for training the neural net model with a combined set consisting of (i) the new training data and (ii) the initial set of training data.

20. The system of claim 18, wherein the trained model is updated by using the stored partial products along with a forgetting factor α

.

21. The system of claim 18, wherein the neural net model includes a functional link net.

22. The system of claim 21, wherein the weights of the updated model are computed using an orthogonal least squares technique.

23. The system of claim 21, wherein the updated model has more functional link nodes than the trained model.

24. The system of claim 21, further comprising:

determining a plurality of candidate functions,

wherein selected ones of the candidate functions are used to create the functional link net model.

25. The system of claim 24, further comprising generating reserve candidate functions after the neural net model is trained, until a number of unused candidate functions reaches a predetermined threshold number.

26. The system of claim 18, further comprising:

determining additional partial products by using the new training data;

determining updated partial products for the updated model by using the stored partial products and the additional partial products; and

storing the updated partial products.

27. The system of claim 26, further comprising updating further the updated model, when additional new training data become available, by using the additional new training data and the updated partial products.

28. Logic for automated maintenance of a neural net model, the logic encoded in a medium and operable to:

train a neural net model with an initial set of training data, the neural net model having one or more original weights associated with the neural net model;

store partial products of the trained model, the partial products comprising a portion of the initial training set of training data and a new set of training data; and

update the trained model by using the stored partial products and new training data to compute adjusted weights for the updated model.

29. A computer data signal transmitted in one or more segments in a transmission medium which embodies instructions executable by a computer to:

train a neural net model with an initial set of training data, the neural net model having one or more original weights associated with the neural net model;

store partial products of the trained model, the partial products comprising a portion of the initial training set of training data and a new set of training data; and

update the trained model by using the stored partial products and new training data to compute adjusted weights for the updated model.