SYNTHESIZED CASCADED PROCESSOR SYSTEM

• US 3,700,866 A
• Filed: 10/28/1970
• Issued: 10/24/1972
• Est. Priority Date: 10/28/1970
• Status: Expired due to Term

• Current Assignee: Fredrick J. Taylor
• Original Assignee: Fredrick J. Taylor
• Inventors: Taylor, Fredrick J.
• Primary Examiner(s): Botz, Eugene G.
• Application Number:
  US05/084,858
• Time in Patent Office: 727 Days
• Field of Search: 235/150.1 340/146.3T,172.5
• US Class Current: 700/2
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A single trainable nonlinear processor is trained with a single pass of training data through such processor. The single processor is then converted into a system of cascaded processors. In an execution mode of operation, each processor of the synthesized nonlinear cascaded processor system generates a probabilistic signal for the next processor in the cascade which is a best estimate for that processor of some desired response. The last processor in the cascade thereby provides a minimum entropy or minimum uncertainty actual output signal which most closely approximates a desired response for the total system to any input signal introduced into the system. The system is particularly useful for identification, classification, filtering, smoothing, prediction and modeling.

Claims

1. A synthesized cascaded processor system comprising:

a. a trainable nonlinear signal processor, and b. means for converting said trainable nonlinear signal processor into a plurality of executable nonlinear signal processors In cascade.

2. The system of claim 1 wherein the trainable nonlinear signal processor includes means for storing statistical data derived from applied input signals and said conversion means includes means for linking said executable nonlinear signal processors in cascade according to said stored statistical data.

3. The system of claim 2 including means for applying input signals to the system and means for applying corresponding desired response signals to the trainable nonlinear signal processor.

4. The system of claim 3 wherein the last executable nonlinear signal processor in the cascade includes means for generating at least one actual output signal, said actual output signal being the system'"'"''"'"'s best estimate of a desired response to an applied input signal.

5. The system of claim 1 wherein the trainable nonlinear signal processor is comprised of:

a. a plurality of storage registers, b. means for arranging and linking said storage registers into an array according to applied input signals, and c. means for accumulating and storing statistical data in one or more of said storage registers according to applied desired response signals associated with said applied input signals.

6. The system of claim 5 wherein said conversion means includes logic means for relinking said storage registers according to said stored statistical data, thereby providing said plurality of executable nonlinear signal processors in cascade.

7. The system of claim 1 including:

a. means for applying at least one input signal and corresponding desired response associated with such input signal to the system when it is operated in a training mode, and b. means for applying at least one input signal to the system when it is operated in an execution mode.

8. The system of claim 7 wherein the trainable nonlinear signal processor is comprised of:

a. a multi-level tree-arranged storage array having storage registers arranged in at least a root level and a leaf level, and b. means for defining a path through the levels of the tree-arranged storage array from said root level to said leaf level according to said applied input signals.

9. The system of claim 8 wherein said leaf level includes means for accumulating and storing the number of occurrences that a corresponding desired response signal is associated with each of said defined paths during training, thereby providing stored statistical data.

10. The system of claim 9 wherein said conversion means includes logic means for relinking said storage registers according to said stored statistical data, thereby providing said plurality of executable nonlinear signal processors in cascade.

11. The system of claim 9 wherein said conversion means includes:

a. means for deriving probability vectors for each level of the trainable nonlinear processor except the leaf level, b. means for separating each level of said tree-arranged storage array into an executable nonlinear signal processor, and c. logic means for relinking said storage registers in one of such levels to storage registers in the next separated level according to said probability vectors, thereby cascading said executable nonlinear processors.

12. The system of claim 11 wherein said logic means includes means, in each previous executable processor of the cascade, for directly addressing a filial set of registers in the next executable processor of the cascade.

13. The system of claim 8 including preprocessor means for encoding said at least one input signal into one or more key components, said key components providing means for defining said path through the levels of said tree-arranged storage array.

14. The system of claim 13 including means for sequentially comparing the key components of a present input signal with the key components of input signals which have previously defined paths through the levels of said tree-arranged storage array.

15. The system of claim 14 including means for defining a paRtial path through remaining levels of the tree-arranged storage array to said leaf level when a partial path has already been defined through one or more of the levels of the tree-arranged storage array.

16. The system of claim 15 wherein said leaf level includes means for accumulating and storing the number of occurrences that a corresponding desired response signal was associated with each of said defined paths during training, thereby providing stored statistical data.

17. The system of claim 16 wherein said conversion means includes logic means for relinking said storage registers according to said stored statistical data, thereby providing said plurality of executable nonlinear signal processors in cascade.

18. A synthesized cascaded processor system comprising:

a. a plurality of storage registers, b. means for arranging, linking and chaining said storage registers into a tree-structured matrix having nodes in at least a root level and a leaf level including means for defining paths through the levels of the tree-structured matrix from said root level to said leaf level according to applied input signals, c. means for accumulating and storing statistical data in one or more of said storage registers comprising nodes in said leaf level according to applied desired response signals associated with said applied input signals, d. means for combining the statistical data stored in registers of said leaf level nodes to derive probability vectors for each level of the tree-structured matrix except the leaf level, and e. means for merging the nodes of the tree-structured matrix to relink all nodes in the same level having the same probability vector to a common node in the next level of the tree-structured matrix.

19. The system of claim 18 wherein the merger means includes:

a. means for relinking all nodes in the same level having the same probability vector to a common node in the next level of the tree-structured matrix, b. means for chaining all nodes in said next level, previously linked to nodes in said same level having the same probability vector, to said common node, and c. means for eliminating duplicate nodes in said next level chained to said common node.

20. The system of claim 19 wherein said eliminating means includes:

a. means for combining the statistical data stored in the storage registers associated with said duplicate nodes, when said next level is the leaf level, and b. means for storing the combined statistics in storage registers of the first of such duplicate nodes in the leaf level.

21. The system of claim 19 wherein said eliminating means includes:

a. means for chaining all nodes in the level following said next level, linked to said duplicate nodes, to the node in the following level linked to the first of such duplicate nodes in said next level when said next level is not the leaf level, and b. means for eliminating duplicate nodes in said following level chained to the node in said following level which is linked to said first duplicate node in said next level.

22. The system of claim 19 including:

a. means for searching the nodes comprising each level of said relinked tree-structured matrix to find a path to a leaf level node defined according to an applied input signal thereby providing statistical data associated with such applied input signal, and b. means for generating from such provided statistical data an actual output signal, said actual output signal being the system'"'"''"'"'s best estimate of a desired response to such applied input signal.

23. The system of claim 22 including:

a. counter means for generating signals to sequentially operate the system, b. clock means for operating said counter means, and c. time pulse distributor logic circuit means for resetting said counter means and distributing said generated signals to the system.

24. A method of providing a trained and executable system of cascaded nonlinear signal processoRs comprising the steps of:

a. training a single trainable nonlinear signal processor, and b. converting the trained single nonlinear processor into a plurality of executable nonlinear signal processors in cascade.

25. The method of claim 24 wherein the training step includes storing statistical data derived from applied input signals and the conversion step includes linking said executable nonlinear signal processors in cascade according to said stored statistical data.

26. The method of claim 25 including the step of applying input signals and corresponding desired response signals to the trainable nonlinear signal processor.

27. The method of claim 24 wherein the training step includes:

a. arranging and linking a plurality of storage registers into an array according to applied input signals, and b. accumulating and storing statistical data in one or more of said storage registers according to applied desired response signals associated with said applied input signals.

28. The method of claim 27 wherein said conversion step includes relinking said storage registers according to said stored statistical data, thereby providing said trained and executable system of cascaded nonlinear signal processors.

29. The method of claim 24 including the step of applying at least one input signal and corresponding desired response associated with such input signal to the single trainable nonlinear processor.

30. The method of claim 29 wherein the training step includes:

a. arranging storage registers into a multi-level tree-arranged storage array having at least a root level and a leaf level, and b. defining a path through the levels of the tree-arranged storage array from said root level to said leaf level according to said applied input signals.

31. The method of claim 30 wherein the training step further includes the steps of accumulating and storing the number of occurrences that a corresponding desired response signal is associated with each of said defined paths, thereby providing stored statistical data.

32. The method of claim 31 wherein said conversion step includes relinking said storage registers according to said stored statistical data, thereby providing said trained and executable system of cascaded nonlinear signal processors.

33. The system of claim 31 wherein said conversion step includes:

a. deriving probability vectors for each level of the trainable nonlinear processor except the leaf level, and b. separating each level of the tree-arranged storage array into a trained and executable nonlinear signal processor, and c. relinking said storage registers in one of such levels to storage registers in the next separated level according to said derived probability vectors, thereby cascading said trained and executable nonlinear processors.

34. The method of claim 33 wherein the relinking includes storing in a register of said one level the address of the entry register of a filial set of registers in said next level, thereby providing direct addressing between said trained and executable nonlinear processors.

35. The method of claim 34 including the step of executing said trained and executable nonlinear processors.

36. The method of claim 35 wherein the execution step includes:

a. following a defined path through the levels of the tree-arranged storage array from said root level to said leaf level according to an applied input signal, and b. generating from said statistical data stored in said leaf level at least one actual output signal which is the system'"'"''"'"'s best estimate of a desired response to an applied input signal.

37. The method of claim 30 including the step of encoding said at least one input signal into a plurality of key components, said key components being utilized to define said path through the levels of said tree-arranged storage array.

38. The method of claim 37 wherein said training step includes sequentially comparing the key components of a preSent input signal with the key components of input signals which have previously defined paths through the levels of said tree-arranged storage array whereby all or part of a previously defined path is followed according to said present input signal.

39. The method of claim 38 wherein said training step further includes defining a partial path through remaining levels of the tree-arranged storage array to said leaf level when a previously defined path is partially followed through one or more of the levels of the tree-arranged storage array.

40. The method of claim 39 wherein said training step further includes accumulating and storing the number of occurrences that a corresponding desired response signal was associated with each defined path during training, thereby providing stored statistical data.

41. The method of claim 40 wherein said conversion step includes relinking said storage registers according to said stored statistical data, thereby providing said trained and executable system of cascaded nonlinear signal processors.

42. The method of claim 24 wherein said training step includes:

a. arranging, linking and chaining a plurality of storage registers into a tree-structured matrix having nodes in at least a root level and a leaf level, b. defining paths through the levels of the tree-structured matrix from said root level to said leaf level according to applied input signals, and c. accumulating and storing statistical data in one or more of said storage registers comprising nodes in said leaf level according to applied desired response signals associated with said applied input signals.

43. The method of claim 42 wherein the conversion step includes:

a. combining the statistical data stored in registers of said leaf level nodes to derive probability vectors for each level of the tree-structured matrix except the leaf level, and b. merging the nodes of the tree-structured matrix to relink all nodes in the same level having the same probability vector to a common node in the next level of the tree-structured matrix.

44. The method of claim 43 wherein the merger step includes:

a. relinking all nodes in the same level having the same probability vector to a common node in the next level of the tree-structured matrix, b. chaining all nodes in said next level, previously linked to nodes in said same level having the same probability vector, to said common node, and c. eliminating duplicate nodes in said next level chained to said common node.

45. The method of claim 44 wherein said elimination step includes:

a. combining statistical data stored in the storage registers associated with said duplicate nodes, when said next level is the leaf level, and b. storing the combined statistics in storage registers of the first of such duplicate nodes in the leaf level.

46. The method of claim 44 wherein said elimination step includes:

a. chaining all nodes in the level following said next level, linked to said duplicate nodes, to the node in the following level linked to the first of such duplicate nodes in said next level when said next level is not the leaf level, and b. eliminating duplicate nodes in said following level chained to the node in said following level which is linked to said first duplicate node in said next level.

47. The method of claim 44 including:

a. searching the nodes comprising each level of said relinked tree-structured matrix to find a path to a leaf level node defined according to an applied input signal, whereby statistical data associated with such applied input signal is provided, and b. generating from such provided statistical data an actual output signal which is the system'"'"''"'"'s best estimate of a desired response to such applied input signal.