Neural network for classification of patterns with improved method and apparatus for ordering vectors

• US 5,729,662 A
• Filed: 06/07/1995
• Issued: 03/17/1998
• Est. Priority Date: 06/07/1995
• Status: Expired due to Fees

• Current Assignee: Rozmus J. Michael
• Original Assignee: Rozmus J. Michael
• Inventors: Rozmus, J. Michael
• Primary Examiner(s): Downs, Robert W.
• Application Number:
  US08/487,062
• Time in Patent Office: 1,014 Days
• Field of Search: 395/22, 395/23, 395/24, 382/159, 382/160
• US Class Current: 706/20
• CPC Class Codes: G06F 18/2137 based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Abstract

A type of neural network called a self-organizing map (SOM) is useful in pattern classification. The ability of the SOM to map the density of the input distribution is improved with two techniques. In the first technique, the SOM is improved by monitoring the frequency for which each node is the winning node, and splitting frequently winning nodes into two nodes, while eliminating infrequently winning nodes. Topological order is preserved by inserting a link between the preceding and following nodes so that such preceding and following nodes are now adjacent in the output index space. In the second technique, the SOM is trained by applying a weight correction to each node based on the frequencies of that node and its neighbors. If any of the adjacent nodes have a frequency greater than the frequency of the present node, then the weight vector of the present node is adjusted towards the highest-frequency neighboring node. The topological order of the nodes is preserved because the weight vector is moved along a line of connection from the present node to the highest-frequency adjacent node. This second technique is suitable for mapping to an index space of any dimension, while the first technique is practical only for a one-dimensional output space.

Claims

1. In a neural network for the classification of a plurality of patterns, wherein each of said plurality of patterns is represented by a respective multi-dimensional vector in a multi-dimensional space to form a training set of multi-dimensional input vectors corresponding to said plurality of input patterns, said neural network having a plurality of nodes in an ordered array corresponding to a topological ordering of said plurality of input patterns, a method for improving the ordering of said plurality of multi-dimensional input vectors, said method comprising:

determining the frequency that each node in said multi-dimensional space has been previously determined as the closest node to some portion of said plurality of input vectors forming said training set of multi-dimensional input vectors corresponding to said plurality of input patterns, to form a determined frequency for each of said plurality of nodes;

deleting a node if said determined frequency for such node is below a predetermined value, thereby defining a deleted node; and

connecting the node preceding said deleted node to the node following said deleted node along a line corresponding to said topological ordering of said plurality of input patterns to substantially maintain the topologic, further comprising;

splitting a given node into two nodes if said determined frequency of said given node is above a predetermined value, wherein said step of splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, comprises;

placing a first node along a first connecting line from said given node to a preceding adjacent node, said first connecting line corresponding to said topological ordering of said plurality of input patterns;

placing a second node along a second connecting line from said given node to a following adjacent node, said second connecting line corresponding to said topological ordering of said plurality of input patterns; and

deleting said given node.

2. A method in accordance with claim 1, wherein said step of splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

placing said first node at a position along said first connecting line from said given node to said preceeding adjacent node, said position of said first node being one third way along said first connecting line from said given node to preceeding adjacent node; and

placing said second node at a position along said second connecting line from said given node to said following adjacent node, said position of said second node being one third way along said second connecting line from said given node to said following adjacent node.

3. A method in accordance with claim 1, wherein said step of splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

placing said first node at a position along said first connecting line from said given node to said preceeding adjacent node, the distance from said position of said first node to the position of said given node being a monotonically increasing function of the ratio of said determined frequency of said preceding adjacent node to the said determined frequency of said given node, along said first connecting line between said given node and said preceeding adjacent node; and

placing said second node at a position along said second connecting line from said given node to said following adjacent node, the distance from said position of said second node to the position of said given node being a monotonically increasing function of the ratio of said determined frequency of said following adjacent node to the said determined frequency of said given node, along said second connecting line between said given node and said following adjacent node.

4. A method in accordance with claim 1, wherein said step of splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

placing said first node at a position along said first connecting line from said given node to said preceeding adjacent node, the distance from said position of said first node to the position of said given node being a monotonically increasing function of the result of said determined frequency of said preceding adjacent node minus the said determined frequency of said given node, along said first connecting line between said given node and said preceeding adjacent node; and

placing said second node at a position along said second connecting line from said given node to said following adjacent node, the distance from said position of said second node to the position of said given node being a monotonically increasing function of the result of said determined frequency of said following adjacent node minus the said determined frequency of said given node, along said second connecting line between said given node and said following adjacent node.

5. In a neural network for the classification of a plurality of patterns, wherein each of said plurality of patterns is represented by a respective multi-dimensional vector in a multi-dimensional space to form a training set of multi-dimensional input vectors corresponding to said plurality of input patterns, said neural network having a plurality of nodes in an ordered array corresponding to a topological ordering of said plurality of input patterns, a method for improving the ordering of said plurality of multi-dimensional input vectors, said method comprising:

determining the frequency that each node in said multi-dimensional space has been previously determined as the closest node to some portion of said plurality of input vectors forming said training set of multi-dimensional input vectors corresponding to said plurality of input patterns, to form a determined frequency for each of said plurality of nodes;

splitting a given node into two nodes if said determined frequency of said given node is above a predetermined value, wherein said step of splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, comprises;

placing a first node along a first connecting line from said given node to a preceding adjacent node, said first connecting line corresponding to said topological ordering of said plurality of input patterns;

placing a second node along a second connecting line from said given node to a following adjacent node, said second connecting line corresponding to said topological ordering of said plurality of input patterns; and

deleting said given node.

6. A method in accordance with claim 5, wherein said step of splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

placing said first node at a position along said first connecting line from said given node to said preceeding adjacent node, said position of said first node being one third way along said first connecting line from said given node to preceeding adjacent node; and

placing said second node at a position along said second connecting line from said given node to said following adjacent node, said position of said second node being one third way along said second connecting line from said given node to said following adjacent node.

7. A method in accordance with claim 5, wherein said step of splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

placing said first node at a position along said first connecting line from said given node to said preceeding adjacent node, the distance from said position of said first node to the position of said given node being a monotonically increasing function of the ratio of said determined frequency of said preceding adjacent node to the said determined frequency of said given node, along said first connecting line between said given node and said preceeding adjacent node; and

placing said second node at a position along said second connecting line from said given node to said following adjacent node, the distance from said position of said second node to the position of said given node being a monotonically increasing function of the ratio of said determined frequency of said following adjacent node to the said determined frequency of said given node, along said second connecting line between said given node and said following adjacent node.

8. A method in accordance with claim 5, wherein said step of splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

placing said first node at a position along said first connecting line from said given node to said preceeding adjacent node, the distance from said position of said first node to the position of said given node being a monotonically increasing function of the result of said determined frequency of said preceding adjacent node minus the said determined frequency of said given node, along said first connecting line between said given node and said preceeding adjacent node; and

placing said second node at a position along said second connecting line from said given node to said following adjacent node, the distance from said position of said second node to the position of said given node being a monotonically increasing function of the result of said determined frequency of said following adjacent node minus the said determined frequency of said given node, along said second connecting line between said given node and said following adjacent node.

9. In a neural network for the classification of a plurality of patterns, wherein each of said plurality of patterns is represented by a respective multi-dimensional vector in a multi-dimensional space to form a training set of multi-dimensional input vectors corresponding to said plurality of input patterns, said neural network having a plurality of nodes in an ordered array corresponding to a topological ordering of said plurality of input patterns, a method for improving the ordering of said plurality of multi-dimensional input vectors, said method comprising:

determining the frequency that each node in said multi-dimensional space has been previously determined as the closest node to some portion of said plurality of input vectors forming said training set of multi-dimensional input vectors corresponding to said plurality of input patterns, to form a determined frequency for each of said plurality of nodes; and

comparing the determined frequency of a given node to the determined frequency of an adjacent node; and

changing the position of said given node to form a moved node based on said determined frequency of said given node and said determined frequency of said adjacent node.

10. A method in accordance with claim 9, wherein said step of changing the position of said given node to form said moved node further comprises:

changing the position of said given node to form said moved node if the frequency of said adjacent node is greater than the frequency of said given node.

11. A method in accordance with claim 10, wherein said step of changing the position of said given node to form said moved node further comprises:

moving said given node towards said adjacent node along a connecting line from said given node to said adjacent node, said connecting line corresponding to said topological ordering of said plurality of input patterns.

12. A method in accordance with claim 10, wherein said step of changing the position of said given node to form said moved node further comprises:

moving said given node to a position along said connecting line from said given node to said adjacent node, the distance from said position of said moved node to the original position of said given node being a monotonically increasing function of the ratio of said determined frequency of said adjacent node to the said determined frequency of said given node, along said connecting line between said given node and said adjacent node.

13. A method in accordance with claim 10, wherein said step of changing the position of said given node to form said moved node further comprises:

moving said given node to a position along said connecting line from, said given node to said adjacent node, the distance from said position of said moved node to the original position of said given node being a monotonically increasing function of the result of said determined frequency of said adjacent node minus the said determined frequency of said given node.

14. In a neural network for the classification of a plurality of patterns, wherein each of said plurality of patterns is represented by a respective multi-dimensional vector in a multi-dimensional space to form a training set of multi-dimensional input vectors corresponding to said plurality of input patterns, said neural network having a plurality of nodes in an ordered array corresponding to a topological ordering of said plurality of input patterns, an apparatus for improving the ordering of said plurality of multi-dimensional input vectors, said apparatus comprising:

means for determining the frequency that each node in said multi-dimensional space has been previously determined as the closest node to some portion of said plurality of input vectors forming said training set of multi-dimensional input vectors corresponding to said plurality of input patterns, to form a determined frequency for each of said plurality of nodes;

means for deleting a node if said determined frequency for such node is below a predetermined value, thereby defining a deleted node;

means for connecting the node preceding said deleted node to the node following said deleted node along a line corresponding to said topological ordering of said plurality of input of input patterns to substantially maintain the topological ordering of said ordered array; and

means for splitting a given node into two nodes if said determined frequency of said given node is above a predetermined value, wherein said means for splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, comprises;

means for placing a first node along a first connecting line from said given node to a preceding adjacent node, said first connecting line corresponding to said topological ordering of said plurality of input patterns;

means for placing a second node along a second connecting line from said given node to a following adjacent node, said second connecting line corresponding to said topological ordering of said plurality of input patterns; and

means for deleting said given node.

15. An apparatus in accordance with claim 14, wherein said means for splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

means for placing said first node at a position along said first connecting line from said given node to said preceding adjacent node, said position of said first node being one third way along said first connecting line from said given node to preceding adjacent node; and

means for placing said second node at a position along said second connecting line from said given node to said following adjacent node, said position of said second node being one third way along said second connecting line from said given node to said following adjacent node.

16. An apparatus in accordance with claim 14, wherein said means for splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

means for placing said first node at a position along said first connecting line from said given node to said preceding adjacent node, the distance from said position of said first node to the position of said given node being a monotonically increasing function of the ratio of said determined frequency of said preceding adjacent node to the said determined frequency of said given node, along said first connecting line between said given node and said preceding adjacent node; and

means for placing said second node at a position along said second connecting line from said given node to said following adjacent node, the distance from said position of said second node to the position of said given node being a monotonically increasing function of the ratio of said determined frequency of said following adjacent node to the said determined frequency of said given node, along said second connecting line between said given node and said following adjacent node.

17. An apparatus in accordance with claim 14, wherein said means for splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

means for placing said first node at a position along said first connecting line from said given node to said preceding adjacent node, the distance from said position of said first node to the position of said given node being a monotonically increasing function of the result of said determined frequency of said preceding adjacent node minus the said determined frequency of said given node, along said first connecting line between said given node and said preceding adjacent node; and

means for placing said second node at a position along said second connecting line from said given node to said following adjacent node, the distance from said position of said second node to the position of said given node being a monotonically increasing function of the result of said determined frequency of said following adjacent node minus the said determined frequency of said given node, along said second connecting line between said given node and said following adjacent node.

18. In a neural network for the classification of a plurality of patterns, wherein each of said plurality of patterns is represented by a respective multi-dimensional vector in a multi-dimensional space to form a training set of multi-dimensional input vectors corresponding to said plurality of input patterns, said neural network having a plurality of nodes in an ordered array corresponding to a topological ordering of said plurality of input patterns, an apparatus for improving the ordering of said plurality of multi-dimensional input vectors, said apparatus comprising:

means for determining the frequency that each node in said multi-dimensional space has been previously determined as the closest node to some portion of said plurality of input vectors forming said training set of multi-dimensional input vectors corresponding to said plurality of input patterns, to form a determined frequency for each of said plurality of nodes;

means for splitting a given node into two nodes if said determined frequency of said given node is above a predetermined value, wherein said means for splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, comprises;

means for placing a first node along a first connecting line from said given node to a preceding adjacent node, said first connecting line corresponding to said topological ordering of said plurality of input patterns;

means for placing a second node along a second connecting line from said given node to a following adjacent node, said second connecting line corresponding to said topological ordering of said plurality of input patterns; and

means for deleting said given node.

19. An apparatus in accordance with claim 18, wherein said means for splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

means for placing said first node at a position along said first connecting line from said given node to said preceding adjacent node, said position of said first node being one third way along said first connecting line from said given node to preceding adjacent node; and

means for placing said second node at a position along said second connecting line from said given node to said following adjacent node, said position of said second node being one third way along said second connecting line from said given node to said following adjacent node.

20. An apparatus in accordance with claim 18, wherein said means for splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

means for placing said first node at a position along said first connecting line from said given node to said preceding adjacent node, the distance from said position of said first node to the position of said given node being a monotonically increasing function of the ratio of said determined frequency of said preceding adjacent node to the said determined frequency of said given node, along said first connecting line between said given node and said preceding adjacent node; and

means for placing said second node at a position along said second connecting line from said given node to said following adjacent node, the distance from said position of said second node to the position of said given node being a monotonically increasing function of the ratio of said determined frequency of said following adjacent node to the said determined frequency of said given node, along said second connecting line between said given node and said following adjacent node.

21. An apparatus in accordance with claim 18, wherein said means for splitting said given node into two nodes if said determined frequency of said given node is above said predetermined value, further comprises:

means for placing said first node at a position along said first connecting line from said given node to said preceding adjacent node, the distance from said position of said first node to the position of said given node being a monotonically increasing function of the result of said determined frequency of said preceding adjacent node minus the said determined frequency of said given node, along said first connecting line between said given node and said preceding adjacent node; and

means for placing said second node at a position along said second connecting line from said given node to said following adjacent node, the distance from said position of said second node to the position of said given node being a monotonically increasing function of the result of said determined frequency of said following adjacent node minus the said determined frequency of said given node, along said second connecting line between said given node and said following adjacent node.

22. In a neural network for the classification of a plurality of patterns, wherein each of said plurality of patterns is represented by a respective multi-dimensional vector in a multi-dimensional space to form a training set of multi-dimensional input vectors corresponding to said plurality of input patterns, said neural network having a plurality of nodes in an ordered array corresponding to a topological ordering of said plurality of input patterns, an apparatus for improving the ordering of said plurality of multi-dimensional input vectors, said apparatus comprising:

means for determining the frequency that each node in said multi-dimensional space has been previously determined as the closest node to some portion of said plurality of input vectors forming said training set of multi-dimensional input vectors corresponding to said plurality of input patterns, to form a determined frequency for each of said plurality of nodes;

means for comparing the determined frequency of a given node to the determined frequency of an adjacent node; and

means for changing the position of said given node to form a moved node based on said determined frequency of said given node and said determined frequency of said adjacent node.

23. An apparatus in accordance with claim 22, wherein said means for changing the position of said given node to form said moved node further comprises:

means for changing the position of said given node to form said moved node if the frequency of said adjacent node is greater than the frequency of said given node.

24. An apparatus in accordance with claim 23, wherein said means for changing the position of said given node to form said moved node further comprises:

means for moving said given node towards said adjacent node along a connecting line from said given node to said adjacent node, said connecting line corresponding to said topological ordering of said plurality of input patterns.

25. An apparatus in accordance with claim 23, wherein said means for changing the position of said given node to form said moved node further comprises:

means for moving said given node to a position along said connecting line from said given node to said adjacent node, the distance from said position of said moved node to the original position of said given node being a monotonically increasing function of the ratio of said determined frequency of said adjacent node to the said determined frequency of said given node, along said connecting line between said given node and said adjacent node.

26. An apparatus in accordance with claim 23, wherein said means for changing the position of said given node to form said moved node further comprises:

means for moving said given node to a position along said connecting line from said given node to said adjacent node, the distance from said position of said moved node to the original position of said given node being a monotonically increasing function of the result of said determined frequency of said adjacent node minus the said determined frequency of said given node.