System for building an artificial neural network

• US 6,049,793 A
• Filed: 11/13/1997
• Issued: 04/11/2000
• Est. Priority Date: 11/15/1996
• Status: Expired due to Fees

• Current Assignee: Kenichi Tomita
• Original Assignee: Kenichi Tomita
• Inventors: Tomita, Kenichi
• Primary Examiner(s): Hafiz, Tariq R.
• Assistant Examiner(s): PENDER JR, MICHAEL J
• Application Number:
  US08/970,050
• Time in Patent Office: 880 Days
• Field of Search: 706/20, 706/25, 706/15, 706/27, 706/17
• US Class Current: 706/17
• CPC Class Codes: G06F 18/24317 Piecewise classification, i.e. whereby each classification requires several discriminant rules
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/105 Shells for specifying net layout

Abstract

A system for building an artificial neural network is provided which precisely defines the network'"'"'s structure of artificial neurons, and non-iteratively determines the synapse-weights and hard limiter threshold of each artificial neuron of the network. The system includes a computer for analyzing input data, which represents patterns of different classes of signals, to generate one or more data points in two or three dimensions representative of the signals in each of the different classes. A distribution of the data points is visualized on a map on an output device coupled to the computer. The data points are clustered on the map into clusters in accordance with the classes associated with the data points, and the map is then partitioned into regions by defining linear boundaries between clusters. The artificial neural network is configured in accordance with the data points, clusters, boundaries, and regions, such that each boundary represents a different artificial neuron of the artificial neural network, and the geometric relationship of the regions on the map to the classes defines the logic connectivity of the artificial neurons. The synaptic weights and threshold of each artificial neuron in the network are graphically determined based on the data points of the map.

Claims

1. A system for building an artificial neural network of artificial neurons on a computer interactively with a user using input data representing patterns of different classes of signals comprising:

means for analyzing said input data to generate one or more data points in at least two dimensions representative of said signals in each of said different classes;

means for visualizing the distribution of said data points on a map in at least two dimensions using an output device coupled to said computer;

means for clustering said data points on said map provided by said visualizing means into clusters in accordance with the classes associated with said data points;

means for partitioning said map provided by said visualizing means into regions by defining boundaries between said clusters interactively with the user; and

means for configuring said artificial neural network in accordance with said data points, said cluster, said boundaries, and said regions, in which each of said boundaries defines a different one of the artificial neurons of said artificial neural network.

2. The system according to claim 1 wherein said analyzing means comprises means for generating vectors in accordance with said input data to represent said data points.

3. The system according to claim 2 wherein said visualizing means comprises means for plotting said vectors on said map to form said data points in at least two dimensions.

4. The system according to claim 3 wherein said plotting means comprises means for adjusting the distribution of said data points utilizing scaling factors in one or more dimensions of said map.

5. The system according to claim 1 wherein said visualizing means further comprises means for assigning to each of said visualized data points a unique point name which includes the class associated with said data point, and visualizing said unique point name on said map as display codes.

6. The system according to claim 1 wherein said clustering means further comprises:

means for forming clusters of data points on said map of the same class; and

means for assigning a unique cluster name for each cluster.

7. The system according to claim 1 wherein said boundaries are assigned unique names.

8. The system according to claim 1 wherein said boundaries are linear when said map is two-dimensional, and planar when said map is three-dimensional.

9. The system according to claim 1 wherein said partitioning means further comprises means for superimposing half-spaces over the map and numerically defining said boundaries in terms of said half-spaces.

10. The system according to claim 1 further comprising means for determining for each of said boundaries a weight vector and bias in accordance with said data points on said map.

11. The system according to claim 1 wherein each of said boundaries represents a different artificial neuron of said artificial neural network, and said geometric relationship of said clusters to said boundaries on said map defines the logic connectivity of said artificial neurons.

12. The system according to claim 11 wherein said configuring means further comprises means for establishing input nodes connected to one or more of said artificial neurons defined by said boundaries, and output nodes coupled to one or more of said artificial neurons.

13. The system according to claim 12 wherein said configured artificial neural network has at least one layer of connected artificial neurons.

14. The system according to claim 1 wherein a plurality of said maps are generated based on said input data, and each of said maps being operated upon by said visualizing means, said clustering means, said partitioning means, and said configuring means.

15. The system according to claim 1 further comprising means for reanalyzing said input data to generate data points for said visualizing means to visualize another one of said maps when said clustering means is not operative to provide separate ones of said clusters.

32. The system according to claim 1 wherein said clusters are graphically defined on said map provided by said visualizing means.

33. The system according to claim 1 wherein said boundaries are graphically defined on said map provided by said visualizing means between said clusters.

36. The system according to claim 1 further comprising means for non-iteratively determining the weights of the artificial neurons in accordance with the map.

37. The system according to claim 1 wherein said means for analyzing said input data is in accordance with the geometric relationships of the input data.

38. The system according to claim 1 wherein said input data represents a data matrix of columns, and said analyzing means comprises means for transforming the columns of the matrix into data points capable of being visualized on the map in at least two dimensions such that said data points representative of patterns of the same class can be clustered by said clustering mean.

16. A method for building an artificial neural network of artificial neurons using input data representing patterns of different classes of signals comprising the steps of:

analyzing said input data to generate one or more data points in at least two dimensions representative of said signals in each of said different classes;

visualizing said data points on a map in at least two dimensions;

clustering said data points on said map provided by said visualizing step into clusters in accordance with said different classes associated with said data points;

partitioning said map provided by said visualizing step into different regions by defining boundaries between said clusters; and

configuring said artificial neural network of artificial neurons in which said boundaries of said map define said artificial neurons, wherein at least said partitioning step is carried out with the aid of a user.

17. The method according to claim 16 wherein said analyzing step further comprises the step of generating vectors in accordance with said input data to represent said data points.

18. The method according to claim 17 wherein said visualizing step further comprises the step of plotting said vectors on said map to form said data points in at least two dimensions.

19. The method according to claim 18 wherein said plotting step further comprises the step of adjusting the distribution of said data points utilizing scaling factors in one or more dimensions of said map.

20. The method according to claim 16 wherein said visualizing step further comprises the steps of assigning to each of said visualized data points a unique point name which includes the class associated with said data points, and visualizing said unique point name on said map as display codes.

21. The method according to claim 16 wherein said clustering step further comprises the steps of:

forming clusters of data points on said map of the same class; and

assigning a unique cluster name for each cluster.

22. The method according to claim 16 wherein said boundaries are assigned unique names.

23. The method according to claim 16 wherein said boundaries are linear when said map is two-dimensional, and planar when said map is three-dimensional.

24. The method according to claim 16 wherein said partitioning step further comprises the step of superimposing half-spaces over the map and numerically defining said boundaries in terms of said half-spaces.

25. The method according to claim 16 further comprising the step of determining for each of said boundaries a weight vector and bias in accordance with said data points on said map.

26. The method according to claim 16 wherein each of said boundaries represents a different artificial neuron of said artificial neural network, and said geometric relationship of said regions on said map to said classes defines the logic connectivity of said artificial neurons.

27. The method according to claim 16 wherein said configuring step further comprises the step of establishing input nodes connected to one or more of said artificial neurons defined by said boundaries, and output nodes coupled to one or more of said artificial neurons.

28. The method according to claim 27 wherein said configured artificial neural network has at least one layer of connected artificial neurons.

29. The method according to claim 16 wherein a plurality of said maps are generated based on said input data, and each of said maps being operated upon by said visualizing step, said clustering step, said partitioning step, and said configuring step.

30. The method according to claim 16 further comprising the step of reanalyzing said data to generate data points for said visualizing step to visualize another one of said maps when said clustering step is not operative to provide separate ones of said clusters.

34. The method according to claim 16 wherein said clustering step is carried out graphically on said map provided by said visualizing step to locate said clusters on said map.

35. The method according to claim 16 wherein said partitioning step is carried out graphically on said map provided by said visualizing step to locate boundaries between said clusters.

39. The method according to claim 16 wherein said analyzing step analyzes said input data in accordance with the geometric relationships of the input data.

40. The method according to claim 16 wherein said clustering step is carried out before said partitioning step.

31. A method of building an artificial neural network using an input data set comprising the steps of:

creating at least one multi-dimensional map of the input data set by analyzing the geometrical relationships of said input data;

clustering the input data on the map according to different classifications of said input data;

partitioning the map into regions by defining linear boundaries to verify boundary margins of the input data corresponding to the classifications with the aid of a user;

forming artificial neurons corresponding to said linear boundaries;

configuring a network based on relationship between regions defined by said boundaries margins and the class indicators to provide a network as multiple segments of at least one layer; and

calculating synapse weights and thresholds of said artificial neurons based on the form of said linear boundaries.