Hybrid multi-layer neural networks

• US 5,303,330 A
• Filed: 12/17/1992
• Issued: 04/12/1994
• Est. Priority Date: 06/03/1991
• Status: Expired due to Fees

• Current Assignee: Telcordia Technologies Incorporated (Telefonaktiebolaget LM Ericsson)
• Original Assignee: Bell Communications Research, Inc. (Telefonaktiebolaget LM Ericsson)
• Inventors: Gersho, Marvin H., Reiter, Randall R.
• Primary Examiner(s): MacDonald, Allen R.
• Application Number:
  US07/992,868
• Time in Patent Office: 481 Days
• Field of Search: 395/11, 395/22, 395/24
• US Class Current: 706/16
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06N 3/045 Combinations of networks

Abstract

A hybrid network 100 which combines a neural network of the self-organized type 110 with a plurality of neural networks of the supervised learning type 150,160,170 to successfully retrieve building address information from a database using imperfect textual retrieval keys. Generally, the self-organized type is a Kohonen Feature Map network, whereas each supervised learning type is a Back Propagation network. A user query 105 produces an activation response 111,112,113 from the self-organized network 110 and this response, along with a new query 151,161,171 derived from the original query 105, activates a selected one of the learning networks R₁,R₂,R_M to retrieve the requested information.

Claims

1. A neural network for retrieving information from a database, the neural network being trained to retrieve the contents of the database in response to a user input, the neural network comprising:

input means for receiving the user input and converting the user input to produce a first user query,

a self-organized neural network composed of neural nodes, said neural nodes being grouped into classes based on node activation resulting from training said self-organized neural network with reference to the contents of the database, said self-organized neural network including a plurality of outputs in correspondence to the number of classes,

said first user query serving as an input to each of said neural nodes, said self-organized neural network being responsive to said first user query so that one of said classes and, correspondingly, one of said outputs is activated as the result of the user input,

query means for receiving the user input and transforming the user input to produce a second user query different from but related to said first user query, and

a plurality of independently trained supervised learning networks, responsive to said query means and said self-organized neural network such that each of said outputs serves as an input to a corresponding one of said supervised learning networks, said learning networks being trained with reference to the contents of the database and arranged such that only one of said learning networks is activated in correspondence to said activation of said one of said classes, said activated one of said learning networks processing said second user query and emitting the retrieved information in response to the user input.

2. The network as recited in claim 1 wherein said self-organized neural network is a Kohonen Feature Map Network and each of said supervised learning networks is a Back Propagation Network.

3. The network as recited in claim 2 wherein the contents are building addresses and a subset of said Back Propagation Networks is trained with data formed from single character-shift variations in the contents supplied for training, said subset corresponding to the number of said classes indicative of said character-shift variations.

4. The network as recited in claim 2 wherein the contents are building addresses and a subset of said Back Propagation Networks is trained with data formed from random errors in the contents supplied for training, said subset corresponding to the number of said classes indicative of said random errors.

5. The network as recited in claim 2 wherein the contents are building addresses and a subset of said Back Propagation Networks are trained with data formed from single character-shift variations in the contents supplied for training, and the remainder of said Back Propagation Networks are trained with data formed from random errors in the contents supplied for training, said subset corresponding to the number of said classes indicative of said character-shift variations.

6. The network as recited in claim 2 wherein the contents are building addresses and said Kohonen Feature Map is trained with a subset of the contents of the database, said subset provided by the user to satisfy training criteria.

7. A neural network for retrieving information from a database, the neural network being trained to retrieve the contents of the database in response to a user query, the neural network comprising

means for formulating a first query and a second query based upon the user query,

a self-organized neural network, responsive to said means for formulating, composed of neural nodes and a plurality of outputs, said neural nodes being grouped into classes based on node activation resulting from training said self-organized neural network with reference to the contents of the database, wherein said first query serves as an input to each of said neural nodes and one of said classes is activated and, correspondingly, one of said outputs is activated as the result of said first query, and

a plurality of independently trained supervised learning networks, responsive to said means for formulating and said self-organized neural network such that each of said outputs serves as an input to a corresponding one of said supervised learning networks, said supervised learning networks being trained with reference to the contents of the database and arranged such that only one of said supervised learning networks is activated in correspondence to said activation of said one of said outputs, said activated one of said supervised learning networks processing said second query and emitting the retrieved information in response to said second query.

8. The network as recited in claim 7 wherein said self-organized neural network is a Kohonen Feature Map Network and each of said supervised learning networks is a Back Propagation Network.

9. The network as recited in claim 8 wherein the contents are building addresses and a subset of said Back Propagation Networks is trained with data formed from single character-shift variations in the contents supplied for training, said subset corresponding to the number of said classes indicative of said character-shift variations.

10. The network as recited in claim 8 wherein the contents are building addresses and a subset of said Back Propagation Networks is trained with data formed from random errors in the contents supplied for training, said subset corresponding to the number of said classes indicative of said random errors.

11. The network as recited in claim 8 wherein the contents are building addresses and a subset of said Back Propagation Networks are trained with data formed from single character-shift variations in the contents supplied for training, and the remainder of said Back Propagation Networks are trained with data formed from random errors in the contents supplied for training, said subset corresponding to the number of said classes indicative of said character-shift variations.

12. The network as recited in claim 8 wherein the contents are building addresses and said Kohonen Feature Map is trained with a subset of the contents of the database, said subset provided by the user to satisfy training criteria.

13. The network as recited in claim 7 wherein the user query contains alphanumeric information and wherein said means for formulating includes first means for selecting alphabetic data from the user query and second means for selecting alphanumeric data from the user query.

14. A method for retrieving information from a neural network, the neural network being trained to retrieve the contents of a database in response to a user query, the method comprising the steps of

implementing a self-organized neural network composed of neural nodes interconnected in a preselected pattern,

training said self-organized neural network with reference to the contents of the database,

grouping said neural nodes into classes based on node activation resulting from said training,

formulating a first query and a second query from the user query,

submitting said first query to each of said neural nodes and activating one of said classes based on the results of the first query,

implementing a plurality of supervised learning networks, one for each of said classes,

independently training each of said plurality of learning networks with reference to the contents of the database,

activating only one of said learning networks in correspondence to said activation of said one of said classes,

submitting said second query to said activated one of said learning networks, and

emitting the retrieved information in correspondence to an output of said activated one of said learning networks in response to said second query.