Probabilistic learning system

• US 4,599,693 A
• Filed: 01/16/1984
• Issued: 07/08/1986
• Est. Priority Date: 01/16/1984
• Status: Expired due to Fees

• Current Assignee: ITT Corporation (ITT, Inc.)
• Original Assignee: ITT Corporation (ITT, Inc.)
• Inventors: Denenberg, Jeffrey N.
• Primary Examiner(s): Smith, Jerry
• Assistant Examiner(s): Grossman, Jon D.
• Application Number:
  US06/571,222
• Time in Patent Office: 904 Days
• Field of Search: 364/200 MS File, 364/900 MS File, 364/513, 364/515, 364/725, 364/740, 340/146.3 MA, 340/146.3 H, 340/146.3 R, 340/146.3 Q, 382/15
• US Class Current: 706/12
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 10/70 using pattern recognition or machine learning optical pattern recognition or electronic computations therefor G06V10/88
• CPC Class Codes: G06V 10/85 Markov-related models; Markov random fields

Abstract

A probabilistic learning system of the type that receives sequential input data and outputs sequences of recognized patterns. The system includes an array of interconnected probabilistic learning elements of the type that receive sequences of objects and outputs sequences of recognized states, the array of learning elements being interconnected to have a number of input learning elements and a number of output learning elements. The input data is partitioned between the input learning elements of the array so that the partitioned input data forms objects provided to the learning elements in an overlapping and redundant manner. The output sequences of recognized states from the output learning elements are collected and combined to provide a sequence of recognized patterns as an output of the probabilistic learning system. The reliability of the learning system is enhanced due to the overlapping and redundant nature in which the input objects are processed through the system and the time required to perform the system task is reduced through the use of parallel processing through the array. Each element of the array provides a signal correspoding to a rating of confidence in the recognized states and this rating of confidence is fed back to the input of the element to cause the element to learn the recognized states when the rating of confidence exceeds a predetermined threshold level. The rating of confidence is also provided to the inputs of prior elements in the array to cause the prior elements to learn their recognized states when the rating of confidence exceeds the predetermined threshold.

Claims

1. A probabilistic learning system that receives sequential input data and outputs sequences of recognized patterns, comprising:

an array of interconnected probabilistic learning elements, each element sequentially receiving objects and outputting sequences of recognized states, each element including,

means for sequentially receiving objects,

means for storing,

said received objects,

sequences of received objects,

previously learned sequences of states,

states contained in said previously learned sequences of states, and

predetermined types of knowledge relating to,

said previously learned sequences of states,

said states contained in said previously learned sequences of states,

objects contained in said previously learned sequences of states, and

sequences of objects contained in said previously learned sequences of states, whereby current object information relating to said received objects and said sequences of received objects is stored as well as statisitcal information relating to previously learned sequences of states and said states, objects and sequences of objects contained in said previously learned sequences of states,

means for correlating said stored current object information with said stored statistical information for assigning probabilities to possible next states in the sequence of recognized states,

means, responsive to said probabilities of possible next states, for determining a most likely next state,

means, responsive to the stored current object information and statistical information, for providing a signal corresponding to the probability that a state has ended, and

means, responsive to said end of state probability signal, for outputting said most likely next state as a recognized next state in a recognized state sequence, said array of learning elements being interconnected to have a number of input learning elements and a number of output learning elements, the recognized state sequences of predetermined learning elements being combined to form objects to be received by other learning elements in the array;

means for receiving and partitioning the input data between the input learning elements of the array in an overlapping and redundant manner, whereby the partitioned input data become objects provided to the input learning elements; and

means for collecting and combining the recognized state sequences from the output learning elements of the array and for providing a sequence of recognized patterns as an output of the probabilistic learning system, whereby the reliability of the learning system is enhanced due to the overlapping and redundant nature in which the input data are processed through the system and the time required to perform the system task is reduced through the use of parallel processing through the array.

2. A probabilistic learning system as described in claim 1, wherein the array comprises eight probabilistic learning elements arranged in two columns of four elements.

3. A probabilistic learning system as described in claim 1, wherein the array is arranged in n+1 columns with each coulmn having Kⁿ learning elements, wherein n and K are each integers and K is greater than 1.

4. A probabilistic learning system as described in claim 1, wherein each element additionally comprises means for providing a signal corresponding to a rating of confidence in the recognized next state.

5. A probabilistic learning system as described in claim 4, wherein each element additionally comprises:

means for accumualting the ratings of confidence of the recognized states of the recognized state sequence; and

means, responsive to said accumulated rating of confidence, for causing said means for storing to store the recognized state sequence as a previously learned sequence of states, and to store the states in the recognized state sequence, the objects and sequences of objects in the recognized state sequence and the predetermined types of knowledge relating to the objects, states, sequence of objects and sequences of states forming said recognized state sequence as statistical information relating to previously learned sequences of states when the accumulated ratings of confidence exceed a predetermined threshold level.

6. A probabilistic learning system as described in claim 5, wherein the accumulated rating of confidence from a particular learning element is fed back to the responsive means of the element and to the responsive means of each learning element having an output connected to the receiving means of the particular element.

7. A probabilistic learning system as described in claim 5, additionally comprising means for correcting a recognized state sequence prior to initiating an external reinforcement signal.

8. A probabilistic learning system as described in claim 1, additionally comprising a learning supervision means, responsive to external reinforcement signals, for causing said means for storing in each element to store the recognized state sequence as a previously learned sequence of states, and to store the states in the recognized state sequence, the objects and sequences of objects in the recognized state sequence and the predetermined types of knowledge concerning the objects, states, sequence of objects and sequences of states forming said recognized state sequence as statistical information relating to previously learned sequences of states.

9. A probabilistic learning system as described in claim 8, additionally comprising means for correcting a recognized state sequence prior to initiating an external reinforcement signal.

10. A pattern recognition system, comprising:

transducer means for sensing a pattern to be recognized and outputting sequential data signals representative of the pattern to be recognized;

an array of interconnected probabilistic learning elements, each element sequentially receiving objects and outputting sequences of recognized states, each element including,

means for sequentially receiving objects,

means for storing,

said received objects,

sequences of received objects,

previously learned sequences of states,

states contained in said previously learned sequences of states, and

predetermined types of knowledge relating to,

said previously learned sequences of states,

said states contained in previously learned sequences of states,

objects contained in said previously learned sequences of states, and

sequences of objects contained in said previously learned sequences of states, whereby current object information relating to said received objects and sequences of received objects is stored as well as statistical information relating to previously learned sequences of states and said states, objects and sequences of objects contained in said previously learned sequences of states,

means for correlating said stored current object information with said stored statistical information for assigning probabilities to possible next states in the sequence of recognized states,

means, responsive to said probabilities of possible next states, for determining a most likely next state,

means, responsive to the stored current object information, and statistical information for providing a signal corresponding to the probability that a state has ended, and

means, responsive to said end of state probability signal, for outputting said most likely next state as a recognized next state in a recognized state sequence, said array of learning elements being interconnected to have a number of input learning elements and a number of output learning elements, the recognized state sequences of predetermined learning elements being combined to form objects to be received by other learning elements of the array;

means for receiving and partitioning data signals between the input learning elements of the array in an overlapping and redundant manner, whereby partitioned data become objects provided to the input learning elements; and

means for collecting and combining the recognized state sequences from the output learning elements of the array and for providing a sequence of recognized patterns as an output of the probabilistic learning system, whereby the reliability of the learning system is enhanced due to the overlapping and redundant nature in which the data are processed through the system and the time required to perform the system task is reduced through the use of parallel processing through the array.

11. A pattern recognition system as described in claim 10, wherein the pattern to be recognized is a character and the transducer means comprises an optical means for scanning said character.

12. A pattern recognition system as described in claim 10, wherein the transducer comprises a video scanner.

13. A pattern recognition system as described in claim 10, additionally comprising means for displaying the recognized pattern.

14. A pattern recognition system as described in claim 10, wherein the pattern to be recognized is sound and the transducer means is an acoustic sensor.

15. A pattern recognition system as described in claim 14, wherein the pattern is speech.

16. A pattern recognition system as described in claim 10, wherein the transducer comprises an acoustic transducer.

17. A pattern recognition system as described in claim 10, wherein the means for collecting and combining includes an error correcting decoding means.

18. A pattern recognition system as described in claim 17, wherein the error correcting decoding means implements a BCH decoding system.

19. A probabilistic learning system that receives sequential input data and outputs sequences of recognized patterns, comprising:

an array of interconnected probabilistic learning elements that receives sequences of objects and output sequences of recognized states, said array of learning elements being interconnected to have a number of input learning elements and a number of output learning elements the sequences of recognized states from predetermined learning elements being combined to form objects to be received by other learning elements of the array;

means for receiving and partitioning the input data between the input learning elements of the array in an overlapping and redundant manner, whereby the partitioned input data become objects provided to the input learning elements; and

means for collecting and combining the recognized state sequences from the output learning elements of the array and for providing a sequence of recognized patterns as an output of the probabilistic learning system, whereby the reliability of the learning system is enhanced due to the overlapping and redundant nature in which the input data is processed through the system and the time required to perform the system task is reduced through the use of parallel processing through the array.

20. A probabilistic learning system as described in claim 19, wherein each element additionally comprises means for providing a signal corresponding to a rating of confidence in the recognized state sequence outputted by the learning element.

21. A probabilistic learning system as described in claim 20, wherein each element additionally comprises means responsive to the signal corresponding to a rating of confidence to cause said element to learn the recognized state sequence when the rating of confidence exceeds a predetermined threshold level.

22. A probabilistic learning system as described in claim 21, wherein the signal corresponding to a rating of confidence from a particular learning element is fed back to the responsive means of the element and to the responsive means of each learning element having an output connected to a receiving means of the particular element.

23. A probabilistic learning system as described in claim 19, additionally comprising a learning supervision means responsive to external reinforcement signals to cause said elements to learn a recognized state sequence.