Developmental learning machine and method

• US 6,353,814 B1
• Filed: 10/07/1998
• Issued: 03/05/2002
• Est. Priority Date: 10/08/1997
• Status: Expired due to Term

• Current Assignee: Michigan State University
• Original Assignee: Michigan State University
• Inventors: Weng, Juyang
• Primary Examiner(s): Davis, George B.
• Application Number:
  US09/167,751
• Time in Patent Office: 1,245 Days
• Field of Search: 706/12, 700/258
• US Class Current: 706/12
• CPC Class Codes: G05B 13/0265 the criterion being a learning criterion
• CPC Class Codes: G06F 18/21322 Rendering the within-class scatter matrix non-singular
• CPC Class Codes: G06F 18/231 Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06N 3/004 Artificial life, i.e. computing arrangements simulating life

Abstract

A machine and method capable of developing intelligent behavior from interaction with its environment directly using the machine'"'"'s sensors and effectors. The method described is independent of the type of sensors and actuators, or the tasks to be executed, and, therefore, provides a general purpose learner that learns while performing. It senses the world, recalls what is learned, judges what to do and acts according to what it has learned. The machine enables the machine to learn directly from sensory input streams while interacting with the environment, including human teachers. The presented approach enables the system to self-organize its internal representation, and uses a systematic way to automatically build multi-level representation using the Markov random process model. Reward and punishment are combined with sensor-based teaching to develop intelligent behavior.

Claims

1. A machine having developmental learning capability, said machine comprising:

one or more sensors for sensing an environment of the machine and generating one or more sensed signals in response thereto;

one or more effectors for acting on one or more objects in the environment;

a sensor based level builder having one or more level building elements, said sensor based level builder receiving as an input successive frames of said sensed signals one at a time, and generating action signals each having a relative probability, the sensor based level builder autonomously generating representations of tasks to be learned from said one or more sensed signals; and

a confidence accumulator for receiving said action signals and accumulating confidence of said action signals based on priority to determine most probable action signals, said confidence accumulator producing action control signals to control said one or more effectors in response to said determined most probable action signals, wherein said machine learns directly from continuous unsegmented sensory streams on-line while performing an operation and learns new tasks of unconstrained domains without a need for reprogramming, and wherein said learned new tasks include tasks that are not predetermined at the time of machine programming, said learned new tasks comprising at least two of autonomous navigation, speech recognition, and object manipulation.

2. The machine as defined in claim 1 further comprising an integration level builder for receiving state outputs from a plurality of said sensor based level builders, said integration level builder performing integration on said state outputs and producing one or more action signals for input to said confidence accumulator.

3. The machine as defined in claim 2, wherein levels in each integration level builder are automatically constructed such that a higher level corresponds to a longer temporal context.

4. The machine as defined in claim 2, wherein said confidence accumulator resolves ambiguity of multiple actions from a plurality of levels using probability normalization and expectation.

5. The machine as defined in claim 2, wherein said integration level builder comprises level building elements each having one or more states, wherein the states in each level building element has a vector representation that is not task-specific.

6. The machine as defined in claim 2, wherein said integration level builder includes higher level building elements that accept state output from lower level building elements that is not task-specific.

7. The machine as defined in claim 1, wherein levels in each sensor based level builder are automatically constructed such that a higher level corresponds to a longer temporal context.

8. The machine as defined in claim 1, wherein said confidence accumulator resolves ambiguity of multiple actions from a plurality of levels using probability normalization and expectation.

9. The machine as defined in claim 1, wherein said sensor based level builder comprises level building elements each having one or more states, wherein the states in each level building element has a vector representation that is not task-specific.

10. The machine as defined in claim 1, wherein said sensor based level builder includes higher level building elements that accept state output from lower level building elements that is not task-specific.

11. The machine as defined in claim 1 further comprising a plurality of sensor based level builders each associated with different ones of said sensors, wherein states in each-level builder have a vector representation that is not task-specific.

12. The machine as defined in claim 1, wherein said machine is capable of learning and performing concurrently.

13. The machine as defined in claim 1 further comprising a signal preprocessor for processing said sensed signals.

14. The machine as defined in claim 1, wherein said learned new tasks comprise at least all of autonomous navigation, speech recognition, and object manipulation.

15. A method of automatically developing learning capability with a machine, said method comprising the steps of:

sensing an environment with one or more sensors;

inputting successive frames of signal information into one or more sensor based level builders;

deriving action signals with said one or more sensor based level builders while no action is imposed from the environment, each of said action signals having a relative probability;

autonomously generating representations of tasks to be learned from said sensed environment;

updating memory from continuous unsegmented sensory streams on-line and complying with an action when the action is imposed from the environment, wherein new tasks of unconstrained domains are learned, and new tasks include tasks that are not predetermined at the time of machine programming, said learned new tasks comprising at least two of autonomous navigation, speech recognition, and object manipulation;

inputting said action signals to a confidence accumulator;

determining a most probable action based on said probability of said action signals received by said confidence accumulator; and

producing action control signals to control one or more effectors in response to said determined most probable action.

16. The method as defined in claim 15 further comprising the step of recording feedback reward signals from the environment.

17. The method as defined in claim 15 further comprising the step of producing state output signals from said one or more sensor based level builders.

18. The method as defined in claim 17 further comprising the steps of:

integrating a plurality of said state output signals from a plurality of sensor based level builders to produce one or more action signals that depend on multiple sensors; and

inputting said one or more action signals with corresponding confidence to said confidence accumulator.

19. The method as defined in claim 15 further comprising the step of computing the most confident action using probability normalization and expectation.

20. The method as defined in claim 15 further comprising the step of automatically forming a hierarchy of y-clusters from continuous vector outputs.

21. The method as defined in claim 20 further comprising the step of forming a hierarchy of x-clusters from said y-clusters.

22. The method as defined in claim 15 further comprising the step of deleting elements that are not visited often as defined by a memory trace update.

23. The method as defined in claim 15, wherein said machine is capable of leaning and performing concurrently.

24. The method as defined in claim 15, wherein said machine forms new states recursively from previous states using any one or more of uniform and non-uniform resolution reduction and resolution retention.

25. The method as defined in claim 15, wherein said machine forms new states as context of certain temporal extent, thus enabling the machine to learn directly from continuous unsegmented, sensory input streams.

26. The method as defined in claim 15, wherein said machine allows external action imposition and reward to be applied at any time, thus enabling learning and performance to occur in an arbitrary order and to occur concurrently.

27. The method as defined in claim 15, wherein said machine performs action-imposed learning in developmental learning.

28. The method as defined in claim 15, wherein said machine performs reinforcement learning in developmental learning.

29. The method as defined in claim 15, wherein said machine performs communicative learning in developmental learning.

30. The method as defined in claim 15, wherein said learned new tasks comprise at least all of autonomous navigation, speech recognition, and object manipulation.