Machine that learns what it actually does

• US 5,579,440 A
• Filed: 11/22/1993
• Issued: 11/26/1996
• Est. Priority Date: 11/22/1993
• Status: Expired due to Fees

• Current Assignee: Robert A Brown
• Original Assignee: Robert A Brown
• Inventors: Brown, Robert A.
• Primary Examiner(s): Davis, George B.
• Application Number:
  US08/155,587
• Time in Patent Office: 1,100 Days
• Field of Search: 395/22, 395/23, 395/24, 395/27, 395/164, 395/11, 395/20, 395/11, 365/49, 382/15
• US Class Current: 706/14
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

A self-learning machine 10 with one or more sensor(s) 110 and actuator(s) 135, and a plurality of memory cells 20 that select actions for the actuator(s) 135 to be carried out during specific sensing and action periods, the selection being determined by the stored values in the memory cells 20 and the conditions identified by its sensor(s) 110 in these sensing and action periods. The stored values are reduced in the selecting memory cells 20 when they select specific actions, and the stored values are increased in selected memory cells 20 according to what actions actually occur within these specific sensing and action periods.

Claims

1. A self-learning and self-organizing machine that operates in a sequence of sensing and action periods of predetermined duration, said machine comprising:

a. a timing means to herald a beginning and an end of a sensing and action period within said sequence of said sensing and action periods, and

b. at least one sensor means to measure a value of said sensor means at said beginning of said sensing and action period, and

c. at least one actuator means to make an attempt to produce a selected value of said actuator means during said sensing and action period, and

d. a feedback sensing means to measure an actual value of said actuator means after a predetermined delay from said attempt to produce said value of said actuator means, and

e. a self-learning memory means to make a record of a confidence level of said actual value of said actuator means occurring with said value of said sensor means in each said sensing and action period, and

f. said self-learning memory means to select said selected value of said actuator means, said selected value having a highest of said confidence level within said record, and

g. an actuator brake means to restrain said actuator means except when said self-learning memory means makes said attempt to produce said selected value of said actuator means, whereby

h. said self-learning machine attempts to produce a highest historical probability of said actual value of said actuator means occurring with said value of said sensor means in said sensing and action period, said self-learning machine also being self-organizing.

2. The self-learning machine of claim 1 operating more specifically with one sensor means and one actuator means, said self-learning machine comprising:

a. a submatrix means belonging to said sensor means and said actuator means, said submatrix means being made up of a plurality of memory cell means wherein a memory cell means of said plurality of memory cell means is functionally connected to a unique combination of a matrix input terminal means of a plurality of said matrix input terminal means and a matrix output terminal means of a plurality of said matrix output terminal means along with a matrix output co-terminal means of a plurality of said matrix output co-terminal means, wherein

b. said matrix input terminal means represents a unique value of said sensor means, and wherein an active matrix input terminal means of said plurality of said matrix input terminal means is made active by an extant value of said sensor means at a beginning of a sensing and action period, and wherein said matrix input terminal means includes a latching means triggered by said active matrix input terminal means wherein said latching means maintains an active state of said active matrix input terminal means for a remainder of said sensing and action period, and wherein said latching means includes a disconnect means that prohibits any said matrix input terminal means other than said active matrix input terminal means from being made active for said remainder of said sensing and action period, and wherein

c. said matrix output terminal means represents a value of said actuator means, and wherein a selected matrix output terminal means of said plurality of said matrix output terminal means is put into an active state by a forward-selecting memory cell means in said submatrix means, and wherein said matrix output terminal means includes a latching means to maintain said active state of said selected matrix output terminal means for a remainder of said sensing and action period, where said latching means also includes a disconnect means that prohibits any said matrix output terminal means other than said selected matrix output terminal means from being put in said active state for said remainder of said sensing and action period, and wherein

d. said selected matrix output terminal means causes said actuator means to make an attempt to assume a selected value of said actuator means after a predetermined action delay from said beginning of said sensing and action period, said attempt being made for a predetermined action period, and where

e. a voltage ramp means is connected to said plurality of memory cell means, each said memory cell means containing a voltage-sensitive switch means connected to said voltage ramp means, where said voltage-sensitive switch means closes an output contact means when a ramp signal from said voltage ramp means reaches a sufficient voltage, said sufficient voltage being determined by a sensitivity of said voltage-sensitive switch means, where

f. said memory cell means becomes said forward-selecting memory cell means when said output contact means belonging to said forward-selecting memory cell means are closed, when said forward-selecting memory cell means is connected to said active matrix input terminal means, and when no other said matrix output terminal means has been selected in said sensing and action period, where

g. said sensitivity of said forward-selecting memory cell means is higher than said sensitivity in any other said memory cell means connected to said matrix input terminal means made active by said extant value of said sensor means in said sensing and action period, and where

h. each said memory cell means contains a sensitivity reducing means capable of reducing said sensitivity of said voltage-sensitive switch means, said reduction being made in said forward-selecting memory cell means, said reduction being equal to said sensitivity of said voltage-sensitive switch means multiplied by a predetermined constant, said predetermined constant being in a range from one to a value greater than zero, and where

i. said matrix output co-terminal means of said plurality of matrix output co-terminal means represents a measured value of said actuator means, and wherein a feedback-selected matrix output co-terminal means of said plurality of matrix out-put co-terminal means is made active by an actual value of said actuator means after a predetermined feedback delay from when said actuator means attempts to assume said selected value, and wherein said matrix output co-terminal means also includes a disconnect means that prohibits any said matrix output co-terminal other than said feedback selected matrix output co-terminal means from being made active for a remainder of said sensing and action period, and

j. said memory cell means becomes a feedback-selected memory cell means when said matrix input terminal means belonging to said feedback-selected memory cell means is said active matrix input terminal means and said matrix output co-terminal means belonging to said feedback-selected memory cell means is said feedback-selected matrix output co-terminal means, and where

k. each said memory cell means also includes a sensitivity increasing means capable of increasing said sensitivity of said voltage-sensitive switch means, said increase being made in said feedback-selected memory cell means, said increase being equal to a difference between a maximum sensitivity of said voltage-sensitive switch means and said sensitivity of said voltage-sensitivity switch means, said difference being multiplied by said predetermined constant, and where

l. said self-learning machine also contains an actuator brake means to restrain said actuator means except when said selected matrix output terminal means causes said actuator means to make said attempt to assume said selected value of said actuator means, whereby

m. said self-learning machine attempts to produce said selected value of said actuator means with said extant value of said sensor means, said selected value of said actuator means having a highest confidence level of having occurred with said extant value of said sensor means in each said sensing and action period.

3. The self-learning machine of claim 2 operating upon a single sensor means and multiple actuator means in a sensing and action period, wherein each of an actuator means of said multiple actuator means is connected to a submatrix means belonging only to said actuator means, and wherein said single sensor means is connected to all said submatrix means, and wherein:

a. at a beginning of said sensing and action period, a forward-selecting memory cell means in each said submatrix means forward selects a selected matrix output terminal means, where

b. said selected matrix output terminal means causes a related said actuator means to make an attempt to assume a selected value corresponding to said selected matrix output terminal means, said attempt being made after a predetermined action delay from said beginning of said sensing and action period, said attempt being made for a predetermined action period, and where

c. said forward-selecting memory cell means are at an intersection of a first set of memory cell means and a second set of memory cell means, where

d. said first set of memory cell means consists of said memory cell means connected to an active matrix input terminal means, where said active matrix input terminal means is made active by an extant value of said single sensor means at said beginning of said sensing and action period, and where

e. said second set of memory cell means consists of memory cell means connected to said selected matrix output terminal means, and where

f. in a set of said submatrix means belonging to a one said actuator means, said memory cell means in said first set of memory cell means connected to an unselected matrix output terminal means form a plurality of unselecting sets of memory cell means, where

g. in said set of said submatrix means belonging to said one said actuator means, said forward-selecting memory cell means with a lowest sensitivity has a higher sensitivity than said memory cell means with a lowest sensitivity in any one of said plurality of unselecting sets, and where

h. a sensitivity in each said forward-selecting memory cell means is subjected to a reduction, said reduction being equal to said sensitivity multiplied by a predetermined constant, said predetermined constant being in a range from one to a value greater than zero, and

i. a feedback-selected memory cell means in each said submatrix means is back-selected, where said feedback-selected memory cell means is at an intersection of said first set of said memory cell means and a feedback set of said memory cell means, where

j. said feedback set of memory cell means consists of said memory cells means connected to an active matrix output co-terminal means belonging to a related said output terminal means, where said active matrix output co-terminal means is made active by an actual value of said related said actuator means after a predetermined feedback delay from when said actuator means attempts to assume said selected value, and

k. a sensitivity of said feedback-selected memory cell means in each said submatrix means is subjected to an increase, said increase being equal to a difference between a maximum possible sensitivity and said sensitivity, said difference being multiplied by said predetermined constant, and

l. said self-learning machine also contains an actuator brake means to restrain said actuator means except when said selected matrix output terminal means causes said actuator means to make said attempt to assume said selected value of said actuator means, whereby

m. said self-learning machine attempts to produce each said selected value of said actuator means with said extant value of said single sensor means in each said sensing and action period, each said selected value having a highest confidence level of having occurred with said extant value of said single sensor means in each said sensing and action period, said self-learning machine also being self-organizing.

4. The self-learning machine of claim 2 operating upon multiple sensor means and a single actuator means in a sensing and action period, wherein a sensor means of said multiple sensor means is connected to a submatrix means belonging only to said sensor means, and said single actuator means is connected to all said submatrix means, wherein:

a. at a beginning of said sensing and action period, a forward-selecting memory cell means in each said submatrix means forward selects a selected matrix output terminal means belonging to said single actuator means, where

b. said selected matrix output terminal means causes said single actuator means to make an attempt to assume a selected value corresponding to said selected matrix output terminal means, said attempt being made after a predetermined action delay from a beginning of said sensing and action period, said attempt being made for a predetermined action period, and where

c. said forward-selecting memory cell means are at an intersection of a first set of memory cell means and a second set of memory cell means, where

d. said first set of memory cell means consists of said memory cell means connected to an active matrix input terminal means, where said active matrix input terminal means is made active by an extant value of a related said sensor means, and where

e. said second set of memory cell means consists of memory cell means connected to said selected matrix output terminal means, and where

f. said memory cell means in said first set of memory cell means connected to an unselected matrix output terminal means form a plurality of unselecting sets of memory cell means, where

g. said forward-selecting memory cell means with a lowest sensitivity has a higher sensitivity than said memory cell means with a lowest sensitivity in any one of said plurality of unselecting sets, and where

h. a sensitivity in each said forward-selecting memory cell means is subjected to a reduction, said reduction being equal to said sensitivity multiplied by a predetermined constant, said predetermined constant being in a range from one to a value greater than zero, and

i. a feedback-selected memory cell means in each said submatrix means is back-selected, where said feedback-selected memory cell means is at an intersection of said first set of said memory cell means and a feedback set of said memory cell means, where

j. said feedback set of memory cell means consists of said memory cells means connected to an active matrix output co-terminal means belonging to a related said output terminal means, where said active matrix output co-terminal means is made active by an actual value of said single actuator means after a predetermined feedback delay from when said actuator means attempts to assume said selected value, and

k. a sensitivity of said feedback-selected memory cell means in each said submatrix means is subjected to an increase, said increase being equal to a difference between a maximum possible sensitivity and said sensitivity, said difference being multiplied by said predetermined constant, and

l. said self-learning machine also contains an actuator brake means to restrain said actuator means except when said selected matrix output terminal means causes said actuator means to make said attempt to assume said selected value of said actuator means, whereby

m. said self-learning machine attempts to produce said selected value of said single actuator means with each said extant value of said sensor means in said sensing and action period, said selected value having a highest confidence level of having occurred with each said extant value of said sensor means in each said sensing and action period, said self-learning machine also being self-organizing.

5. The self-learning machine of claim 2 operating upon multiple sensor means and multiple actuator means in a sensing and action period, wherein each combination of a sensor means of said multiple sensor means and an actuator means of said multiple actuator means is connected to a submatrix means uniquely, where said submatrix means include memory cell means, and wherein:

a. at a beginning of said sensing and action period, a forward-selecting memory cell means in each said submatrix means forward selects a selected matrix output terminal means, where

b. said selected matrix output terminal means causes a related said actuator means to make an attempt to assume a selected value corresponding to said selected matrix output terminal means, said attempt being made after a predetermined action delay from a beginning of said sensing and action period, said attempt being made for a predetermined action period, and where

c. said forward-selecting memory cell means are at an intersection of a first set of memory cell means and a second set of memory cell means, where

d. said first set of memory cell means consists of said memory cell means connected to an active matrix input terminal means, where said active matrix input terminal means is made active by an extant value of a related said sensor means, and where

e. said second set of memory cell means consists of memory cell means connected to said selected matrix output terminal means, and where

f. in a set of said submatrix means belonging to a one said actuator means, said memory cell means in said first set of memory cell means connected to an unselected matrix output terminal means form a plurality of unselecting sets of memory cell means, where

g. in said set of said submatrix means belonging to said one said actuator means, said forward-selecting memory cell means with a lowest sensitivity has a higher sensitivity than said memory cell means with a lowest sensitivity in any one of said plurality of unselecting sets, and where

h. a sensitivity in said forward-selecting memory cell means is subjected to a reduction, said reduction being equal to said sensitivity multiplied by a predetermined constant, said predetermined constant being in a range from one to a value greater than zero, and

i. a feedback-selected memory cell means in each said submatrix means is back-selected, where said feedback-selected memory cell means is at an intersection of said first set of said memory cell means and a feedback set of said memory cell means, where

j. said feedback set of memory cell means consists of said memory cells means connected to an active matrix output co-terminal means belonging to a related said output terminal means, where said active matrix output co-terminal means is made active by an actual value of said related said actuator means after a predetermined feedback delay from when said actuator means attempts to assume said selected value, and

k. a sensitivity of said feedback-selected memory cell means in each said sub-matrix means is subjected to an increase, said increase being equal to a difference between a maximum possible sensitivity and said sensitivity, said difference being multiplied by said predetermined constant,

l. said self-learning machine also contains an actuator brake means to restrain said actuator means except when said selected matrix output terminal means causes said actuator means to make said attempt to assume said selected value of said actuator means, whereby

m. said self-learning machine attempts to produce each said selected value of said actuator means with each said extant value of said sensor means in each said sensing and action period, said selected value having a highest confidence level of having occurred with said extant value of said sensor means in each said sensing and action period, said self-learning machine also being self-organizing.

6. The self-learning machine of claim 5, further including a digitizing means, forming a digitized self-learning machine, where in a sensing and action period:

a. an extant value of a sensor means is decomposed by an encoding means into a unique combination of values of a plurality of aggregate matrix input variable means, and

b. a selected value of an actuator means is synthesized by a decoding means from a unique combination of values of a plurality of aggregate matrix output variable means, and a measured value of said actuator means is decomposed by an encoding means into a unique combination of values of a plurality of aggregate matrix output co-variable means, and

c. a plurality of a forward-selecting memory cell means in a self-learning memory memos forward-select said unique combination of values of said plurality of aggregate matrix output variable means for said unique combination of values of said plurality of aggregate matrix input variable means, said unique combination of values of said plurality of aggregate matrix output variable means being representative of said unique combination of values of said plurality of said aggregate matrix output co-variable means, said unique combination of values of said plurality of said aggregate matrix output co-variable means having a highest confidence level of having occurred with said unique combination of values of said plurality of aggregate input variable means within said sensing and action period, whereby

d. said digitized self-learning machine attempts to produce said selected value of said actuator means, said selected value of said actuator means having occurred with said extant value of said sensor means with a highest historical probability in each said sensing and action period, said digitized self-learning machine requiring fewer said memory cell means than an undigitized self-learning machine, said digitized self-learning machine also being self-organizing.