Apparatus and methods for removal of learned behaviors in robots

• US 9,579,790 B2
• Filed: 09/17/2014
• Issued: 02/28/2017
• Est. Priority Date: 09/17/2014
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Laurent, Patryk, Petre, Csaba, Izhikevich, Eugene M., Polonichko, Vadim
• Primary Examiner(s): Kiswanto, Nicholas
• Application Number:
  US14/489,373
• Publication Number:
  US 20160075017A1
• Time in Patent Office: 895 Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G05B 2219/40116 Learn by operator observation, symbiosis, show, watch
• CPC Class Codes: Y10S 901/05 Machine driven lead through

Abstract

Computerized appliances may be operated by users remotely. In one implementation, a learning controller apparatus may be operated to determine association between a user indication and an action by the appliance. The user indications, e.g., gestures, posture changes, audio signals may trigger an event associated with the controller. The event may be linked to a plurality of instructions configured to communicate a command to the appliance. The learning apparatus may receive sensory input conveying information about robot'"'"'s state and environment (context). The sensory input may be used to determine the user indications. During operation, upon determine the indication using sensory input, the controller may cause execution of the respective instructions in order to trigger action by the appliance. Device animation methodology may enable users to operate computerized appliances using gestures, voice commands, posture changes, and/or other customized control elements.

Claims

1. A method of operating a learning process for a computerized apparatus, the method comprising:

operating the learning process to determine a plurality of respective associations between a plurality of commands and a plurality of contexts;

storing the plurality of associations, one or more of the plurality of associations being configured to enable provision of one or more of the plurality of commands based on an occurrence of one or more of the plurality of contexts;

determining the one or more of the plurality of contexts via one or more sensory inputs;

determining a first user indication corresponding to the one or more sensory inputs;

when a given association between the one or more of the plurality of commands and the one or more of the plurality of contexts occurs, executing one or more tasks corresponding to the one or more of the plurality of commands;

responsive to a second user indication, updating the one or more of the plurality of associations, the updating the one or more of the plurality of associations being configured to reduce a probability of the provision of at least one of the plurality of commands based on another occurrence of the one or more of the plurality of contexts subsequent to the updating the one or more of the plurality of associations, the second user indication comprising an unlearn indication; and

when the unlearn indication is present, inhibiting execution of at least one of the one or more tasks.

2. The method of claim 1, wherein:

the determining the one or more of the plurality of contexts is based on an action by the user; and

further comprising;

receiving the plurality of commands on a wave transmission caused by the user; and

the determining the plurality of associations are based on a plurality of substantially simultaneous occurrences of the action and detection of the wave transmission.

3. The method of claim 2, wherein:

the operating the learning process is configured based on a look up table comprising a plurality of entries;

the operating the learning process to determine the plurality of respective associations comprises identifying an entry of the plurality of entries, the entry characterizing a number of the plurality of substantially simultaneous occurrences of the action and detection of the transmission; and

the updating the the one or more of the plurality of associations comprises reducing the number.

4. The method of claim 3, wherein:

the operating the learning process further comprises selecting an output;

wherein the output selection is configured based on a comparison of a given entry to another entry of the plurality of entries; and

the provision of at least one of the plurality of commands is based on the given entry being selected as the output.

5. The method of claim 4, wherein:

the operating the learning process comprises executing a winner takes all process; and

the selecting of the given entry as the output is configured based on an entry value of the given entry exceeding one or more individual values of the plurality of entries.

6. The method of claim 3, wherein:

the operating the learning process is configured based on a network of artificial neurons;

the one or more of the plurality of commands are based on an activity of an output neuron of the plurality of artificial neurons;

the association comprises an efficacy associated with the output neuron; and

the updating the the one or more of the plurality of associations comprises reducing the efficacy.

7. The method of claim 6, wherein:

the operating the learning process comprises executing a winner takes all process; and

when activated, the output neuron is configured based on the efficacy exceeding one or more efficacies of other output neurons of the plurality of neurons.

8. The method of claim 3, wherein:

the one or more of the plurality of commands are configured to cause execution of a task by the computerized apparatus.

9. The method of claim 8, wherein:

the operating the learning process comprises selecting one of a first or a second output;

the command is associated with the first output selected;

a null command is associated with the second output selected; and

the null command is characterized by an absence of task execution by the computerized apparatus.

10. The method of claim 9, wherein:

the updating the the one or more of the plurality of associations is configured to increase a probability of the second output being selected relative the first output.

11. The method of claim 1, wherein:

the computerized apparatus comprises a computerized appliance apparatus; and

the command is configured to cause the appliance to execute an action.

12. The method of claim 11, wherein:

the plurality of contexts are determined based on an analysis of sensory information, where the analysis comprises comparing a reference instance of the sensory information and a current instance of the sensory information, the reference instance preceding the current instance; and

the plurality of commands are characterized by a transmission comprising a plurality of pulses.

13. The method of claim 12, wherein:

the sensory information comprises a sequence of images provided by a camera;

the current instance of the sensory information comprises a current frame of the sequence of frames;

the context is characterized by a plurality of pixels of the current frame, the plurality of pixels representing a user gesture; and

the association is determined based on a plurality of simultaneous occurrences of the user gesture and detection of the transmission.

14. A home automation apparatus, comprising:

a camera component configured to provide a sequence of frames to a learning process;

a user interface device configured to receive an event indication and an undo indication from a user;

a communications apparatus configured to provide a command to a remote device; and

a processing component configured to operate the learning process and to detect an action of a user based on an analysis of the sequence of frames;

wherein the processing component is configured to;

based on the detection of the action of the user and the event indication and in an absence of a contemporaneous undo indication, update a configuration of the learning process to increase a probability of the command being communicated to the remote device based on a subsequent detection of the action of the user; and

based on the detection of the action of the user and the event indication and the contemporaneous undo indication, update the configuration of the learning process to decrease the probability of the command being communicated to the remote device based on the subsequent detection of the action of the user.

15. A home automation apparatus, comprising:

a sensor component configured to provide a sensory input to a learning process;

a user interface configured to receive a plurality of action indications and an undo indication from a user;

a communications apparatus configured to provide a command of a plurality of commands to a remote device; and

a processing component configured to operate the learning process and to determine one or more context instances in the sensory input;

wherein;

the learning process is operable to determine an association between a given context and a given action of the plurality of actions, the given action having a respective command of the plurality of commands associated therewith; and

the processing component is configured to, based on detection of the undo indication, update a configuration of the learning process to prevent transmission of at least one command of the plurality of commands during an interval subsequent to receipt of the undo indication and in the given context corresponding to the respective command.

16. The apparatus of claim 15, wherein the interval is selected based on a data parameter associated with sensory input provided to the learning process for determination of the context.

17. The apparatus of claim 15, wherein:

the user interface is configured to receive another undo indication from the user; and

the interval comprises a time between a receipt of the undo indication and a receipt of the another undo indication.

18. A method of removing learned behavior of a robotic apparatus, the method comprising:

determining a context in a sensory input, the context having one or more actions executable by the apparatus associated therewith based on a previously learned first association;

based on an occurrence of the context, causing the apparatus to execute the at least one of the one or more actions based on at least one control command transmitted to the apparatus;

determining an occurrence of an idle indication;

based on the occurrence of the idle indication, producing a second association between the context and a null action; and

based on the second association between the context and the null action, preventing transmission of at least one of the at least one control command to the apparatus;

wherein;

the previously learned first association is configured to trigger the transmission of the at least of the at least one control command to the apparatus;

and

the learned behavior comprises an execution of the one or more actions by the apparatus based on the occurrence of the context.

19. The method of claim 18, wherein the robotic apparatus comprises a home appliance configured to execute a household task.