Apparatus and methods for remotely controlling robotic devices

• US 9,849,588 B2
• Filed: 09/17/2014
• Issued: 12/26/2017
• Est. Priority Date: 09/17/2014
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Izhikevich, Eugene M., Laurent, Patryk, Petre, Csaba, Hylton, Todd, Polonichko, Vadim
• Primary Examiner(s): Bendidi, Rachid
• Application Number:
  US14/489,242
• Publication Number:
  US 20160075015A1
• Time in Patent Office: 1,196 Days
• Field of Search: 700253, 700245, 700250, 700255, 700257, 700258, 901 5, 901 3, 901 9
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G05B 2219/40411 Robot assists human in non-industrial environment like home or office
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 2012/2841 Wireless
• CPC Class Codes: H04L 2012/2849 Audio/video appliances
• CPC Class Codes: Y10S 901/05 Machine driven lead through

Abstract

Computerized appliances may be operated by users remotely. A learning controller apparatus may be operated to determine association between a user indication and an action by the appliance. The user indications, e.g., gestures, posture changes, audio signals may trigger an event associated with the controller. The event may be linked to a plurality of instructions configured to communicate a command to the appliance. The learning apparatus may receive sensory input conveying information about robot'"'"'s state and environment (context). The sensory input may be used to determine the user indications. During operation, upon determine the indication using sensory input, the controller may cause execution of the respective instructions in order to trigger action by the appliance. Device animation methodology may enable users to operate computerized appliances using gestures, voice commands, posture changes, and/or other customized control elements.

Claims

1. A method of providing transmissions to a remotely controlled apparatus by a learning controller, the method comprising:

configuring the learning controller to monitor one or more sensors configured to provide information related to a state of the remotely controlled apparatus disposed remotely from the learning controller;

enabling the learning controller to transmit a wireless command to the remotely controlled apparatus;

performing a plurality of training sessions, each of the plurality of training sessions comprising;

determining the state of the remotely controlled apparatus based on the information from the one or more sensors;

receiving an indication of an activation of a user command; and

incrementing an efficacy parameter corresponding to a contemporaneous occurrence of (i) the transmission of the wireless command to the remotely controlled apparatus and (ii) the receiving of the identification of the activation of the user command;

based on the incrementing of the efficacy parameter corresponding to the contemporaneous occurrence, developing an association between the wireless command and the information from the one or more sensors;

assigning a tag configured to identify the developed association between the wireless command and the information from the one or more sensors; and

causing the remotely controlled apparatus to execute an action based on the assigned tag.

2. The method of claim 1, wherein at least one of the one or more sensors comprises a video camera.

3. The method of claim 2, wherein transmission of the wireless command comprises an infrared signal transmission.

4. The method of claim 2, wherein transmission of the wireless command comprises a radio frequency signal transmission.

5. The method of claim 1, further comprising:

recording user commands and the state of the remotely controlled apparatus via the learning controller; and

transmitting the wireless command via the learning controller.

6. The method of claim 1, further comprising:

recording the state of the remotely controlled apparatus via a computing cloud entity; and

providing the wireless command via the computing cloud entity;

wherein the learning controller comprises the computing cloud entity in data communication with the remotely controlled apparatus.

7. The method of claim 1, further comprising:

receiving the wireless command from a computing cloud entity, the wireless command based on a profile configured to associate the state with the wireless command;

wherein the learning controller is disposed remote from the computing cloud entity; and

the profile is configured in accordance with a characteristic of the remotely controlled apparatus.

8. A method of remotely controlling a robotic device by a universal controller, the method comprising:

determining a set of instructions based on a user indication provided via the universal controller;

developing an association between the set of instructions and a context determined at least in part on information received by one or more sensors, the developing of association comprising a learning process, the learning process comprising;

detecting the user indication contemporaneously with the context; and

when the contemporaneous user indication and the context are detected, incrementing an entry in a look up table, the entry being associated with performing a task correlated to the user indication;

based on the incrementing of the entry in the look up table, and upon a subsequent occurrence of the context, loading the set of instructions within the universal controller, the set of instructions configured to control the robotic device;

monitoring a state of the robotic device;

determining a command according to the set of instructions; and

transmitting the command to the robotic device, the transmission causing the robotic device to execute the command to perform the task.

9. The method of claim 8, wherein the state of the robotic device comprises a position of the robotic device in an environment of the robotic device.

10. The method of claim 8, wherein the state of the robotic device comprises a parameter characterizing an event that is external to the robotic device.

11. The method of claim 10, wherein the parameter is configured to convey one or more of a position, a gesture, or a movement of a person.

12. The method of claim 8, wherein:

the robotic device is configured to perform the task within an environment comprising one or more objects; and

the task comprises the robotic device displacing at least one of the one or more objects within the environment.

13. The method of claim 8, wherein:

the robotic device is configured to perform the task within an environment comprising one or more objects characterized by corresponding object motion parameters;

the command execution is characterized by a first motion parameter of the robotic device; and

the command execution is configured based on a modification of the first motion parameter based on value associated with at least one of the corresponding object motion parameters.

14. The method of claim 8, where the set of instructions was previously configured for another robotic device or another task.

15. The method of claim 8, wherein the loading of the set of instructions further comprises automatically triggering the loading of the set of instructions based on a detection of a remote transmission or a pre-defined context in a sensory input.

16. The method of claim 8, wherein monitoring the state of the robotic device comprises receiving an input from a video camera.

17. The method of claim 16, wherein:

the robotic device is configured to perform the task within an environment comprising one or more objects; and

the monitored state comprises a position of the robotic device relative to a position of the one or more objects determined based on the input from the video camera.

18. The method of claim 8, wherein:

the robotic device is configured to perform the task within an environment comprising one or more objects characterized by corresponding object motion parameters; and

the task comprises removing one of the one or more objects from a subset of the environment.

19. The method of claim 8;

wherein;

the robotic device is configured to perform the task within an environment comprising a first area and a second area;

the monitoring comprises monitoring the state of the robotic device by the universal controller, the universal controller being in the first area, and where the state of the robotic device is monitored by a second universal controller in the second area; and

the transmission of the command is effectuated by one of the first or second universal controllers when the robotic device is within the first area.

20. The method of claim 19, wherein:

the robotic device is configured to receive the transmission of the command from the universal controller when the robotic device is within the first monitoring area; and

the robotic device is configured to receive a second transmission from the second universal controller when the robotic device is within the second monitoring area.

21. The method of claim 8, wherein:

the transmitting the command comprises transmitting an infrared frequency signal;

and the state of the robotic device is determined based on an output provided by the one or more sensor.

22. The method of claim 8, wherein:

the robotic device comprises a household appliance; and

the task comprises one or more of a cleaning task.