FACILITATING DEVICE CONTROL

• US 20160243701A1
• Filed: 02/23/2016
• Published: 08/25/2016
• Est. Priority Date: 02/23/2015
• Status: Active Grant

• Current Assignee: Ocado Innovation Limited (Ocado Group PLC)
• Original Assignee: Kindred Systems Inc (LifePoint Health, Inc.)
• Inventors: Gildert, Suzanne, Rose, George Samuel, Taylor, Graham William, Bergstra, James
• Granted Patent: US 10,216,177 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G05B 19/42 Recording and playback systems, i.e. in which the programme is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine
• CPC Class Codes: G05B 2219/36184 Record actions of human expert, teach by showing
• CPC Class Codes: G05B 2219/36442 Automatically teaching, teach by showing
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

A method of deriving autonomous control information involves receiving one or more sets of associated environment sensor information and device control instructions. Each set of associated environment sensor information and device control instructions includes environment sensor information representing an environment associated with an operator controllable device and associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information. The method also involves deriving autonomous control information from the one or more sets of associated environment sensor information and device control instructions, the autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, the autonomous device control signals configured to cause the autonomous device to take at least one autonomous action.

Claims

1. A method of deriving autonomous control information, the method comprising:

receiving one or more sets of associated environment sensor information and device control instructions, wherein each set of associated environment sensor information and device control instructions comprises;

environment sensor information representing an environment associated with an operator controllable device; and

associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information;

deriving autonomous control information from the one or more sets of associated environment sensor information and device control instructions, said autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, said autonomous device control signals configured to cause the autonomous device to take at least one autonomous action.

2. The method of claim 1 wherein the autonomous control information comprises a representation of an artificial intelligence model derived from the one or more sets of associated environment sensor information and device control instructions.

3. The method of claim 1 further comprising:

receiving first autonomous environment sensor information representing an environment associated with a first autonomous device;

deriving first autonomous device control instructions from the first autonomous environment sensor information using the autonomous control information;

producing first autonomous device control signals representing the first autonomous device control instructions for causing the first autonomous device to take at least one autonomous action.

4. The method of claim 1 wherein receiving the one or more sets of associated environment sensor information and device control instructions comprises:

receiving first environment sensor information representing a first environment of a first operator controllable device;

producing first operator interface control signals derived from the first environment sensor information for causing a first operator interface to simulate the first environment for a first operator interacting with the first operator interface;

receiving first operator interface sensor information representing at least one action taken by the first operator in response to the first operator interface simulating the first environment;

deriving first device control instructions from the first operator interface sensor information, said first device control instructions configured to cause the first operator controllable device to simulate the at least one action taken by the first operator;

wherein one of the one or more sets of associated environment sensor information and device control instructions comprises the first environment information and the first device control instructions.

5. The method of claim 1 wherein the environment sensor information for each of the one or more sets of associated environment sensor information and device control instructions comprises at least one of the following:

visual information;

audio information;

location information;

feature proximity information;

force feedback information;

chemical information;

temperature information;

kinematic information; and

orientation information.

6. The method of claim 1 wherein the device control instructions for each of the one or more sets of associated environment sensor information and device control instructions represents signals for causing the device to take at least one of the following actions:

gripping a feature;

pushing a feature;

pulling a feature;

propelling the device;

moving the device;

moving a feature; and

emitting communication signals.

7. A computer-readable medium having stored thereon codes which, when executed by at least one processor, cause the at least one processor to:

receive one or more sets of associated environment sensor information and device control instructions, wherein each set of associated environment sensor information and device control instructions comprises;

environment sensor information representing an environment associated with an operator controllable device; and

associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information;

derive autonomous control information from the one or more sets of associated environment sensor information and device control instructions, said autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, said autonomous device control signals configured to cause the autonomous device to take at least one autonomous action.

8. A system for deriving autonomous control information, the system comprising:

means for receiving one or more sets of associated environment sensor information and device control instructions, wherein each set of associated environment sensor information and device control instructions comprises;

environment sensor information representing an environment associated with an operator controllable device; and

associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information;

means for deriving autonomous control information from the one or more sets of associated environment sensor information and device control instructions, said autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, said autonomous device control signals configured to cause the autonomous device to take at least one autonomous action.

9. A system for deriving autonomous control information, the system comprising at least one processor configured to:

receive one or more sets of associated environment sensor information and device control instructions, wherein each set of associated environment sensor information and device control instructions comprises;

environment sensor information representing an environment associated with an operator controllable device; and

associated device control instructions configured to cause the operator controllable device to simulate at least one action taken by at least one operator experiencing a representation of the environment generated from the environment sensor information;

derive autonomous control information from the one or more sets of associated environment sensor information and device control instructions, said autonomous control information configured to facilitate generating autonomous device control signals from autonomous environment sensor information representing an environment associated with an autonomous device, said autonomous device control signals configured to cause the autonomous device to take at least one autonomous action.

10. An apparatus for sensing rotation of a first body part relative to a second body part, the apparatus comprising;

a first rotator configured to be coupled to the first body part to translate first body part rotation of the first body part relative to the second body part into first rotator rotation of the first rotator relative to the second body part, wherein said first rotator rotation has a rotation axis generally corresponding to a rotation axis of the first body part rotation;

a second rotator having;

a rotation translator coupled to the first rotator and configured to rotate in response to the rotation of the first rotator about a rotation translator axis offset from the first rotator axis;

a sensor coupled to the rotation translator and configured to be coupled to the second body part, said sensor being configured to sense the rotation of the rotation translator and produce at least one signal representing the rotation of the rotation translator.

11. The apparatus of claim 10 wherein the rotation translator axis is non parallel to the axis of the first rotator.

12. The apparatus of claim 10 wherein the sensor comprises an actuator configured to rotate the rotation translator.

13. A method of operation in a robotic system including an operator controllable device and a processor in communication with the operator controllable device, the method comprising:

receiving, by the processor, environment sensor information that represents an environment associated with the operator controllable device;

causing, by the processor, the display of a representation of the environment to a human operator;

receiving, by the processor, device control instructions which when executed by the operator controllable device, cause the operator controllable device to simulate at least one action taken by the human operator in response to the representation of the environment;

deriving, by the processor from the environment sensor information and the device control instructions, autonomous control instructions, which, when executed by the operator controllable device, cause the operator controllable device to take at least one autonomous action; and

producing, by the processor, at least one signal that represents the autonomous control instructions.

14. The method of claim 13 further comprising:

receiving, by the processor, internal sensor information that represents at least one internal state of the operator controllable device.

15. The method of claim 14, wherein deriving autonomous control instructions further comprises:

deriving the autonomous control instructions, by the processor, from the internal sensor information.

16. The method of claim 14, wherein the robotic system further includes an operator interface in communication with the operator controllable device and the processor, the method further comprising:

causing, by the processor, the display of a representation of the at least one internal state of the operator controllable device at the operator interface.

17. The method of claim 13, wherein the robotic system further includes an operator interface in communication with the operator controllable device and the processor and wherein causing, by the processor, the display of the representation of the environment comprises:

causing, by the at least one processor, the display of the representation of the environment at the operator interface.

18. The method of claim 13, wherein the robotic system further includes an operator interface in communication with the operator controllable device and the processor, the method further comprising:

receiving, by the processor, the device control instructions from the operator interface.

19. The method of claim 13, wherein deriving the autonomous control instructions comprises:

deriving, by the processor, a representation of an artificial intelligence model from the environment sensor information and the device control instructions.

20. The method of claim 19, wherein deriving the representation of the artificial intelligence model comprises:

deriving, by the processor, a plurality of probabilities and a plurality of autonomous actions, wherein a representative probability from the plurality of probabilities is associated with a representative action from the plurality of autonomous actions and wherein a representative probability varies with the environment sensor information.

21. The method of claim 19, wherein deriving the representation of the artificial intelligence model comprises:

deriving, by the processor, a first plurality of features from the environment sensor information and the device control instructions;

applying, by the processor, one or more non-linear transformations to the first plurality of features to derive information characterizing a high-level representation of the first plurality of features; and

deriving, by the processor, the autonomous control instructions from information that includes the information characterizing a high-level representation of the first plurality of features.

22. The method of claim 19, wherein deriving the representation of the artificial intelligence model comprises:

deriving, by the processor, the artificial intelligence model from a model that comprises at least one of the following;

a conditional deep belief network;

a conditional restricted Boltzmann machine;

a long short-term memory neural network;

a recurrent neural network;

a deep hierarchical learning system; and

a variational auto-encoder.

23. The method of claim 13 further comprising:

receiving, by the processor, information comprising at least one of the following;

visual information;

audio information;

location information;

proximity information;

force feedback information;

chemical information;

temperature information;

kinematic information; and

orientation information.

24. The method of claim 13, wherein the device control instructions represent instructions which when executed cause the operator controllable device to take at least one of the following actions:

gripping a feature of the environment;

pushing a feature of the environment;

pulling a feature of the environment;

propelling the device;

moving the device;

moving a feature of the environment; and

emitting communication signals.

25. The method of claim 24, wherein the autonomous control instructions represent instructions which when executed cause the operator controllable device to autonomously take at least one of the following actions:

gripping a feature of the environment;

pushing a feature of the environment;

pulling a feature of the environment;

propelling the device;

moving the device;

moving a feature of the environment; and

emitting communication signals.

26. The method of claim 13 further comprising:

in response to execution of the autonomous control instructions by the operator controllable device, causing the production of a plurality of autonomous device control signals that cause the operator controllable device to take at least one autonomous action.

27. The method of claim 13 further comprising:

sending, by the processor, the at least one signal that represents the autonomous control instructions across a network.

28. The method of claim 13 further comprising:

storing, by the processor, the autonomous control instructions on at least one tangible computer readable storage media.

29-65. -65. (canceled)