Autonomous robot using data captured from a living subject

• US 10,166,680 B2
• Filed: 07/30/2016
• Issued: 01/01/2019
• Est. Priority Date: 07/31/2015
• Status: Active Grant

• Current Assignee: Heinz Hemken
• Original Assignee: Heinz Hemken
• Inventors: Hemken, Heinz
• Primary Examiner(s): Caputo, Lisa
• Assistant Examiner(s): Cotey, Philip
• Application Number:
  US15/224,519
• Publication Number:
  US 20170028563A1
• Time in Patent Office: 885 Days
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 5/1121 Determining geometric values, e.g. centre of rotation or angular range of movement
• CPC Class Codes: A61B 5/45 For evaluating or diagnosing the musculoskeletal system or teeth A61B5/1036, A61B5/1074, A61B7/006 take precedence
• CPC Class Codes: A61B 5/6801 specially adapted to be attached to or worn on the body surface
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: B25J 11/0005 Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/162 Mobile manipulator, movable base with manipulator arm mounted on it
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: G05B 2219/33277 Distributed system with host as leader, host with multiple of agents
• CPC Class Codes: G05B 2219/39252 Autonomous distributed control, task distributed into each subsystem, task space
• CPC Class Codes: G05B 2219/40116 Learn by operator observation, symbiosis, show, watch
• CPC Class Codes: G05B 2219/40264 Human like, type robot arm
• CPC Class Codes: G05B 2219/40304 Modular structure
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/46 Sensing device

Abstract

Using various embodiments, an autonomous robot using data captured from a living subject are disclosed. In one embodiment, an autonomous robot is described comprising a robotic skeleton designed similar to that of a human skeleton to simulate similar movements as performed by living subjects. The movements of the robotic skeleton are resultant due to control signals received by effectors present near or on the robotic skeleton. The robot can be configured to receive sensor data transmitted from a sensor apparatus that periodically gathers the sensor data from a living subject. The robot can then process the sensor data to transmit control signals to the effectors to simulate the actions performed by the living subject and perform a predictive analysis to learn the capability of generating spontaneous and adaptive actions, resulting in an autonomous robot that can adapt to its surroundings.

Claims

1. An autonomous robot system comprising:

a robot comprising a robotic skeleton designed to simulate similar actions as performed by a living subject;

a first set of sensors, coupled to the robotic skeleton, wherein the first set of sensors periodically gathering a first set of sensor data from the first set of sensors to capture motions of the robotic skeleton wherein motions of the robotic skeleton are resultant to control signals received by effectors present near or on the robotic skeleton; and

a processing system coupled to the robot, the processing system configured to;

receive a second set of sensor data, the second set of sensor data generated from a sensor apparatus worn by the living subject,

process the second set of sensor data to transmit control signals to the effectors, wherein the control signals move the effectors to simulate actions performed by the living subject,

gather the first set of sensor data based on movement of the effectors, wherein the first set of sensor data generated by the effectors is substantially similar to the second set of sensor data generated from the sensor apparatus worn by the living subject, and

iteratively, perform a predictive analysis, on the first set of sensor data, to learn a capability of generating actions that are spontaneous and adaptive to an immediate environment of the robot, as well as its ongoing interactions with living or inanimate elements that surround it, wherein the processing system further receives and processes sensor data resulting from the spontaneous and adaptive actions of the robot.

2. The system of claim 1, wherein the control signals transmitted to the effectors, for a next time interval, can be represented by a conversion function represented by:

y=ƒ

(x)

wherein y represents a vector containing all signals that need to be sent to the effectors of the robot or audio generation devices in the next time interval to make the robot simulate similar actions as performed by the living subject, and x represents a vector that includes sensor readings that are produced at any given moment by the sensor apparatus, and wherein ƒ

( ) represents a machine learning algorithm function that can receive sensor data and perform computations on the sensor data to yield the control signals to send to the effectors of the robot during the next time interval.

3. The system of claim 1, wherein the predictive analysis is derived from at least a first neural network that uses layers of non-linear hidden units between its inputs and its outputs.

4. The system of claim 3, wherein each unit is assigned a weight during an initial training stage.

5. The system of claim 4, wherein during a learning stage the robot adapts the weight on incoming connections of hidden units to learn feature detectors that enable it to predict a correct output when given an input vector.

6. The system of claim 1, wherein the predictive analysis is derived from at least a first and second neural network, wherein the at least first neural network receives a feed from the first set of sensor data and wherein at least the second neural network receives a feed related to the immediate environment of the robot.

7. The system of claim 1, wherein the robotic skeleton is at least partly formed using universal joint segments.

8. A method to train an autonomous robot using data gathered from a living subject comprising:

receiving a first set of sensor data, by a processing system coupled to the autonomous robot, the first set of sensor data received from a sensor apparatus, wherein the sensor apparatus periodically gathers the first set of sensor data from the living subject;

processing the first set of sensor data and transmitting control signals to effectors of the autonomous robot to simulate actions performed by the living subject; and

iteratively, performing a predictive analysis to learn a capability of generating actions that are spontaneous and adaptive to an immediate environment of the autonomous robot, as well as its ongoing interactions with living or inanimate elements that surround it, wherein the processing system further receives and processes a second set of sensor data resulting from the spontaneous and adaptive actions of the autonomous robot;

wherein the autonomous robot includes a robotic skeleton to simulate similar actions as performed by the living subject, wherein motions of the robotic skeleton are resultant to control signals received by the effectors of the autonomous robot, the effectors present near or on the robotic skeleton.

9. The method of claim 8, wherein the control signals transmitted to the effectors, for a next time interval, can be represented by a conversion function represented by:

y=ƒ

(x)

wherein y represents a vector containing all signals that need to be sent to the effectors of the autonomous robot or audio generation devices in the next time interval to make the autonomous robot simulate similar actions as performed by the living subject, and x represents a vector that includes sensor readings that are produced at any given moment by the sensor apparatus, and wherein ƒ

( ) represents a machine learning algorithm function that can receive sensor data and perform computations on the sensor data to yield the control signals to send to the effectors of the autonomous robot during the next time interval.

10. The method of claim 8, wherein the predictive analysis is derived from at least a first neural network that uses layers of non-linear hidden units between its inputs and its outputs.

11. The method of claim 10, wherein each unit is assigned a weight during an initial training stage.

12. The method of claim 11, wherein during a learning stage the autonomous robot adapts the weight on incoming connections of hidden units to learn feature detectors that enable it to predict a correct output when given an input vector.

13. The method of claim 8, wherein the predictive analysis is derived from at least a first and second neural network, wherein the at least first neural network receives a feed from the second set of sensor data and wherein at least the second neural network receives a feed related to the immediate environment of the autonomous robot.

14. The method of claim 8, wherein the robotic skeleton is at least partly formed using universal joint segments.

15. A non-transitory computer readable medium comprising instructions which when executed by a processing system coupled to an autonomous robot performs a method to train the autonomous robot using data gathered from a living subject, the method comprising:

receiving a first set of sensor data, the first set of sensor data received from a sensor apparatus, wherein the sensor apparatus periodically gathers the first set of sensor data from the living subject;

processing the first set of sensor data and transmitting control signals to effectors of the autonomous robot to simulate actions performed by the living subject; and

iteratively, performing a predictive analysis to learn a capability of generating actions that are spontaneous and adaptive to an immediate environment of the autonomous robot, as well as its ongoing interactions with living or inanimate elements that surround it, wherein the processing system further receives and processes a second set of sensor data resulting from the spontaneous and adaptive actions of the autonomous robot;

wherein the autonomous robot includes a robotic skeleton to simulate similar actions as performed by a living subject, wherein motions of the robotic skeleton are resultant to control signals received by the effectors of the autonomous robot, the effectors present near or on the robotic skeleton.

16. The non-transitory computer readable medium of claim 15, wherein the control signals transmitted to the effectors, for a next time interval, can be represented by a conversion function represented by:

y=ƒ

(x)

wherein y represents a vector containing all signals that need to be sent to the effectors of the autonomous robot or audio generation devices in the next time interval to make the autonomous robot simulate similar movements as performed by the living subject, and x represents a vector that includes sensor readings that are produced at any given moment by the sensor apparatus, and wherein ƒ

( ) represents a machine learning algorithm function that can receive sensor data and perform computations on the sensor data to yield the control signals to send to the effectors of the autonomous robot during the next time interval.

17. The non-transitory computer readable medium of claim 15, wherein the predictive analysis is derived from at least a first neural network that uses layers of non-linear hidden units between its inputs and its outputs.

18. The non-transitory computer readable medium of claim 17, wherein each unit is assigned a weight during an initial training stage.

19. The non-transitory computer readable medium of claim 18, wherein during a learning stage the autonomous robot adapts the weight on incoming connections of hidden units to learn feature detectors that enable it to predict a correct output when given an input vector.

20. The non-transitory computer readable medium of claim 15, wherein the predictive analysis is derived from at least a first and second neural network, wherein the at least first neural network receives a feed from the second set of sensor data and wherein at least the second neural network receives a feed related to the immediate environment of the autonomous robot.