System and method of pervasive developmental disorder interventions

• US 10,176,725 B2
• Filed: 08/29/2012
• Issued: 01/08/2019
• Est. Priority Date: 08/29/2011
• Status: Active Grant

• Current Assignee: Worcester Polytechnic Institute
• Original Assignee: Worcester Polytechnic Institute
• Inventors: Fischer, Gregory S., Su, Hao, Dickstein-Fischer, Laurie, Harrington, Kevin, Alexander, Elizabeth V.
• Primary Examiner(s): Gebremichael, Bruk
• Application Number:
  US13/598,066
• Publication Number:
  US 20130078600A1
• Time in Patent Office: 2,323 Days
• Field of Search: 434236, 434238
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26

Abstract

Methods and systems for observing/analyzing interactive behavior are presented. In one instance, the method for observing/analyzing interactive behavior includes interacting, using a robot, with a subject and obtaining data from interaction between the subject and the robot, the data from the data acquisition components, the data being used for diagnosis and/or charting progress. In one instance, the robot includes data acquisition components, interaction inducing components (such as, but not limited to, movable eyelids, movable appendages, sound generating components), a control component operatively connected to the interaction inducing components and a processing component operatively connected to the control component and the data acquisition components, the processing component being configured to obtain data from the data acquisition components, the data being used for diagnosis and/or charting progress. In one instance, the robot is integrated with a computer-aided system for diagnosis, monitoring, and therapy.

Claims

1. A robot configured to provide or assist in providing applied behavior analysis therapy and generate applied behavior analysis therapy data, the robot comprising:

a plurality of actuators;

a plurality of sensors including at least one camera;

interaction inducing components operatively connected to the plurality of actuators;

a control component operatively connected to the plurality of actuators;

a processing component operatively connected to the control component and to the plurality of sensors, the processing component comprising;

a processor; and

a non-transitory computer usable media having computer readable code embodied therein which, when executed by the processor, cause the processor to perform operations for operating the interaction inducing components to perform one of providing and assisting in providing applied behavior analysis therapy between the robot and a human subject, the operations comprising;

generating, as part of the applied behavior analysis therapy, an instructive command for the interaction inducing components, the instructive command selected from a plurality of predetermined commands designed to cause a behavioral response by the human subject;

transmitting the instructive command to at least a portion of the interaction inducing components and the plurality of actuators, to produce at least one of a motion, a gesture, and a sound;

receiving, via the plurality of sensors, and in response to the instructive command, a behavioral response of the human subject, the receiving comprising tracking gaze direction of the human subject via the camera, the tracking of the gaze direction comprising capturing images of a face of the human subject in a video feed and tracking eye motion of the human subject by generating coordinate frames that represent eyes of the human subject, wherein the behavioral response comprises a result generated from the tracking of the gaze direction;

generating, based on the behavioral response, including the result, a quantitative measure of the human subject,

wherein the quantitative measure is associated with at least one of cognitive development, behavioral development, emotional development, and social development; and

utilizing the quantitative measure to perform at least one of charting progress of the applied behavior analysis therapy and diagnosis of the human subject.

2. The robot of claim 1, wherein the computer readable code, when executed by the processor, causes the processor to perform additional operations comprising:

receiving, from an external system, supervisor instructions, wherein the supervisor instructions are used in the generating of the instructive command.

3. The robot of claim 1, wherein the computer readable code, when executed by the processor, causes the processor to perform additional operations comprising:

communicating with a display; and

,

wherein the behavioral response further comprises an interaction between the human subject and objects displayed on the display.

4. The robot of claim 3, further comprising:

a supervisory external system; and

wherein the computer readable code, when executed by the processor, causes the processor to perform additional operations comprising;

receiving, from the supervisory external system, at the processor, additional instructions for operating the interaction inducing components, wherein the additional instructions are based on a tailored therapeutic applied behavior analysis therapy program.

5. The robot of claim 3, wherein a game is displayed on the display;

and

wherein a game character displayed in the game represents at least one of the human subject or the robot.

6. The robot of claim 1, further comprising:

at least one appendage operatively connected to at least one of the plurality of actuators, the at least one appendage configured to provide appendage motion.

7. The robot of claim 1, further comprising a component configured to provide positive reinforcement to the human subject based on the behavioral response.

8. The robot of claim 1, wherein one or more of the plurality of actuators are coupled to an interaction component, wherein the interaction component is configured with adjustable stiffness to render tactile sensation during physical contact with the human subject.

9. The robot of claim 1, wherein said robot further comprises:

a base;

a rotatable structure disposed above the base;

a head structure operatively connected to the rotatable structure and configured to perform a tilt motion and a roll motion, the head structure comprising two eye-shaped components, wherein each eye-shaped component is configured to move with at least one degree of freedom and one of the eye-shaped components comprising the at least one camera;

at least one actuator in the plurality of actuators being operatively connected to the rotatable structure and the head structure.

10. The robot of claim 6, wherein at least one sensor in the plurality of sensors is disposed on the appendages and is operatively connected to the processing component.

11. The robot of claim 1, wherein the at least one camera is located in eye-shaped component of the robot.

12. The robot of claim 1, wherein the processing component also comprises a data storage component;

and

wherein the quantitative measure is used to provide a quantitative assessment of the human subject with respect to a disorder.

13. The robot of claim 3, wherein the display is a touch sensitive screen.

14. The robot of claim 1, wherein the behavioral response further comprises one or more of a change in facial expression, head pose, body posturing, and gestures.

15. The robot of claim 9, wherein the applied behavior analysis therapy guides social skills intervention for the human subject.

16. The robot of claim 1, wherein the tracking occurs over multiple frames of the video feed.

17. The robot of claim 1, wherein the computer readable code, when executed by the processor, causes the processor to perform additional operations comprising:

identifying treatment progress of the human subject based on the quantitative measure.

18. The robot of claim 1, wherein the quantitative measure is based on a psychological assessment criteria for diagnosing pervasive development disorders.

19. The robot of claim 1, further comprising:

changing robot behavior of the robot based on treatment progress of the human subject in the applied behavior analysis therapy.

20. The robot of claim 19, wherein the changing of the robot behavior comprises modifying the instructive command based on the behavioral response of the human subject.

21. The robot of claim 1, further comprising utilizing treatment progress of the human subject to update and modify the robot as the human subject'"'"'s level of interaction improves.

22. A method comprising:

identifying, using a plurality of sensors on a robot, a human subject interacting with the robot, the plurality of sensors comprising at least one camera;

generating, via a processor on the robot, and as part of applied behavior analysis therapy between the robot and the human subject, an instructive command selected from a plurality of predetermined commands designed to cause a behavioral response in the human subject;

transmitting the instructive command to at least one actuator in a plurality of actuators in the robot, such that the at least one actuator produces at least one of a motion, a gesture, and a sound;

receiving, via the plurality of sensors in the robot, and in response to the instructive command, a behavioral response of the human subject, the receiving comprising tracking gaze direction of the human subject via the at least one camera, the tracking of the gaze direction involving capturing images of the human subject'"'"'s face in a video feed, and thereby tracking the subject'"'"'s eye motion by generating coordinate frames that represent the subject'"'"'s eyes, wherein the behavioral response comprises a result generated from the tracking of the gaze direction;

generating, based on the behavioral response, including the result, a quantitative measurement related to at least one of cognitive development, behavioral development, emotional development, and social development of the human subject;

performing at least one of diagnosing a developmental disorder based on the quantitative measurement using a psychological assessment criteria and charting progress of the applied behavior analysis therapy using the quantitative measurement.

23. The method of claim 22, further comprising communicating with an external system, the external system being external to the robot.

24. The method of claim 23, wherein the external system is a display, and wherein the behavioral response of the human subject further comprises an interaction between the human subject and objects displayed on the display.

25. The method of claim 24, further comprising receiving, from an additional supervisory external system, supervisor instructions for engaging with the human subject via the external system, wherein the supervisor instructions are provided by a supervisor.

26. The method of claim 24, wherein a game is displayed on the display, wherein a game character in the game represents one of the human subject and the robot.

27. The method of claim 26, wherein the human subject operates the game through interactions with the robot.

28. The method of claim 26, wherein the human subject interacts with the game via a game controller.

29. The method of claim 23, wherein the communicating with the external system further comprises transmitting the behavioral interaction to the external system from the robot.

30. The method of claim 22, further comprising:

recording the behavioral response in a non-transitory computer-readable storage medium.

31. The method of claim 23, further comprising receiving, from an additional supervisory external system, therapy guidance used in generating the instructive command.

32. The method of claim 22, wherein the behavioral response further includes interactions between the robot, another robot, and the human subject.

33. The method of claim 22, wherein the behavioral response includes interactions between the human subject, one or more human beings, and the robot.

34. The method of claim 22, wherein the plurality of actuators are connected to a plurality of interaction inducing components.

35. The method of claim 22, wherein the applied behavior analysis therapy targets at least one disorder from the group consisting of general health, self-regulation, motor development, social interaction, communication, cognitive development, and daily living.

36. The method of claim 22, further comprising making an assessment based on quantitative measurement;

and

altering future instructive commands based on the assessment.

37. The method of claim 24, wherein the interaction between the human subject and the objects displayed on the display relates to the human subject selecting an object from a set of displayed objects.

38. The method of claim 23, wherein

wherein the external system is configured for reviewing recorded data and performing real-time processing the recorded data.

39. The method of claim 38, wherein the external system generates the quantitative measures, and the quantitative measure is related to at least one of cognitive development, behavioral development, emotional development, and social development of the human subject.

40. The method of claim 30, wherein the behavioral response is stored as one of audio, video, 3D imaging, and robot configuration, and the behavioral response is identified based on one of times or frame identifiers.

41. The method of claim 22, further comprising:

identifying treatment progress of the human subject based on the quantitative measurement.

42. A method comprising:

receiving, at an external system, sensor data, the sensor data from an interaction between a human subject and a robot, the interaction comprising;

the robot generating, via interaction inducing components, an action as part of an applied behavior analysis therapy, the action is generated via at least one actuator of a plurality of actuators of the robot, said action comprising one of a motion, a gesture, and a sound; and

the robot recording, via a plurality of sensors, including at least one camera, behavior of the human subject in response to the action, the recording including tracking of gaze direction of the human subject via the at least one camera, the tracking of the gaze direction comprising capturing images of a face of the human subject in a video feed and tracking eye motion of the human subject by generating coordinate frames representing eyes of the human subject to yield the sensor data and a result generated from the tracking of the gaze direction;

generating, based on the sensor data and the result, a quantitative measurement associated with at least one of cognitive development, behavioral development, emotional development, and social development of the human subject;

performing one of a diagnosis of the human subject based on a comparison of the quantitative measurement to psychological assessment criteria and charting progress of the applied behavior analysis therapy using the quantitative measurement.

43. The method of claim 42, further comprising identifying treatment progress of the human subject based on the quantitative measurement.

44. The method of claim 43, wherein previous data from previous interactions between the human subject and the robot is stored at the external system;

and

wherein the sensor data from the interaction between the human subject and the robot and the previous data from previous interactions are used to chart progress of the human subject.

45. The method of claim 44, wherein the sensor data is further used to modify behavior of the robot and modify a course of therapy of the human subject.

46. The method of claim 42, wherein the sensor data is used to perform at least one aspect of a diagnosis of a plurality of disorders.

47. The method of claim 42, wherein the robot is preconfigured with a plurality of actions to be used as part of the applied behavior analysis therapy, the plurality of actions designed to evaluate disorders within the human subject.

48. The method of claim 42, wherein the quantitative measurement is used in evaluating treatment progress of the human subject in one or more areas comprising speech, language, communication, sociability, cognitive awareness, health, physical and behavior.

49. The method of claim 48, wherein the evaluating is based on a standard psychological metric from the psychological assessment criteria.

50. The method of claim 48, wherein continuous assessment of treatment progress is used to modify the robot'"'"'s behavior as the human subject'"'"'s level of interaction changes.

51. The method of claim 42, wherein the psychological assessment criteria includes physiologic parameters.

52. A system comprising:

a plurality of actuators;

a plurality of sensors, including at least one camera;

interaction inducing components operatively connected to at least one actuator in the plurality of actuators;

a processor operatively connected to the plurality of actuators and to the plurality of sensors; and

a non-transitory computer usable media having computer readable code embodied therein which, when executed by the processor, causes the processor to perform operations comprising;

operating the interaction inducing components according to predetermined instructions as part of applied behavior analysis therapy for a human subject, such that the interaction inducing components generate one of a robotic motion and a sound;

recording, via the plurality of sensors, a behavioral reaction of the human subject in response to the operation of the interaction inducing components, the recording comprising tracking gaze direction of the human subject via the at least one camera, the tracking of the gaze direction comprising capturing images of a face of the human subject in a video feed and tracking eye motion of the human subject by generating coordinating frames that represent eyes of the human subject, the behavioral reaction comprising a result generated from the tracking of the gaze direction;

generating behavior analysis therapy data based upon the behavioral reaction, including the result, the behavior analysis therapy data comprising a measurement of at least one of cognitive development, behavioral development, emotional development, and social development of the human subject; and

using the behavior analysis therapy data for at least one of a diagnosis of a disorder of the human subject and a charting of the applied behavior analysis therapy.

53. The system of claim 52, further comprising a haptic device.

54. The system of claim 53, wherein the haptic device is remotely controlled by a supervisor.

55. The system of claim 52, wherein the predetermined instructions are augmented by supervisor instructions received from an external device.

56. The system of claim 55, wherein the supervisor instructions are provided from a puppet-like device.

57. The system of claim 52, wherein the computer readable code further instructs the processor to record movement patterns of the human subject after the operation of the interaction inducing components via the at least one camera.

58. The system of claim 52, wherein the behavior analysis therapy data is communicated to an external device after completion of a therapy session.

59. The system of claim 52, further comprising a display, wherein the display shows at least one of a video, a stereo camera view, a robot status, a human subject'"'"'s progress according to caregiver feedback, an automatic analysis and diagnosis of the human subject'"'"'s response, a behavior associated with the applied behavior analysis therapy, and a movement associated with the behavioral reaction.

60. The system of claim 59, wherein specific behaviors are identified automatically;

and wherein recorded video segment associated with the specific behaviors is shown on the display.