System and method for automated monitoring, recognizing, supporting, and responding to the behavior of an actor

• US 20040030531A1
• Filed: 01/10/2003
• Published: 02/12/2004
• Est. Priority Date: 03/28/2002
• Status: Abandoned Application

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Riley, Victor A., Toms, David C., King, Lawrence A., Dewing, Wende L., Miller, Christopher A., Whitlow, Stephen D., Whillock, Rand P., Phelps, John A., Geib, Christopher W., Allen, John A., Wu, Peggy, Haigh, Karen Z., Richardson, Rose Mae M., Metz, Stephen V.
• Application Number:
  US10/341,335
• Publication Number:
  US 20040030531A1
• Time in Patent Office: Days
• US Class Current: 702/182
• CPC Class Codes: A61B 2503/08 Elderly
• CPC Class Codes: A61B 2560/0242 adapted to measure environmental factors, e.g. temperature, pollution
• CPC Class Codes: A61B 5/0002 Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network A61B5/07 takes precedence; transmission systems for measured values G08C; transmission H04B; transmission of digital information H04L; wireless communication networks H04W
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/021 Measuring pressure in heart or blood vessels A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/024 Detecting, measuring or recording pulse rate or heart rate A61B5/0205, A61B5/021 take precedence
• CPC Class Codes: A61B 5/1112 Global tracking of patients, e.g. by using GPS
• CPC Class Codes: A61B 5/1117 Fall detection
• CPC Class Codes: A61B 5/14532 for measuring glucose, e.g. by tissue impedance measurement
• CPC Class Codes: A61B 5/4088 Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G08B 21/0423 detecting deviation from an expected pattern of behaviour or schedule
• CPC Class Codes: G08B 21/0469 Presence detectors to detect unsafe condition, e.g. infrared sensor, microphone G08B21/0476 takes precedence
• CPC Class Codes: G08B 21/0476 Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: G08B 21/0484 Arrangements monitoring consumption of a utility or use of an appliance which consumes a utility to detect unsafe condition, e.g. metering of water, gas or electricity, use of taps, toilet flush, gas stove or electric kettle
• CPC Class Codes: G16H 20/00 ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

An automated system and method for monitoring and supporting and actor in an environment, such as a daily living environment. The system includes at least one sensor, at least one effector and a controller adapted to provide monitoring, situation assessment, response planning, and plan execution functions. In one preferred embodiment, the controller provides a layered architecture allowing multiple modules to interact and perform the desired monitoring and support functions.

Claims

1. An automated monitoring and support system for an actor in an environment, the system comprising:

a sensor;

an effector; and

a controller for receiving information from the sensor to monitor at least one of the actor and the environment and for controlling operation of the effector based upon the monitored information, the controller including;

a situation assessor for determining a current situation based upon the sensor information,

a response planner for automatically generating an appropriate current response plan to the current situation as determined by the situation assessor,

a plan executor for prompting the effector to execute the generated current response plan.

2. The system of claim 1, wherein the system includes a plurality of different sensors each providing sensor information to the controller, and further wherein the situation assessor is adapted to selectively evaluate all sensor information.

3. The system of claim 2, wherein the situation assessor is further adapted to aggregate information from at least two of the sensors to designate a single event as occurring.

4. The system of claim 3, wherein the response planner is further adapted to rely upon the designated single event in generating an appropriate response.

5. The system of claim 1, wherein the situation assessor is further adapted to determine the current situation based upon at least one item selected from the group consisting of an activity of the actor, an intended activity of the actor, an inactivity of the actor, the status of the actor, a future status of the actor, a status of the environment, and a future status of the environment.

6. The system of claim 5, wherein the situation assessor is further adapted to consider periods of user inactivity in determining the current situation.

7. The system of claim 1, wherein the response planner is further adapted to coordinate a plurality of possible response plans.

8. The system of claim 7, wherein the response planner is further adapted to prioritize the plurality of possible response plans.

9. The system of claim 1, wherein the system includes a plurality of effectors, and further wherein the response planner is further adapted to select one or more of the plurality of effectors to execute the current response plan.

10. The system of claim 1, wherein the response planner is further adapted to consider previous re-actions of the actor to previous response plans in generating the current response plan.

11. The system of claim 1, wherein the situation assessor and the response planner are further adapted to monitor a response of the actor to the current response plan following execution by the plan executor.

12. The system of claim 1, wherein the system includes a plurality of effectors, and further wherein the plan executor is further adapted to control operations of each of the plurality of effectors.

13. The system of claim 12, wherein the plan executor is adapted to coordinate operation of the plurality of effectors.

14. The system of claim 1, wherein the controller further includes a machine learning device adapted to establish a behavioral model of at least one of the actor and the environment, and further wherein at least one of the situation assessor and the response planner is further adapted to utilize information from the machine learning device in determining the current situation and generating a current response plan, respectively.

15. The system of claim 1, wherein the system includes at least one sensor adapted to provide information relating to the actor and at least one sensor adapted to provide information related to the environment, and further wherein the situation assessor is further adapted to process the actor-related information and the environment-related information.

16. The system of claim 1, wherein the situation assessor, the response planner, and the plan executor are provided as a layered architecture.

17. The system of claim 16, wherein the controller further includes modules adapted to provide protocol constraints relating to designated subject matters, and further wherein the situation assessor, the response planner, and the plan executor define categories of capabilities made available to each of the modules by the controller.

18. The system of claim 17, wherein the controller further includes a first subject matter module and a second subject matter module each adapted to process information within at least one of the architecture layers.

19. The system of claim 18, wherein the first and second subject matter modules are communicatively linked.

20. The system of claim 18, wherein the first subject matter module is adapted to perform situation assessment operations related to a first domain subject matter and the second subject matter module is adapted to perform situation assessment operations relating to a second domain subject matter.

21. The system of claim 20, wherein the first and second subject matter modules are adapted to perform response planning operations relating to the first and second domain subject matters, respectively.

22. The system of claim 21, wherein the controller further includes a plan coordination module adapted to process response plans generated by the first and second subject matter modules.

23. The system of claim 21, wherein the controller further includes an intent recognition module adapted to determine an intent of the user, and further wherein the first and second subject matter modules utilize information generated by the intent recognition module in performing respective situation assessment and response planning operations.

24. The system of claim 21, wherein the controller is adapted to communicatively link a third, newly added domain subject matter module with the first and second subject matter modules.

25. The system of claim 19, wherein the first subject matter module is further adapted to perform response planning operations relating to a first subject matter utilizing information from the second subject matter module.

26. The system of claim 25, wherein the second subject matter module relates to actor information and the first subject matter module is adapted to process information relating to a subject matter selected from the group consisting of fire safety, home security, medication management, telephone interaction, eating, mobility, cognitive disorders, and toileting.

27. An automated monitoring and support system for an actor in an environment, the system comprising:

a sensor located in the environment;

an effector adapted to interface with at least one of the actor and the environment; and

a controller adapted to provide a layered architecture including a sensing layer for receiving information from the sensor, a situation assessment layer for determining a current situation of at least one of the actor and the environment based upon the sensor information, a response planning layer for automatically generating a response plan to the determined situation, and a plan execution layer for prompting the effector to execute the generated response plan, the controller further including;

a first domain subject matter module operating across at least the sensing, situation assessment, and response planning layers, the first domain subject matter module adapted to process information relating to a first subject matter and generate a response plan relating to the first subject matter.

28. The system of claim 27, wherein the first domain subject matter is selected from the group consisting of fire safety, home security, medication management, telephone interaction, eating, mobility, cognitive disorders, and toileting.

29. The system of claim 27, wherein the controller further includes a second domain subject matter module adapted to process information relating to a second subject matter and generate a response plan relating to the second subject matter.

30. The system of claim 29, wherein the controller further includes a plan coordination module for prioritizing response plans generated by the first and second domain subject matter modules.

31. The system of claim 27, wherein the controller further includes a second subject matter module adapted to process information relating to a second subject matter, and further wherein the first domain subject matter module utilizes information from the second subject matter module in performing situation assessment operations relating to the first subject matter.

32. The system of claim 27, wherein the controller further includes a second subject matter module adapted to process information relating to a second subject matter and further wherein the first domain subject matter utilizes information from the second subject matter module in performing response planning operations relating to the first subject matter.

33. The system of claim 27, wherein the controller further includes an intent recognition module adapted to recognize an intended activity of the actor based upon sensor information, and further wherein the first domain subject matter module utilizes information from the intent recognition module in performing situation assessment operations relating to the first subject matter.

34. The system of claim 27, wherein the controller further includes an intent recognition module adapted to recognize an intended activity of the actor based upon sensor information, and further wherein the first domain subject matter module utilizes information from the intent recognition module in performing response planning operations relating to the first subject matter.

35. The system of claim 27, wherein the controller further includes a behavioral database that establishes a behavioral model for at least one of the actor and the environment, and further wherein the first domain subject matter module utilizes information from the behavioral database in performing situation assessment operations relating to the first subject matter.

36. The system of claim 35, wherein the controller further includes a machine learning module adapted to generate the behavioral database.

37. The system of claim 27, wherein the controller further includes a behavioral database that establishes a behavioral model for at least one of the actor and the environment, and further wherein the first domain subject matter module utilizes information from the behavioral database in performing response planning operations relating to the first subject matter.

38. The system of claim 37, wherein the controller further includes a machine learning module adapted to generate the behavioral database.

39. The system of claim 27, wherein the system further includes a plurality of sensors, and further wherein the controller includes an event recognition module adapted to aggregate information from multiple ones of the sensors to designate a single event as occurring, and further wherein the first domain subject matter module utilizes information from the event recognition module in performing situation assessment operations relating to the first subject matter.

40. The system of claim 27, wherein the system further includes a plurality of effectors adapted to interface with the actor, and further wherein the controller includes a plan coordination module for implementing the response plan via at least two of the effectors.

41. An automated monitoring and support system for an actor in an environment, the system comprising:

a plurality of sensors adapted to sense information relating to the actor and the environment;

a plurality of effectors adapted to interface with the actor; and

a controller adapted to provide a layered architecture including a sensing layer for receiving information from the plurality of sensors, a situation assessment layer for determining a current situation of at least one of the actor and the environment based upon the sensor information, a response planning layer for automatically generating a response plan to the determined situation, and a plan executing layer for prompting at least one of the effectors to execute the generated response plan, the controller further including;

the first domain subject matter module operating across at least the sensing, situation assessment and response planning layers, the first domain subject matter module adapted to process information relating to a first subject matter and generate a response plan relating to the first subject matter,

a second domain subject matter module operating across at least the sensing, situation assessment, and response planning layers, the second domain subject matter module adapted to process information and generate a response plan relating to a second subject matter,

a third domain subject matter module operating across at least the situation assessment layer and adapted to process information relating to the actor,

a fourth subject matter module operating across at least the situation assessment module and adapted to process information relating to the environment,

a response execution module adapted to process response plans generated by the first and second domain subject matter modules.

42. The system of claim 41, further comprising a machine learning module communicatively linked to at least one of the first, second, third, and fourth modules.

43. The system of claim 41, further comprising an intent recognition module communicatively linked to at least one of the first, second, third, and fourth modules.

44. The system of claim 41, wherein the controller is adapted to load and communicatively link additional subject matter modules.

45. A method of automatically monitoring and supporting an actor in an environment, the method comprising:

receiving information from a plurality of sensors located in the environment;

automatically assessing a situation relating to at least one of the actor and the environment based upon information from the plurality of sensors;

automatically generating a response plan based upon the assessed situation; and

automatically executing the response plan by operating at least one of a plurality of effectors in the environment.

46. The method of claim 45, wherein automatically assessing a situation includes automatically assessing a situation of a plurality of subject matters.

47. The method of claim 45, further comprising:

providing a plurality of subject matter modules each adapted to assess a situation of a different subject matter.

48. The method of claim 47, wherein at least one of the plurality of subject matter modules utilizes information from at least another one of the plurality of subject matter modules in assessing a situation of the corresponding subject matter.

49. The method of claim 45, wherein automatically assessing a situation relating to the actor or the environment includes determining an intent of the actor.

50. The method of claim 45, wherein automatically generating a response plan includes generating a plurality of response plans relating to a plurality of different subject matters.

51. The method of claim 50, further comprising:

providing a plurality of subject matter modules each adapted to generate a response plan relating to a different subject matter.

52. The method of claim 50, further comprising:

evaluating each of the plurality of response plans; and

designating a primary response plan based upon the evaluation.

53. The method of claim 45, wherein generating a response plan includes referring to a machine learning database.

54. The method of claim 53, wherein generating a response plan further includes:

adapting the response plan to capabilities of the actor as provided by the machine learning database.

55. The method of claim 45, further comprising:

monitoring a response of the actor to the response plan following execution of the response plan.