Non-contact activity sensing network for elderly care

• US 10,410,498 B2
• Filed: 10/01/2018
• Issued: 09/10/2019
• Est. Priority Date: 10/06/2017
• Status: Active Grant

• Current Assignee: TellUs You Care, Inc.
• Original Assignee: TellUs You Care, Inc.
• Inventors: Coke, Tania Abedian, Hsu, Kevin
• Primary Examiner(s): Terrell, Emily C
• Application Number:
  US16/148,418
• Publication Number:
  US 20190108740A1
• Time in Patent Office: 344 Days
• Field of Search: None
• CPC Class Codes: G01S 13/726 Multiple target tracking
• CPC Class Codes: G01S 13/886 for alarm systems alarms with electrical actuation G08B13/22
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 7/415 Identification of targets based on measurements of movement associated with the target
• CPC Class Codes: G08B 21/0423 detecting deviation from an expected pattern of behaviour or schedule
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 21/0461 integrated or attached to an item closely associated with the person but not worn by the person, e.g. chair, walking stick, bed sensor
• CPC Class Codes: G08B 21/0476 Cameras to detect unsafe condition, e.g. video cameras
• CPC Class Codes: G08B 29/186 Fuzzy logic; neural networks
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Determining a physical state of a person includes detecting positions of different portions of the person, transforming detected positions of the person into a point cloud having a density that varies according to movement of each of the portions, correlating movement and position data from the point cloud with known physical state positions and transitions between different states, and choosing a particular physical state by matching the data from the point cloud with the particular physical state. Positions of different portions of the person may be detected using a tracking device. The tracking device may be a non-contact tracking device. The tracking device may include at least one wide band radar. The tracking devices may communicate wirelessly with at least one server in a cloud computing system. The states may include walking, standing, sitting, laying down, turning in bed, falling, and/or departed.

Claims

1. A method of determining a physical state of a person, comprising:

detecting positions of different portions of the person;

transforming detected positions of the person into a point cloud having a density that varies according to movement of each of the portions;

correlating movement and position data from the point cloud with known physical state positions and transitions between different states, wherein each of the physical state positions is associated with particular point densities, sizes, orientations, centers of gravity, and dispositions of bounding boxes of the point cloud; and

choosing a particular physical state by matching the data from the point cloud with the particular physical state.

2. A method, according to claim 1, wherein positions of different portions of the person are detected using a tracking device.

3. A method, according to claim 2, wherein the tracking device is a non-contact tracking device.

4. A method, according to claim 2, wherein the tracking device includes at least one wide band radar.

5. A method, according to claim 4, wherein the tracking devices communicate wirelessly with at least one server in a cloud computing system.

6. A method, according to claim 1, wherein the states include at least one of:

walking, standing, sitting, laying down, turning in bed, falling, and departed.

7. A method, according to claim 6, wherein falling is detected in response to the person transitioning from the walking state to the laying down state.

8. A method, according to claim 7, wherein during the transitioning, the person is detected as having a progressively lower center of gravity of the point cloud.

9. A method, according to claim 7, wherein an audio generating device communicates with the person following a fall to confirm the fall and offer basic instructions to the person.

10. A method, according to claim 9, wherein a caregiver is contacted if the person does not move or communicate following the audio generating device communicating with the person.

11. A method, according to claim 10, wherein the audio generating device includes a microphone that receives audio communication from the person.

12. A method, according to claim 1, wherein parametric representations of the bounding boxes, the point densities and positions of the centers of gravity of samples of different states are provided as input to a neural network classifier.

13. A method, according to claim 12, wherein the neural network is trained by providing the neural network on a server in a cloud computing system that receives data from tracking devices that detect positions of different portions of the person and communicate wirelessly with the cloud computing system.

14. A method, according to claim 13, wherein the neural network is a long short-term memory recurrent neural network.

15. A method, according to claim 12, wherein the neural network classifier correlates movement and position data from the point cloud with known physical state positions and transitions between different states to choose the particular physical state.

16. A method, according to claim 1, further comprising:

maintaining information corresponding to customary routine state transitions and locations of the person.

17. A method, according to claim 16, wherein customary routine state transitions and locations are determined by detecting clusters of points in a multi-dimensional space of sequences of objects, time intervals, locations, and state transitions that represent complex user behaviors.

18. A method, according to claim 16, wherein an alarm is provided to a caretaker in response to the person deviating from the customary routine state transitions and locations.

19. A method, according to claim 18, wherein deviating from the customary routine state transitions and locations includes cycling around a room for a prolonged period of time or repetitively moving back and forth between two objects.

20. A method, according to claim 17, wherein the clusters of points corresponding to customary routines are provided to a cloud computing system for comparison with clusters of points corresponding to customary routines for other people to further categorize behaviors and improve detection of dangerous situations.

21. A method, according to claim 1, wherein the person is in a room and objects in the room are initially detected by monitoring absolute coordinates of a bounding box of the point cloud in various user states.

22. A method, according to claim 21, wherein the objects include at least one of:

a bed, a table, a chair, a bookshelf, a door, or a window.

23. A method, according to claim 21, wherein objects in the room are detected by subtracting multiple positions of the bounding box from the area of the room.

24. A method, according to claim 23, wherein a bed is detected by observing the person in a laying down state at a certain height off the floor.

25. A method, according to claim 24, wherein boundaries of the bed are determined by tracking bounding boxes corresponding to a laying down state, a sitting state either before or after entering the laying down state, and a standing state prior to entering the sitting.

26. A method, according to claim 23, wherein a bed or a couch is detected by observing adjacent states of standing, sitting and laying down at a position corresponding to the bed or the couch.

27. A method, according to claim 23, wherein a window is detected by observing the person standing a relatively long time at a boundary of the room.

28. A non-transitory computer-readable medium containing software that determines a physical state of a person, the software comprising:

executable code that detects positions of different portions of the person;

executable code that transforms detected positions of the person into a point cloud having a density that varies according to movement of each of the portions;

executable code that correlates movement and position data from the point cloud with known physical state positions and transitions between different states, wherein each of the physical state positions is associated with particular point densities, sizes, orientations, centers of gravity, and dispositions of bounding boxes of the point clouds; and

executable code that chooses a particular physical state by matching the data from the point cloud with the particular physical state.