APPARATUS AND METHOD FOR REFINING SUBJECT ACTIVITY CLASSIFICATION FOR RECOGNITION OF DAILY ACTIVITIES, AND SYSTEM FOR RECOGNIZING DAILY ACTIVITIES USING THE SAME

• US 20090082699A1
• Filed: 04/18/2008
• Published: 03/26/2009
• Est. Priority Date: 09/21/2007
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Park, Seon Hee, Park, Soo Jun, Kim, Min Ho, Bang, Sun Lee
• Granted Patent: US 8,079,277 B2
• Time in Patent Office: Days
• US Class Current: 600/595
• CPC Class Codes: A61B 2505/07 Home care
• CPC Class Codes: A61B 2562/08 Sensors provided with means for identification, e.g. barcodes or memory chips
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1117 Fall detection
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/6887 mounted on external non-worn devices, e.g. non-medical devices
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

Apparatus and method for refining subject activity classification for the recognition of daily activities of a subject, and a system for recognizing daily activities using the same. The refining apparatus improves the correctness of subject activity classification using daily activities of a subject, activation time information of sensors mounted on objects associated with the daily activities of the subject, and the suitability of a continuous activity pattern in relation to the daily activities. This improves the correctness of subject activity classification that becomes basic information in daily activity analysis.

Claims

1. An apparatus for refining subject activity classification, comprising:

an input buffer receiving and storing activity classification values of activity data of a subject, measured by a sensor attached to a body of the subject, and object response data of objects associated with the activity data of the subject and measured by at least one sensor mounted on the objects;

a first refining module extracting a predetermined section of the activity classification values, stored in the input buffer, extracting a sudden significant activity data, which occurs in a time shorter than a minimum time necessary for carrying out an activity corresponding to an activity classification value in the predetermined section, and refining the extracted sudden significant activity data to be identical with activity classification values preceding and following the sudden significant activity data; and

a second refining module, if the sudden significant activity data is absent in the predetermined section, extracting an unsuitable pattern, which is not suitable for a continuous activity pattern of the activity classification values in the predetermined section and refining the unsuitable pattern to be identical with the continuous activity pattern by considering an activation time of the sensor mounted on the objects associated with the activity data of the subject.

2. The apparatus of claim 1, wherein each of the object response data and the activity classification values includes occurrence time information of an activity and an object response.

3. The apparatus of claim 2, wherein the first refining module organizes triple data by receiving, from the input buffer, three continuous activity classification values, which are extracted from the predetermined section and are time sequential, compares a short time activity data with a preceding activity classification value just prior thereto if the short time activity data is present in the triple data, judges the short time activity data as a sudden significant data if the short time activity data is not identical with the preceding activity classification value just prior thereto, and refines the sudden significant data using preceding and following activity classification values, which are classified as identical activities.

4. The apparatus of claim 3, wherein the second refining module judges whether or not the triple data are suitable for a continuous activity pattern if the sudden significant data is absent in the triple data, and if there is an unsuitable pattern, refines the unsuitable pattern using a reference activity set to the triple data and an activation time of at least one sensor mounted on an object associated with the reference activity.

5. The apparatus of claim 4, wherein the second refining module refines the activity classification value following the reference activity, using the reference activity, if at least one sensor mounted on an object associated with the reference activity is kept activated until an occurrence of the following activity classification value.

6. The apparatus of claim 5, wherein the second refining module refines the unsuitable pattern by matching a suitable continuous activity pattern with the triple data, if at least one sensor mounted on an object associated with the reference activity is not kept activated until an occurrence of the following activity classification value.

7. The apparatus of claim 4, wherein the second refining module preferentially sets an activity, in which an object response data is simultaneously activated during an occurrence time of an associated activity data of the subject, as the reference activity, and otherwise, sets an activity, which has a highest classification value in the triple data, or an activity, which is carried out for a predetermined time, as the reference activity.

8. A system for recognizing daily activities, comprising:

an activity classification module generating activity classification values by classifying activity data of a subject, received from a sensor attached to a body of the subject;

an activity refining module receiving and storing the activity classification values and object response data of objects associated with the activity data of the subject and measured by at least one sensor mounted on the objects, extracting a sudden significant activity data and an unsuitable pattern, which is not suitable for a continuous activity pattern, from the activity classification values, and refining the activity classification values using an activation time of the at least one sensor, mounted on the objects associated with the activity data of the subject; and

a daily activity recognition module recognizing daily activities of the subject using the refined activity classification values.

9. The system of claim 8, wherein the activity classification module classifies the daily activities of the subject into motions including one of ‘

sit,’

‘

walk,’

‘

run,’

‘

stand,’

‘

lie’ and

‘

fall’

, postures indicating progressive motions including one of ‘

sitting,’

‘

standing’ and

‘

lying;

’ and

‘

unknown’

activities.

10. The system of claim 9, wherein the activity classification module includes:

an input buffer receiving and storing the activity classification values and the object response data;

a first refining module extracting a predetermined section of the activity classification values, stored in the input buffer, extracting a sudden significant activity data, which occurs in a time shorter than a minimum time necessary for carrying out an activity corresponding to an activity classification value in the predetermined section, and refining the extracted sudden significant activity data to be identical with an activity classification values preceding and following the sudden significant activity data; and

a second refining module, if the sudden significant activity data is absent in the predetermined section, extracting an unsuitable pattern, which is not suitable for a continuous activity pattern of the activity classification values in the predetermined section and refining the unsuitable pattern to be identical with the continuous activity pattern by considering an activation time of the sensor mounted on the objects associated with the activity data of the subject.

11. The system of claim 10, wherein the first refining module organizes triple data by receiving, from the input buffer, three continuous activity classification values, which are extracted from the predetermined section and are time sequential, compares a short time activity data with a preceding activity classification value just prior thereto if the short time activity data is present in the triple data, judges the short time activity data as a sudden significant data if the short time activity data is not identical with the preceding activity classification value just prior thereto, and refines the sudden significant data using preceding and following activity classification values, which are classified as identical activities.

12. The system of claim 11, wherein the second refining module judges whether or not the triple data are suitable for a continuous activity pattern if the sudden significant data is absent in the triple data, and if there is an unsuitable pattern, refines the unsuitable pattern using a reference activity set to the triple data and an activation time of at least one sensor mounted on an object associated with the reference activity.

13. The system of claim 12, wherein the second refining module preferentially sets an activity, in which an object response data is simultaneously activated during an occurrence time of an associated activity data of the subject, as the reference activity, and otherwise, sets an activity, which has a highest classification value in the triple data, or an activity, which is carried out for a predetermined time, as the reference activity.

14. A method for refining subject activity classification, comprising:

receiving and storing activity classification values of activity data of a subject, measured by a sensor attached to a body of the subject, and object response data measured by at least one sensor mounted on objects;

extracting a predetermined section of the stored activity classification values of the subject, judging whether or not a sudden significant activity data is present in the predetermined section, and if the sudden significant activity data is present, refining the sudden significant activity data; and

if the sudden significant activity data is absent, judging whether or not activity classification data in the predetermined section are suitable for a continuous activity pattern, if an unsuitable pattern is present, refining the unsuitable pattern by considering an activation time of at least one sensor mounted on an object associated with an activity of the subject.

15. The method of claim 14, wherein the refining of the sudden significant activity data comprises:

organizing triple data using continuous three activity classification data in the predetermined section;

comparing a short time activity data, which occurs for a time shorter than a minimum time necessary for carrying out an activity included in the triple data, with a preceding activity classification value just prior thereto, and if the short time activity data is not identical with the preceding activity classification value just prior thereto, judging the short time activity data as a sudden significant activity data; and

if preceding and following activity classification values are identical, refining the sudden significant activity data using the preceding and following activity classification values.

16. The method of claim 15, wherein the refining of the sudden significant activity data comprises:

if the short time activity data is identical with the preceding activity classification value just prior thereto, or the preceding and following activity classification values are not identical, judging suitability for a continuous activity pattern.

17. The method of claim 14, wherein the refining of the unsuitable pattern comprises:

if the sudden significant activity data is absent, searching the triple data for an unsuitable pattern, which is not suitable for a continuous activity pattern;

setting a reference activity for the triple data; and

if an activation time of at least one sensor, mounted on an object associated with the set reference activity, is identical with an occurrence time of the unsuitable pattern, refining the unsuitable pattern using the reference activity.

18. The method of claim 17, wherein the refining of the unsuitable pattern further comprises:

if an activation time of at least one sensor is not identical with an occurrence time of a next continuous activity, refining the unsuitable pattern by matching the triple data with a suitable continuous activity pattern.

19. The method of claim 17, wherein the setting of a reference activity comprises:

setting an activity, in which an object response data is simultaneously activated during an occurrence time of an associated activity data of the subject, as the reference activity, and if an activity classification value is not delivered, setting an activity, which has a highest classification value in the triple data, or an activity, which is carried out for a predetermined time, as the reference activity.

20. The method of claim 18, wherein the suitable continuous pattern comprises at least one selected from the group consisting of ‘

lie’

-‘

lying,’

‘

lying’

-[‘

sit’

or ‘

stand’

], ‘

sit’

-‘

sitting,’

‘

sitting’

-[‘

stand’

or ‘

lie’

], ‘

stand’

-‘

standing,’

‘

standing’

-[‘

sit’

or ‘

lie’

or ‘

walk’

or ‘

run’

or ‘

fall’

], ‘

fall’

-[‘

sit’

or ‘

stand’

], ‘

walk’

-[‘

stand’

or ‘

run’

] and ‘

run’

-[‘

walk’

or ‘

stand’

or ‘

fall’

].