Apparatus and method for refining subject activity classification for recognition of daily activities, and system for recognizing daily activities using the same

• US 8,079,277 B2
• Filed: 04/18/2008
• Issued: 12/20/2011
• Est. Priority Date: 09/21/2007
• Status: Expired due to Fees

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Bang, Sun Lee, Kim, Min Ho, Park, Soo Jun, Park, Seon Hee
• Primary Examiner(s): Hindenburg, Max
• Assistant Examiner(s): Pani, John
• Application Number:
  US12/105,327
• Publication Number:
  US 20090082699A1
• Time in Patent Office: 1,341 Days
• Field of Search: 600/595, 738/654, 702/19, 702/93, 702189-197
• US Class Current: 73/865.4
• CPC Class Codes: A61B 2505/07 Home care
• CPC Class Codes: A61B 2562/08 Sensors provided with means for identification, e.g. barcodes or memory chips
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/1117 Fall detection
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/6887 mounted on external non-worn devices, e.g. non-medical devices
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G16H 20/70 relating to mental therapies, e.g. psychological therapy or autogenous training
• CPC Class Codes: G16H 50/20 for computer-aided diagnosis, e.g. based on medical expert systems

Abstract

Apparatus and method for refining subject activity classification for the recognition of daily activities of a subject, and a system for recognizing daily activities using the same. The refining apparatus improves the correctness of subject activity classification using daily activities of a subject, activation time information of sensors mounted on objects associated with the daily activities of the subject, and the suitability of a continuous activity pattern in relation to the daily activities. This improves the correctness of subject activity classification that becomes basic information in daily activity analysis.

Claims

1. An apparatus for refining subject activity classification, comprising:

a computer system comprising a processor and a memory;

an input buffer on the computer system and configured to receive and store activity classification values of activity data of a subject, wherein the activity data is measured by a sensor attached to a body of the subject and the activity classification values represent activities of the subject, and to receive and store object response data of objects associated with the activity data of the subject, wherein the object response data is measured by at least one object response sensor mounted on the objects and includes an activation time of the sensor;

a first refining module on the computer system and configured to extract a predetermined number of sequential activity classification values having a pattern and stored in the input buffer;

to compare a duration of an activity classification value of the extracted predetermined number of sequential activity classification values to a preset reference time considered necessary for carrying out the activity represented by the activity classification value;

to judge the activity represented by the activity classification value as a short time activity when the duration of the activity classification value is shorter than the preset time;

to judge the short time activity to be sudden significant activity data when the short time activity is not identical with a preceding activity; and

to refine the sudden significant activity data to be identical with activity classification values preceding and following the sudden significant activity data when the preceding and following activities are identical; and

a second refining module on the computer system and configured to, in response to the first refining module judging there is no sudden significant activity in the predetermined number of sequential activity classification values, judge whether the pattern of the extracted predetermined number of sequential activity classification values is suitable or unsuitable for a continuous activity pattern by comparing the pattern of the extracted sequential activity classification values to stored predetermined suitable or unsuitable continuous activity patterns;

check whether the activation time of the at least one object response sensor corresponds to a time when an unsuitable pattern occurs in the extracted predetermined number of sequential activity classification values; and

refine the unsuitable pattern to a predetermined suitable pattern corresponding to a reference activity associated with the object response sensor when the object response sensor is activated during the time when the unsuitable pattern occurs.

2. The apparatus of claim 1, wherein the object response data includes occurrence time information of an object response and the activity classification values include occurrence time information of an activity.

3. The apparatus of claim 2, wherein the predetermined section of the activity classification values extracted by the first refining module is triple data, the triple data being three continuous and time sequential activity classification values.

4. The apparatus of claim 3, wherein the second refining module is configured to refine the unsuitable pattern by matching a suitable continuous activity pattern with the triple data when at least one sensor mounted on an object associated with the reference activity is not activated until an occurrence of a following activity classification value.

5. The apparatus of claim 3, wherein the second refining module is configured to set the reference activity to be an activity having a highest classification value in the triple data or an activity carried out for a predetermined time.

6. A system for recognizing daily activities, comprising:

a computer system comprising a processor and a memory

an activity classification module on the computer system and configured to generate activity classification values by classifying activity data of a subject, wherein the activity data is received from a sensor attached to a body of the subject and the activity classification values represent activities of the subject;

an activity refining module on the computer system and configured to receive and store the activity classification values and to receive and store object response data of objects associated with the activity data of the subject, wherein the object response data is measured by at least one object response sensor mounted on the objects, the activity refining module comprising;

a first refining module configured to extract a predetermined number of sequential activity classification values having a pattern and stored in the input buffer;

to compare a duration of an activity classification value of the extracted predetermined number of sequential activity classification values to a preset reference time considered necessary for carrying out the activity represented by the activity classification value;

to judge the activity represented by the activity classification value as a short time activity when the duration of the activity classification value is shorter than the preset time;

to judge the short time activity to be sudden significant activity data when the short time activity is not identical with a preceding activity; and

to refine the sudden significant activity data to be identical with activity classification values preceding and following the sudden significant activity data when the preceding and following activities are identical; and

a second refining module configured to, in response to the first refining module judging there is no sudden significant activity in the predetermined number of sequential activity classification values, judge whether the pattern of the extracted predetermined number of sequential activity classification values is suitable or unsuitable for a continuous activity pattern by comparing the pattern of the extracted sequential activity classification values to stored predetermined suitable or unsuitable continuous activity patterns;

check whether the activation time of the at least one object response sensor corresponds to a time when an unsuitable pattern occurs in the extracted predetermined number of sequential activity classification values; and

refine the unsuitable pattern to a predetermined suitable pattern corresponding to a reference activity associated with the object response sensor when the object response sensor is activated during the time when the unsuitable pattern occurs; and

a daily activity recognition module on the computer system and configured to recognize daily activities of the subject using the refined activity classification values.

7. The system of claim 6, wherein the activity classification module is configured to classify the daily activities of the subject into motions including any one of ‘

sit,’

‘

walk,’

‘

run,’

‘

stand,’

‘

lie’ and

‘

fall’

, and postures indicating progressive motions, the postures including any one of ‘

sitting,’

‘

standing’

, ‘

lying;

’ and

‘

unknown’

activities.

8. The system of claim 7, wherein the activity classification refining module includes:

an input buffer configured to receive and store the activity classification values and the object response data.

9. The system of claim 8, wherein the predetermined section of the activity classification values extracted by the first refining module is triple data, the triple data being three continuous and time sequential activity classification values.

10. The system of claim 9, wherein the second refining module is configured to set the reference activity to be an activity having a highest classification value in the triple data or an activity carried out for a predetermined time.

11. A system for recognizing daily activities, comprising:

a first sensor for attaching to a body of a subject and configured to measure movement of the subject to output activity data of the subject;

at least one object response sensor for installing on objects which the subject interacts with, the at least one object response sensor configured for measuring response of the objects to output object response data;

an activity classification module configured to classify the activity data obtained by the first sensor into activity classification values each representing an activity of the subject;

an activity refining module configured to refine erroneously-sensed activity classification values using activation time information of the object response sensor and a suitable continuous activity pattern, the activity refining module comprising;

an input buffer configured to receive the activity classification values from the activity classification module and to store the received activity classification values in a time sequence, and to receive and store the object response data;

a first refining module configured to extract a predetermined number of sequential activity classification values having a pattern and stored in the input buffer;

to compare a duration of an activity classification value of the extracted predetermined number of sequential activity classification values to a preset reference time considered necessary for carrying out the activity represented by the activity classification value;

to judge the activity represented by the activity classification value as a short time activity when the duration of the activity classification value is shorter than the preset time;

to judge the short time activity to be sudden significant activity data when the short time activity is not identical with a preceding activity; and

to refine the sudden significant activity data to be identical with activity classification values preceding and following the sudden significant activity data when the preceding and following activities are identical; and

a second refining module configured to, in response to the first refining module judging there is no sudden significant activity in the predetermined number of sequential activity classification values, judge whether the pattern of the extracted predetermined number of sequential activity classification values is suitable or unsuitable for a continuous activity pattern by comparing the pattern of the extracted sequential activity classification values to stored predetermined suitable or unsuitable continuous activity patterns;

check whether the activation time of the at least one object response sensor corresponds to a time when an unsuitable pattern occurs in the extracted predetermined number of sequential activity classification values; and

refine the unsuitable pattern to a predetermined suitable pattern corresponding to a reference activity associated with the object response sensor when the object response sensor is activated during the time when the unsuitable pattern occurs.