Activity recognition apparatus, method and program

• US 20060284979A1
• Filed: 05/18/2006
• Published: 12/21/2006
• Est. Priority Date: 06/09/2005
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Clarkson, Brian
• Granted Patent: US 7,421,369 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: A61B 2562/0219 Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: A61B 5/1038 Measuring plantar pressure during gait
• CPC Class Codes: A61B 5/1118 Determining activity level
• CPC Class Codes: A61B 5/1123 Discriminating type of movement, e.g. walking or running A61B5/1116, A61B5/112 take precedence
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G01C 21/183 Compensation of inertial measurements, e.g. for temperature effects
• CPC Class Codes: G01C 22/006 Pedometers
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition

Abstract

There is provided an activity recognition apparatus for detecting an activity of a subject. The apparatus includes: a sensor unit including a plurality of linear motion sensors configured to detect linear motions and a plurality of rotational motion sensors, the linear motions being orthogonal to each other, the rotational motions being orthogonal to each other; and a computational unit configured to receive and process signals from the sensors included in the sensor unit so as to detect an activity of the subject. The sensor unit is directly or indirectly supported by the subject with an arbitrary orientation with respect to the subject. The computational unit performs a calculation that uses the signals from both linear motion sensors and rotational motion sensors to determine the activity of the subject independent of the orientation of the sensor unit.

Claims

1. An activity recognition apparatus for detecting an activity of a subject, the apparatus comprising:

a sensor unit including a plurality of linear motion sensors configured to detect linear motions and a plurality of rotational motion sensors, the linear motions being orthogonal to each other, the rotational motions being orthogonal to each other; and

a computational unit configured to receive and process signals from the sensors included in the sensor unit so as to detect an activity of the subject;

wherein the sensor unit is directly or indirectly supported by the subject with an arbitrary orientation with respect to the subject; and

wherein the computational unit performs a calculation that uses the signals from both linear motion sensors and rotational motion sensors to determine the activity of the subject independent of the orientation of the sensor unit.

2. The activity recognition apparatus according to claim 1, wherein:

the plurality of linear motion sensors are three accelerometers for measuring accelerations in three orthogonal axes;

the plurality of rotational motion sensors are three gyro sensors for measuring angular velocities around the three orthogonal axes; and

the computational unit uses six outputs from the accelerometers and the gyro sensors to determine a current activity of the subject.

3. The activity recognition apparatus according to claim 1 or 2, wherein:

the computational unit includes

a buffer configured to store an observation sequence formed of a plurality set of the sensor outputs for a predetermined period of time;

a storage configured to store a plurality of reference sequences, the reference sequences corresponding to motion signatures of different activities; and

a matching processor configured to match the observation sequence against the reference sequences to find a best matched reference sequence;

wherein the matching processor finds an optimal rotation and temporal correspondence between the observation sequence and one of the reference sequences so as to obtain the best matched reference sequence.

4. The activity recognition apparatus according to claim 3, wherein:

the reference sequence includes a plurality of vectors representing states in Hidden Markov Model; and

the matching between the observation sequence and the reference sequence is performed using Viterbi algorithm so as to obtain an optimal state path.

5. The activity recognition apparatus according to claim 1 or 2, wherein:

the computational unit further detects a rate of the subject'"'"'s gait by calculating an autocorrelation of the signals from the linear motion sensors and an autocorrelation of the signals from the rotation motion sensors, and summing these autocorrelations to obtain a summary autocorrelation function; and

the summary autocorrelation function is used to determine the rate of the subject'"'"'s gait.

6. The activity recognition apparatus according to claim 1 or 2, wherein:

the computational unit further detects a rotation of the sensor unit by calculating a first rotation by integrating the signals from the rotational motion sensors, calculating a second rotation by using a gravitation model and the signals from the linear motion sensors, and performing weighted interpolation of the first and second rotation;

the weight for the first rotation is larger if variations in the signals from the rotational motion sensors are smaller; and

the weight for the second rotation is larger if variations in the signals from the linear motion sensors are smaller.

7. The activity recognition apparatus according to claim 1 or 2, further comprising:

a storage configure to store reference motion signatures and corresponding locations for a reference activity;

wherein

the computational unit further detects a location of the sensor unit on the subject by determining if a current activity represented with the signals from the sensors is the reference activity, and, if the current activity is the reference activity, matching a motion signature corresponding to the current activity against the reference motion signature so as to determine a location of the sensor unit, which corresponds to a best matched reference motion signature.

8. An activity recognition method of determining an activity of a subject, the method comprising:

sampling accelerations in three orthogonal axes and angular velocities around the three orthogonal axes; and

determining an activity of the subject using all the sampled accelerations and angular velocities;

wherein the accelerations and angular velocities are detected by sensors integrated into a single unit, which is directly or indirectly supported by the subject with an arbitrary orientation with respect to the subject; and

wherein the determination of activity is performed by using both the accelerations and angular velocities to determine the activity independent of the orientation of the sensor unit.

9. A recording medium that stores a program that causes a computer to perform activity recognition by determining an activity of a subject, the program comprising:

receiving data of accelerations in three orthogonal axes and angular velocities around the three orthogonal axes; and

determining an activity of the subject using all the received data of accelerations and angular velocities;

wherein the accelerations and angular velocities are detected by sensors integrated into a single unit, which is directly or indirectly supported by the subject with an arbitrary orientation with respect to the subject; and

wherein the determination of activity is performed by using both the accelerations and angular velocities to determine the activity independent of the orientation of the sensor unit.