WORK RECOGNITION SYSTEM, WORK RECOGNITION DEVICE, AND WORK RECOGNITION METHOD

• US 20110128363A1
• Filed: 05/21/2010
• Published: 06/02/2011
• Est. Priority Date: 06/08/2009
• Status: Active Grant

• Current Assignee: Sovereign Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Mizutani, Kenji
• Granted Patent: US 8,654,187 B2
• Time in Patent Office: Days
• US Class Current: 348/77
• CPC Class Codes: A61B 2505/07 Home care
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/6824 Arm or wrist
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device

Abstract

A work recognition device that recognizes processes in user'"'"'s work includes: a whole-state observation time determination unit (303) that determines, as a whole-state observation time, from among observation times of chronological first observation values generated by a first sensor (201) to represent movements of a user'"'"'s hand, an observation time of observing a first observation value representing a predetermined movement of the hand; an observation value selection unit (304) that selects a second observation value corresponding to the time determined by the determination unit (303) from among chronological second observation values generated by a second sensor (202) to represent scenes (states) in the user'"'"'s work; and a process recognition unit (305) that specifies a process in the work, by comparing the second observation value selected by the selection unit (304) to pieces of knowledge data associated with work processes in which the user performs predetermined movements of the hand.

Claims

1. A work recognition device comprising:

a whole-state observation time determination unit configured to determine, as a whole-state observation time, from among observation times of a plurality of first observation values generated by a speed sensor to represent respective movements of a hand of a user in chronological order, an observation time of observing a first observation value representing a predetermined movement of the hand, based on magnitudes of the plurality of first observation values and an angular difference between the plurality of first observation values; and

an observation value selection unit configured to select a second observation value corresponding to the whole-state observation time determined by said whole-state observation time determination unit, from among a plurality of second observation values generated by an imaging unit to represent respective states in work of the user in chronological order.

2. The work recognition device according to claim 1, further comprising

a process recognition unit configured to recognize a process in the work, by comparing the second observation value selected by said observation value selection unit to pieces of knowledge data associated with respective processes in the work, in each of the respective processes the user performing the predetermined movement of the hand.

3. The work recognition device according to claim 1,

wherein said whole-state observation time determination unit is configured to determine, as the whole-state observation time, a time having a local minimum value of an amount of a change between the plurality of first observation values.

4. The work recognition device according to claim 1,

wherein said whole-state observation time determination unit is configured to determine the whole-state observation time based on magnitudes of at least a part of the plurality of first observation values, the at least the part being generated during a time period after a first time until a predetermined time, and the first time being a time having a local minimum value of an amount of a change between the plurality of the first observation values.

5. The work recognition device according to claim 4,

wherein said whole-state observation time determination unit is configured to determine, as the whole-state observation time, a time closest to the first time from among times having local maximum values or maximum values of the magnitudes of the at least a part of the plurality of first observation values generated during the time period after the first time until the predetermined time.

6. The work recognition device according to claim 1,

wherein said speed sensor is a triaxial acceleration sensor that generates accelerations in chronological order as the plurality of first observation values.

7. The work recognition device according to claim 6,

wherein the user wears said speed sensor on a dominant hand of the user.

8. The work recognition device according to claim 6,

said whole-state observation time determination unit is configured to determine, as the whole-state observation time, based on the plurality of the first observation values generated in chronological order, an observation time having a local minimum value representing a change of a direction of the hand, when a magnitude of an acceleration of the hand becomes locally maximum after the observation time having the local minimum value.

9. The work recognition device according to claim 8,

wherein said whole-state observation time determination unit includes:

a magnitude estimation unit configured to estimate a magnitude of an acceleration that is obtained by subtracting a gravity acceleration from each of the accelerations generated by said speed sensor;

an angular difference calculation unit configured to calculate, for each of the accelerations generated by said speed sensor, an angle between (a) one acceleration at a corresponding observation time and (b) another acceleration at an observation time prior to the corresponding observation time; and

a time determination unit configured to determine, as the whole-state observation time, the corresponding observation time, when (a) a magnitude of the one acceleration which is estimated by said magnitude estimation unit and (b) the angle calculated by said angular difference calculation unit show a predetermined pattern.

10. The work recognition device according to claim 9,

wherein said time determination unit is configured to determine, as the whole-state observation time, a time at which (b) the angle calculated by said angular difference calculation unit approaches asymptotically to 0 degree, when (a) the magnitude of the one acceleration which is estimated by said magnitude estimation unit becomes locally maximum within a predetermined time period immediately after the time.

11. The work recognition device according to claim 10,

wherein said time determination unit is configured to determine the time at which (a) the angle calculated by said angular difference calculation unit approaches asymptotically to 0 degree, using a learning machine that learns angles at respective times at which the angles approaches asymptotically to 0 degree.

12. The work recognition device according to claim 9,

wherein said magnitude estimation unit is configured to approximate magnitudes of accelerations that are obtained by subtracting a magnitude of the gravity acceleration from each of the magnitudes of the accelerations generated by said speed sensor.

13. The work recognition device according to claim 1,

wherein said imaging unit is a camera that captures an image of a range of a process in the work of the user.

14. A work recognition device comprising:

a whole-state observation time determination unit configured to determine, as a whole-state observation time, from among observation times of a plurality of first observation values generated by a speed sensor to represent respective movements of a hand of a user in chronological order, an observation time of observing a first observation value representing a predetermined movement of the hand; and

an observation value selection unit configured to select a second observation value corresponding to the whole-state observation time determined by said whole-state observation time determination unit, from among a plurality of second observation values generated by an imaging unit to represent respective states in work of the user in chronological order.

15. A work recognition system that recognizes processes in work performed by a user, said work recognition system comprising:

a speed sensor that generates, in chronological order, a plurality of first observation values representing respective movements of a hand of the user performing the work;

an imaging unit configured to generate, in chronological order, a plurality of second observation values representing respective states in the work; and

a work recognition device that recognizes a process in the work based on the plurality of first observation values generated in chronological order and the plurality of second observation values generated in chronological order,

wherein said work recognition device includes;

a whole-state observation time determination unit configured to determine, as a whole-state observation time, from among observation times of a plurality of first observation values generated in chronological order, an observation time of observing a first observation value representing a predetermined movement of the hand;

an observation value selection unit configured to select a second observation value corresponding to the whole-state observation time determined by said whole-state observation time determination unit, from among a plurality of second observation values generated in chronological order; and

a process recognition unit configured to specify a process in the work, by comparing the second observation value selected by said observation value selection unit to pieces of knowledge data associated with respective processes in the work, in each of the respective processes the user performing the predetermined movement of the hand.

16. A work recognition method of recognizing, by using a computer, processes in work performed by a user, said work recognition method comprising:

determining, as a whole-state observation time, from among observation times of a plurality of first observation values generated by a speed sensor to represent respective movements of a hand of the user in chronological order, an observation time of observing a first observation value representing a predetermined movement of the hand, based on the plurality of first observation values;

selecting a second observation value corresponding to the whole-state observation time determined in said determining, from among a plurality of second observation values generated by an imaging unit to represent respective states in work of the user; and

specifying a process in the work, by comparing the second observation value selected in said selecting to pieces of knowledge data associated with respective processes in the work, in each of the respective processes the user performing the predetermined movement of the hand.

17. A computer program which is recorded on a non-transitory computer-readable recording medium for use in a computer and is used for recognizing processes in work performed by a user, said computer program causing the computer to execute:

determining, as a whole-state observation time, from among observation times of a plurality of first observation values generated by a speed sensor to represent respective movements of a hand of the user in chronological order, an observation time of observing a first observation value representing a predetermined movement of the hand, based on the plurality of first observation values;

selecting a second observation value corresponding to the whole-state observation time determined in said determining, from among a plurality of second observation values generated by an imaging unit to represent respective states in work of the user; and

specifying a process in the work, by comparing the second observation value selected in said selecting to pieces of knowledge data associated with respective processes in the work, in each of the respective processes the user performing the predetermined movement of the hand.