PERSON BEHAVIOR ANALYSIS DEVICE, PERSON BEHAVIOR ANALYSIS SYSTEM, PERSON BEHAVIOR ANALYSIS METHOD, AND MONITORING DEVICE

• US 20150010204A1
• Filed: 06/30/2014
• Published: 01/08/2015
• Est. Priority Date: 07/02/2013
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: IWAI, Kazuhiko
• Granted Patent: US 9,558,398 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A behavior analysis/monitoring device includes: a person detection unit configured to detect a person(s) from image information obtained by capturing images covering an area around an item placement area; a part-of-interest detection unit configured to detect, for each person detected by the person detection unit, a part of interest set in a part of an upper body of the person excluding hands and arms; a position measurement unit configured to measure a position of the part of interest detected by the part-of-interest detection unit; and an item pick-up action determination unit configured to obtain a displacement of the part of interest based on the position of the part of interest obtained by the position measurement unit and to determine whether each person detected by the person detection unit performed an item pick-up action based on the displacement of the part of interest of the person.

Claims

1. A person behavior analysis device for performing analysis of item pick-up actions, which are actions of picking up an item placed in an item placement area, the device comprising:

a person detection unit configured to detect a person(s) from image information obtained by capturing images covering an area around the item placement area;

a part-of-interest detection unit configured to detect, for each person detected by the person detection unit, a part of interest set in a part of an upper body of the person excluding hands and arms;

a position measurement unit configured to measure a position of the part of interest detected by the part-of-interest detection unit;

an item pick-up action determination unit configured to obtain a displacement of the part of interest based on the position of the part of interest obtained by the position measurement unit and to determine whether each person detected by the person detection unit performed an item pick-up action based on the displacement of the part of interest of the person; and

an output information generation unit configured to generate output information representing a result of analysis based on a result of determination by the item pick-up action determination unit.

2. The person behavior analysis device according to claim 1, wherein the part of interest is a head or shoulder of a person.

3. The person behavior analysis device according to claim 1, wherein the item placement area includes a display shelf having a plurality of shelf boards on which items are placed such that the shelf boards are spaced apart in a vertical direction.

4. The person behavior analysis device according to claim 1, further comprising:

a vicinity area setting unit configured to set, around the item placement area, a vicinity area which is a maximum area in which a person can touch an item in the item placement area by extending a hand; and

an area entry determination unit configured to determine that, of the persons detected by the person detection unit, a person who did not enter the vicinity area is a person who could not perform the item pick-up action.

5. The person behavior analysis device according to claim 4, wherein the vicinity area setting unit sets the vicinity area based on an arm length estimated from a result of measurement by the position measurement unit in accordance with correlation between height and arm length.

6. The person behavior analysis device according to claim 4, further comprising:

a staying time measurement unit configured to measure, for each person entering the vicinity area, a staying time in the vicinity area; and

a staying time assessment unit configured to determine that, of the persons detected by the person detection unit, a person whose staying time does not reach a predetermined threshold value is a person who could not perform the item pick-up action.

7. The person behavior analysis device according to claim 4, wherein the item pick-up action determination unit determines whether each person entering the vicinity area performed the item pick-up action based on a displacement of the part of interest of the person with respect to a position of the part of interest when the person entered the vicinity area in an upright posture.

8. The person behavior analysis device according to claim 1, wherein the item pick-up action determination unit determines whether each person detected by the person detection unit performed the item pick-up action based on a positional relationship between a reference position set in the item placement area and the part of interest of the person.

9. The person behavior analysis device according to claim 2, wherein the position measurement unit measures a central point of a head of each person.

10. The person behavior analysis device according to claim 1, wherein the position measurement unit obtains information relating to an apparent size of the part of interest in captured images from the image information, and measures the position of the part of interest based on a change in the apparent size of the part of interest.

11. The person behavior analysis device according to claim 1, wherein:

the person detection unit detects a plurality of persons who are present in the vicinity of the item placement area at the same time;

the part-of-interest detection unit detects the part of interest of each person detected by the person detection unit;

the position measurement unit measures a position of the part of interest of each person detected by the part-of-interest detection unit; and

the item pick-up action determination unit determines whether each person detected by the person detection unit performed the item pick-up action.

12. The person behavior analysis device according to claim 1, wherein:

a plurality of item placement areas are arranged for respective categories of items; and

the output information generation unit generates output information relating to a result of totaling of at least one of a number of occurrences of item pick-up actions for each item placement areas and staying times of detected persons in an area around each item placement area.

13. The person behavior analysis device according to claim 12, wherein the output information generation unit generates output information relating to a display image for displaying the result of totaling superimposed on an image representing the plurality of item placement areas arranged in accordance with an actual arrangement of the item placement areas.

14. A person behavior analysis system for performing analysis of item pick-up actions, which are actions of picking up an item placed in an item placement area, the system comprising:

an imaging device configured to capture an image of an area around the item placement area; and

a plurality of information processing devices,

wherein the plurality of information processing devices jointly comprise;

a person detection unit configured to detect a person(s) from image information obtained by capturing images by the imaging device;

a part-of-interest detection unit configured to detect, for each person detected by the person detection unit, a part of interest set in a part of an upper body of the person excluding hands and arms;

a position measurement unit configured to measure a position of the part of interest detected by the part-of-interest detection unit;

an item pick-up action determination unit configured to obtain a displacement of the part of interest based on the position of the part of interest obtained by the position measurement unit and to determine whether each person detected by the person detection unit performed an item pick-up action based on the displacement of the part of interest of the person; and

an output information generation unit configured to generate output information representing a result of analysis based on a result of determination by the item pick-up action determination unit.

15. A person behavior analysis method for performing, with an information processing device, analysis of item pick-up actions, which are actions of picking up an item placed in an item placement area, the method comprising:

a first step of detecting a person(s) from image information obtained by capturing images covering an area around the item placement area;

a second step of detecting, for each person detected by the first step, a part of interest set in a part of an upper body of the person excluding hands and arms;

a third step of measuring a position of the part of interest detected by the second step;

a fourth step of obtaining a displacement of the part of interest from the position of the part of interest obtained by the third step and determining whether each person detected by the first step performed an item pick-up action based on the displacement of the part of interest of the person; and

a fifth step of generating output information representing a result of analysis based on a result of determination by the fourth step.

16. A monitoring device for notifying a monitoring person of a person performing an item pick-up action which is an action of picking up an item placed in a placement area, comprising:

a person detection unit configured to detect a person(s) from image information obtained by capturing images covering an area around the item placement area;

a part-of-interest detection unit configured to detect, for each person detected by the person detection unit, a part of interest set in a part of an upper body of the person excluding hands and arms;

a position measurement unit configured to measure a position of the part of interest detected by the part-of-interest detection unit;

an item pick-up action determination unit configured to obtain a displacement of the part of interest based on the position of the part of interest obtained by the position measurement unit and to determine whether each person detected by the person detection unit performed an item pick-up action based on the displacement of the part of interest of the person; and

a notification unit configured to generate notification information relating to a person performing the item pick-up action based on a result of determination by the item pick-up action determination unit.