Behavior monitoring system and behavior monitoring method

• US 8,525,678 B2
• Filed: 07/18/2008
• Issued: 09/03/2013
• Est. Priority Date: 08/21/2007
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Monden, Akira
• Primary Examiner(s): Nguyen, Tai T
• Assistant Examiner(s): NWUGO, OJIAKO K
• Application Number:
  US12/674,295
• Publication Number:
  US 20110115634A1
• Time in Patent Office: 1,873 Days
• Field of Search: 340/573.1, 340/5.7, 340/540, 340/541, 340/573.4
• US Class Current: 340/573.1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion

Abstract

A storage element stores identification information identifying to-be-monitored objects (TBMO) and prohibited behavior conditions indicating behaviors respectively prohibited to the TBMO in an area. Entry monitoring elements monitor as specific TBMO, TBMO which enter the area, and judge whether or not specific identification information identifying the specific to TBMO exists in the identification information stored in the storage element. When identification information coincident with the specific identification information exists in the identification information, behavior monitoring element monitor based on the specific identification information, the specific TBMO which behave in the area, and generate behavior information indicating behaviors of the specific TBMO. Behavior judging element select specific prohibited behavior conditions corresponding to the specific identification information from the plurality of prohibited behavior conditions stored in the storage element, and produce an alarm when behaviors indicated by the specific prohibited behavior conditions coincide with behaviors indicated by the behavior information.

Claims

1. A behavior monitoring system, comprising:

a storage unit which stores a plurality of pieces of identification information, each piece of identification information respectively identifying a corresponding one of a plurality of persons and one of a plurality of prohibited behavior conditions indicating behaviors respectively prohibited to said plurality of persons in an area and being correlated with interrelationships of the persons;

an entry monitoring unit configured to monitor as specific persons persons which enter said area, and to judge whether or not a specific piece of identification information identifying said specific persons exists in said plurality of pieces of identification information stored in said storage unit, and to specify said interrelationships of said specific persons;

a behavior monitoring unit configured to monitor, based on said specific piece of identification information, said specific persons which behave in said area and to generate a piece of behavior information indicating behaviors of said specific persons, when a piece of identification information coincident with said specific piece of identification information exists in said plurality of pieces of identification information; and

a behavior judging unit configured to select specific prohibited behavior conditions corresponding to said specific piece of identification information from said plurality of prohibited behavior conditions stored in said storage unit based on said specified interrelationships of said specific persons and to produce an alarm when behaviors indicated by said specific prohibited behavior conditions coincide with behaviors indicated by the piece of said behavior information.

2. The behavior monitoring system according to claim 1, further comprising:

a monitoring camera for taking images of said area; and

a monitoring camera control unit which controls said monitoring camera to take images of said area and to generate dynamic image data indicating a dynamic image of said area, wherein each of said plurality of persons holds specific information of itself,

said plurality of pieces of identification information stored in said storage unit respectively include a plurality of specific information indicating said plurality of persons and a plurality of face image data indicating face images of said plurality of persons,

said entry monitoring unit judges whether or not specific information of said specific persons exist in said plurality of specific information stored in said storage unit,

said behavior monitoring unit selects selected face image data corresponding to specific information of said specific persons from said plurality of face image data stored in said storage unit when specific information coincident with specific information of said specific persons exists in said plurality of specific information,

said behavior monitoring unit executes a face recognition process to judge whether or not face images indicated by said selected face image data are included in a dynamic image indicated by said dynamic image data in order to monitor said specific persons which behave in said area,

said behavior monitoring unit generates as said behavior information, behaviors of said specific persons included in a dynamic image indicated by said dynamic image data when face images indicated by said selected face image data are included in a dynamic image indicated by said dynamic image data, and

said behavior judging unit selects said specific prohibited behavior conditions corresponding to said specific piece of identification information including specific information of said specific persons and said selected face image data from said plurality of prohibited behavior conditions stored in said storage unit.

3. The behavior monitoring system according to claim 2, further comprising a communication unit provided at an entrance of said area in order to monitor said specific persons which enter said area,

wherein each of said plurality of persons possesses a holding unit in which said specific information of oneself is held,

specific information held in said holding unit of each of said plurality of persons is readable by said communication unit, and

said entry monitoring unit controls said communication unit to communicate with said holding unit of said specific persons which enter said area and to read specific information of said specific persons from a holding unit of said specific persons.

4. The behavior monitoring system according to claim 2, further comprising a biometrics reader provided at an entrance of said area in order to monitor said specific persons which enter said area,

wherein each of said plurality of persons holds specific information of oneself,

specific information of each of said plurality of persons indicates at least one biometrics of a fingerprint, an iris, a voiceprint and handwriting and is readable by said biometrics reader, and

said entry monitoring unit controls said biometrics reader to read specific information of said specific persons which enter said area.

5. The behavior monitoring system according to claim 2,

wherein a gate is provided at an entrance of said area, and

said entry monitoring unit controls said gate to refuse said specific persons to enter said area when said specific piece of identification information does not exist.

6. The behavior monitoring system according to claim 3,

wherein a gate is provided at an entrance of said area, and

said entry monitoring unit controls said gate to refuse said specific persons to enter said area when said specific piece of identification information does not exist.

7. The behavior monitoring system according to claim 1, further comprising:

a monitoring camera for taking images of said area;

a monitoring camera control unit which controls said monitoring camera to take images of said area and to generate dynamic image data indicating a dynamic image of said area; and

a camera provided at an entrance of said area in order to monitor said specific persons which enter said area,

wherein said plurality of pieces of identification information stored in said storage unit respectively include a plurality of face image data indicating face images of said plurality of persons,

said entry monitoring unit controls said camera to take images of persons which enters said area as said specific persons and to generate image data indicating an image including face images of said specific persons,

said entry monitoring unit judges whether or not face image data indicating face images of said specific persons exists in said plurality of face image data stored in said storage unit,

when face image data indicating face images coincident with face images of said specific persons exists in said plurality of face image data, said behavior monitoring unit selects said face image data as selected face image data,

said behavior monitoring unit executes a face recognition process to judge whether or not face images indicated by said selected face image data are included in a dynamic image indicated by said dynamic image data in order to monitor said specific persons which behave in said area,

said behavior monitoring unit generates as said behavior information, behaviors of said specific persons included in a dynamic image indicated by said dynamic image data when face images indicated by said selected face image data are included in a dynamic image indicated by said dynamic image data, and

said behavior judging unit selects said specific prohibited behavior information corresponding to said specific piece of identification information including specific information of said specific persons and said selected face image data from said plurality of prohibited behavior conditions stored in said storage unit.

8. The behavior monitoring system according to claim 7,

wherein a prohibited behavior condition of said plurality of prohibited behavior conditions stored in said storage unit is an outsider prohibited behavior condition indicating a behavior prohibited to said outsider,

when face image data indicating face images coincident with face images of said specific persons do not exist in said plurality of face image data, said entry monitoring unit stores as a piece of outsider identification information for identifying outsider, outsider face image data indicating face images of said specific persons such that said outsider face image data is correlated to said outsider prohibited behavior condition,

said behavior monitoring unit executes a face recognition process to judge whether or not face images indicated by said outsider face image data are included in a dynamic image indicated by said dynamic image data in order to monitor said specific persons which behave in said area,

said behavior monitoring unit generates as said behavior information, behaviors of said specific persons included in a dynamic image indicated by said dynamic image data when face images indicated by said outsider face image data are included in a dynamic image indicated by said dynamic image data,

said behavior judging unit selects said outsider prohibited behavior condition corresponding to said outsider face image data from said plurality of prohibited behavior conditions stored in said storage unit, and

said behavior judging unit produces an alarm when a behavior indicated by said outsider prohibited behavior condition coincides with behaviors indicated by said behavior information.

9. The behavior monitoring system according to claim 1,

wherein a gate is provided at an entrance of said area, and

said entry monitoring unit controls said gate to refuse said specific persons to enter said area when said specific piece of identification information does not exist.

10. The behavior monitoring system according to claim 1,

wherein said area includes predetermined region, and

said plurality of prohibited behavior conditions stored in said storage unit indicates permission or prohibition of entry to said predetermined region, for said plurality of persons, respectively.

11. The behavior monitoring system according to claim 10, wherein permission of entry to said predetermined region is determined based on date or time zone in a day.

12. The behavior monitoring system according to claim 10,

wherein said area further includes a plurality of regions, and

said plurality of prohibited behavior conditions stored in said storage unit respectively indicate for said entry-permitted persons group, orders for passing through said plurality of regions in entering and leaving said predetermined region.

13. The behavior monitoring system according to claim 1,

wherein said area includes predetermined region, and

said plurality of prohibited behavior conditions stored in said storage unit indicates times in which said plurality of persons can stay in said predetermined region, respectively.

14. The behavior monitoring system according to claim 1,

wherein said area includes a predetermined region, and

said interrelationships indicate that a first person and a second person of said plurality of persons approach one another in said predetermined region such that a distance between said first person and said second person is shorter than a predetermined distance.

15. The behavior monitoring system according to claim 1,

wherein said area includes a predetermined region, and

said interrelationships indicate whether or not a first person and a second person of said plurality of persons belong to a same group in said predetermined region.

16. The behavior monitoring system according to claim 1, wherein said interrelationships indicate that a first person and a second person of said plurality of persons depart from one another in said area such that a distance between said first person and said second person is a predetermined distance or longer.

17. The behavior monitoring system according to claim 1,

wherein said area includes a specific place, and

said interrelationships indicate that a first person and a second person of said plurality of persons depart from one another in said area except said specific place such that a distance between said first person and said second person is a predetermined distance or longer.

18. A behavior monitoring method comprising:

monitoring as specific persons, persons which enter an area;

judging, by referring to a database in which a plurality of pieces of identification information respectively identifying a plurality of persons and a plurality of prohibited behavior conditions indicating behaviors respectively prohibited to said plurality of persons in said area and being correlated with interrelationships of the persons, whether or not a specific piece of identification information identifying said specific persons exists in said plurality of pieces of identification information so as to specify said interrelationships of said specific persons;

monitoring, based on said specific piece of identification information, said specific persons which behave in said area when a piece of identification information coincident with said specific piece of identification information exists in said plurality of pieces of identification information;

generating behavior information indicating behaviors of said specific persons;

selecting, by referring to said database, specific prohibited behavior conditions corresponding to said specific piece of identification information from said plurality of prohibited behavior conditions based on said specified interrelationships of said specific persons; and

producing an alarm when behaviors indicated by said specific prohibited behavior conditions coincide with behaviors indicated by the piece of said behavior information.

19. A non-transitory computer-readable recording medium which records a computer program for causing a computer to execute a behavior monitoring method comprising:

monitoring as specific persons persons which enter an area;

judging by referring to a database in which a plurality of pieces of identification information respectively identifying a plurality of persons and a plurality of prohibited behavior conditions indicating behaviors respectively prohibited to said plurality of persons in said area and being correlated with interrelationships of the persons, whether or not a specific piece of identification information identifying said specific persons exists in said plurality of pieces of identification information so as to specify said interrelationships of said specific persons;

monitoring, based on said specific piece of identification information, said specific persons which behave in said area when a piece of identification information coincident with said specific piece of identification information exists in said plurality of pieces of identification information;

generating behavior information indicating behaviors of said specific persons;

selecting, by referring to said database, specific prohibited behavior conditions corresponding to said specific piece of identification information from said plurality of prohibited behavior conditions based on said specified interrelationships of said specific persons; and

producing an alarm when behaviors indicated by said specific prohibited behavior conditions coincide with behaviors indicated by the piece of said behavior information.