Person search method and device for searching person staying on platform

• US 9,589,181 B2
• Filed: 02/18/2014
• Issued: 03/07/2017
• Est. Priority Date: 02/28/2013
• Status: Active Grant

• Current Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Original Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Inventors: Akimoto, Yohei, Ito, Wataru
• Primary Examiner(s): Bekele, Mekonen
• Application Number:
  US14/771,004
• Publication Number:
  US 20160012280A1
• Time in Patent Office: 1,113 Days
• Field of Search: 382/196, 382/305, 382/195, 382/115, 382/103
• US Class Current: 1/1
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion

Abstract

Provided is a suspicious person detection method. First, a normal similar facial image search is carried out. Next, facial images, which are detected automatically from the input images and specified manually, are specified to be determined. Next, similar faces are searched for limited time on a time axis on the database. Next, the number of search results that distance between the features is lower than predetermined value is calculated and it is determined that the number of appearances is large and a possibility of a prowling person is high if the number of cases is large, and otherwise a possibility of prowling person is low. Last, a similarity between a facial image of a pre-registered residents and a facial image of a person whose number is large is calculated, and it is re-determined that the person is residents if the similarity is high, regardless of the determination.

Claims

1. A person search method comprising:

a first step of receiving face detection data including a feature extracted from a facial image detected from images of a plurality of cameras, an attribute of each camera and time information, and registering records including the face detection data in a first database or a second database depending on the attributes of the cameras;

a second step of grouping records of a same person by searching for similar features in at least one of the first database and the second database using, as a search key, a feature of any record which is registered in the first database and has undetermined estimated person ID, assigning an estimated person ID to the record used as the search key based on a predetermined first rule, and updating a white list retaining the assigned estimated person ID in association with the feature thereof;

a third step of detecting a suspicious person candidate by creating a suspicious person appearance list based on a predetermined second rule, and searching for similar features in the suspicious person appearance list using, as a search key, a feature of a record newly registered in at least the second database; and

a fourth step of adding an appearance history to the record having the feature used as the search key in the third step when the suspicious person candidate is detected, and determining whether or not the suspicious person candidate corresponds to a suspicious person based on a predetermined third rule from the appearance history retained in the suspicious person appearance list.

2. The person search method of claim 1, wherein the face detection data further includes orientation information of the detected facial image, and the records of the white list retain, as the features, representative features corresponding to the information for respective facial orientations and a total representative feature corresponding to all the facial orientations.

3. The person search method of claim 1, wherein the predetermined second rule is characterized in that if there is no result with similarity greater than a predetermined value by searching for similar features in the white list using, as the search key, the feature of the record newly registered in the second database, the record having the feature used as the search key is newly registered in the suspicious person appearance list.

4. The person search method of claim 2, wherein the predetermined first rule includes at least one of:

a first sub-rule of extracting, from results of the search for the first database in the second step, first candidates for the same person which have similarity with the total representative feature being equal to or greater than a first threshold and to which the estimated person IDs have been assigned, and if records of the most frequent estimated person ID account for more than a predetermined first percentage in the first candidates, or if records of the estimated person ID having a same camera ID and continuity in capturing time are found in the first candidates, assigning the same ID as the estimated person ID to the record having the feature used as the search key and a record whose estimated person ID is undetermined among the first candidates for the same person, and

a second sub-rule of extracting, from results of the search for at least one of the first database and the second database in the second step, second candidates for the same person which have similarity with the representative feature having facial orientation closest to facial orientation of the feature used as the search key being equal to or greater than a second threshold, and if records to which the same estimated person ID has been assigned account for more than a predetermined second percentage in the second candidates, assigning the same ID as the estimated person ID to the record having the feature used as the search key and a record whose estimated person ID is undetermined among the second candidates for the same person.

5. The person search method of claim 1, wherein the predetermined third rule is to check whether or not at least one of the following conditions is satisfied,

(i) when a person enters premises or a building, or is moving therein in a legitimate manner, it does not match the order in which the person should be captured by the plurality of cameras,

(ii) it is in a time zone that an owner, a resident or a relevant person of the premises or building rarely appear,

(iii) the person is slow compared to a normal moving speed, or turns back on the way,

(iv) no specific event that would occur before and after appearance of the owner, the resident or the relevant person, is detected, and

(v) the person appears at a time different from a scheduled time of a visitor notified previously, and there are no other records that the person is captured by the same monitoring camera at that time.

6. A device for searching a person staying on a platform, comprising:

means for creating a database by detecting facial images in input images from cameras for capturing a station platform, extracting features from the facial images, and registering the features in the database together with time information;

means for searching for, in the database created in the creating step, similar features among the features registered in a predetermined time period longer than a time period between arrival and departure of a train;

means for determining whether or not a person is staying on the platform by comparison between the number of the features of the person detected by said means for searching and the number of registrations which allows a person to be expected as a staying person;

means for registering the person who is determined as the staying person in a black list; and

means for, when a feature similar to the features of the registered person is retrieved from the database, notifying that.