PERSON SEARCH METHOD AND DEVICE FOR SEARCHING PERSON STAYING ON PLATFORM

• US 20160012280A1
• Filed: 02/18/2014
• Published: 01/14/2016
• Est. Priority Date: 02/28/2013
• Status: Active Grant

• Current Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Original Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Inventors: ITO, Wataru, AKIMOTO, Yohei
• Granted Patent: US 9,589,181 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion

Abstract

Provided is a suspicious person detection method. First, a normal similar facial image search is carried out. Next, facial images, which are detected automatically from the input images and specified manually, are specified to be determined. Next, similar faces are searched for limited time on a time axis on the database. Next, the number of search results that distance between the features is lower than predetermined value is calculated and it is determined that the number of appearances is large and a possibility of a prowling person is high if the number of cases is large, and otherwise a possibility of prowling person is low. Last, a similarity between a facial image of a pre-registered residents and a facial image of a person whose number is large is calculated, and it is re-determined that the person is residents if the similarity is high, regardless of the determination.

Claims

1. A person search method comprising the steps of:

creating a database by detecting facial images in input images, extracting features from the facial images, and registering the features in the database together with time information;

specifying facial images which are detected automatically or specified manually from the input images as facial images to be determined;

searching the database created in the creating step for similar faces for limited time on a time axis;

calculating the number of cases with higher similarity than a predetermined value among search results of the searching step, and determining that the number of appearances is large and a possibility of a prowling person is high if the number of cases is large, and that the number of appearances is small and a possibility of prowling person is low if the number of cases is small; and

calculating similarity between a facial image of a pre-registered non-suspicious person and a facial image of a person whose appearance number is large, and re-determining that the person is not a suspicious person if the similarity is high, regardless of determination of the determining step.

2. A person search method comprising:

a first step of receiving face detection data including time information and a feature extracted from a facial image detected from images of a plurality of cameras, and registering the face detection data in a first database and a second database depending on attributes of the cameras;

a second step of grouping records of a same person by performing similarity search on at least one of the first database and the second database using, as a key, a face of one record which is registered in the first database and has undetermined estimated person ID, assigning an estimated person ID to the record used as a search key based on a predetermined first rule, and updating a white list retaining the assigned estimated person ID in association with a features thereof;

a third step of detecting a suspicious person candidate by creating a suspicious person appearance list using the white list based on a predetermined second rule, and performing similarity search for a suspicious person list using, as a key, a face of face detection data newly registered in at least the second database; and

a fourth step of adding at least a portion of the face detection data, used as a key in the third step, as appearance history to the suspicious person appearance list when the suspicious person candidate is detected, and determining whether or not the suspicious person candidate corresponds to a suspicious person based on a predetermined third rule from the appearance history retained in the suspicious person appearance list.

3. The person search method of claim 2, wherein the face detection data includes orientation information of the detected face, and the white list retains representative features corresponding to the information for respective facial orientations and a total representative feature corresponding to all the facial orientations.

4. The person search method of claim 2, wherein the predetermined second rule is characterized in that if there is no result with similarity greater than a predetermined value by performing similarity search on the white list using, as a key, a face of face detection data newly registered in the second database, the face is newly registered in the suspicious person appearance list.

5. The person search method of claim 3, wherein the predetermined first rule includes at least one of:

a sub-rule of extracting, from results of the search for the first database in the second step, candidates for the same person which have similarity with the total representative feature being equal to or greater than a first threshold and to which the estimated person IDs have been assigned, and if records of the most frequent estimated person ID account for more than a predetermined first percentage in the candidates, or if records of the estimated person ID having a same camera ID and continuity in capturing time are found in the candidates, assigning the same ID as the estimated person ID to a record used as the key face and a record whose estimated person ID is undetermined among candidates for the same person, and

a sub-rule of using, from results of the search for at least one of the first database and the second database in the second step, the results having similarity with the representative feature having facial orientation close to facial orientation of the key face being equal to or greater than a second threshold, as candidates for the same person, and if records to which the same estimated person ID has been assigned account for more than a predetermined second percentage in the results, assigning the same ID as the estimated person ID to a record used as the key face and a record whose estimated person ID is undetermined among candidates for the same person.

6. The person search method of claim 2, wherein the predetermined third rule is, without being directly related to a similarity learning machine, based on whether or not one or more of following propositions is true or false,

(i) when a person enters premises or a building, or is moving therein in a legitimate manner, it does not match the order in which the person should be captured by the plurality of cameras,

(ii) it is in a time zone that an owner, a resident or a relevant person of the premises or building rarely appear,

(iii) the person is slow compared to a normal moving speed, or turns back on the way,

(iv) no specific event that would occur before and after appearance of the owner, the resident or the relevant person, is detected, and

(v) the person appears at a time different from a scheduled time of a visitor notified previously, and there are no other records that the person is captured by the same monitoring camera at that time.

7. A device for searching a person staying on a platform comprising the steps of:

creating a database by detecting facial images in input images from cameras for capturing a station platform, extracting features from the facial images, and registering the features in the database together with time information;

searching for similar features, which are similar, on the database created in the creating step for a limited time on a time axis wider than a time period between arrival and departure of a train;

determining whether or not a person is staying on the platform by comparison between the number of extractions obtained by the searching step and the number of registrations which allows a person to be expected as a staying person;

registering the person who is determined as the staying person in a black list; and

when a feature similar to the registered person is retrieved from the database, notifying that.