Information processing apparatus, method of controlling the same, and storage medium

• US 10,353,954 B2
• Filed: 09/09/2016
• Issued: 07/16/2019
• Est. Priority Date: 09/11/2015
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Matsushita, Masahiro, Shiiyama, Hirotaka
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US15/260,822
• Publication Number:
  US 20170075993A1
• Time in Patent Office: 1,040 Days
• Field of Search: None
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/96 Management of image or video recognition tasks
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

This invention prevents a collation database from becoming bulky and shortening the delay time needed from person detection to registration in the collation database. An information processing apparatus comprises an acquiring unit which acquires a video, a detecting unit which detects a whole body or a part of a person from at least one frame of the acquired video, a tracking unit which tracks the whole body or the part of the person detected, and a registering unit which registers, in a database, a feature amount extracted from the whole body or the part of the person tracked during a first period from a timing of a start of tracking of the whole body or the part of the person by the tracking unit to a timing before an end of the tracking by the tracking unit.

Claims

1. An information processing apparatus which extracts a feature amount of a person in frames received from at least one camera and registers the feature amount in a collation database so as to search for a person inputted as a query by referring to the collation database, comprising:

one or more processors; and

at least one memory coupled to the one or more processors, the at least one memory storing instructions which, when executed by the one or more processors, cause the information processing apparatus to;

receive, via a communication interface for communicating with at least one camera, a frame sequentially from the camera as a current frame of a live video;

detect a person from the current frame;

if a new person is detected, set the new person as a target to be tracked;

track, in the frames following the current frame, the person set as the target to be tracked until the person is not detected;

extract a feature amount of the person from each frame during the tracking of the person;

obtain an elapsed time from a start of the tracking of the person set as the target; and

if the tracking of the person is ended, select a predetermined number of feature amounts among feature amounts extracted from frames received during the tracking of the person and register the selected feature amounts in the collation database; and

if the elapsed time from the start of the tracking of the person reaches a first period though the tracking of the person is continued, select the predetermined number of feature amounts among the feature amounts extracted from frames received during the first period and register the selected feature amounts in the collation database,

wherein the features amounts are registered by causing the information processing apparatus to;

classify faces of the tracked persons into a plurality of predetermined groups on the basis of each of directions of the faces; and

select, from the classified groups, faces in a number not more than the predetermined number, and

wherein, in the collation database, the feature amounts of the selected faces are registered.

2. The information processing apparatus according to claim 1, wherein the person set as the target to be tracked is detected after the first period, in the collation database, the predetermined number of feature amounts among features amounts extracted from frames received during a second period after the end of the first period is registered if the tracking of the person is continued, and

wherein the first period is shorter than the second period.

3. The information processing apparatus according to claim 2, wherein in the collation database, the predetermined feature amounts among feature amounts extracted from frames every time the second period has elapsed are registered after the end of the first period if the tracking of the person is continued.

4. The information processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, further cause the information processing apparatus to:

hold the feature amount extracted from the tracked person in the current frame so as to belong to a classified group.

5. The information processing apparatus according to claim 4, wherein the groups in the number not more than a first predetermined number are selected based on at least one of a size of the face, a blur, a closed eye, and an open mouth.

6. The information processing apparatus according to claim 4, wherein, from the plurality of predetermined groups, groups in a number not more than a second predetermined number are selected based on priorities, and feature amounts in the number not more than a first predetermined number for each of the selected groups are selected, and

wherein the priorities are set in advance for the plurality of groups.

7. The information processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, further cause the information processing apparatus to:

input a query video; and

perform collation with the feature amounts registered in the collation database based on a feature amount of a person in the input query video.

8. A method of controlling an information processing apparatus which extracts a feature amount of a person in frames received from at least one camera and registers the feature amount in a collation database so as to search for a person inputted as a query by referring to the collation database, comprising:

receiving, via a communication interface for communicating with at least one camera, a frame sequentially from the camera as a current frame of a live digital image video;

detecting a person from the received current frame;

setting, if a new person is detected in the detecting, the new person as a target to be tracked;

tracking, in the frames following the current frame, the person set as the target to be tracked;

extracting a feature amount of the person from each frame during tracking the person;

obtaining an elapsed time from a start of the tracking of the person set as the target to be tracked;

selecting, if the tracking of the person is ended, a predetermined number of feature amounts among feature amounts extracted from frames received during tracking the person and registering the selected feature amounts in the collation database; and

selecting, if the elapsed time from the start of the tracking of the person reaches a first period while tracking the person continues, the predetermined number of feature amounts among the feature amounts extracted from frames received during the first period and registering the selected features amounts in the collation database,

wherein the features amounts are registered by;

classifying faces of the tracked persons into a plurality of predetermined groups on the basis of each of directions of the faces; and

selecting, from the classified groups, faces in a number not more than a first predetermined number, and

wherein, in the collation database, the feature amounts of the selected faces are registered.

9. A non-transitory computer-readable storage medium storing a computer program which causes a computer to executes steps of a method of controlling an information processing apparatus which extracts a feature amount of a person in frames received from at least one camera and registers the feature amount in a collation database so as to search for a person inputted as a query by referring to the collation database, the method comprising:

receiving, via a communication interface for communicating with at least one camera, a frame sequentially from the camera as a current frame of a live video;

detecting a person from the received current frame;

setting, if a new person is detected in the detecting, the new person as a target to be tracked;

tracking, in the frames following the current frame, the person set as the target to be tracked;

extracting a feature amount of the person from each frame during tracking the person;

obtaining an elapsed time from a start of the tracking of the person set as the target to be tracked;

selecting, if the tracking of the person is ended, a predetermined number of feature amounts among feature amounts extracted from frames received during tracking the person and registering the selected feature amounts in the collation database; and

selecting, if the elapsed time from the start of the tracking of the person reaches a first period while tracking the person continues, the predetermined number of feature amounts among the feature amounts extracted from frames received during the first period and registering the selected features amounts in the collation database,

wherein the features amounts are registered by;

classifying faces of the tracked persons into a plurality of predetermined groups on the basis of each of directions of the faces; and

selecting, from the classified groups, faces in a number not more than a first predetermined number, and

wherein, in the collation database, the feature amounts of the selected faces are registered.

10. An information processing apparatus which extracts a feature amount of a person in frames received from at least one camera and registers the feature amount in a collation database so as to search for a person inputted as a query by referring to the collation database, the information processing apparatus comprising:

one or more processors; and

at least one memory coupled to the one or more processors, the at least one memory storing instructions which, when executed by the one or more processors, cause the information processing apparatus to;

receive a frame sequentially from the camera as a video;

detect a person from the frame;

if a new person is detected, set the new person as a target to be tracked;

track, in the frames following the frame, the person set as the target to be tracked until the person is not detected;

extract a feature amount of the person set as the target to be tracked;

obtain an elapsed time from a start of the tracking of the person set as the target;

select, if the tracking of the person is ended a predetermined number of feature amounts among feature amounts extracted from frames received during tracking the person and register the selected feature amounts in the collation database; and

select, if the elapsed time from the start of the tracking the person reaches a first period while tracking the person continues, the predetermined number of feature amounts among feature amounts extracted from frames received during the first period and register the selected features amounts in the collation database,

wherein the feature amounts are registered by causing the information processing apparatus to;

classify faces of the tracked persons into a plurality of predetermined groups on the basis of each of directions of the faces; and

select, from the classified groups, faces in a number not more than a first predetermined number, and

wherein, in the collation database, the feature amounts of the selected faces are registered.