SURVEILLANCE IMAGE RETRIEVAL APPARATUS AND SURVEILLANCE SYSTEM

• US 20110019003A1
• Filed: 07/21/2010
• Published: 01/27/2011
• Est. Priority Date: 07/22/2009
• Status: Active Grant

• Current Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Original Assignee: Hitachi Kokusai Electric Incorporated (Hitachi, Ltd.)
• Inventors: Asa, Yasuhiro, Hirai, Seiichi
• Granted Patent: US 9,342,744 B2
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: G06F 18/40 Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
• CPC Class Codes: G06V 10/987 with the intervention of an operator
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G06V 40/179 metadata assisted face recognition
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Disclosed is a surveillance image retrieval apparatus capable of generating an image representing a search result that a surveillant can visually confirm, even if a face image used is not suitable for the visual identification of a person. The surveillance image retrieval apparatus includes a surveillance image acquirer, which acquires surveillance images; a feature amount generator, which extracts a face image from the surveillance images and generates the feature amount of the extracted face image; an image feature amount memory for storing a face image, a surveillance image, and other metadata associated with an acquired feature amount; a face image designator, which selects an appropriate face image in accordance with a user'"'"'s input, and designates the selected face image as a query image for a person search; a similar image searcher for searching the series of surveillance images stored in the image feature amount memory for a face image similar to the designated face image; a presentation image selector, which examines surveillance images that are close to the search result in imaging time, and selects presentation surveillance images that include face images of the same person; and a screen data generator for outputting the presentation surveillance images selected.

Claims

1. A surveillance image retrieval apparatus comprising:

a surveillance image acquirer that acquires a series of surveillance images picked up by an imager;

a feature amount generator that extracts a face image from the surveillance images and generates the feature amount of the extracted face image;

an image feature amount memory that stores a face image and other metadata associated with an acquired feature amount;

a face image designator that selects an appropriate face image from face images stored in the image feature amount memory or in a surveillant device in accordance with an external input, and designates the selected face image as a query image for a person search;

a similar image searcher that searches the series of surveillance images stored in the image feature amount memory for a face image similar to the designated face image;

a presentation image selector that examines surveillance images that are close in imaging time to the similar image retrieved as a result of the search, and selects presentation surveillance images that contain face images of the same person as a person depicted by the similar image; and

a screen data generator that outputs the presentation surveillance images selected by the presentation image selector.

2. The surveillance image retrieval apparatus according to claim 1, wherein the presentation image selector includes:

a face image selector that calculates similarity between a face image representing the search result and a face image in a surveillance image preceding a surveillance image containing the face image representing the search result, and, when the calculated similarity is not greater than a threshold value, concludes that the face image in the surveillance image preceding the surveillance image containing the face image representing the search result depicts the same person as the face image representing the search result, and selects the face image in the surveillance image preceding the surveillance image containing the face image representing the search result;

a movement direction calculator that calculates the direction of movement of the region of the face image selected by the face image selector in a surveillance image; and

a determiner that selects as an optimum image a surveillance image containing a largest-size face image whose movement direction calculated by the movement direction calculator is deviated from the downward direction in the surveillance image by a value not greater than a threshold value.

3. The surveillance image retrieval apparatus according to claim 1, wherein the presentation image selector includes:

a face image selector that calculates similarity between a face image representing the search result and a face image in a surveillance image succeeding a surveillance image containing the face image representing the search result, and, when the calculated similarity is not greater than a threshold value, concludes that the face image in the surveillance image succeeding the surveillance image containing the face image representing the search result depicts the same person as the face image representing the search result, and selects the face image in the surveillance image succeeding the surveillance image containing the face image representing the search result;

a movement direction calculator that calculates the direction of movement of the region of the face image selected by the face image selector in a surveillance image; and

a determiner that selects as an optimum image a surveillance image containing a largest-size face image whose movement direction calculated by the movement direction calculator is deviated from the downward direction in the surveillance image by a value not greater than a threshold value.

4. The surveillance image retrieval apparatus according to claim 1, wherein the screen data generator generates a screen that simultaneously presents a search result surveillance image retrieved by the similar image searcher and an optimum image selected by the presentation image selector.

5. The surveillance image retrieval apparatus according to claim 1, wherein the surveillance images that are close in imaging time to the similar image retrieved as a result of the search are derived from images picked up by the same camera as the camera used for the surveillance images.

6. A surveillance system comprising:

a plurality of cameras that pick up surveillance images;

a surveillant device that designates a query image and outputs a presentation image; and

a surveillance image retrieval apparatus;

wherein the surveillance image retrieval apparatus includes

a surveillance image acquirer that acquires a series of surveillance images picked up by an imager,

a feature amount generator which extracts a face image from the surveillance images and generates the feature amount of the extracted face image,

an image feature amount memory that stores a face image and other metadata associated with an acquired feature amount,

a face image designator that selects an appropriate face image from face images stored in the image feature amount memory or in the surveillant device in accordance with an external input, and designates the selected face image as a query image for a person search,

a similar image searcher that searches the series of surveillance images stored in the image feature amount memory for a face image similar to the designated face image,

a presentation image selector that examines surveillance images that are close in imaging time to the similar image retrieved as a result of the search, and selects presentation surveillance images that contain face images of the same person as a person depicted by the similar image, and

a screen data generator that outputs the presentation surveillance images selected by the presentation image selector.

7. The surveillance system according to claim 6, wherein the surveillance images that are close in imaging time to the similar image retrieved as a result of the search are derived from images picked up by the same camera as the camera used for the surveillance images.