Device and method for detecting specified objects in images using metadata

• US 9,665,798 B2
• Filed: 05/24/2012
• Issued: 05/30/2017
• Est. Priority Date: 05/24/2012
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Watanabe, Yuki, Hiroike, Atsushi
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): BEG, SAMAH A
• Application Number:
  US14/402,819
• Publication Number:
  US 20150286896A1
• Time in Patent Office: 1,832 Days
• Field of Search: None
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 2201/07 Target detection
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries

Abstract

The purpose of the present invention is to provide an image analysis technique enabling a detection subject to be rapidly detected from image data. This image analysis device generates metadata for a query image containing the detection subject, and using the metadata, narrows down the image data serving as the search subject beforehand and then conducts object detection.

Claims

1. An image analysis device comprising:

at least one processor executing stored instructions to;

receive query image data including an image of an object to be searched;

determine and identify one or more images similar to the image of the object to be searched, wherein the one or more similar images are selected by a user during a first interactive selection;

generate metadata of the query image data using image data and its bibliographic information associated with the selected one or more similar images, the one or more similar images being stored in an image database;

calculate a score for each metadata based on a frequency by which each of the metadata appears in the bibliographic information of the image data;

determine and identify one or more metadata to be used as a search key, the determination being based on one or more of the calculated score for each metadata and a second interactive selection different from the first interactive selection by the user of the one or more metadata, wherein the one or more metadata of the search key is selected by the user and differently combinable when used;

extract one or more of the image data stored in the image database, the extracted image data having the bibliographic information matching with the one or more metadata of the search key;

detect, in one or more of the extracted image data, a region including the image of the object; and

output a detected result.

2. The image analysis device according to claim 1, wherein the at least one processor further executes stored instructions to:

search the image data stored in the image database, the searched image data being similar to the query image data,

and generate the metadata using the bibliographic information of the searched image data.

3. The image analysis device according to claim 2, wherein the at least one processor further executes stored instructions to:

receive a similar image designation specifying the image data among the searched image data that is used for generating the metadata of the query image data,

search, among the image data stored in the image database, the image data similar to the query image data and the image data specified by the similar image designation,

and generate the metadata using the bibliographic information of the image data acquired by the search.

4. The image analysis device according to claim 1, wherein the at least one processor further executes stored instructions to

extract the image data associated with the bibliographic information matching with the metadata, using the metadata within a predetermined range sequentially from higher one of the scores as a search key.

5. The image analysis device according to claim 1, wherein the at least one processor further executes stored instructions to:

receive a metadata designation specifying the metadata used for extracting the image data matching with the bibliographic information,

and extract the image data that is associated with the bibliographic information matching with the specified metadata.

6. The image analysis device according to claim 5,

further comprising a display that displays a number of the image data which is a target for the at least one processor to detect a region including an image of the object, and that displays a processing time to perform the detection,

and wherein the at least one processor further executes stored instructions to recalculate the number of the image data and the processing time every time when receiving the metadata designation, and reflects a result of the recalculation on the display.

7. The image analysis device according to claim 1, wherein the at least one processor further executes stored instructions to:

calculate a distance between a feature vector of a partial region of the image data and a feature vector of the query image data,

and determine, depending on whether the distance is within a predetermined range, whether the object included in the query image data is included in the partial region.

8. The image analysis device according to claim 1, wherein the at least one processor further executes stored instructions to

output, along with a detected result, a number of the detected object in the image data.

9. The image analysis device according to claim 1, wherein the at least one processor further executes stored instructions to

output, along with a detected result, a detection reliability of the detected object in the image data.

10. The image analysis device according to claim 1, wherein the at least one processor further executes stored instructions to:

generate metadata of one of the image data stored in the image database using other one of the image data stored in the image database, and add the generated metadata as the bibliographic information,

and extract, using the bibliographic information where the metadata is added, the image data among the image data stored in the image database, the bibliographic information of the extracted image data matching with the metadata of the query image data.

11. The image analysis device according to claim 1,

wherein the at least one processor further executes stored instructions to detect, among one or more of the extracted image data, a region including an image of the object and a region including an image of an object included in the acquired image data in the search.

12. The image analysis device according to claim 11, wherein the at least one processor further executes stored instructions to:

receive a detection target designation specifying, among the acquired image data, the image data including an object that is to be detected along with the object included in the query image data,

and detect, among one or more of the extracted image data, a region including an image of the object and a region including an image of an object included in the specified image data.

13. An image analysis system comprising:

the image analysis device according to claim 1; and

an image database that stores image data and its bibliographic information associated with each other,

wherein the at least one processor further executes stored instructions to generate metadata of the query image data using the image database.

14. An image analysis method comprising:

receiving query image data including an image of an object to be searched;

determining and identifying one or more images similar to the image of the object to be searched, wherein the one or more similar images are selected by a user during a first interactive selection;

generating metadata of the query image data using image data and its bibliographic information associated with the selected one or more similar images, the one or more similar images being stored in an image database;

calculating a score for each metadata based on a frequency by which each of the metadata appears in the bibliographic information of the image data;

determining and identifying one or more metadata to be used as a search key, the determining being based on one or more of the calculated score for each metadata and a second interactive selection different from the second interactive selection by the user of the one or more metadata, wherein the one or more metadata of the search key is selected by the user and differently combinable when used;

extracting one or more of the image data stored in the image database, the extracted image data having the bibliographic information matching with the one or more metadata of the search key;

detecting, in one or more of the extracted image data;

a region including an image of the object; and

outputting a result detected in the object region detection step.