Image recognition device and image recognition method

• US 8,108,408 B2
• Filed: 06/12/2008
• Issued: 01/31/2012
• Est. Priority Date: 06/14/2007
• Status: Active Grant

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Kondo, Satoshi, Yoneda, Aki, Mizuno, Ko, Ogura, Takashi
• Primary Examiner(s): Ng, Amy
• Application Number:
  US12/521,891
• Publication Number:
  US 20100077003A1
• Time in Patent Office: 1,328 Days
• Field of Search: 707/705, 707/706, 707/732, 707/728, 707/749
• US Class Current: 707/749
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations

Abstract

An image recognition device includes: an image input unit obtaining an input image corresponding to a target image to be recognized; a similar image search unit searching out at least one similar image similar to the target image obtained by the image input unit from among images generated independently of the image recognition device, and unintentionally obtained; a keyword extraction unit extracting a plurality of keywords corresponding to candidates for a recognition result of the target image; a keyword analysis unit determining at least one likely keyword likely to be the recognition result of the target image from among the plurality of keywords extracted by the keyword extraction unit by analyzing the plurality of keywords; and a presentation unit outputting at least one likely keyword determined by the keyword analysis unit, as the recognition result of the target image.

Claims

1. An image recognition device which outputs, as a recognition result, information indicating what an input image is, said image recognition device comprising:

a processor;

an image input unit configured to obtain the input image which is a target image to be recognized;

a similar image search unit configured to (i) search out a plurality of similar images which are similar to the target image obtained by said image input unit from among images generated independently of said image recognition device, stored in a server device accessible via the Internet, and unintentionally obtained, and (ii) calculate a similarity between each of the similar images and the target image;

a keyword extraction unit configured to (i) extract a plurality of keyword sets each of which includes one or more keywords which are candidates for the recognition result of the target image, from attached information which is attached to each of the similar images searched out by said similar image search unit, and (ii) store each of the keyword sets of the respective similar images, in association with the similarity of a corresponding one of the similar images calculated by said similar image search unit;

a keyword analysis unit configured to (i) calculate weights of the respective keywords in each of the keyword sets extracted by said keyword extraction unit, the weights of the respective keywords in each of the keyword sets being based on the similarity between the target image and each of the similar images associated with a corresponding one of the keyword sets, and (ii) calculate sums of the weights of the respective keywords in each of the keyword sets; and

a presentation unit configured to output a plurality of the keywords in a descending order of the sums of the weights, the plurality of keywords being output as the candidates for the recognition result of the target image.

2. The image recognition device according to claim 1,

wherein said similar image search unit is configured to search out an image included in an Exif file as one of the plurality of similar images, and

said keyword extraction unit is configured to extract the keyword sets from header information in the Exif file.

3. The image recognition device according to claim 1,

wherein said similar image search unit is configured to search out at least one of the plurality of similar images from a Web page on the server device, and

said keyword extraction unit is configured to extract the plurality of keyword sets using, as a clue, tag information included in either the Web page or an other Web page associated with the Web page.

4. The image recognition device according to claim 1,

wherein said similar image search unit is configured to search out at least one of the plurality of similar images from a Web page on the server device, and

said keyword extraction unit is configured to extract, as each of the keywords which are included in the keyword sets, a peripherally-positioned word or phrase included in the Web page or an other Web page associated with the Web page.

5. The image recognition device according to claim 1, further comprising

an image specification unit configured to obtain a specification of a target area to be recognized within the target image obtained by said image input unit,

wherein said similar image search unit is configured to search out at least one similar image similar to a sub-image included in the target area obtained by said image specification unit.

6. The image recognition device according to claim 1, further comprising:

a classification database in which information indicating an association between each of image templates and a category of the image template are stored; and

a simple classification unit configured to determine the category of the target image by verifying the target image obtained by said image input unit with each of the image templates stored in said classification database,

wherein said similar image search unit is configured to search out at least one similar image which is similar to the target image and belongs to the category obtained as a result of the category determination made by said simple classification unit, using the result of the category determination as a keyword for search.

7. The image recognition device according to claim 1, further comprising:

a detailed information database in which attached information of each of target image candidates and detailed information of the attached information are stored in an associated manner; and

a pre-processing unit configured to obtain the attached information attached to the target image obtained by said image input unit, and obtain the detailed information of the attached information attached to the target image obtained by said image input unit by verifying the obtained attached information with the attached information registered in said detailed information database,

wherein said similar image search unit is configured to search out at least one similar image which is similar to the target image and relates to the detailed information obtained by said pre-processing unit using the detailed information as a keyword for search.

8. The image recognition device according to claim 7,

wherein each of the target image candidates is a shot image,

the attached information attached to each of the target image candidates indicates a geographical location at which the target image candidate was shot, and

the detailed information database includes, as the detailed information, information identifying a landmark located at or around the geographical location.

9. The image recognition device according to claim 8,

wherein the target image obtained by said image input unit is included in an Exif file, and

said pre-processing unit is configured to obtain the attached information from header information of the Exif file including the target image.

10. An image recognition method for outputting, as a recognition result, information indicating what an input image is, said image recognition method comprising:

obtaining the input image which is a target image to be recognized;

searching out a plurality of similar images which are similar to the target image obtained in said obtaining from among images generated independently of said image recognition method, stored in a server device accessible via the Internet, and unintentionally obtained;

calculating a similarity between each of the similar images and the target image;

extracting a plurality of keyword sets each of which includes one or more keywords which are candidates for the recognition result of the target image, from attached information which is attached to each of the similar images searched out in said searching out;

storing each of the keyword sets of the respective similar images, in association with the similarity of a corresponding one of the similar images calculated in said calculating;

calculating weights of the respective keywords in each of the keyword sets extracted in said extracting, the weights of the respective keywords in each of the keyword sets being based on the similarity between the target image and each of the similar images associated with a corresponding one of the keyword sets;

calculating sums of the weights of the respective keywords in each of the keyword sets; and

outputting a plurality of the keywords in a descending order of the sums of the weights, the plurality of keyword being output as the candidates for the recognition result of the target image.

11. A program recorded on a non-transitory computer-readable recording medium, said program causing a computer to execute the steps included in the image recognition method according to claim 10.