Image retrieving system, image classifying system, image retrieving program, image classifying program, image retrieving method and image classifying method

• US 7,440,638 B2
• Filed: 05/28/2004
• Issued: 10/21/2008
• Est. Priority Date: 05/30/2003
• Status: Expired due to Fees

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Nagahashi, Toshinori, Hyuga, Takashi
• Primary Examiner(s): Le; Brian Q
• Assistant Examiner(s): BLOOM, NATHAN J
• Application Number:
  US10/857,638
• Publication Number:
  US 20050008263A1
• Time in Patent Office: 1,607 Days
• Field of Search: 382/118, 382/181, 382/203, 382/206, 382/209, 382224-228, 707/3, 707/4, 707/6, 348/145
• US Class Current: 382/305
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

An image retrieving system is provided which is suitable for acquiring a retrieving result or a classifying result according to the desire of a user. The system extracts a noticing area from a retrieving key image and each retrieving object image, and a characteristic vector V of these images is generated on the basis of the extracted noticing area. An image similar to the retrieving key image is retrieved from a retrieving object image registration database on the basis of the generated characteristic vector V.

Claims

1. An image retrieving system for retrieving an image adapted for a given retrieving key image from plural retrieving object images on the basis of the given retrieving key image, the system comprising:

an extractor extracting a noticing area from the retrieving key image and each retrieving object image;

a face information judging section for judging face information of a person image included in the retrieving key image and each retrieving object image;

a similar degree judging section for judging similar degrees of a face of a person image included in the retrieving object image and a face of a person image included in the retrieving key image for each retrieving object image;

a generator generating a characteristic vector showing a character of the images on the basis of the extracted noticing area, on the basis of the face information judged by the face information judging section, and on the basis of a judging result of the similar degree judging section, with respect to the retrieving key image and each retrieving object image, the characteristic vector being constructed by a first element group according to a conspicuous degree of the extracted noticing area, a second element group according to face information f1 to f9 according to a direction, a size, and a gravity center position of the face of the person image included in the retrieving key image and judged by the face information judging section, and a third element group according to a similar degree judged by the similar degree judging section, f1 being a sum total of an area occupied by the face of the person image, f2 being an average value of the area occupied by the face of the person image, f3 being a dispersion of the area occupied by the face of the person image, f4 being an average of a horizontal direction front face directing degree, f5 being a dispersion of the horizontal direction front face directing degree, f6 being an average value of a perpendicular direction front face directing degree, f7 being a dispersion of the perpendicular direction front face directing degree, f8 being an average value of each gravity center position of the face of the person image, and f9 being a dispersion of each gravity center position of the face of the person image; and

a retriever retrieving the image adapted for the retrieving key image from the plural retrieving object images on the basis of the generated characteristic vector.

2. An image retrieving system for retrieving an image adapted for a given retrieving key image from plural retrieving object images on the basis of the given retrieving key image, the image retrieving system comprising:

a retrieving object image storage database for storing the plural retrieving object images;

a retrieving key image input device for inputting the retrieving key image;

an extractor for extracting a noticing area from the images with respect to the retrieving key image inputted by the retrieving key image input device and each retrieving object image of the retrieving object image storage database;

a face information judging section for judging face information of a person image included in the images with respect to the retrieving key image and each retrieving object image;

a similar degree judging section for judging similar degrees of a face of a person image included in the retrieving object image and a face of a person image included in the retrieving key image for each retrieving object image;

a characteristic vector generator for generating a characteristic vector showing a character of the images on the basis of the noticing area extracted by the extractor, on the basis of the face information judged by the face information judging section, and on the basis of the judging result of the similar degree judging section, with respect to the retrieving key image and each retrieving object image, the characteristic vector being constructed by a first element group according to a conspicuous degree of the noticing area, a second element group according to face information f1 to f9 according to a direction, a size, and a gravity center position of the face of the person image included in the retrieving key image for each retrieving object image and judged by the face information judging section, and a third element group according to a similar degree judged by the similar degree judging section f1 being a sum total of an area occupied by the face of the person image, f2 being an average value of the area occupied by the face of the person image, f3 being a dispersion of the area occupied by the face of the person image, f4 being an average of a horizontal direction front face directing degree, f5 being a dispersion of the horizontal direction front face directing degree, f6 being an average value of a perpendicular direction front face directing degree, f7 being a dispersion of the perpendicular direction front face directing degree, f8 being an average value of each gravity center position of the face of the person image, and f9 being a dispersion of each gravity center position of the face of the person image; and

an image retrieving section for retrieving the image adapted for the retrieving key image from the retrieving object image storage database on the basis of the characteristic vector generated by the characteristic vector generator.

3. The image retrieving system according to claim 2, wherein the image retrieving section is constructed such that the retrieving object image corresponding to the characteristic vector having a smallest inter-vector distance with respect to the characteristic vector of the retrieving key image is retrieved and outputted from the retrieving object image storage database.

4. The image retrieving system according to claim 2, wherein the image retrieving section is constructed such that each retrieving object image is classified into plural groups on the basis of the inter-vector distance between the characteristic vectors of the retrieving object image, and all the retrieving object images to which the retrieving key image belongs within the plural groups are retrieved and outputted from the retrieving object image storage database.

5. The image retrieving system according to claim 2, wherein, when the aspect ratios of the retrieving key image and the retrieving object image or the aspect ratios of the retrieving object images are different from each other, the characteristic vector generator overlaps first and second images having the different aspect ratios, and generates the characteristic vector of the first image with respect to the overlapping area within the first image, and also generates the characteristic vector of the second image with respect to the overlapping area within the second image.

6. An image classifying system for classifying plural classifying object images comprising:

an extractor extracting a noticing area from the classifying object images for each classifying object image;

a generator generating a characteristic vector showing a character of the classifying object images on the basis of the extracted noticing area, the characteristic vector being constructed by a first element group according to a conspicuous degree of the extracted noticing area, a second element group according to face information f1 to f9 according to a direction, a size, and a gravity center position of a face of a person image included in each classifying object image, and a third element group according to a similar degree judged by a similar degree judging section, f1 being a sum total of an area occupied by the face of the person image, f2 being an average value of the area occupied by the face of the person image, f3 being a dispersion of the area occupied by the face of ther person image, f4 being an average of a horizontal direction front face directing degree, f5 being a dispersion of the horizontal direction front face directing degree, f6 being an average value of a perpendicular direction front face directing degree, f7 being a dispersion of the perpendicular direction front face directing degree, f8 being an average value of each gravity center position of the face of the person image, and f9 being a dispersion of each gravity center position of the face of the person image; and

a classifier classifying each classifying object image into plural groups on the basis of the generated characteristic vector.

7. An image classifying system for classifying plural classifying object images comprising:

a classifying object image storage database for storing the plural classifying object images;

an extractor for extracting a noticing area from the classifying object images for each classifying object image of the classifying object image storage database;

a characteristic vector generator for generating a characteristic vector showing a character of the classifying object image for each classifying object image on the basis of the noticing area extracted by the extractor, the characteristic vector being constructed by a first element group according to a conspicuous degree of the noticing area extracted by the extractor, a second element group according to face information f1 to f9 according to a direction, a size, and a gravity center position of a face of a person image included in each classifying object image, and a third element group according to a similar degree judged by a similar degree judging section, f1 being a sum total of an area occupied by the face of the person image, f2 being an average value of the area occupied by the face of the person image, f3 being a dispersion of the area occupied by the face of ther person image, f4 being an average of a horizontal direction front face directing degree, f5 being a dispersion of the horizontal direction front face directing degree, f6 being an average value of a perpendicular direction front face directing degree, f7 being a dispersion of the perpendicular direction front face directing degree, f8 being an average value of each gravity center position of the face of the person image, and f9 being a dispersion of each gravity center position of the face of the person image; and

a classifier for classifying each classifying object image into plural groups on the basis of the characteristic vector generated by the characteristic vector generator.

8. The image classifying system according to claim 7, wherein the image classifying system further comprises:

a face information judging section for judging face information of a person image included in the classifying object image for each classifying object image; and

wherein the characteristic vector generator generates the characteristic vector showing the character of the classifying object image for each classifying object image on the basis of the face information judged by the face information judging section.

9. The image classifying system according to claim 7, wherein the image classifying system further comprises:

the similar degree judging section for judging similar degrees of a face of a person image included in the classifying object image for each classifying object image and a face of a person image included in a specific image; and

wherein the characteristic vector generator generates the characteristic vector showing the character of the classifying object image for each classifying object image on the basis of the judging result of the similar degree judging section.

10. The image classifying system according to claim 7, wherein the classifier classifies each classifying object image into the plural groups on the basis of the inter-vector distance between the characteristic vectors of the classifying object image, and a predetermined number of classifying object images are retrieved and outputted from the classifying object image storage database for each group.

11. The image classifying system according to claim 7, wherein, when the aspect ratios of the classifying object images are different from each other, the characteristic vector generator overlaps first and second images having the different aspect ratios, and generates the characteristic vector of the first image with respect to the overlapping area within the first image, and also generates the characteristic vector of the second image with respect to the overlapping area within the second image.

12. A computer-readable medium storing an image retrieving program for retrieving an image adapted for a given retrieving key image from plural retrieving object images on the basis of the given retrieving key image, the image retrieving program comprising:

an extractor for extracting a noticing area from the images with respect to the retrieving key image inputted by a retrieving key image input device and each retrieving object image of a retrieving object image storage database about a computer capable of utilizing the retrieving object image storage database for storing the plural retrieving object images and the retrieving key image device for inputting the retrieving key image;

a generator for generating a characteristic vector showing a character of the images on the basis of the noticing area extracted by the extractor with respect to the retrieving key image and each retrieving object image, the characteristic vector being constructed by a first element group according to a conspicuous degree of the noticing area, a second element group according to face information f1 to f9 according to a direction, a size, and a gravity center position of a face of a person image included in the retrieving key image and judged by a face information judging section, and a third element group according to a similar degree judged by a similar degree judging section, f1 being a sum total of an area occupied by the face of the person image, f2 being an average value of the area occupied by the face of the person image, f3 being a dispersion of the area occupied by the face of ther person image, f4 being an average of a horizontal direction front face directing degree, f5 being a dispersion of the horizontal direction front face directing degree, f6 being an average value of a perpendicular direction front face directing degree, f7 being a dispersion of the perpendicular direction front face directing degree, f8 being an average value of each gravity center position of the face of the person image, and f9 being a dispersion of each gravity center position of the face of the person image; and

a retriever for retrieving the image adapted for the retrieving key image from the retrieving object image storage database on the basis of the characteristic vector generated by the generator.

13. A computer-readable medium storing an image classifying program for classifying plural classifying object images, the image classifying program comprising:

an extractor for extracting a noticing area from the classifying object image for each classifying object image of a classifying object image storage database with respect to a computer capable of utilizing the classifying object image storage database for storing the plural classifying object images;

a generator for generating a characteristic vector showing a character of the classifying object image for each classifying object image on the basis of the noticing area extracted by the extractor, the characteristic vector being constructed by a first element group according to a conspicuous degree of the noticing area, a second element group according to face information f1 to f9 according to a direction, a size, and a gravity center position of a face of a person image included in each classifying object image, and a third element group according to a similar degree judged by a similar degree judging section, f1 being a sum total of an area occupied by the face of the person image, f2 being an average value of the area occupied by the face of the person image, f3 being a dispersion of the area occupied by the face of ther person image, f4 being an average of a horizontal direction front face directing degree, f5 being a dispersion of the horizontal direction front face directing degree, f6 being an average value of a perpendicular direction front face directing degree, f7 being a dispersion of the perpendicular direction front face directing degree, f8 being an average value of each gravity center position of the face of the person image, and f9 being a dispersion of each gravity center position of the face of the person image; and

a classifier for classifying each classifying object image into plural groups on the basis of the characteristic vector generated by the generator.

14. An image retrieving method for retrieving an image adapted for a given retrieving key image from a retrieving object image storage database storing plural retrieving object images thereto on the basis of the given retrieving key image, the method comprising:

inputting the retrieving key image;

extracting a noticing area from the inputted retrieving key image;

generating a characteristic vector showing a character of the retrieving key image on the basis of the extracted noticing area;

extracting the noticing area from the retrieving object image;

generating a characteristic vector showing a character of the retrieving object image on the basis of the extracted noticing area, the characteristic vector being constructed by a first element group according to a conspicuous degree of the noticing area, a second element group according to face information f1 to f9 according to a direction, a size, and a gravity center position of a face of a person image included in the retrieving key image, and a third element group according to a similar degree based on judging face information of the person image in the retrieving key image and in each retrieving object image, f1 being a sum total of an area occupied by the face of the person image, f2 being an average value of the area occupied by the face of the person image, f3 being a dispersion of the area occupied by the face of ther person image, f4 being an average of a horizontal direction front face directing degree, f5 being a dispersion of the horizontal direction front face directing degree, f6 being an average value of a perpendicular direction front face directing degree, f7 being a dispersion of the perpendicular direction front face directing degree, f8 being an average value of each gravity center position of the face of the person image, and f9 being a dispersion of each gravity center position of the face of the person image;

repeatedly executing the extracting and generating for each retrieving object image of the retrieving object image storage database; and

retrieving the image adapted for the retrieving key image from the retrieving object image storage database on the basis of the generated characteristic vectors.

15. An image classifying method for classifying plural classifying object images, the method comprising:

extracting a noticing area from the classifying object images;

generating a characteristic vector showing a character of the classifying object image on the basis of the extracted noticing area, the characteristic vector being constructed by a first element group according to a conspicuous degree of the extracted noticing area, a second element group according to face information f1 to f9 according to a direction, a size, and a gravity center position of a face of a person image included in each classifying object image, and a third element group according to a similar degree based on judging face information of the person image in each classifying object image, f1 being a sum total of an area occupied by the face of the person image, f2 being an average value of the area occupied by the face of the person image, f3 being a dispersion of the area occupied by the face of ther person image, f4 being an average of a horizontal direction front face directing degree, f5 being a dispersion of the horizontal direction front face directing degree, f6 being an average value of a perpendicular direction front face directing degree, f7 being a dispersion of the perpendicular direction front face directing degree, f8 being an average value of each gravity center position of the face of the person image, and f9 being a dispersion of each gravity center position of the face of the person image;

repeatedly executing the extracting and generating for each classifying object image; and

classifying each classifying object image into plural groups on the basis of the generated characteristic vector.