Image searching method and image processing method

• US 6,813,395 B1
• Filed: 07/14/2000
• Issued: 11/02/2004
• Est. Priority Date: 07/14/1999
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Kinjo, Naoto
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US09/617,309
• Time in Patent Office: 1,572 Days
• Field of Search: 707/3, 707/7, 707/6, 382/305, 382/282, 382/286, 382/291, 382/203, 382/218, 382/219, 382/283, 382/295, 382/296
• US Class Current: 382/305
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

The image searching method extracts or recognizes specific information for an image that exists in the image and/or accompanies it, allows either the specific information itself or related information thereof to be appendant to image data for the image, stores the image data as well as either information appendant to the image data as accessory information in a database, designates at least a portion of the accessory information as a searching condition, searches through the database under the designated searching condition, determines a degree of agreement with the accessory information appendant to a selected image data and reads an image having the degree of agreement not lower than a predetermined value. The image processing method utilizes the image searching method. These methods are capable of efficient image detection or search using as a key specific information such as the shape of subjects.

Claims

1. An image searching method, comprising the steps of:

extracting or recognizing specific information for an image that either exists in the image or accompanies it or both exists in the image and accompanies it;

allowing either the specific information as-obtained or related information relative to said specific information to be appendant to image data for said image;

storing the image data as well as either information appendant to the image data as accessory information in a database;

designating at least a portion of the accessory information as a searching condition;

searching through said database under the designated searching condition;

determining a degree of agreement with said accessory information appendant to a selected image data; and

reading an image having the degree of agreement not lower than a predetermined value.

2. The image searching method according to claim 1, wherein the specific information existing in said image is M (M≧

1) kinds of specific geometric figures predetermined for said image, said related information to be stored as said accessory information is either a combination of position and size or a combination of position, size and direction or a combination of position, size and color or a combination of position, size, direction and color that correspond to N (N≧

0) specific geometric figures that have been obtained as a result of extraction by processing said image with a treatment of extracting said M kinds of said specific geometric figures, and the designating step of at least a portion of said accessory information as said searching condition comprises designating, for L (L is from one to M) kinds of said specific geometric figures, presence or absence of said specific geometric figures or an outline of at least one element selected from among said position, size, direction and color.

3. The image searching method according to claim 1, wherein the specific information existing in said image is M (M≧

1) kinds of specific subjects predetermined for said image, said related information to be stored as said accessory information is either a combination of position and size or a combination of position, size and direction or a combination of position, size and color or a combination of position, size, direction and color that correspond to N (N≧

0) specific subjects that have been obtained as a result of extraction by processing said image with a treatment of extracting said M kinds of said specific subjects, and the step of designating at least a portion of said accessory information as said searching condition comprises designating, for L (L is from one to M) kinds of said specific subjects, presence or absence of said specific subjects or an outline of at least one element selected from among said position, size, direction and color.

4. The image searching method according to claim 1, wherein said image is temporally continuous images, the specific information existing in said image is a specific geometric figure predetermined for said image, said related information to be stored as said accessory information comprises information about said specific geometric figure and a direction of movement of an image corresponding to said specific geometric figure that have been obtained by processing said temporally continuous images by a treatment of extracting said specific geometric figure from said temporally continuous images at predetermined intervals of time or frame numbers and determining the direction of movement of the image corresponding to said geometric figure based on comparison of a position of said extracted specific geometric figure between frames, and the step of designating at least a portion of said accessory information as said searching condition comprises designating, for said temporally continuous images, outlines of the specific geometric figure and the direction of its motion vector at the predetermined intervals of time or frame numbers.

5. The image searching method according to claim 1, wherein said image is temporally continuous images, the specific information existing in said image is a specific subject predetermined for said image, said related information to be stored as said accessory information comprises information about said specific subject and a direction of movement of an image corresponding to said specific subject that have been obtained by processing said temporally continuous images by a treatment of extracting said specific subject from said temporally continuous images at predetermined intervals of time or frame numbers and determining the direction of movement of the image corresponding to said specific subject based on comparison of a position of said extracted subject between frames, and the step of designating at least a portion of said accessory information as said searching condition comprises designating, for said temporally continuous images, outlines of the specific subject and the direction of its motion vector at the predetermined intervals of time or frame numbers.

6. The image searching method according to claim 1, wherein said image has audio information, said specific information accompanying said image is said audio information, said accessory information is said audio information that has been recognized for the image having said audio information, the step of designating at least portion of said accessory information as said searching condition comprises designating an outline of audio information for the image having said audio information, and said degree of agreement is with speech recognition information that is appendant to said stored image data.

7. The image searching method according to claim 6, wherein said accessory information comprises not only said audio information but also said related information which is either a combination of position and size or a combination of position, size and direction or a combination of position, size and color or a combination of position, size, direction and color which correspond to a specific geometric figure, and said searching condition comprises not only said audio information but also presence or absence of the specific geometric figure or at least one element selected from among position, size, direction and color which correspond to said specific geometric figure.

8. The image searching method according to claim 6, wherein said accessory information comprises not only said audio information but also said related information which is either a combination of position and size or a combination of position, size and direction or a combination of position, size and color or a combination of position, size, direction and color which correspond to a specific subject, and said searching condition comprises not only said audio information but also presence or absence of the specific subject or at least one element selected from among position, size, direction and color which correspond to said specific subject.

9. An image searching method according to claim 1, wherein the specific information which both exists in and accompanies said image is person'"'"'s information comprising facial image of a particular person and his or her name, said related information to be stored as said accessory information is presence or absence of said particular person or either a combination of position and size of said particular person or a combination of position, size and direction of said particular person or a combination of position, size and color of said particular person or a combination of position, size, direction and color of said particular person and obtained by the steps of preliminarily registering person'"'"'s information comprising the facial images of said A particular persons (A≧

1) and their names and performing extraction of persons and a person identifying treatment using said person'"'"'s information at the time of storing said image, the step of designating at least a portion of said accessory information as said searching condition comprising designating presence or absence of B particular persons (B is from one to A) or at least one element selected from among positions, sizes and directions of the particular persons, and said degree of agreement is with the accessory information to each of the stored images.

10. The image searching method according to claim 9, wherein said image is temporally continuous images, and said related information to be stored as said accessory information is obtained by processing said temporally continuous images by said person identifying treatment using said person'"'"'s information at predetermined intervals of time or frame numbers and comprises information about said particular parsons and a direction of the movement of said particular parsons, the step of designating at least a portion of said accessory information as said searching condition comprising designating presence or absence of said particular persons or at least one element selected from among positions, sizes and colors of the particular persons, and the direction of its motion vector.

11. The image searching method according to claim 9, wherein said image has audio information, said accessory information comprises said related information obtained for the image having said audio information by a person identifying treatment using said person'"'"'s information and audio information recognized for the image having said audio information, and the step of designating at least a portion of said accessory information as said searching condition comprises designating an outline of said related information containing the presence or absence of said particular persons or at least one element selected from among positions, sizes, directions and colors or said audio information as said searching condition applicable to the image having said audio information.

12. The image searching method according to claim 1, wherein said image has message information, the specific information accompanying said image is said message information, said accessory information is said message information recognized for the image having said message information, the step of designating at least a portion of said accessory information as said searching condition comprises designating an outline of message information for the image having said message information, and said degree of agreement is with message recognition information appendant to said stored image data.

13. The image searching method according to claim 12, wherein said message information is text data or appendant image data.

14. The image searching method according to claim 1, wherein said image is a computer graphics image, the specific information appendant to said image is editing data used to create said image, said accessory information is said editing data, the step of designating at least a portion of said accessory information as said searching condition has the substeps of designating a scene of said computer graphics image by at least one element selected from among position, size and direction of at least one member selected from among a specific geometric figure, a specific physical object and a specific color and automatically transforming the designated scene designation data to a computer graphics command, and the step of determining said degree of agreement is for determining the degree of agreement between the resulting computer graphics command and said editing data appendant to a selected item of said stored image data.

15. The image searching method according to claim 14, wherein said computer graphics image is temporally continuous computer graphics images, said accessory information comprises not only said editing data but also either said specific geometric figure or said specific physical object or both and a direction of movement of an image that have been obtained by determining, for said temporally continuous computer graphics images, the direction of movement of the image corresponding to either said specific geometric figure or said physical object or both at predetermined intervals of time or frame numbers based on comparison between frames of the position of either said specific geometric figure or said specific physical object or both, and outlines of either the specific geometric figure or the specific physical object or both and the direction of their motion vector are designated as said searching condition at predetermined intervals of time or frame numbers for said temporally continuous computer graphics images.

16. The image searching method according to claim 1, wherein said step of designation has the substeps of designating at least a portion of the accessory information by at least one means selected from among symbols, language and speech and automatically transforming at least one designated element selected from among symbols, language and speech into said specific information or accessory information therefor.

17. The image searching method according to claim 1, wherein an area in said image data that meets said searching condition is displayed in temporal preference.

18. The image searching method according to claim 1, wherein an area in said image data that meets said searching condition is displayed as enhanced by a different image processing scheme than applied to the other areas.

19. The image searching method according to claim 1, wherein said image data is processed in accordance with a display performance of a display terminal.

20. The image searching method according to claim 1, wherein an image areas that have been searched through are managed area by area and a priority of reference in search mode is set in accordance with access frequency.

21. The image searching method according to claim 1, wherein an image areas that have been searched through are managed area by area and an image resolution for data storage is set in accordance with access frequency.

22. The image searching method according to claim 1, wherein the specific information accompanying said image is a name of a shot place where an image was shot or a subject and their position on a map, said accessory information is a result obtained by extracting the name of said shot place or said subject and theirs position on said map from shooting information when the image was shot and map information, and the step of designating at least a portion of said accessory information as said searching condition comprises designating the position or the name of said shot place or the position or the name of the subject as reference data.

23. The image searching method according to claim 22, wherein said shooting information is acquired in association with the shot image and is either a first data composition consisting of shooting position and shooting direction or a third data composition consisting of shooting position and magnification rate or a fourth data composition consisting of shooting position, shooting direction and magnifying power of shooting.

24. The image searching method according to claim 22, wherein an outline of depth is designated as reference data concerning said shooting position and subject'"'"'s position by arranging patterned subjects in a virtual map space.

25. The image searching method according to claim 22, wherein the shot place or the position and size of the subject within the screen are extracted and added to said composition consisting of shooting position or a second accessory information together with said extracted information.

26. The image searching method according to claim 22, wherein the shot place or the position and size of the subject within the screen are extracted and added to the designated information together with said extracted information.

27. The image searching method according to claim 22, wherein a difference between image data of the actual shot image and image data simulating the shot image and which has been created from position information and camera information when the image was shot, and said map information is stored and when the image which is a result of searching is outputted, said stored difference is added to the image data simulating the shot image and which has been created from said position information, said camera information and said map information and the resulting image is output.

28. The image searching method according to claim 1, wherein the specific information accompanying said image is information representing a distance from a screen recording said image, said accessory information is a result obtained by deriving the information representing the distance from said image recording screen, and the step of designating at least a portion of said accessory information as said searching condition comprises designating the information representing the distance from said image recording screen as reference data.

29. The image searching method according to claim 28, wherein if said image is a multi-stage focused image or a stereoscopic image, the information representing the distance from said image recording screen is derived from said multi-stage focused image or stereoscopic image.

30. The image searching method according to claim 28, wherein the information representing the distance from said image recording screen is derived from the position information when the image was shot and map information or from the position information, the map information and the camera information.

31. The image searching method according to claim 28, wherein an outline of the depth for each of the areas in the image recording screen is designated as reference data concerning the information representing the distance from said image recording screen.

32. The image searching method according to claim 28, wherein an outline of the depth is designated as reference data concerning the information representing the distance from said image recording screen by arranging patterns of subjects in a virtual map space.

33. The image searching method according to claim 26, wherein a difference between image data the actual recorded image and image data simulating the recorded image and which has been created from position information, camera information and map information that are relevant to a shooting time is stored and to output the image which is a result of searching, said stored difference is added to the image data simulating the recorded image and which has been created from the position information, camera information and map information that are relevant to said shooting time and the resulting image is output.

36. The image searching method according to claim 1, wherein the specific information accompanying representing a distance from a screen recording said image.

37. The image searching method according to claim 1, wherein the step of reading includes reading all images in the database having the degree of agreement not lower than a predetermined value.

40. The image searching method according to claim 1, wherein the specific information existing in said image is M (M≧

1) kinds of specific geometric figures, said related information to be stored as said accessory information is at least one of position, size, direction, and color corresponding to N (N≧

0) specific geometric figures obtained as a result of the step of extracting.

41. The image searching method according to claim 1, wherein sa specific information includes audio information.

42. The image searching method according to claim 41, wherein the said degree of agreement includes speech recognition information.

43. The image searching method according to claim 1, wherein the specific information is person'"'"'s information comprising a facial image of a particular person and his or her name, said related information to be stored as said accessory information is presence or absence of said particular person or at least one of position, size, direction, and color of said particular person.

44. The image searching method according to claim 1, wherein the specific information accompanying said image is a geographic name of a location where an image was shot.

34. An image processing method, comprising the steps of:

extracting specific figures or specific subjects in a printed image during a previous printing;

storing the extracted specific figures or subjects in a database as accessory information that is keyed to image processing conditions either alone or in combination with the image data;

reading image data from a previous print or captured image data recording medium to extract the specific figures or specific subjects for processing prints upon reorder;

determining a degree of agreement between the extracted specific figures or subjects and the accessory information stored in said database;

searching for an image having the degree of agreement not lower than a predetermined value; and

processing a hit image under the image processing conditions employed for the previous printing.

38. The image processing method according to claim 34, wherein the step of searching includes searching for all images in the database having the degree of agreement not lower than a predetermined value.

35. An image processing method, comprising the steps of:

storing image processing conditions for a previous printing in a database in such a manner that they are keyed to image data;

reading image data from a previous print or captured image data reading medium to extract a first specific figure or subject for processing prints upon reorder;

also extracting a second specific figure or subject from the image data stored in said database;

determining a degree of agreement between said first specific figure or subject and said second specific figure or subject;

searching for an image having the degree of agreement not lower than a predetermined value;

reading from said database said image data for a hit image together with said stored image processing conditions keyed to said image data; and

subjecting the thus read image data to image processing under the image processing conditions as read from said data base.

39. The image processing method according to claim 35, wherein the step of reading includes reading all images in the database having the degree of agreement not lower than a predetermined value.