Image search apparatus and method, and computer readable memory

• US 6,731,826 B1
• Filed: 08/30/1999
• Issued: 05/04/2004
• Est. Priority Date: 08/31/1998
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Yamamoto, Kunihiro, Kusama, Kiyoshi, Enokida, Miyuki, Matsumoto, Kentaro
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US09/385,430
• Time in Patent Office: 1,709 Days
• Field of Search: 382/181, 382/187, 382/189, 382/202, 382/209, 382/218, 382/219, 382/220, 382/278, 382/298, 382/305, 345/700, 345/835, 345/838, 358/403, 707/1-7
• US Class Current: 382/305
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A plurality of image data are stored in a hard disk device in correspondence with their image feature amounts. An image is using an input window. A CPU computes the image feature amount of the input image. The CPU computes image similarity on the basis of the computed image feature amount and the image feature amounts of the stored image data.

Claims

1. An image search apparatus for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:

input means for inputting a stroke or strokes that form an image using an input window displayed on a display unit wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;

image feature amount computing means for computing an image feature amount of an image drawn every time a stroke that forms a partial of the image is input by said input means;

image similarity computing means for computing image similarity every time a stroke that forms the image is input by said input means on the basis of the image feature amount computed by said image feature amount computing means; and

image search means for searching for desired image data from the image database every time a stroke that forms the image is input by said input means on the basis of the image similarity computed by said image similarity computing means.

2. The apparatus according to claim 1, further comprising image display means for displaying a list of image data as search results on the basis of the image similarity computed by said image similarity computing means.

3. The apparatus according to claim 2, wherein said image display means displays reduced-scale images corresponding to the image data as search results.

4. The apparatus according to claim 3, wherein said image display means comprises display control means for displaying a detailed image which is associated with each of the reduced-scale images, and

said display control means displays the detailed image associated with the selected reduced-scale image, when one or a plurality of the reduced-scaled images are selected.

5. The apparatus according to claim 2, wherein the input window and a display window displayed by said image display means are simultaneously displayed on a display unit.

6. The apparatus according to claim 2, wherein the input window and a display window displayed by said image display means are alternately displayed on a display unit in accordance with an instruction.

7. The apparatus according to claim 1, wherein said input means comprises a pointing device.

8. The apparatus according to claim 7, wherein said pointing device comprises a mouse.

9. The apparatus according to claim 7, wherein said pointing device comprises a touch screen.

10. The apparatus according to claim 7, wherein said pointing device comprises a pen tablet.

11. The apparatus according to claim 2, wherein said image display means displays the image data as the search results to line up one-dimensionally.

12. The apparatus according to claim 2, wherein said image display means displays the image data as the search results to line up two-dimensionally.

13. The apparatus according to claim 2, wherein said image display means displays the image data as the search results to line up three-dimensionally.

14. The apparatus according to claim 2, wherein said image display means displays the image data as the search results in descending order of image similarity.

15. An image search method for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:

an inputting step, of inputting a stroke or strokes that form an image using an input window displayed on a display unit, wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;

an image feature amount computing step, of computing an image feature amount of an image drawn every time a stroke that forms a partial image of the image is input on the input window;

an image similarity computing step, of computing image similarity every time a stroke that forms the image is input on the basis of the image feature amount computed in said image feature amount computing step; and

an image searching step, of searching for desired image data from the image database every time a stroke that forms the image is input, on the basis of the image similarity computed in said image similarity computing step.

16. The method according to claim 15, further comprising an image display step of displaying a list of image data as search results on the basis of the image similarity computed in said image similarity computing step.

17. The method according to claim 16, wherein said image display step includes the step of displaying reduced-scale images corresponding to the image data as search results.

18. The method according to claim 17, wherein said image display step comprises a display control step of displaying a detailed image which is associated with each of the reduced-scale images, and

said display control step includes the step of displaying the detailed image associated with the selected reduced-scale image, when one or a plurality of the reduced-scaled images are selected.

19. The method according to claim 16, wherein the image input on the input window and the list of image data as the search results displayed in said image display step are simultaneously displayed on a display unit.

20. The method according to claim 16, wherein the image input on the input window and the list of image data as the search results displayed in said image display step are alternately displayed on a display unit in accordance with an instruction.

21. The method according to claim 15, wherein the image is input by a pointing device.

22. The method according to claim 21, wherein the pointing device comprises a mouse.

23. The method according to claim 21, wherein the pointing device comprises a touch screen.

24. The method according to claim 21, wherein the pointing device comprises a pen tablet.

25. The method according to claim 16, wherein said image display step includes the step of displaying the image data as the search results to line up one-dimensionally.

26. The method according to claim 16, wherein said image display step includes the step of displaying the image data as the search results to line up two-dimensionally.

27. The method according to claim 16, wherein said image display step includes the step of displaying the image data as the search results to line up three-dimensionally.

28. The method according to claim 16, wherein said image display step includes the step of displaying the image data as the search results in descending order of image similarity.

29. A computer readable memory that stores a program code of an image search for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:

a program code of an inputting step, of inputting a stroke or strokes that form an image using an input window displayed on a display unit, wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;

a program code of an image feature amount computing step, of computing an image feature amount of an image drawn every time a stroke that forms a partial image of the image is input on the input window;

a program code of an image similarity computing step, of computing image similarity every time a stroke that forms the image is input on the basis of the image feature amount computed in said image feature amount computing step; and

a program code for an image searching step, of searching for desired image data from the image database every time a stroke that forms the image is input, on the basis of the image similarity computed in said image similarity computing step.

30. An image search apparatus for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:

input means for inputting a stroke or strokes that form an image using an input window displayed on a display unit wherein the stroke or strokes are handwritten stroke or strokes and can be imputted succesively and directly in the input window;

image feature amount computing means for computing an image feature amount of an image drawn every time a stroke that forms a partial of the image is input by said input means;

image similarity computing means for computing image similarity every time a stroke that forms the image is input by said input means, on the basis of the image feature amount computed by said image feature amount computing means;

image search means for searching for desired image data from the image database every time a stroke that forms the image is input by said input means, on the basis of the image similarity computed by said image similarity computing means;

image display means for displaying a list of image data as search results obtained by said image search means; and

display control means for controlling to display the input window and a display window displayed by said image display means on a single screen of a display unit.

31. The apparatus according to claim 30, wherein said image feature amount computing means computes the image feature amount of an image drawn every time the image is modified by said input means.

32. The apparatus according to claim 30, further comprising instruction means for instructing start of a search using the image input by said input means as a query criteria, and

wherein said image feature amount computing means computes the image feature amount of the image input by said input means upon receiving the instruction from said instruction means.

33. The apparatus according to claim 30, wherein said image feature amount computing means comprises:

management means for managing an input start time of the image by said input means; and

computing means for computing a time elapsed from the start time every time a stroke that forms the image is input by said input means, and

wherein said image feature amount computing means computes the image feature amount of the image input by said input means on the basis of the elapsed time computed by said computing means.

34. The apparatus according to claim 30, wherein said image feature amount computing means computes the image feature amount of the image input by said input means at a predetermined time interval.

35. The apparatus according to claim 30, wherein said image feature amount computing means comprises monitoring means for monitoring a load on said image search apparatus, and

wherein said image feature amount computing means computes the image feature amount of the image input by said input means in accordance with a monitoring result of said monitoring means.

36. The apparatus according to claim 30, wherein said image feature amount computing means computes the image feature amount of the image input by said input means so far in accordance with the number of strokes that form the image input by said input means.

37. An image search method for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:

an inputting step, of inputting a stroke or strokes that form an image using an input window displayed on a display unit, wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;

an image feature amount computing step, of computing an image feature amount of an image drawn every time a stroke that forms a partial image of the image is input on the input window;

an image similarity computing step, of computing image similarity every time a stroke that forms the image is input, on the basis of the image feature amount computed in said image feature amount computing step;

an image search step, of searching for desired image data from the image database every time a stroke that forms the image is input, on the basis of the image similarity computed in said image similarity computing step;

an image display step, of displaying a list of image data as search results obtained in said image search step; and

an display control step, of controlling to display the input window and a display window displayed in the image display step on a single screen of a display unit.

38. The method according to claim 37, wherein said image feature amount computing step includes the step of computing the image feature amount of an image drawn every time the image input on the input window is modified.

39. The method according to claim 37, further comprising an instruction step of instructing start of a search using the image input on the input window as a query criteria,

wherein said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window upon receiving the instruction issued in said instruction step.

40. The method according to claim 37, wherein, said image feature amount computing step comprises:

a management step of managing an input start time of the image on the input window; and

a computing step of computing a time elapsed from the start time every time a stroke that forms the image is input on the input window, and

said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window on the basis of the elapsed time computed in said computing step.

41. The method according to claim 37, wherein said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window at a predetermined time interval.

42. The method according to claim 37, wherein said image feature amount computing step comprises the monitoring step of monitoring a load on an image search apparatus, and

said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window in accordance with a monitoring result in said monitoring step.

43. The method according to claim 37, wherein said image feature amount computing step includes the step of computing the image feature amount of the image input on the input window so far in accordance with the number of strokes that form the image input on the input window.

44. A computer readable memory that stores a program code of an image search for searching for desired image data from an image database that stores a plurality of image data in correspondence with image feature amounts of the image data, comprising:

a program code of an inputting step, of inputting a stroke or strokes that form an image using an input window displayed on a display unit, wherein the stroke or strokes are handwritten stroke or strokes and can be inputted successively and directly in the input window;

a program code of an image feature amount computing step, of computing an image feature amount of an image drawn every time a stroke that forms a partial of the image is input on the input window;

a program code of an image similarity computing step, of computing image similarity every time a stroke that forms the image is input, on the basis of the image feature amount computed in said image feature amount computing step;

a program code of an image search step, of searching for desired image data from the image database every time a stroke that forms the image is input, on the basis of the image similarity computed in said image similarity computing step;

a program code of an image display step, of displaying a list of image data as search results obtained in said image search step; and

a program code of a display control step, of controlling to display the input window and a display window displayed in said image display step on a single screen of a display unit.