Information processing apparatus for retrieving image data similar to an entered image

• US 7,593,961 B2
• Filed: 04/21/2004
• Issued: 09/22/2009
• Est. Priority Date: 04/30/2003
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Uzawa, Mitsuru, Eguchi, Takami
• Primary Examiner(s): Mofiz; Apu M
• Assistant Examiner(s): Le; Jessica N
• Application Number:
  US10/828,166
• Publication Number:
  US 20040220898A1
• Time in Patent Office: 1,980 Days
• Field of Search: 707/104, 707 1- 6, 707100-102, 382/22, 382/225, 382/248, 382/305, 235/436, 395/600, 705/67
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

More accurate retrieval of original documents is conducted by adaptively evaluating retrieval results, which are obtained by retrieving attributes, in accordance with layout information. To achieve this, there is provided an information processing method for retrieving image data that is similar to an entered document image, the method including a step (S402) of segmenting the entered document image into a plurality of areas on a per-attribute basis; a step of calculating degree of similarity, for every area obtained by segmentation, using a retrieval step suited to the attribute; and a step (S406) of calculating overall degree of similarity in which the degree of similarity calculated for every area obtained by segmentation is weighted.

Claims

1. An image processing apparatus, including a computer-readable storage medium encoded with a computer program which is executable on a computer for retrieving retrieval-target image data that is similar to an entered document image, which operates as a scanner and a printer, comprising:

a scanning unit for scanning a document and generating the entered document image;

a segmentation unit for segmenting the entered document image and the retrieval-target image data into a plurality of areas on a per-attribute basis;

an area specification unit for specifying an area to be emphasized of the plurality of areas segmented by said segmentation unit based on a user'"'"'s instruction;

a detection unit for detecting pointer information in a predetermined area of the plurality of areas of the entered document image, which indicates storage location in a storage device storing original image data of the entered document image;

a first image data retrieving unit for retrieving the original image data of the entered document image based on the pointer information if the pointer information is detected by said detection unit; and

a second image data retrieving unit for retrieving image data that is similar to the entered document image if the pointer information is not detected by the detection unit, including

1) a layout-similarity calculation unit for calculating a layout degree of similarity, which is a degree of similarity between a layout of areas obtained by segmentation in the entered document image and a layout of areas obtained by segmentation in the retrieval-target image data, and

2) a similarity calculation unit for calculating a degree of similarity, with regard to the retrieval-target image data for which the calculated layout degree of similarity is greater than a predetermined threshold value, for every area obtained by segmentation, using a comparison unit suited to the attribute; and

an overall-similarity calculation unit for calculating an overall degree of similarity, with regard to the retrieval-target image data for which the calculated layout degree of similarity is greater than a predetermined threshold value, based on the degree of similarity calculated for every area obtained by segmentation and a weighting coefficient corresponding to the degree of similarity calculated for every area obtained by segmentation,

wherein the weighting coefficient is calculated depending on a ratio between a sum of sizes of all segmented areas in the entered document image and each size of each segmented area in the entered document image, and is increased if the area corresponding to the weighting coefficient is the area specified by the area specification unit and decreased if the area corresponding to the weighting coefficient is not the area specified by the area specification unit, and

wherein the entered document image is converted to vector data by a vector data conversion unit and registered as retrieval-target image data if the overall degree of similarity calculated by the overall-similarity calculation unit is lower than a predetermined threshold, and

wherein said vector data conversion unit converts the document image to vector data in a case where an original data file corresponding to the entered document image could not be found based upon a result of calculation by said overall-similarity calculation unit; and

a storage unit for storing the entered document image that has been converted to the vector data.

2. The apparatus according to claim 1, wherein said comparison unit includes:

a unit for conducting a full-text search based upon text data extracted by character recognition from a text area among the areas obtained by said segmentation unit;

a unit for conducting a concept search based upon the text data; and

a unit for conducting an image search based upon a feature extracted from an image area among the areas obtained by said segmentation unit.

3. The apparatus according to claim 1, further comprising a communication unit for communicating with a server that stores information relating to a copyright royalty of each item of the retrieval-target image data;

wherein the information relating to the copyright royalty being acquired with regard to the retrieval-target image data.

4. The apparatus according to claim 3, wherein in a case where the information relating to copyright royalty exists, no processing regarding the retrieval-target image data is executed as long as permission is not received from the server.

5. The apparatus according to claim 3, further comprising a billing unit for executing billing processing of a user that is based upon the copyright royalty.

6. The apparatus according to claim 1, further comprising:

a billing unit for billing a user for a fee when the entered document image is converted to the vector data;

the vector data conversion unit for converting the entered document image to the vector data in a case where a procedure for payment of the fee billed by said billing unit has been executed.

7. The apparatus according to claim 6, wherein said billing unit bills a fee when a copyright royalty of the entered document image that has been converted to the vector data is managed.

8. The apparatus according to claim 1, further comprising:

a candidate category output unit for outputting candidate categories of storage destinations based upon a feature of the entered document image that has been converted to the vector data.

9. The apparatus according to claim 8, further comprising:

a category selection unit for allowing a user to select a category of a storage destination from among the candidate categories of storage destinations that have been output by said candidate category output unit; and

a storage unit for storing the entered document image, which has been converted to the vector data, at the category of storage destination.

10. The apparatus according to claim 8, further comprising a storage unit for automatically storing the entered document image, which has been converted to the vector data, with respect to a first-ranked candidate category that has been output by said candidate category output unit.

11. The apparatus according to claim 8, wherein in a case where the original data file corresponding to the document image is an image file, said vector data conversion unit converts the document image to vector data.

12. The apparatus according to claim 1, further comprising a selection unit for allowing a selection of an area, which is used in calculation of degree of similarity, from among the plurality of areas obtained by segmentation by said segmentation unit;

wherein said similarity calculation unit calculates degree of similarity, using a comparison unit suited to the attribute, with regard to the area that has been selected by said selection unit.

13. An image processing method for retrieving retrieval-target image data that is similar to an entered document image, the method being used with an image processing apparatus that has a scanner and a printer, and comprising:

a scanning step of scanning a document and generating the entered document image, the scanning step being performed by the scanner;

a segmentation step of segmenting the entered document image and the retrieval-target image data into a plurality of areas on a per-attribute basis;

an area specification step of specifying an area to be emphasized of the plurality of areas segmented in said segmentation step based on a user'"'"'s instruction;

a detection step of detecting pointer information in a predetermined area of the plurality of areas of the entered document image, which indicates storage location in a storage device storing original image data of the entered document image;

a first image data retrieving step of retrieving the original image data of the entered document image based on the pointer information if the pointer information is detected in said detection step; and

a second image data retrieving step of retrieving image data stored in the storage device that is similar to the entered document image if the pointer information is not detected in the detection step, including

1) a layout-similarity calculation step of calculating a layout degree of similarity, which is a degree of similarity between a layout of the areas obtained by segmentation in the entered document image and a layout of areas obtained by segmentation in the retrieval-target image data;

2) a similarity calculation step of calculating a degree of similarity, with regard to the retrieval-target image data for which the calculated layout degree of similarity is greater than a predetermined threshold value, for every area obtained by segmentation, using a comparison step suited to the attribute; and

an overall-similarity calculation step of calculating an overall degree of similarity, with regard to the retrieval-target image data for which the calculated layout degree of similarity is greater than a predetermined threshold value, based on the degree of similarity calculated for every area obtained by segmentation and a weighting coefficient corresponding to the degree of similarity calculated for every area obtained by segmentation,

wherein the weighting coefficient is calculated depending on a ratio between a sum of sizes of all segmented areas in the entered document image and each size of each segmented area in the entered document image, and is increased if the area corresponding to the weighting coefficient is the area specified in the area specification step and decreased if the area corresponding to the weighting coefficient is not the area specified in the area specification step, and

wherein a vector data conversion step for converting the entered document image to vector data and registered as retrieval-target image data stored in the storage device if the overall degree of similarity calculated in the overall-similarity calculation step is lower than a predetermined threshold, and

wherein said vector data conversion step converts the entered document image to the vector data in a case where an original data file corresponding to the document image could not be found based upon a result of calculation in said overall-similarity calculation step; and

a storage step for storing the entered document image that has been converted to the vector data.

14. The method according to claim 13, wherein said comparison step includes:

a step of conducting a full-text search based upon text data extracted by character recognition from a text area among the areas obtained at said segmentation step;

a step of conducting a concept search based upon the text data; and

a step of conducting an image search based upon a feature extracted from an image area among the areas obtained at said segmentation step.

15. The method according to claim 13, further comprising a communication step of communicating with a server that stores information relating to a copyright royalty of each item of the retrieval-target image data;

wherein the information relating to the copyright royalty being acquired with regard to the retrieval-target image data.

16. The method according to claim 15, wherein in a case where the information relating to the copyright royalty exists, no processing regarding the retrieval-target image data is executed as long as permission is not received from the server.

17. The method according to claim 15, further comprising a billing step of executing billing processing of a user that is based upon the copyright royalty.

18. The method according to claim 13, further comprising:

a billing step of billing a user for a fee when the entered document image is converted to the vector data;

wherein said vector data conversion step of converting the entered document image to vector data in a case where a procedure for payment of the fee billed at said billing step has been executed.

19. The method according to claim 18, wherein said billing step bills a fee when a copyright royalty of the entered document image that has been converted to the vector data is managed.

20. The method according to claim 13, further comprising:

a candidate category output step of outputting candidate categories of storage destinations based upon a feature of the entered document image that has been converted to the vector data.

21. The method according to claim 20, further comprising:

a category selection step of allowing a user to select a category of a storage destination from among the candidate categories of storage destinations that have been output at said candidate category output step; and

a storage step of storing the entered document image, which has been converted to the vector data, at the category of storage destination.

22. The method according to claim 20, further comprising a storage step of automatically storing the entered document image, which has been converted to the vector data, with respect to a first-ranked candidate category that has been output at said candidate category output step.

23. The method according to claim 20, wherein in a case where the original data file corresponding to the document image is an image file, said vector data conversion step converts the document image to the vector data.

24. The method according to claim 13, further comprising a selection step of allowing selection of an area, which is used in calculation of the degree of similarity, from among the plurality of areas obtained by segmentation at said segmentation step;

wherein said similarity calculation step calculates the degree of similarity, using a comparison step suited to the attribute, with regard to the area that has been selected at said selection step.

25. A storage medium storing a control program for execution by a computer to implement the image processing method set forth in claim 13.