Semantic classification and enhancement processing of images for printing applications

• US 7,505,178 B2
• Filed: 01/07/2005
• Issued: 03/17/2009
• Est. Priority Date: 09/25/2003
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Wolff, Gregory J., Hull, Jonathan J., Erol, Berna, Lee, Dar-Shyang
• Primary Examiner(s): Lee; Thomas D
• Application Number:
  US11/031,516
• Publication Number:
  US 20060256388A1
• Time in Patent Office: 1,530 Days
• Field of Search: 358/1.1, 358/1.9, 358/2.1, 358/3.24, 358/3.27, 358/1.13, 358/1.15, 358/1.18, 358/448, 358/462, 382/224, 382/228, 382/229, 382/254
• US Class Current: 358/3.27
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/162 Quantising the image signal
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition

Abstract

A printing system enables the printing of enhanced documents using a semantic classification scheme. A printing system receives an image to be printed. The system classifies the image according to the semantic classification scheme and, based on this classification, performs enhancement processing on the image. Depending on the desired application, the printing system may recognize and classify any number of image types and may then perform various enhancement processing functions on the image, where the type of enhancement processing performed is based on the classification of the image.

Claims

1. A system for organizing images captured in an environment, the system comprising:

an interface for receiving an image from an image capture device;

a semantic classifier module coupled to the interface, the semantic classifier module configured to determine the semantic category for the image, at least in part, by;

identifying text-containing regions in the image;

recognizing at least some textual content in the text-containing regions; and

applying a set of heuristics to the recognized textual content to determine the semantic classification for the image; and

an enhancement processing module coupled to the semantic classifier module, the enhancement processing module configured to enhance the image based on the semantic category into which the image was classified.

2. The system of claim 1, further comprising:

an output system coupled to receive the enhanced image from the enhancement processing module and prepare the enhanced image for printing.

3. The system of claim 1, wherein the semantic categories include at least whiteboard images, business card images, document images, slide images, and regular images.

4. The system of claim 1, wherein the enhancement processing module enhances the image, at least in part, by performing at least one of:

scaling, cropping, foreground subtraction, background subtraction, contrast adjustment, and text rotation.

5. The system of claim 1, wherein the enhancement processing module enhances the image, at least in part, by appending information to the image to enhance the image.

6. The system of claim 1, wherein the image is a whiteboard image, and the enhancement processing module computes a background of the image and subtracts the background from the image to enhance textual content in the image.

7. A system for organizing images captured in an environment, the system comprising:

an interface for receiving an image from an image capture device;

a semantic classifier module coupled to the interface, the semantic classifier module configured to determine a semantic category for the image based at least in part on content within the image; and

an enhancement processing module coupled to the semantic classifier module, the enhancement processing module configured to enhance the image based on the semantic category into which the image was classified, at least in part, by;

recognizing content within the image;

associating the image with a stored document based on the recognized content; and

replacing the captured image with the stored document.

8. The system of claim 7, wherein the semantic classifier module determines the semantic category for the image, at least in part, by:

extracting a plurality of features from the image;

combining the extracted features into a feature vector; and

comparing the feature vector with information in a trained database to determine the semantic classification for the image.

9. The system of claim 7, wherein the semantic classifier module determines the semantic category for the image, at least in part, by comparing the heights of a plurality of markings on the image to determine whether the image belongs to a semantic category typically comprising machine-generated text or a semantic category typically comprising handwriting.

10. The system of claim 7, further comprising:

an output system configured to select a layout orientation for a printed output of the image based on the semantic category into which the image was classified.

11. The system of claim 7, further comprising:

an output system configured to select a paper source and/or ink selection for a printed output of the image based on the semantic category into which the image was classified.

12. The system of claim 7, further comprising:

an output system coupled to receive the enhanced image from the enhancement processing module and prepare the enhanced image for printing, at least in part, by;

retrieving a set of default printing preferences associated with the semantic category into which the image was classified; and

applying the set of default printing preferences to generate a printed output of the image.

13. The system of claim 7, wherein the captured image is received from a digital camera.

14. The system of claim 7, further comprising:

an output system configured to generate a printed output of the enhanced image.

15. The system of claim 7, further comprising:

an output system configured to deliver the enhanced image to one or more users, the delivered enhanced image capable of being printed.

16. A method for organizing images captured in an environment, the method comprising:

receiving a captured image;

classifying the image into a semantic category based at least in part on content within the image, wherein classifying the image comprises;

identifying text-containing regions in the image;

recognizing at least some textual content in the text-containing regions; and

applying a set of heuristics to the recognized textual content to determine the semantic classification for the image; and

enhancing the image based on the semantic category into which the image was classified.

17. The method of claim 16, further comprising:

preparing the enhanced image for printing.

18. The method of claim 16, wherein the semantic categories include at least whiteboard images, business card images, document images, slide images, and regular images.

19. The method of claim 16, wherein the enhancing comprises performing at least one of:

scaling, cropping, foreground subtraction, background subtraction, contrast adjustment, and text rotation.

20. The method of claim 16, wherein the enhancing comprises appending information to the image.

21. The method of claim 16, wherein the image is a whiteboard image, and enhancing a whiteboard image comprises computing a background of the image and subtracting the background from the image to enhance textual content in the image.

22. A method for organizing images captured in an environment, the method comprising:

receiving a captured image;

classifying the image into a semantic category based at least in part on content within the image; and

enhancing the image based on the semantic category into which the image was classified, wherein the enhancing comprises;

recognizing content within the image;

associating the image with a stored document based on the recognized content; and

replacing the captured image with the stored document.

23. The method of claim 22, wherein the classifying comprises:

extracting a plurality of features from the image;

combining the extracted features into a feature vector; and

comparing the feature vector with information in a trained database to determine the semantic classification for the image.

24. The method of claim 22, wherein the classifying comprises comparing the heights of a plurality of markings on the image to determine whether the image belongs to a semantic category typically comprising machine-generated text or a semantic category typically comprising handwriting.

25. The method of claim 22, further comprising:

preparing the enhanced image for printing by selecting a layout orientation for a printed output of the image based on the semantic category into which the image was classified.

26. The method of claim 22, further comprising:

preparing the enhanced image for printing by selecting a paper source and/or ink selection for a printed output of the image based on the semantic category into which the image was classified.

27. The method of claim 22, further comprising:

preparing the enhanced image for printing, wherein preparing the enhanced image comprises;

retrieving a set of default printing preferences associated with the semantic category into which the image was classified; and

applying the set of default printing preferences to generate a printed output of the image.

28. The method of claim 22, wherein the captured image is received from a digital camera.

29. The method of claim 22, further comprising:

printing the enhanced image to generate a printed output thereof.

30. The method of claim 22, further comprising:

delivering the enhanced image to one or more users, the delivered enhanced image capable of being printed.