Generation of documents from images

• US 8,509,563 B2
• Filed: 02/02/2006
• Issued: 08/13/2013
• Est. Priority Date: 02/02/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Robinson, Merle Michael, Uyttendaele, Matthieu T., Zhang, Zhengyou, Simard, Patrice Y.
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US11/275,908
• Publication Number:
  US 20070177183A1
• Time in Patent Office: 2,749 Days
• Field of Search: 382/284
• US Class Current: 382/284
• CPC Class Codes: G06F 40/279 Recognition of textual entities

Abstract

A system for generating soft copy (digital) versions of hard copy documents uses images of the hard copy documents. The images may be captured using a device suitable for capturing images, like a camera phone. Once available, the images may be processed to improve their suitability for document generation. The images may then be processed to recognize and generate soft copy versions of the documents represented by the images.

Claims

1. A method for generating a soft copy document from a hard copy document, comprising:

acquiring via photographs taken by a user at least two images of a single page of the hard copy document;

providing feedback to the user indicating any parts of the single page not photographed and further indicating any of the at least two images that are of insufficient quality for generating the soft copy document;

processing the acquired images to be better suited for soft copy document generation;

combining the processed images resulting in a composite image that represents the single page of the hard copy document;

recognizing one or more particular text elements in the composite image;

identifying, by a document type module, one or more document types of the composite image, wherein the document type module is configured for identifying a plurality of document types including a word processor document type, a spreadsheet document type, a presentation document type, and form types of forms including business cards, facsimiles, and a form complying with a pre-defined template;

recognizing further particular text elements in the composite image according to characteristics of the identified one or more document types;

generating the soft copy document as at least one of the identified one or more document types, the soft copy document including the recognized one or more particular text elements and the recognized further particular text elements; and

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein the processing step further comprises one or more of the following operations:

removing keystone effects from the two or more images; and

removing shadowing from the two or more images.

3. The method of claim 1, wherein the processing step further comprises increasing a resolution of at least one of the images.

4. The method of claim 1, wherein the recognizing particular elements step further comprises:

if a particular element cannot be identified with a defined level of certainty, presenting to a user one or more selectable choices for the particular element;

receiving a selected choice of the one or more selectable choices; and

recognizing the particular element as the selected choice.

5. The method of claim 1, further comprising:

identifying one or more background images in the composite image;

removing the background images before recognizing elements; and

adding the background images to the soft copy document after recognizing elements.

6. The method of claim 1, wherein the steps of processing the images, creating the composite image, recognizing elements, and generating the soft copy document are performed automatically without human intervention.

7. The method of claim 1, wherein the step of acquiring the images further comprises taking two or more photographs of the one or more hard copy documents using a digital camera.

8. The method of claim 7, wherein the digital camera further comprises a camera integrated with a mobile telephone.

9. The method of claim 1, wherein the step of acquiring the images further comprises scanning the one or more hard copy documents using a device with document scanning functionality.

10. A device, comprising:

a memory;

a document generation module implemented at least in part in the memory and configured for generating a soft copy document from a hard copy document;

an image acquisition module configured for acquiring via photographs taken by a user at least two images of a single page of the hard copy document;

an image feedback module configured for providing feedback to the user indicating any parts of the single page not photographed and further indicating any of the at least two images that are of insufficient quality for generating the soft copy document;

an image processing module configured for processing the acquired images to be better suited for soft copy document generation;

an image creation module configured for combining the processed images resulting in a composite image that represents the single page of the hard copy document;

a document type module configured for identifying one or more document types of the composite image, wherein the document type module is configured for identifying a plurality of document types including a word processor document type, a spreadsheet document type, a presentation document type, and form types of forms including business cards, facsimiles, and a form complying with a pre-defined template;

an element recognition module configured for recognizing one or more particular text elements in the composite image according to characteristics of the identified one or more document types; and

wherein at least one of the modules is configured to operate on the device.

11. The device of claim 10, wherein the device further comprises a mobile telephone.

12. The device of claim 10, wherein the image acquisition module is further configured to acquire images generated by a second device.

13. The device of claim 12, wherein the second device includes document scanning functionality.

14. The device of claim 10, further comprising an image feedback module configured to provide feedback to a user acquiring the images, said feedback indicating that an image of at least a portion of the hard copy document has not been satisfactorily acquired.

15. One or more data storage devices that are not signal or carrier wave comprising computer-executable instructions that, when executed by a computer, cause the computer to perform a method comprising:

acquiring via photographs taken by a user at least two images of a single page of a hard copy document;

providing feedback to the user indicating any parts of the single page not photographed and further indicating any of the at least two images that are of insufficient quality for generating the soft copy document;

processing the acquired images to be better suited for soft copy document generation;

combining the processed images resulting in a composite image that represents the single page of the hard copy document;

recognizing particular text elements in the composite image;

identifying, by a document type module, one or more document types of the composite image, wherein the document type module is configured for identifying a plurality of document types including a word processor document type, a spreadsheet document type, a presentation document type, and form types of forms including business cards, facsimiles, and a form complying with a pre-defined template;

recognizing further particular text elements in the composite image according to characteristics of the identified one or more document types; and

generating the soft copy document as at least one of the identified one or more document types, the generated soft copy document including the recognized particular text elements and the recognized further particular text elements.

16. The one or more data storage devices of claim 15, wherein the processing step further comprises one or more of the following operations:

removing keystone effects; and

removing shadowing.

17. The one or more data storage devices of claim 15, wherein the processing step further comprises increasing the resolution of at least one of the images.

18. The one or more data storage devices of claim 15, wherein the creating step further comprises joining two or more images to create the composite image.