Lifting ink annotations from paper

• US 7,526,129 B2
• Filed: 06/23/2005
• Issued: 04/28/2009
• Est. Priority Date: 06/23/2005
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bargeron, David M.
• Primary Examiner(s): Bella; Matthew C
• Assistant Examiner(s): Seth; Manav
• Application Number:
  US11/165,070
• Publication Number:
  US 20060291727A1
• Time in Patent Office: 1,405 Days
• Field of Search: 382101-102, 382/119, 382137-140, 382/179, 382186-189, 382/257, 382294-296, 382/116, 715230-233
• US Class Current: 382/179
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition

Abstract

A computer-implemented system for obtaining ink annotations from paper comprises a comparator that compares an initial electronic version of a document with an electronic image of an annotated version of the document to identify a handwritten annotation. The system also includes an annotation lifter that extracts the handwritten annotation from the electronic image and converts the handwritten annotation into a computer-usable form. Methods for using the system are also provided.

Claims

1. A computer-implemented system having a processor communicatively coupled to a memory for facilitating obtaining ink annotations from paper, comprising:

a comparator that compares an initial electronic version of a document with an electronic image of an annotated version of the document to identify an ink annotation supplementing original text of the initial electronic version of the document, the ink annotation including at least one of emphasis highlighting, typewritten information, stamped information, or printed information placed over, near, or around original text;

an annotation lifter that extracts the ink annotation from the electronic image and converts the ink annotation into an independent computer-usable object and assigns a document-altering command to the object based on an effect on the electronic image determined for the ink annotation;

an image registration component that registers the electronic image of an annotated version of the document with the initial electronic version of the document;

an image subtractor that subtracts an image of the initial electronic version of the document from the electronic image of an annotated version;

a stroke joiner that joins a broken stroke of the ink annotation, the broken stroke being included in the ink annotation; and

,

an integrator that inserts the computer-usable object representing the ink annotation at an appropriate nonstatic anchor position within an editing version of the document.

2. The system of claim 1, further comprising a preprocessor that performs a preprocessing task on the image of the annotated version of the document, the preprocessing task including at least one of white balancing, Bayer'"'"'s effect removal, and perspective correction.

3. The system of claim 2, further comprising a caching component that creates the initial electronic version.

4. The system of claim 3, further comprising a search component that locates the initial electronic version of the document for use by the comparator.

5. A computer-implemented method for lifting handwritten annotations from a document comprising:

using the computer to perform the actions of;

comparing an initial electronic version of a document with an electronic image of an annotated version of the document;

identifying a handwritten annotation in the electronic image;

converting the handwritten annotation into an object form usable by a computer; and

associating a document-altering command to the object form;

registering the electronic image of the annotated version of the document with the initial electronic version;

subtracting the image of the initial electronic version of the document from the electronic image of the annotated version;

joining a broken stroke of the ink annotation, the broken stroke being included in the ink annotation; and

inserting the converted handwritten annotation at a nonstatic position in an editing version of the document.

6. The method of claim 5, further comprising a preprocessor that performs a preprocessing task on the image of the annotated version of the document, the preprocessing task including at least one of white balancing, Bayer'"'"'s effect removal, and perspective correction.

7. A computer-implemented system having a processor communicatively coupled to a memory for lifting handwritten annotations from a document, comprising:

means for comparing an initial electronic version of a document with an electronic image of an annotated version of the document;

means for identifying a handwritten annotation in the electronic image;

means for converting the handwritten annotation into an independent data representation and for assigning a document-altering command to the data representation;

means for registering the electronic image of the annotated version of the document with the initial electronic version;

means for subtracting the image of the initial electronic version of the document from the electronic image of the annotated version;

means for joining a broken stroke of the ink annotation, the broken stroke being included in the ink annotation; and

means for inserting the converted handwritten annotation into a nonstatic position in an editing version of the document.

8. The system of claim 7, further comprising a preprocessor that performs a preprocessing task on the image of the annotated version of the document, the preprocessing task including at least one of white balancing, Bayer'"'"'s effect removal, and perspective correction.