Synchronizing annotations between printed documents and electronic documents

• US 9,436,665 B2
• Filed: 02/28/2013
• Issued: 09/06/2016
• Est. Priority Date: 02/28/2013
• Status: Active Grant

• Current Assignee: Thomson Reuters Enterprise Centre GmbH (The Woodbridge Co. Ltd.)
• Original Assignee: Thomson Reuters Global Resources
• Inventors: Pircher, Peter, Leidner, Jochen Lothar, Cooper, Aaron, Dabney, Daniel, Medin, Jonathan, Conrad, Jack G.
• Primary Examiner(s): Hong, Stephen
• Assistant Examiner(s): Khan, Shahid
• Application Number:
  US13/781,446
• Publication Number:
  US 20140245123A1
• Time in Patent Office: 1,286 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/171 by use of digital ink

Abstract

An image of a printed document portion is provided to a synchronizer. The synchronizer retrieves an electronic version of the printed document and identifies an electronic text portion that is textually similar to a printed text portion. The synchronizer detects an annotation in the printed document portion and inserts a corresponding digital annotation into the electronic document.

Claims

1. A computer-implemented method for synchronizing annotations between a printed document and an electronic document, the method comprising:

receiving, at a server, an image of a printed document portion having a printed text portion, wherein the printed document portion includes an annotation in proximity to the printed text portion;

accessing at least a portion of the electronic document from a memory device, wherein the electronic document comprises an electronic version of the printed document;

identifying an electronic text portion within the electronic document that corresponds to the printed text portion;

detecting the annotation in the printed document portion;

facilitating insertion of a digital annotation into at least one of the electronic document and a copy of the electronic document, the digital annotation corresponding to the detected annotation, wherein the digital annotation is inserted in proximity to the identified electronic text portion at a location that at least substantially corresponds to a location of the detected annotation in the printed document portion;

identifying an additional digital annotation in the electronic document;

determining, by analyzing the image of the printed document portion, that the printed document portion does not include an additional annotation corresponding to the additional digital annotation;

determining whether the additional digital annotation is a migrated annotation or a direct digital annotation, wherein a migrated annotation is an annotation inserted during a synchronization process to correspond to an annotation in a printed document, and wherein a direct digital annotation is an annotation added directly to an electronic document by a reader; and

removing the additional digital annotation from the electronic document if the additional digital annotation is a migrated annotation, wherein the additional digital annotation is not removed if the additional digital annotation is a direct digital annotation.

2. The method of claim 1, wherein determining whether the additional digital annotation is a migrated annotation or a direct digital annotation comprises determining a value of a tag associated with the additional digital annotation.

3. The method of claim 1, further comprising:

identifying an annotation type of the detected annotation, wherein the annotation type comprises at least one of a highlight, a handwritten text, an underline, and a flag, and wherein facilitating insertion of the digital annotation into the electronic document comprises creating a digital annotation of the identified annotation type.

4. The method of claim 1, wherein facilitating insertion of the digital annotation in the electronic document comprises inserting electronic text into a margin of the electronic document.

5. The method of claim 1, wherein the electronic document comprises a platform-independent document format.

6. The method of claim 1, wherein identifying the corresponding electronic text portion comprises identifying a textual similarity between the printed text portion and the corresponding electronic text portion.

7. The method of claim 6, wherein identifying the textual similarity comprises:

recognizing a part of the printed text portion by performing an optical character recognition (OCR) procedure on the image of the printed document portion;

converting the recognized part of the printed text portion to recognized electronic text using the OCR procedure;

identifying at least one character sequence in the recognized electronic text that includes a recognition error;

creating pruned recognized electronic text, wherein the pruned recognized electronic text comprises the recognized electronic text from which the at least one character sequence has been removed; and

searching the electronic document using at least one search query that comprises the pruned recognized electronic text.

8. One or more non-transitory computer-readable media having computer-executable instructions embodied thereon for facilitating synchronization of annotations between a printed document and an electronic document, wherein the instructions include a synchronizer having a plurality of program components, the plurality of program components comprising:

a matching component that (1) receives an image of a printed document portion having a printed text portion and (2) identifies an electronic text portion within the electronic document that corresponds to the printed text portion; and

a digital annotation component that facilitates insertion of a digital annotation into the electronic document in proximity to the identified corresponding electronic text portion;

wherein the synchronizer is further configured to;

identify an additional digital annotation in the electronic document;

determine, by analyzing the image of the printed document portion, that the printed document portion does not include an annotation corresponding to the additional digital annotation;

determine whether the additional digital annotation is a migrated annotation or a direct digital annotation, wherein a migrated annotation is an annotation inserted during a synchronization process to correspond to an annotation in a printed document, and wherein a direct digital annotation is an annotation added directly to an electronic document by a reader; and

remove the additional digital annotation from the electronic document if the additional digital annotation is a migrated annotation, wherein the additional digital annotation is not removed if the additional digital annotation is a direct digital annotation.

9. The media of claim 8, wherein the matching component utilizes an optical character recognition (OCR) procedure to recognize the printed text portion within the printed document portion, and wherein the matching component utilizes one or more search queries to identify the corresponding electronic text portion.

10. The media of claim 9, wherein the matching component utilizes a pruning procedure to remove recognition errors from the recognized printed text portion.

11. The media of claim 8, further comprising:

a detection component that detects an annotation in the printed document portion by analyzing at least the image of the printed document portion, wherein the printed document portion includes the annotation in proximity to the printed text portion.

12. The media of claim 11, wherein the detection component partitions the image of the printed document portion into a plurality of zones, the plurality of zones comprising at least one text zone and at least one candidate zone.

13. The media of claim 12, wherein the detection component analyzes the at least one candidate zone using at least one of a handwriting recognition procedure, an OCR procedure, a statistical language model, a bitmap overlay comparison, and a statistical classifier.

14. The media of claim 11, wherein the detection component detects the annotation based, at least in part, on feedback received from at least one reviewer via a crowd-sourcing platform.

15. A system that facilitates synchronization of annotations between a printed document and an electronic document, the system comprising:

a server configured to receive, from an imaging device, an image of a printed document portion having a printed text portion, the server comprising a processor configured to instantiate a synchronizer configured to;

(a) identify a corresponding electronic text portion in the electronic document, wherein the corresponding electronic text portion is textually similar to the printed text portion,

(b) identify a digital annotation in the electronic document,

(c) determine, by analyzing the image of the printed document portion, that the printed document portion does not include an annotation corresponding to the digital annotation,

(d) determine whether the digital annotation is a migrated annotation or a direct digital annotation, wherein a migrated annotation is an annotation inserted during a synchronization process to correspond to an annotation in a printed document, and wherein a direct digital annotation is an annotation added directly to an electronic document by a reader, and

(e) remove the digital annotation from the electronic document if the digital annotation is a migrated annotation, wherein the additional digital annotation is not removed if the additional digital annotation is a direct digital annotation.

16. The system of claim 15, wherein the synchronizer is configured to:

(f) detect an annotation in the printed document portion by utilizing at least one of a handwriting recognition procedure, an OCR procedure, a statistical language model, a bitmap overlay comparison, and a statistical classifier; and

(g) facilitate insertion of an additional digital annotation into the electronic document, wherein the additional digital annotation corresponds to the detected annotation.

17. The system of claim 16, wherein the synchronizer is configured to facilitate insertion of the digital annotation into the electronic document by (1) associating a metadata file containing the annotation with the electronic document, and (2) inserting a pointer to the metadata file into the electronic document.

18. The system of claim 15, wherein the imaging device comprises at least one of a camera disposed in a reading device and an industrial scanning device.