System and method for comparing and reviewing documents

• US 8,196,030 B1
• Filed: 11/14/2008
• Issued: 06/05/2012
• Est. Priority Date: 06/02/2008
• Status: Active Grant

• Current Assignee: PwC Product Sales LLC
• Original Assignee: PricewaterhouseCoopers LLP (Pricewaterhousecoopers International Limited)
• Inventors: Wang, Lever, Ricart, Glenn, Thompson, Cynthia Ann, Wishon, Keith, Laube, Sheldon
• Primary Examiner(s): Sajous, Wesner
• Application Number:
  US12/271,159
• Time in Patent Office: 1,299 Days
• Field of Search: 715/200, 715/243, 715/251, 715/254, 715/273, 715/700, 715276-277, 715/713, 345/581, 345/617, 345/619, 345/635, 382/165, 382/170, 382/173, 382/177, 382180-181, 382/276, 382/282, 382/290
• US Class Current: 715/200
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition

Abstract

A document processing system for accurately and efficiently analyzing documents and methods for making and using same. Each incoming document includes at least one section of textual content and is provided in an electronic form or as a paper-based document that is converted into an electronic form. Since many categories of documents, such as legal and accounting documents, often include one or more common text sections with similar textual content, the document processing system compares the documents to identify and classify the common text sections. The document comparison can be further enhanced by dividing the document into document segments and comparing the document segments; whereas, the conversion of paper-based documents likewise can be improved by comparing the resultant electronic document with a library of standard phrases, sentences, and paragraphs. The document processing system thereby enables an image of the document to be manipulated, as desired, to facilitate its review.

Claims

1. A processor-implemented method for comparing and viewing documents, comprising the steps of:

classifying each page of a first document into a category;

segmenting each page of the first document into logical word sequences;

creating a hierarchical table of contents for the first document;

comparing the text of the first document to the text of a second document to determine segments that are changed versus segments that are unchanged; and

displaying summary views of the first or second documents on a display device with words shown in differing contrast to indicate matching text versus novel text.

2. The processor-implemented method of claim 1, further comprising enabling reviewer searching for highly matching text between the first and second documents.

3. The processor-implemented method of claim 1, further comprising navigating to highly matching text between the first and second documents.

4. The processor-implemented method of claim 1, further comprising comparing recognitions of phrases, sentences, and paragraphs of the first document with previously encountered or standard phrases, sentences, and paragraphs.

5. The processor-implemented method of claim 1, wherein the first document is an electronic document.

6. The processor-implemented method of claim 1, wherein the first document is an imaged document.

7. The processor-implemented method of claim 1, wherein the first document is an optical character recognition (OCR)-processed document.

8. The processor-implemented method of claim 1, where the step of classifying each page is performed with a multi-class classifier.

9. The processor-implemented method of claim 1, where the step of classifying each page is performed with a set of binary classifiers that vote to determine with which class a page should be labeled.

10. The processor-implemented method of claim 1, where the step of comparing the text of the first document the second document includes detecting places where the documents contain exactly the same content.

11. The processor-implemented method of claim 1, where the step of segmenting each page is performed with a binary classifier.

12. The processor-implemented method of claim 11, where the step of segmenting each page includes using text and image features to determine that a new segment in a document has started.

13. The processor-implemented method of claim 12, wherein the text features include the density or occurrence of a word or a word class on a page.

14. The processor-implemented method of claim 12, where the step of segmenting each page includes using page class labels.

15. The processor-implemented method claim 1, where the step of segmenting each page is performed in phases using different classifiers for each level of the document hierarchy.

16. The processor-implemented method of claim 15, further comprising the step of extracting the document hierarchy from the first document.

17. The processor-implemented method of claim 1, where the step of creating a hierarchical table of contents includes generating components of the hierarchical table of contents where the first document lacks physical or traditional components of a table of contents.

18. The processor-implemented method of claim 17, where the step of generating said components of the hierarchical table of contents comprises using a binary classifier to generate said components of the hierarchical table.

19. The processor-implemented method of claim 17, where the step of generating said components of the hierarchical table of contents comprises using a multi-class classifier to generate said components of the hierarchical table.

20. The processor-implemented method of claim 1, where the step of comparing the first and second documents includes detecting places where the documents contain exactly or substantially the same text content.

21. The processor-implemented method of claim 20, where the step of comparing the first and second documents includes determining word-level distance between segments of the first and second documents.

22. The processor-implemented method of claim 21, where the step of determining word-level distance includes comparing the words in every segment of the first document to every segment of the second document.

23. The processor-implemented method of claim 22, where the step of determining word-level distance further comprises determining whether two segments have a minimal amount of common text.

24. The processor-implemented method of claim 23, further comprising the step of determining whether two segments have N occurrences of at least K consecutive identical words, where N and K are integers.

25. The processor-implemented method of claim 24, further comprising the step of calculating an edit distance between segments by determining the insertion, deletion, or replacement cost for each word that is inserted, deleted or replaced from one segment to another.

26. The processor-implemented method of claim 25, further comprising the step of calculating the edit distance between segments by dividing the edit distance of the segment by the number of words in the segment.

27. The processor-implemented method of claim 26, where the step of calculating the edit distance between segments includes calculating distances in multiple dimensions including row segments and column segments in a table.

28. The processor-implemented method of claim 25, further comprising the step of aligning each segment in the first document with a most similar segment in the second document.

29. The processor-implemented method of claim 28, further comprising the step of determining the longest common subsequence between pairs of segments in the first and second documents.

30. The processor-implemented method of claim 29, further comprising the step of determining the segment level edit distance between the first and second documents.

31. The processor-implemented method of claim 1, further comprising the step of locating moved text between the first and second documents.

32. The processor-implemented method of claim 1, further comprising the step of locating the most similar text in the second document to a selected area of text in the first document.

33. The processor-implemented method of claim 1, further comprising the step of locating deleted text between the first and second documents.

34. The processor-implemented method of claim 1, further comprising the step of locating inserted text between the first and second documents.

35. The processor-implemented method of claim 1, further comprising the step of detecting places where the documents contain conceptually the same content for a reasonably large span of content.

36. The processor-implemented method of claim 20, wherein symbols are considered words.

37. The processor-implemented method of claim 36, further comprising the step of comparing matching symbols for equality.

38. The processor-implemented method of claim 4 further comprising improving optical character recognition when comparing documents against known phrases, sentences, paragraphs, sections or documents by considering alternative recognitions.

39. The processor-implemented method of claim 1 further comprising the merging of segments which optimally increase the similarity between documents segments.

40. The processor-implemented method of claim 4, further comprises comparing recognitions of phrases, sentences, and paragraphs of the first document with new documents and user feedback.

41. The processor-implemented method of claim 1, where the step of displaying the first document includes using a view of the first document that is substantially similar to the original form of the first document.

42. The processor-implemented method of claim 1, further comprising the step of enabling user text searching for text belonging to a predetermined category using one or more classifiers.

43. The processor-implemented method of claim 42, further comprising the step of utilizing user-labeled examples to enhance the classifiers.

44. The processor-implemented method of claim 12, further comprising the step of utilizing user-labeled examples to enhance the classifiers.

45. The processor-implemented method of claim 15, further comprising the step of utilizing user-labeled examples to enhance the classifiers.

46. The processor-implemented method of claim 1, where the step of comparing considers common classes of words as equivalent for text comparison purposes.

47. The processor-implemented method of claim 1, where the step of comparing utilizes classifiers to find similar text and display it to the user as matching text.

48. The processor-implemented method of claim 1, where the step of displaying utilizes different colors or textures to indicate matching text.

49. The processor-implemented method of claim 1, further comprising the step of enabling navigation by double-clicking on an area of text in the first document, to navigate to and highlight the matching text in the second document.

50. The processor-implemented method of claim 1, further comprising the step of enabling navigation by selecting an area of text in the first document and then navigating to and highlighting the matching text in the second document.

51. The processor-implemented method of claim 1, further comprising the step of enabling navigation by creating hyperlinks between documents and document segments.

52. The processor-implemented method of claim 1, further comprising the step of ordering documents in terms of their similarities to each other.

53. The processor-implemented method of claim 1, further comprising the step of enabling the comparison of documents originating in different file formats.