Signature Detection in E-Mails

• US 20120254166A1
• Filed: 08/29/2011
• Published: 10/04/2012
• Est. Priority Date: 03/30/2011
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kumar, Shailesh, AGARWAL, Gaurav
• Application Number:
  US13/219,864
• Publication Number:
  US 20120254166A1
• Time in Patent Office: Days
• US Class Current: 707/730
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]

Abstract

In an electronic discovery search tool, non-substantive information, such as signatures in e-mail, can bias a search tool and add processing time. A method and system for identifying recurring non-substantive text in documents has been developed so that non-substantive text may be processed or ignored by the search tool, as needed.

Claims

1. A method of identifying non-substantive text in a document, comprising:

capturing, by a processor, one or more blocks of text from each document in a set of documents;

determining, by the processor, the frequency of occurrence for each captured block of text in the set of documents;

identifying, by the processor one or more blocks of text in the set of documents as non-substantive text when the frequency of occurrence is above a received threshold; and

tagging, by the processor, the non-substantive text in each document in the set of documents.

2. The method of claim 1, wherein the block of text is one or more lines of a document.

3. The method of claim 1, further comprising filtering, by the processor, a corpus of documents in accordance with specified criteria to create the set of documents.

4. The method of claim 1, wherein the set of documents is a first set of documents, and further comprising tagging non-substantive text in a second set of documents when the text is identified as non-substantive in the first set of documents.

5. The method of claim 1, wherein the step of determining the frequency of occurrence for each captured block in the set of documents comprises:

generating a checksum for each captured block of text in the set of documents; and

determining the frequency of occurrence for each checksum in the set of documents.

6. The method of claim 5, wherein the checksum is an MD5 checksum.

7. The method of claim 5, wherein the frequency is determined by the percentage of e-mails in which the checksum occurs.

8. The method of claim 5, wherein generating a checksum comprises generating a checksum for each unit of text in each document in a first set of documents, and tagging comprises tagging blocks of text in a second set of documents.

9. The method of claim 8, wherein the first set of documents is a subset of the second set of documents.

10. A system for identifying non-substantive text in a document, comprising:

a capturer to capture one or more blocks of text from each document in a set of documents;

a generator that generates a checksum for each captured block of text;

a calculator to calculate a frequency of occurrence of each checksum in the set of documents; and

a tagger for tagging blocks of text as non-substantive when a frequency of occurrence is above a threshold.

11. The system of claim 10, wherein the block of text is one or more lines of a document.

12. The system of claim 10, further comprising a filter module that filters a corpus of documents in accordance with specified criteria to create the set of documents.

13. The system of claim 10, wherein the generated checksum is an MD5 checksum.

14. The system of claim 10, wherein the calculated frequency of occurrence is determined by the percentage of documents in which the checksum occurs.

15. A system, comprising:

a processor; and

a memory, the memory having instructions stored thereon that, when executed by the processor, cause the processor to perform a method of identifying non-substantive text in a document, the method comprising;

capturing one or more blocks of text from each document in a set of documents;

determining the frequency of occurrence for each captured block of text in the set of documents;

identifying one or more blocks of text in the set of documents as non-substantive text when the frequency of occurrence is above a received threshold; and

tagging the non-substantive text in each document in the set of documents.

16. The system of claim 15, wherein the block of text is one or more lines of a document.

17. The system of claim 15, the method further comprising filtering a corpus of documents in accordance with specified criteria to create the set of documents.

18. The system of claim 15, wherein the set of documents is a first set of documents, and wherein the method further comprises tagging non-substantive text in a second set of documents when the text is identified as non-substantive in the first set of documents.

19. The system of claim 15, wherein the step of determining the frequency of occurrence for each captured block in the set of documents comprises:

generating a checksum for each captured block of text in the set of documents; and

determining the frequency of occurrence for each checksum in the set of documents.

20. The system of claim 19, wherein the checksum is an MD5 checksum.

21. The system of claim 19, wherein the frequency is determined by the percentage of documents in which the checksum occurs.

22. The system of claim 19, wherein the step of generating a checksum comprises generating a checksum for each unit of text in each document in a first set of documents, and the step of tagging comprises tagging blocks of text in a second set of documents.

23. The system of claim 18, wherein the first set of documents is a subset of the second set of documents.