Classifying documents based on automatically detected rules

• US 8,903,752 B1
• Filed: 02/09/2012
• Issued: 12/02/2014
• Est. Priority Date: 02/09/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Pacovsk, Ondej, Aberdeen, Douglas Alexander, Takcsi, Pl
• Primary Examiner(s): Gaffin, Jeffrey
• Assistant Examiner(s): Bharadwaj, Kalpana
• Application Number:
  US13/370,264
• Time in Patent Office: 1,027 Days
• Field of Search: 706/20
• US Class Current: 706/20
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management

Abstract

Systems and methods for classifying a set of documents are provided. In some aspects, a method includes receiving a subset of the set of documents. The method also includes automatically determining at least one classification rule for the subset of documents based on the documents in the subset. At least a true-positive threshold proportion of documents in the subset of documents follows the at least one classification rule. At most a false-positive threshold proportion of documents in the set of documents and not in the subset of documents follow the at least one classification rule. The method also includes storing the at least one classification rule in association with the subset of documents.

Claims

1. A computer-implemented method for classifying a set of documents, the method comprising:

receiving an identified subset of documents within a set of documents;

automatically creating at least one classification rule for the subset of documents based on the documents in the identified subset, wherein at least a true-positive threshold proportion of documents in the subset of documents follows the at least one classification rule, further wherein at most a false-positive threshold proportion of documents in the set of documents and outside the identified subset of documents follow the at least one classification rule;

storing the at least one classification rule in association with the identified subset of documents;

receiving an input indicating removing a target document from the identified subset of documents; and

modifying the at least one classification rule based on the input indicating removing the target document from the identified subset of documents.

2. The method of claim 1, further comprising:

determining that the target document in the set of documents and outside the identified subset of documents follows the at least one classification rule; and

automatically placing the target document into the identified subset of documents based on the target document following the at least one classification rule.

3. The method of claim 1, further comprising:

determining that the target document in the set of documents and outside the identified subset of documents does not follow the at least one classification rule; and

foregoing placing the target document into the identified subset of documents based on the target document not following the at least one classification rule.

4. The method of claim 3, further comprising:

receiving an input indicating adding the target document to the identified subset of documents;

placing the target document into the identified subset of documents based on the input indicating adding the target document to the identified subset of documents; and

modifying the at least one classification rule based on the input indicating adding the tartlet document to the identified subset of documents.

5. The method of claim 1, wherein the at least one classification rule comprises at least one Boolean expression.

6. The method of claim 1, wherein the set of documents comprises electronic messages in an electronic messaging account and the identified subset of documents comprises electronic messages in the electronic messaging account that are associated with a category.

7. The method of claim 6, wherein the category comprises a folder, a label or a mark.

8. The method of claim 6, wherein the category is associated with an action applied to the electronic messages in the electronic messaging account that are associated with the category.

9. The method of claim 8, wherein the action comprises one or more of:

marking the electronic messages as read, foregoing placing the electronic messages in the inbox, or sending an automatic reply message.

10. The method of claim 6, wherein the at least one classification rule comprises one or more of a sender of the document being a member of a set of senders, a recipient of the document being a member of a set of recipients, a keyword being present in a subject of the document, or a keyword being present within the document.

11. The method of claim 1, wherein the set of documents comprises files in a directory and the identified subset of documents comprises the files in a subdirectory of the directory.

12. The method of claim 11, wherein the at least one classification rule comprises one or more of the document having one of one or more file types, a keyword being present in the filename of the document, or a keyword being present within the document.

13. A non-transitory computer-readable medium for classifying a set of documents, the computer-readable medium comprising instructions that, when executed by a computer, cause the computer to:

receive a plurality of identified subsets of documents within a set of documents;

automatically create at least one classification rule corresponding to each identified subset in at least a portion of the plurality of identified subsets of documents, the at least one classification rule being based on the documents in the corresponding identified subset, wherein at least one document in each identified subset in the at least a portion of the plurality of identified subsets follows the at least one corresponding classification rule;

store the at least one corresponding classification rule in association with each identified subset of documents in the at least the portion of the plurality of identified subsets of documents;

receive an input indicating removing a target document from the identified subset of documents; and

modify the at least one classification rule based on the input indicating removing the target document from the identified subset of documents.

14. The computer-readable medium of claim 13, wherein at least a true-positive threshold proportion of the documents in a first identified subset of documents in the at least a portion of the plurality of identified subsets of documents follow the at one corresponding classification rule.

15. The computer-readable medium of claim 13, wherein at most a false-positive threshold proportion of the documents in the set of documents and outside a first identified subset of documents in the at least a portion of the plurality of identified subsets of documents follow the at one corresponding classification rule.

16. The computer-readable medium of claim 13, wherein the plurality of identified subsets are mutually exclusive.

17. The computer-readable medium of claim 13, wherein the at least the portion of the plurality of identified subsets comprises four or more identified subsets.

18. The computer-readable medium of claim 13, wherein the instructions to automatically create at least one classification rule corresponding to each identified subset in at least a portion of the plurality of identified subsets of documents comprise instructions that, when executed by the computer, cause the computer to:

receive an input corresponding to an action taken by a user, the action comprising placing one or more documents into or outside of a first identified subset in the at least the portion of the plurality of identified subsets of documents; and

create, in response to the action taken by the user, at least one classification rule corresponding to the first identified subset.

19. The computer-readable medium of claim 18, wherein the action taken by the user comprises one or more of placing a document in a folder, adding a label to a document, or adding a mark to a document.

20. A system for classifying a set of electronic messages, the system comprising:

one or more processors; and

a memory comprising instructions that, when executed by the one or more processors, cause the one or more processors to;

receive an identified subset of electronic messages within a set of electronic messages;

automatically create at least one classification rule for the subset of electronic messages based on the electronic messages in the identified subset, wherein at least a true-positive threshold proportion of electronic messages in the identified subset of electronic messages follows the at least one classification rule, further wherein at most a false-positive threshold proportion of electronic messages in the set of electronic messages and outside the identified subset of electronic messages follow the at least one classification rule;

storing the at least one classification rule in association with the identified subset of electronic messages;

receive an input indicating removing a target document from the identified subset of documents; and

modify the at least one classification rule based on the input indicating removing the target document from the identified subset of documents.

21. The system of claim 20, wherein the memory further comprises instructions that, when executed by the one or more processors, cause the one or more processors to:

take an additional action associated with the electronic messages in the identified subset of electronic messages.

22. The system of claim 21, wherein the additional action comprises one or more of:

placing the electronic messages in the subset into a folder, marking the electronic messages in the identified subset as important, flagging the electronic messages in the identified subset, foregoing placing the electronic messages in the identified subset in the inbox, marking the electronic messages in the identified subset as read, or sending an automatic reply message to the electronic messages in the identified subset.