Methods and apparatus for extraction of content from an email or email threads for use in providing implicit profile attributes and content for recommendation engines

• US 10,318,617 B2
• Filed: 01/15/2016
• Issued: 06/11/2019
• Est. Priority Date: 06/02/2015
• Status: Active Grant

• Current Assignee: Gartner, Inc.
• Original Assignee: Gartner, Inc.
• Inventors: Gelfand, Michael, Michel, Zeevi, Polonsky, Nir
• Primary Examiner(s): Hong, Stephen S
• Assistant Examiner(s): Robinson, Marshon L
• Application Number:
  US14/996,329
• Publication Number:
  US 20160357718A1
• Time in Patent Office: 1,243 Days
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/151 Transformation
• CPC Class Codes: G06F 40/154 Tree transformation for tree-structured or markup documents, e.g. XSLT, XSL-FO or stylesheets
• CPC Class Codes: H04L 51/063 Content adaptation, e.g. replacement of unsuitable content
• CPC Class Codes: H04L 51/42 Mailbox-related aspects, e.g. synchronisation of mailboxes

Abstract

Methods and apparatus for extracting content from an email or email thread are provided. Any replied content is removed from the email based on at least one of known delimiters and known email thread patterns used for separating original and reply emails. Any signature content is removed based on at least one of generated signature patterns for the sender and known signature patterns. Any greeting content is removed based on at least one of generated greeting patterns for the recipient and known greeting patterns. Any sent from content identifying a device or an email client from which the email was sent is removed based on known sent from content. The remaining email text can then be output for further processing, such as analyzing the text for use in augmenting a sender'"'"'s implicit profile, and analyzing the text for question or answer content on specific topics.

Claims

1. An automated, computerized method for extracting content from an email, comprising:

removing any replied content from the email based on at least one of known delimiters and known email thread patterns used for separating original and reply emails;

analyzing the email to locate any generated signature patterns for the sender and any known signature patterns, and removing any signature content from the email based on any of the located generated signature patterns for the sender and any of the located known signature patterns;

analyzing the email to locate any generated greeting patterns for the recipient and any known greeting patterns, and removing any greeting content from the email based on any of the located generated greeting patterns for the recipient and any of the located known greeting patterns;

removing any sent from content identifying a device or an email client from which the email was sent based on known sent from content; and

outputting remaining email text for further processing;

wherein said further processing comprises;

analyzing the remaining email text for use in augmenting a sender'"'"'s implicit profile by deriving key words from the remaining email text and storing the key words as a part of the sender'"'"'s implicit profile, the sender'"'"'s implicit profile being located in a recommendation engine database; and

analyzing the remaining email text for question and answer content on specific topics and storing the question and answer content in the recommendation engine database.

2. The method in accordance with claim 1, wherein, if the email comprises HTML content, further comprising:

initially at least one of deleting any HTML content and converting the HTML content into corresponding text; and

restoring the HTML content to the remaining text of the email to preserve original formatting.

3. The method in accordance with claim 2, wherein the converting of the HTML content into the corresponding text comprises removing any HTML tags or elements from the email and converting the HTML tags or elements into the corresponding text or new line characters.

4. The method in accordance with claim 3, further comprising:

storing the HTML tags or elements together with identifiers linking the HTML tags or elements to the corresponding text in a storage bank; and

retrieving the stored email tags or elements from the storage bank for use in the restoring of the HTML content to the remaining email text.

5. The method in accordance with claim 1, wherein the removing of the replied content comprises:

analyzing the email to determine if any of the known delimiters of replied content are present;

if any of the known delimiters of replied content are present, removing any replied content text appearing in the email after the known delimiters;

analyzing the email text to determine the presence of any of the known email thread patterns; and

if any of the known email thread patterns are present, extracting original email text from the email text based on the known email thread pattern.

6. The method in accordance with claim 5, further comprising:

storing the known delimiters of replied content and the known email thread patterns in a storage bank.

7. The method in accordance with claim 5, further comprising:

analyzing sets of emails from corresponding users;

determining user specific email thread patterns or styles of email thread patterns for each of the sets of emails and the corresponding users; and

storing each of the user specific email thread patterns or styles of email thread patterns as a known email thread pattern associated with the corresponding user.

8. The method in accordance with claim 1, wherein the removing of the signature content comprises:

generating one or more signature patterns or signature lines for a sender based on the sender name and the email text using the known signature patterns or signature line templates;

searching the email text to locate the one or more generated signature patterns or signature lines;

defining a signature block based on a located signature pattern or signature line;

determining whether text is present before the signature block; and

if text is present before the signature block, removing the signature block from the email text.

9. The method in accordance with claim 8, further comprising:

storing the known signature patterns or the signature line templates in a storage bank; and

storing rules defining various types of signature blocks which include definitive characteristics for each of the types of signature block.

10. The method in accordance with claim 8, further comprising:

analyzing sets of emails from corresponding users;

determining at least one of user specific signature patterns, user specific signature lines, and user specific signature blocks for each of the sets of emails and the corresponding users; and

storing the user specific signature patterns, user specific signature lines, and user specific signature blocks as known signature patterns, signature lines, or signature blocks for the corresponding user.

11. The method in accordance with claim 1, wherein the removing of the greeting content comprises:

analyzing the email text to determine the presence of any known greetings or the known greeting patterns; and

if any of the known greetings or the known greeting patterns are present, removing the greeting content comprising the known greeting or the known greeting pattern from the email text.

12. The method in accordance with claim 1, wherein the removing of the greeting content comprises:

storing known greetings or the known greeting patterns in a storage bank;

generating one or more specific greetings or greeting patterns for the recipient based on the recipient name, the email text and the known greetings or the known greeting patterns;

searching the email text for the generated greetings or greeting patterns;

if one of the generated greetings or greeting patterns is located, determining the presence of a greeting block based on a located generated greeting or greeting pattern;

determining whether text is present after the greeting block; and

if text is present after the greeting block, removing the greeting block.

13. The method in accordance with claim 12, further comprising:

storing rules defining various types of greeting blocks which include definitive characteristics for each of the types of the greeting blocks.

14. The method in accordance with claim 12, further comprising:

analyzing sets of emails from corresponding users;

determining at least one of user specific greetings, user specific greeting patterns, and user specific greeting blocks for each of the sets of emails and the corresponding users; and

storing the user specific greetings, user specific greeting patterns, and user specific greeting blocks as known greetings, greeting patterns, and greeting blocks for the corresponding users.

15. The method in accordance with claim 1, wherein the removing of the sent from content comprises:

analyzing the email text to determine the presence of any of the known sent from content; and

if any of the known sent from content is located, extracting all text occurring before the located sent from content for output as the remaining email text;

wherein the known sent from content comprises one of device generated sent from content and personalized sent from content created by a user.

16. The method in accordance with claim 15, further comprising:

storing the known sent from content and templates for the personalized sent from content in a storage bank.

17. The method in accordance with claim 1, wherein the email is part of an email thread comprised of multiple emails, reply emails, and/or forwarded emails, further comprising:

initially extracting the email from the email thread prior to extracting the content from the email.

18. The method in accordance with claim 17, wherein:

the extracting of the email from the email thread comprises;

analyzing the email thread to determine the presence of any of the known email thread patterns; and

if the email thread contains any of the known email thread patterns, splitting the email thread up into a first appearing message and a remainder of the thread; and

the first appearing message is then processed to remove the replied content, the signature content, the greeting content, and the sent from content.

19. The method in accordance with claim 18, further comprising:

stripping off the sender, recipient, and date corresponding to the first appearing message from the remainder of the email thread; and

extracting the next appearing message from the remainder of the email thread.

20. The method in accordance with claim 18, wherein the known email thread patterns comprise at least one of standard email thread patterns for particular email clients in specific languages, and user specific email patterns determined after analysis of sets of user emails.

21. The method in accordance with claim 1, further comprising:

analyzing sets of emails for each of a plurality of users to determine at least one of user specific email thread patterns, user specific greetings, and user specific signatures;

storing the user specific email thread patterns, the user specific greetings, and the user specific signatures in a storage bank together with a user identifier or as part of a user profile associated with a recommendation engine; and

using the at least one of the user specific email thread patterns, the user specific greetings, and the user specific signatures in the removing of the corresponding content from the email text.

22. The method in accordance with claim 21, wherein the analyzing of the sets of emails to determine the user specific greetings comprises:

analyzing a first line of each email from the set of emails for the user to locate all common words that appear in a percentage threshold of the emails;

modifying the first lines of the emails by replacing any words that are not the common words with a match any word indicator; and

storing the modified first lines of the emails as the user specific greetings.

23. The method in accordance with claim 21, wherein the analyzing of the sets of emails to determine the user specific signatures comprises:

analyzing the set of emails for the user to locate all common lines that appear in a percentage threshold of emails;

analyzing the set of emails to locate sequences of the common lines;

identifying any of the sequences of the common lines that appear at an end of one of the emails; and

storing each of the sequences of the common lines that appear at the end of emails as the user specific signatures.

24. The method in accordance with claim 21, wherein the analyzing of the sets of emails to determine the user specific email thread patterns comprises:

identifying all emails that contain either of the terms forward and reply;

analyzing all of the identified emails to locate all common line starts comprising words that appear as a first word in a line, excluding lines identified as a user specific signature;

locating all sequences of lines containing one of the common line starts;

identifying, for each of the sequences, all common words that appear in a percentage threshold of emails;

modifying each of the identified sequence by replacing all words that are not identified as one of the common words by a match any word indicator; and

storing any of the modified sequences that appear immediately after any user specific signature pattern in a percentage threshold of the emails as the user specific email thread patterns.

25. The method in accordance with claim 21, wherein the sets of emails for each user are obtained from an email source comprising at least one of a computer, a tablet computer, a laptop computer, a smartphone, an email server, an Internet enabled device, a corporate email database, an email client, and a user specific email account from an email provider.

26. The method in accordance with claim 25, further comprising:

enabling access to the email source by at least one of a system administrator, a user, or via a software application running on at least one of the computer, the tablet computer, the laptop computer, the smartphone, and the Internet enabled device.

27. The method in accordance with claim 1, wherein the sender'"'"'s implicit profile comprises user-related information derived by:

tracking the user'"'"'s behavior on at least one of one or more electronic devices, an electronic communications network, and a website used to access a recommendation engine;

analyzing the user-related information derived from the user behavior to extract or derive key words therefrom which are used to characterize user interests, expertise, and skills; and

storing the key words in the recommendation engine database as the implicit profile.

28. The method in accordance with claim 1, wherein the augmenting of the sender'"'"'s implicit profile further comprises:

applying natural language processing to the remaining email text to identify a specific concept or topic;

applying a weighting to the concept or topic; and

storing the weighted concept or topic in the sender'"'"'s implicit profile.

29. The method in accordance with claim 28, further comprising assigning an expertise score to at least one of the sender and a recipient in relation to the weighted concept or topic.

30. The method in accordance with claim 1, wherein the analyzing of the remaining email text for question and answer content comprises:

applying natural language processing to the remaining email text to classify the remaining email text as a question or an answer on a specific concept or topic;

linking related questions and answers; and

storing the linked questions and answers in the recommendation engine database for use by the recommendation engine.

31. A computerized system for extracting content from an email, comprising:

a computer processor running an email accumulation software application which is enabled to access stored user emails;

an email account management module adapted to receive the stored user emails from the email accumulation software application;

a self-learning module in communication with the email account management module for analyzing the user emails to determine at least one of user specific email thread patterns, user specific greetings, and user specific signatures;

a storage bank for storing the user specific email thread patterns, the user specific greetings, and the user specific signatures together with a user identifier or as part of a user profile associated with a recommendation engine; and

an email extraction module adapted for processing each of the emails, said processing comprising;

removing any replied content from the email based on at least one of known delimiters for separating original and reply emails, known email thread patterns, and user specific email thread patterns;

analyzing the email to locate any generated signature patterns for the sender, any known signature patterns, and any user specific signature patterns, and removing any signature content from the email based on any of the located generated signature patterns for the sender, any of the located known signature patterns, and any of the located user specific signatures;

analyzing the email to locate any generated greeting patterns for the recipient, any known greeting patterns, and any user specific greeting patterns, and removing any greeting content from the email based on any of the located generated greeting patterns for the recipient, any of the located known greeting patterns, and any of the located user specific greetings;

removing any sent from content identifying a device or an email client from which the email was sent based on known sent from content; and

outputting remaining email text for further processing;

wherein said further processing comprises;

analyzing the remaining email text for use in augmenting a sender'"'"'s implicit profile by deriving key words from the remaining email text and storing the key words as a part of the sender'"'"'s implicit profile, the sender'"'"'s implicit profile being located in a recommendation engine database; and

analyzing the remaining email text for question and answer content on specific topics and storing the question and answer content in the recommendation engine database.

32. The system in accordance with claim 31, wherein the email accumulation software application is running on an email source comprising at least one of a computer, a tablet computer, a laptop computer, a smartphone, an email server, an Internet enabled device, a corporate email database, an email client, and a user specific email account from an email provider.