COGNITIVE LEXICON LEARNING AND PREDICTIVE TEXT REPLACEMENT

• US 20180267954A1
• Filed: 03/17/2017
• Published: 09/20/2018
• Est. Priority Date: 03/17/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Catalano, Pasquale A., Crimmins, Andrew G., Tsfasman, Arkadiy O., Werner, John S.
• Granted Patent: US 10,460,032 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04W 4/14 Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD]

Abstract

A method comprising of receiving a first communication content directed to a user. The first communication content includes one or a combination of the following: content read by the user and content written by the user. The method also comprises of generating tokens corresponding to the first communication content by applying natural language processing and generating a token frequency index for the user, based on the tokens generated from the first communication content. The method determines a lexicon reading level for the user, based on the token frequency index generated for the user. The lexicon reading level indicates a reading level of the user. The method adds the lexicon reading level to a lexicon profile of the user. The method modifies a second communication content by replacing tokens with synonyms of the tokens based on comparing the difficulty ratings of the tokens with the user'"'"'s lexicon reading level.

Claims

1. A computer-implemented method of adjusting content of a communication to align with a lexicon reading level of a user, the computer-implemented method comprising:

receiving a first communication content directed to the user;

generating tokens corresponding to the first communication content by applying natural language processing;

generating a token frequency index for the user, based on the tokens generated from the first communication content;

determining the lexicon reading level for the user, based on the token frequency index generated for the user; and

adding the lexicon reading level to a lexicon profile of the user;

wherein;

the first communication content includes one or a combination of the following;

content read by the user and content written by the user; and

the lexicon reading level indicates a reading level of the user.

2. The computer-implemented method of claim 1 further comprising:

receiving a second communication content selected by the user;

receiving the lexicon reading level of the user;

performing a tokenization of the second communication content that is received, by applying natural language processing to the second communication content;

generating a plurality of tokens, based on the tokenization of the second communication content;

determining a difficulty rating of a first token of the plurality of tokens;

determining whether the difficulty rating for the first token of the plurality of tokens differs from the lexicon reading level of the user;

responsive to determining the difficulty rating of the first token of the plurality of tokens exceeds the lexicon reading level of the user, replacing the first token of the plurality of tokens with a replacement token; and

modifying the second communication content to include the replacement token for the first token of the plurality of tokens;

wherein;

the replacement token includes a synonym of the first token of the plurality of tokens that is consistent with the lexicon reading level of the user.

3. The computer-implemented method of claim 1, wherein a source of the first communication content is a member of groups consisting of:

a history of websites viewed;

a history of books read,

an online journal history,

a publication,

a social media post,

an email,

a SMS text message, and

a local stored document.

4. The computer-implemented method of claim 1, wherein tokenization includes accessing a source of words to determine tokens from text.

5. The computer-implemented method of claim 4, wherein the source of words includes computer-readable dictionaries and word lists.

6. The computer-implemented method of claim 1, wherein determining the difficulty rating of the first token of the plurality of tokens includes accessing a source of difficulty ratings of tokens.

7. The computer-implemented method of claim 1, wherein generating the lexicon reading level of the user includes applying a member of a group consisting of:

the token frequency index of the user,

the source of difficulty ratings of tokens, and

a context of the source of the first communication content.

8. The computer-implemented method of claim 1 further comprising:

responsive to determining the difficulty rating of the first token of the plurality of tokens does not exceeds the lexicon reading level of the user, replacing the first token of the plurality of tokens with a replacement token;

wherein;

the replacement token includes a synonym of the first token of the plurality of tokens that is rated a higher difficulty rating than the lexicon reading level of the user to further educational opportunity of the user.

9. The computer-implemented method of claim 1, wherein the source of the first communication content and a source of the second communication content includes audio content.

10. A computer program product for adjusting content of a communication to align with a lexicon reading level of a user, the computer program product comprising:

one or more computer-readable storage media, and program instructions stored on the one or more computer-readable storage media, executable by one or more processors;

the stored program instructions comprising;

program instructions to receive a first communication content directed to the user;

program instructions to generate tokens corresponding to the first communication content by applying natural language processing;

program instructions to generate a token frequency index for the user, based on the tokens generated from the first communication content;

program instructions to determine the lexicon reading level for the user, based on the token frequency index generated for the user; and

program instructions to add the lexicon reading level to a lexicon profile of the user;

wherein;

the first communication content includes one or a combination of the following;

content read by the user and content written by the user; and

the lexicon reading level indicates a reading level of the user.

11. The computer program product of claim 10, the stored program instructions further comprising:

program instructions to receive a second communication content selected by the user;

program instructions to receive the lexicon reading level of the user;

program instructions to perform a tokenization of the second communication content that is received, by applying natural language processing to the second communication content;

program instructions to generate a plurality of tokens, based on the tokenization of the second communication content;

program instructions to determine a difficulty rating of a first token of the plurality of tokens;

program instructions to determine whether the difficulty rating for the first token of the plurality of tokens differs from the lexicon reading level of the user;

responsive to determining the difficulty rating of the first token of the plurality of tokens exceeds the lexicon reading level of the user, program instructions to replace the first token of the plurality of tokens with a replacement token; and

program instructions to modify the second communication content to include the replacement token for the first token of the plurality of tokens;

wherein;

the replacement token includes a synonym of the first token of the plurality of tokens that is consistent with the lexicon reading level of the user.

12. The computer program product of claim 10, wherein a source of the first communication content is a member of groups consisting of:

a history of websites viewed,

a history of books read,

an online journal history,

a publication,

a social media post,

an email,

a SMS text message, and

a local stored document.

13. The computer program product of claim 10, wherein tokenization includes accessing a source of words to determine tokens from text.

14. The computer program product of claim 13, wherein the source of words includes computer-readable dictionaries and word lists.

15. The computer program product of claim 10, wherein determining the difficulty rating of the first token of the plurality of tokens includes accessing a source of difficulty ratings of tokens.

16. The computer program product of claim 10, wherein generating the lexicon reading level of the user includes applying a member of a group consisting of:

the token frequency index of the user,

the source of difficulty ratings of tokens, and

a context of the source of the first communication content.

17. The computer program product of claim 10, the stored program instructions further comprising:

responsive to determining the difficulty rating of the first token of the plurality of tokens does not exceeds the lexicon reading level of the user, program instructions to replace the first token of the plurality of tokens with a replacement token

wherein;

the replacement token includes a synonym of the first token of the plurality of tokens that is rated a higher difficulty rating than the lexicon reading level of the user to further educational opportunity of the user.

18. The computer program product of claim 10, wherein the source of the first communication content and a source of the second communication content includes audio content.

19. A computer system for adjusting content of a communication to align with a lexicon reading level of a user, the computer system comprising:

one or more computer processors;

one or more computer readable storage device;

program instructions stored on the one or more computer readable storage devices for execution by at least one of the one or more computer processors, the stored program instructions comprising;

program instructions to receive a first communication content directed to the user;

program instructions to generate tokens corresponding to the first communication content by applying natural language processing;

program instructions to generate a token frequency index for the user, based on the tokens generated from the first communication content;

program instructions to determine the lexicon reading level for the user, based on the token frequency index generated for the user; and

program instructions to add the lexicon reading level to a lexicon profile of the user;

wherein;

the first communication content includes one or a combination of the following;

content read by the user and content written by the user; and

the lexicon reading level indicates a reading level of the user.

20. The computer system of claim 19, the stored program instructions further comprising:

program instructions to receive a second communication content selected by the user;

program instructions to receive the lexicon reading level of the user;

program instructions to perform a tokenization of the second communication content that is received, by applying natural language processing to the second communication content;

program instructions to generate a plurality of tokens, based on the tokenization of the second communication content;

program instructions to determine a difficulty rating of a first token of the plurality of tokens;

program instructions to determine whether the difficulty rating for the first token of the plurality of tokens differs from the lexicon reading level of the user;

responsive to determining the difficulty rating of the first token of the plurality of tokens exceeds the lexicon reading level of the user, program instructions to replace the first token of the plurality of tokens with a replacement token; and

program instructions to modify the second communication content to include the replacement token for the first token of the plurality of tokens;

wherein;

the replacement token includes a synonym of the first token of the plurality of tokens that is consistent with the lexicon reading level of the user.