NATURAL LANGUAGE PROCESSING SOCIAL-BASED MATRIX REFACTORIZATION

• US 20190325050A1
• Filed: 04/19/2018
• Published: 10/24/2019
• Est. Priority Date: 04/19/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: DIAMANTI, Gary F., MARZORATI, Mauro, BAUGHMAN, Aaron K., HATANAKA, Iwao
• Granted Patent: US 10,783,141 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/24534 Query rewriting; Transformation
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/56 Natural language generation

Abstract

Methods and systems for natural language processing are disclosed. A method includes: receiving, by a computing device, an initial question from a first user device; receiving, by the computing device, a plurality of additional questions from a plurality of additional user devices; decomposing, by the computing device, the initial question and each of the plurality of additional questions using natural language processing; determining, by the computing device, an overall ranking for each of a plurality of words in the plurality of decomposed questions; and generating, by the computing device, a new question by combining words based on the overall ranking.

Claims

1. A method comprising:

receiving, by a computing device, an initial question from a first user device;

receiving, by the computing device, a plurality of additional questions from a plurality of additional user devices;

decomposing, by the computing device, the initial question and each of the plurality of additional questions using natural language processing;

determining, by the computing device, an overall ranking for each of a plurality of words in the plurality of decomposed questions; and

generating, by the computing device, a new question by combining words based on the overall ranking.

2. The method according to claim 1, further comprising:

performing a search using the new question; and

providing a result of the search to the first user device.

3. The method according to claim 1, wherein the decomposing the initial question and each of the plurality of questions comprises, for each question, performing tokenization and vectorization.

4. The method according to claim 3, wherein the decomposing the initial question and each of the plurality of questions further comprises, for each question, generating a document-term matrix.

5. The method according to claim 4, wherein the document-term matrix includes a ranking of each of a plurality of words in the question.

6. The method according to claim 1, wherein the determining the overall ranking for each of the plurality of words in the plurality of decomposed questions is based on matrix refactorization of the plurality of decomposed questions.

7. The method according to claim 6, wherein the determining the overall ranking for each of the plurality of words in the plurality of decomposed questions is further based on slang semantics.

8. The method according to claim 7, wherein the determining the overall ranking for each of the plurality of words in the plurality of decomposed questions further comprises using a time associated with the slang semantics.

9. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

receive an initial question from a first user device;

receive a plurality of additional questions from a plurality of additional user devices;

decompose the initial question and each of the plurality of additional questions using natural language processing;

determine an overall ranking for each of a plurality of words in the plurality of decomposed questions; and

generate a new question by combining words based on the overall ranking.

10. The computer program product according to claim 9, the program instructions further being executable by the computing device to cause the computing device to:

perform a search using the new question; and

provide a result of the search to the first user device.

11. The computer program product according to claim 9, wherein the decomposing the initial question and each of the plurality of questions comprises, for each question, performing tokenization and vectorization.

12. The computer program product according to claim 11, wherein the decomposing the initial question and each of the plurality of questions further comprises, for each question, generating a document-term matrix.

13. The computer program product according to claim 12, wherein the document-term matrix includes a ranking of each of a plurality of words in the question.

14. The computer program product according to claim 9, wherein the determining the overall ranking for each of the plurality of words in the plurality of decomposed questions is based on matrix refactorization of the plurality of decomposed questions.

15. The computer program product according to claim 14, wherein the determining the overall ranking for each of the plurality of words in the plurality of decomposed questions is further based on slang semantics.

16. The computer program product according to claim 15, wherein the determining the overall ranking for each of the plurality of words in the plurality of decomposed questions further comprises using a time associated with the slang semantics.

17. A system comprising:

a hardware processor, a computer readable memory, and a computer readable storage medium associated with a computing device;

program instructions of a question receiver configured to receive an initial question from a first user device and a plurality of additional questions from a plurality of additional user devices;

program instructions of a question decomposer configured to decompose the initial question and each of the plurality of additional questions received by the question receiver using natural language processing; and

program instructions of a social refactorer configured to determine an overall ranking for each of a plurality of words in the plurality of questions decomposed by the question decomposer and generate a new question by combining words based on the overall ranking,

wherein the program instructions are stored on the computer readable storage medium for execution by the hardware processor via the computer readable memory.

18. The system according to claim 17, wherein the question decomposer is further configured to performing tokenization and vectorization on each question received by the question receiver.

19. The system according to claim 18, wherein the question decomposer is further configured to generate a document-term matrix.

20. The system according to claim 17, wherein the social refactorer is configured to determine the overall ranking for each of the plurality of words based on matrix refactorization of the plurality of questions decomposed by the question decomposer.