Determining whether a comment represented as natural language text is prescriptive

• US 10,248,648 B1
• Filed: 03/30/2017
• Issued: 04/02/2019
• Est. Priority Date: 07/11/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Thomas, Christopher Johannes, Wen, Chih Po, Kurra, Goutham
• Primary Examiner(s): Albertalli, Brian L
• Application Number:
  US15/475,015
• Time in Patent Office: 733 Days
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

An online system receives comments provided by users and analyzes them. The users may be associated with an organization, for example, employees of an enterprise may provide comments related to the enterprise. The online system classifies the comments to determine whether the comments are prescriptive or non-prescriptive. The online system may generate reports based on the classification of the comments. The online system may use a machine learning model for classifying the comments. The features used for the machine learning model include an indication of whether the input comment is associated with a question, n-grams from the comment, location of verbs in sentences, and so on.

Claims

1. A method for classifying comments, the method comprising:

receiving, by an online system, a plurality of comments provided by users associated with an organization;

inputting a first comment of the plurality of comments and a second comment of the plurality of comments to a model trained using machine learning, the model configured to assign a classification to an input comment received from a user;

determining, using the model, that the first comment is a conditionally prescriptive comment representing a prescriptive comment provided in response to a question presented to the user, and responsively assigning the first comment to a first category, wherein the prescriptive comment is a comment that specifies an action to be performed;

determining, using the model, that the second comment is an unconditionally prescriptive comment representing a prescriptive comment that is not associated with a question presented to the user, and responsively assigning the second comment to a second category;

receiving output from the model, the output comprising indicia that the first comment is assigned to the first category and the second comment is assigned to the second category;

generating a report based on the output; and

sending for presentation, results of the report.

2. The method of claim 1, wherein features used for the model comprise n-grams associated with a given comment of the plurality of comments with an indication of whether an n-gram is from a question sentence or from an answer sentence.

3. The method of claim 1, wherein features used for the model comprise n-grams associated with a given comment of the plurality of comments with information indicating a part of speech associated with an n-gram.

4. The method of claim 1, wherein features used for the model comprise an indication of whether terms of a given comment match an action expression.

5. The method of claim 1, wherein features used for the model comprise an indication of whether a given comment of the plurality of comments includes a representation of an action word.

6. The method of claim 5, wherein features used for the model comprise a location of the action word in a sentence.

7. The method of claim 1, wherein features used for the model comprise a form of a verb of a sentence of a given comment.

8. The method of claim 1, wherein the plurality of comments comprises comments received from users in response to a survey presented to the users.

9. A non-transitory computer-readable storage medium storing instructions that cause a processor to:

receive a plurality of comments provided by users associated with an organization;

input a first comment of the plurality of comments and a second comment of the plurality of comments to a model trained using machine learning, the model configured to assign a classification to an input comment received from a user;

determine, using the model, that the first comment is a conditionally prescriptive comment representing a prescriptive comment provided in response to a question presented to the user, and responsively assigning the first comment to a first category, wherein the prescriptive comment is a comment that specifies an action to be performed;

determine, using the model, that the second comment is an unconditionally prescriptive comment representing a prescriptive comment that is not associated with a question presented to the user, and responsively assigning the second comment to a second category;

receive output from the model, the output comprising indicia that the first comment is assigned to the first category and the second comment is assigned to the second category;

generate a report based on the output; and

send for presentation, results of the report.

10. The non-transitory computer-readable storage medium of claim 9, wherein features used for the model comprise n-grams associated with a given comment of the plurality of comments with an indication of whether an n-gram is from a question sentence or from an answer sentence.

11. The non-transitory computer-readable storage medium of claim 9, wherein features used for the model comprise n-grams associated with a given comment of the plurality of comments with information indicating a part of speech associated with an n-gram.

12. The non-transitory computer-readable storage medium of claim 9, wherein features used for the model comprise an indication of whether terms of a given comment of the plurality of comments match an action expression.

13. The non-transitory computer-readable storage medium of claim 9, wherein features used for the model comprise an indication of whether a given comment of the plurality of comments includes a representation of an action word.

14. The non-transitory computer-readable storage medium of claim 9, wherein features used for the model comprise a form of a verb of a sentence of a given comment of the plurality of comments.

15. A computer system comprising:

a computer processor; and

a non-transitory computer-readable storage medium storing instructions for execution by the computer processor, the instructions when executed by the computer processor cause the computer processor to;

receive a plurality of comments provided by users associated with an organization;

input a first comment of the plurality of comments and a second comment of the plurality of comments to a model trained using machine learning, the model configured to assign a classification to an input comment received from a user;

determine, using the model, that the first comment is a conditionally prescriptive comment representing a prescriptive comment provided in response to a question presented to the user, and responsively assigning the first comment to a first category, wherein the prescriptive comment is a comment that specifies an action to be performed;

determine, using the model, that the second comment is an unconditionally prescriptive comment representing a prescriptive comment that is not associated with a question presented to the user, and responsively assigning the second comment to a second category;

receive output from the model, the output comprising indicia that the first comment is assigned to the first category and the second comment is assigned to the second category;

generate a report based on the output; and

send for presentation, results of the report.

16. The system of claim 15, wherein features used for the model comprise n-grams associated with a given comment of the plurality of comments with an indication of whether an n-gram is from a question sentence or from an answer sentence.

17. The system of claim 15, wherein features used for the model comprise n-grams associated with a given comment of the plurality of comments with information indicating a part of speech associated with an n-gram.

18. The system of claim 15, wherein features used for the model comprise an indication of whether terms of a given comment of the plurality of comments match an action expression.

19. The system of claim 15, wherein features used for the model comprise an indication of whether a given comment of the plurality of comments includes a representation of an action word.

20. The system of claim 15, wherein features used for the model comprise a form of a verb of a sentence of a given comment of the plurality of comments.