PROVIDING CROWDSOURCED ANSWERS TO INFORMATION NEEDS PRESENTED BY SEARCH ENGINE AND SOCIAL NETWORKING APPLICATION USERS

• US 20140279996A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Liebling, Daniel, Bernstein, Michael S., Teevan, Jaime, Dumais, Susan T., Horvitz, Eric, Morris, Meredith, Jeong, Jin-Woo
• Granted Patent: US 9,424,354 B2
• Time in Patent Office: Days
• US Class Current: 707/706
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9536 Search customisation based on social or collaborative filtering
• CPC Class Codes: G06F 16/9538 Presentation of query results

Abstract

Technologies pertaining to generating crowd-sourced answers are described herein. A text string is received, and the text string is parsed to determine if the text string represents an information need that is desirably answered by a collective of crowd workers. When it is determined that the information need is desirably answered by the collective of crowd workers, a query or question that represents the information need is provided to a first plurality of crowd workers, who generate proposed answers for the information need. The proposed answers are provided to a second plurality of crowd workers, who vote on the proposed answers. An answer to the information need is output based upon responses of the crowd workers.

Claims

1. A method, comprising:

receiving, at a computing device, a text string that comprises a sequence of words;

parsing the text string;

based at least in part upon the parsing of the text string, identifying that the text string represents an information need that is desirably answered by a collective of crowd workers;

responsive to the identifying that the text string represents an information need that is desirably answered by the collective of crowd workers, transmitting content to a first plurality of computing devices operated by a respective first plurality of crowd workers from amongst the collective of crowd workers, the content comprising first instructions that instruct each crowd worker in the first plurality of crowd workers to perform a same task;

receiving respective responses to the task from each crowd worker in the first plurality of crowd workers; and

outputting an answer to the information need based upon the responses to the task from the first plurality of crowd workers.

2. The method of claim of claim 1, wherein the text string comprises a query submitted to a search engine, the method further comprising:

responsive to identifying that the query represents the information need that is desirably answered by the collective of crowd workers, identifying at least one resource related to the information need; and

transmitting the at least one resource to each crowd worker in the first plurality of crowd workers, wherein the first instructions instruct each crowd worker in the first plurality of crowd workers to generate a proposed answer based upon the query and the at least one resource.

3. The method of claim 2, wherein the first instructions instruct each crowd worker in the first plurality of crowd workers to generate the proposed answer by extracting content from the at least one resource.

4. The method of claim 3, wherein the at least one resource is a resource retrieved by executing a search over a search engine index based upon the query, the method further comprising:

receiving portions of the resource extracted therefrom by respective crowd workers in the first plurality of crowd workers;

transmitting the portions, the query, and second instructions to a second plurality of computing devices operated by a second plurality workers, the second instructions requesting each crowd worker in the second plurality of crowd workers to set forth an indication of a preference as to which option of several options best answers the information need, the several options comprising the portions of the document extracted therefrom by respective crowd workers in the first plurality of crowd workers;

receiving indications of preferences from the second plurality of computing devices; and

outputting the answer based at least in part upon the indications of the preferences.

5. The method of claim 1, wherein the text string is received from a public feed of a social networking application, and wherein the text string comprises an explicit or implied question.

6. The method of claim 5, wherein the task is for each crowd worker from the first plurality of crowd workers to submit a respective proposed answer to the question.

7. The method of claim 6, further comprising:

responsive to receiving proposed answers to the question submitted by the first plurality of crowd workers, transmitting the proposed answers and second instructions to a second plurality of computing devices operated by a respective second plurality of crowd workers, the second instructions instructing each crowd worker in the second plurality of crowd workers to submit a respective vote as to which of several options best answers the information need, the several options comprising the proposed answers;

receiving votes submitted by the second plurality of crowd workers, wherein the answer is output based at least in part upon the votes.

8. The method of claim 1, wherein the identifying that the text string represents an information need that is desirably answered by the collective of crowd workers comprises:

determining that the text string comprises at least one predefined feature;

responsive to determining that the text string comprises the at least one predefined feature, transmitting at least a portion of the text string and second instructions to a computing device of a crowd worker from the collective of crowd workers, the second instructions instructing the crowd worker to indicate whether or not the information need is answerable by the collective of crowd workers; and

receiving a response from the crowd worker that the information need is answerable by the collective of crowd workers.

9. The method of claim 1, wherein identifying that the text string represents an information need that is desirably answered by the collective of crowd workers comprises identifying that the text string comprises a question or query that is time-critical in nature.

10. The method of claim 1, wherein the outputting of the answer to the information need comprises including the answer to the information need in a search engine results page, the search engine results page comprising a ranked list of URLs, and wherein the answer is presented in the search engine results page to visually differentiate the answer from URLs in the ranked list of URLs.

11. The method of claim 10, wherein the answer is displayed above the URLs in the ranked list of URLs.

12. A system, comprising:

a processor; and

a memory that comprises a plurality of components that are executed by the processor, the plurality of components comprising;

a classifier component that receives a text string and classifies the text string as representing an information need that is desirably answered by a collective of crowd workers;

an instruction transmitter component that transmits the text string and first instructions to a first plurality of computing devices operated by a first plurality of crowd workers in the collective of crowd workers, the first instructions instructing each crowd worker in the first plurality of crowd workers to perform a same task with respect to the information need;

a response receiver component that receives responses to the task submitted by the first plurality of crowd workers; and

an output component that outputs an answer to the information need based at least in part upon the responses to the task received by the response receiver component.

13. The system of claim 12 being comprised by a search engine, wherein the text string comprises a query submitted to the search engine.

14. The system of claim 12, wherein the text string comprises a question included in a message posted to a feed of a social networking application, the system further comprising an analyzer component that analyzes the feed and identifies that the message includes the question, wherein the classifier receives the question responsive to the analyzer component identifying that the message includes the question.

15. The system of claim 12, wherein the classifier component classifies the text string as representing an information need that is desirably answered by the collective of crowd workers based at least in part upon a non-alphanumerical character being included in the text string.

16. The system of claim 12, wherein text string comprises a question, and wherein the task is to submit a proposed answer to the question.

17. The system of claim 16, wherein responsive to the response receiver component receiving proposed answers to the question, the instruction transmitter component transmits the question, the proposed answers, and second instructions to a second plurality of computing devices operated by a second plurality of crowd workers, the second instructions instructing each crowd worker in the second plurality of crowd workers to submit a vote as to which proposed answer best answers the information need, and wherein the output component outputs the answer to the information need based at least in part upon votes submitted by the second plurality of crowd workers.

18. The system of claim 12, further comprising an accounting component that tracks payment to be provided to each crowd worker in the first plurality of crowd workers for their respective responses.

19. The system of claim 12, wherein the output component outputs the answer to a public feed of a social networking application, the answer displayed in the public feed in correspondence with a message that includes the text string.

20. A computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:

monitoring a public feed of a social networking application for questions included in messages posted to the public feed;

determining that a message posted to the public feed by a user of the social networking application comprises a question;

determining that the message posted to the public feed by the user of the social networking application comprises a hash tag;

identifying that the question is desirably answered by a collective of crowd workers based upon the determining that the message comprises the question and the determining that the message comprises the hash tag;

transmitting the question and first instructions to a first plurality of computing devices operated by a respective first plurality of crowd workers, the first instructions instructing each crowd worker in the first plurality of crowd workers to generate a respective proposed answer to the question;

receiving proposed answers to the question submitted by the first plurality of crowd workers;

responsive to receiving the proposed answers to the question, transmitting the proposed answers to the question and second instructions to a second plurality of computing devices operated by a respective second plurality of crowd workers, the second instructions instructing each crowd worker in the second plurality of crowd workers to submit a vote as to which of the proposed answers best answers the question;

receiving votes submitted by the second plurality of crowd workers;

selecting a final answer to the question based at least in part upon the votes; and

outputting the final answer to the question to the public feed.