Generating solutions to problems via interactions with human responders

• US 7,899,694 B1
• Filed: 06/30/2006
• Issued: 03/01/2011
• Est. Priority Date: 06/30/2006
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Pope, Elmore Eugene, Merritt, Eric B., Scofield, Christopher L., Marshall, Bradley E.
• Primary Examiner(s): DIXON, THOMAS A
• Application Number:
  US11/479,861
• Time in Patent Office: 1,705 Days
• Field of Search: 705/7
• US Class Current: 705/7.11
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management

Abstract

Techniques are described for using multiple successive interactions with multiple responders in order to perform a variety of types of tasks, such as to generate solutions to abstract or conceptual problems. In some situations, at least some of the multiple responders are humans available to perform tasks and/or provide information, such as to provide responses to questions in exchange for payment. The multiple successive interactions with the responders may be used to iteratively refine the problem and its abstract intermediate solutions until the refinements are sufficiently detailed and tangible as to allow detailed final solutions to be generated. In some situations, the interactions with the responders may be performed as part of an automated service for generating solutions to a variety of types of problems, such as in exchange for payment from the users or others who specify the problems.

Claims

1. A computer-implemented method for managing generation of solutions for problems via repeated interactions with multiple unrelated humans who generate successively more defined refinements of the problem, the method comprising:

identifying a group of multiple unrelated humans who are available to perform tasks that include supplying requested information;

receiving information from a user about an abstract problem for which a solution is desired, the abstract problem being such that the solution cannot be discerned without refinement to further define the problem, the receiving being performed by one or more configured computing systems;

automatically managing generation of the solution to the problem by,

identifying a first group of one or more first refinements to the problem that provide additional definition of the problem to be solved, the identifying of the first group being performed by the one or more configured computing systems and including interacting with some of the multiple humans of the group to obtain multiple proposed first refinements and interacting with other of the multiple humans of the group to obtain evaluations of the multiple proposed first refinements, the one or more first refinements of the identified first group being a subset of the multiple proposed first refinements that is selected based at least in part on the evaluations of the multiple proposed first refinements;

determining by the one or more configured computing systems that the first refinements of the identified first group are not sufficiently defined to allow the solution to the problem to be discerned;

identifying a second group of one or more second refinements to the problem that each provide additional definition relative to at least one of the first refinements of the identified first group, the identifying of the second group being performed by the one or more configured computing systems and including interacting with some of the multiple humans of the group to obtain multiple proposed second refinements and interacting with other of the multiple humans of the group to obtain evaluations of the multiple proposed second refinements, the one or more second refinements of the identified second group being a subset of the multiple proposed second refinements that is selected based at least in part on the evaluations of the multiple proposed second refinements;

determining by the one or more configured computing systems that the second refinements of the identified second group are sufficiently defined to allow the solution to the problem to be discerned; and

identifying one or more solutions to the problem that each are based on the definition provided by at least one of the second refinements of the identified second group, the identifying of the one or more solutions being performed by the one or more configured computing systems and including interacting with some of the multiple humans of the group to obtain multiple proposed solutions and selecting at least one of the multiple proposed solutions as the identified one or more solutions; and

providing to the user the identified one or more solutions to the problem for use in implementing the identified solutions.

2. The method of claim 1 wherein one of the first refinements of the first group includes a sub-problem that may be solved to obtain a partial solution to the problem, and wherein at least one of the second refinements of the second group each includes a refinement of the sub-problem that provides the partial solution to the problem.

3. The method of claim 1 wherein one of the first refinements of the first group includes an abstract solution to the problem, and wherein at least one of the second refinements of the second group each includes a refinement of the abstract solution so as to provide a less abstract solution.

4. The method of claim 1 wherein the selecting of the at least one of the multiple proposed solutions as the identified one or more solutions includes interacting with some of the multiple humans of the group to obtain evaluations of the multiple proposed solutions and using the evaluations during the selecting.

5. The method of claim 1 wherein the determining that the second refinements of the identified second group are sufficiently defined to allow the solution to the problem to be discerned is based on feedback obtained from at least some of the multiple humans of the group.

6. The method of claim 1 wherein the managing of the generation of the solution to the problem includes obtaining feedback from the user for use with at least one of the determining that the first refinements of the identified first group are not sufficiently defined to allow the solution to the problem to be discerned, and of the determining that the second refinements of the identified second group are sufficiently defined to allow the solution to the problem to be discerned.

7. The method of claim 1 wherein the managing of the generation of the solution to the problem includes, after the interacting with some of the multiple humans of the group to obtain multiple proposed first refinements, interacting with one or more of the humans of the group to obtain categorization information for the multiple proposed first refinements, and wherein the obtained categorization information is used for at least one of selecting the one or more first refinements of the identified first group from the multiple proposed first refinements, of the determining that the first refinements of the identified first group are not sufficiently defined to allow the solution to the problem to be discerned, and of the identifying of the second group of second refinements that each provide additional definition relative to at least one of the first refinements of the identified first group.

8. The method of claim 1 wherein the abstract problem is related to at least one of a product to be developed, research to be performed, and computer software to be created.

9. The method of claim 1 wherein the automatic managing of the generation of a solution to a problem is performed to generate solutions to distinct problems for each of multiple users in exchange for payment from the user, and wherein, for each distinct problem, the automatic managing of the generation of the solution to that problem includes providing payments to humans of the group for performing tasks that facilitate the generation of the solution for that problem.

10. A non-transitory computer-readable medium whose contents configure a computing device to facilitate problem solving, by performing a method comprising:

receiving an indication of an abstract problem to be solved;

interacting with a first group of one or more of multiple responders who are available to provide responses in order to obtain one or more initial refinements to the problem such that the initial refinements are less abstract than the problem, the interacting with the first group being performed by the configured computing device;

interacting with a second group of one or more of the multiple responders in order to obtain one or more further refinements that are each based on at least one of the obtained initial refinements, the interacting with the second group being performed by the configured computing device;

interacting with a third group of one or more of the multiple responders in order to obtain one or more solutions that are each based on at least one of the obtained further refinements, the interacting with the third group being performed by the configured computing device; and

providing at least one of the obtained solutions for use as at least part of the solution to the problem.

11. The non-transitory computer-readable medium of claim 10 wherein the multiple responders are part of a group of multiple humans who are unrelated to each other, and wherein the first, second and third groups of responders are each distinct.

12. The non-transitory computer-readable medium of claim 10 wherein the abstract problem lacks sufficient definition to allow a detailed final solution to the problem from being discernable without refinement of the problem, and wherein the provided at least one solutions are part of the detailed final solution to the abstract problem.

13. The non-transitory computer-readable medium of claim 10 wherein the computer-readable medium is a memory of a computing device.

14. The non-transitory computer-readable medium of claim 10 wherein the contents are instructions that when executed cause the computing device to perform the method.

37. The non-transitory computer-readable medium of claim 10 wherein the multiple responders are multiple unaffiliated humans, wherein the second group includes a plurality of the multiple responders, and wherein the interacting with the second group includes interacting repeatedly with multiple sub-groups of the second group to iteratively obtain the further refinements.

38. The non-transitory computer-readable medium of claim 10 wherein the multiple responders are multiple humans, wherein the obtaining of the one or more further refinements includes interacting with at least one of the multiple humans to obtain evaluations of one or more proposed refinements from other of the multiple humans and using those obtained evaluations to select the obtained one or more further refinements from the proposed refinements, and wherein the obtaining of the one or more solutions includes interacting with at least one of the multiple humans to obtain evaluations of the one or more proposed solutions from other of the multiple humans and using those obtained evaluations to select the obtained one or more solutions.

39. The non-transitory computer-readable medium of claim 10 wherein the computing device operates on behalf of an iterative problem solution generation service that provides solutions to abstract problems for multiple users in exchange for fees paid by the multiple users.

40. The non-transitory computer-readable medium of claim 10 wherein the problem is at least one of a product to be developed, of research to be performed, of an object to be designed, and of computer software to be created.

41. The non-transitory computer-readable medium of claim 10 wherein the interacting with the one or more responders of the second group to obtain the further refinements includes obtaining categorization information for at least one of the problem and of the further refinements by interacting with at least some of the multiple responders, and wherein the interacting with the third group is based at least in part on the categorization information.

42. The non-transitory computer-readable medium of claim 10 wherein the problem includes providing structure for an uncategorized collection of information that includes multiple information items, and wherein the at least one obtained solution includes one or more categories discovered for the multiple information items.

43. The non-transitory computer-readable medium of claim 42 wherein the multiple responders are multiple humans, wherein the interacting with the second group to obtain the one or more further refinements includes obtaining information from each of a plurality of the multiple humans that indicates multiple groups created by the human, each created group including a distinct subset of the multiple information items, and includes identifying one or more item combinations that each include two or more items grouped together in the created groups by multiple of the plurality of humans, the identified item combinations each representing a possible category that is one of the obtained further refinements, and wherein the obtained one or more solutions are each a category for the multiple information items that includes at least one of the possible categories represented by the identified item combinations.

44. The non-transitory computer-readable medium of claim 43 wherein the two or more items in each of the identified one or more item combinations are automatically determined based on the multiple groups created by the plurality of humans to have a likelihood of being part of a common category that exceeds a pre-determined threshold.

45. The non-transitory computer-readable medium of claim 43 wherein the obtaining of the one or more solutions includes automatically identifying an item that is a canonical example for the category of each of the obtained solutions based on the multiple groups created by the plurality of humans.

46. The non-transitory computer-readable medium of claim 42 wherein the multiple information items include multiple items having a common format, the common format being at least one of images, text, and audio.

15. A computing system configured to facilitate problem solving, comprising:

one or more processors;

a first component that is configured to, when executed by at least one of the one or more processors, identify multiple humans who are available to provide information; and

an iterative problem solution generation system that is configured to, when executed by at least one of the one or more processors, and in response to an indication of a problem for which a tangible solution is not initially discernable,

interact with one or more of the multiple humans to obtain one or more proposed refinements to the problem, and select at least one of the proposed problem refinements;

interact with one or more of the multiple humans to obtain one or more proposed tangible solutions based on the selected problem refinements, and select at least one of the proposed tangible solutions; and

provide the selected tangible solutions for use as at least part of the tangible solution to the problem.

16. The computing system of claim 15 wherein the first component is at least part of a distributed answer providing service distinct from the iterative problem solution generation system.

17. The computing system of claim 15 wherein the iterative problem solution generation system is further configured to, after the selecting of the at least one proposed problem refinements and before the obtaining of the one or more proposed tangible solutions, perform one or more additional interactions with the multiple humans in order to obtain one or more additional refinements to the problem, and wherein the obtaining of the one or more proposed tangible solutions is further based in part on the obtained additional problem refinements.

18. The computing system of claim 15 wherein the first component and the iterative problem solution generation system each includes software instructions for execution by the one or more processors of the computing system.

19. The computing system of claim 15 wherein the first component consists of a means for identifying multiple humans who are available to provide information, and wherein the iterative problem solution generation system consists of a means for, in response to an indication of a problem for which a tangible solution is not initially discernable, interacting with one or more of the multiple humans to obtain one or more proposed refinements to the problem, selecting at least one of the proposed problem refinements, interacting with one or more of the multiple humans to obtain one or more proposed tangible solutions based on the selected problem refinements, selecting at least one of the proposed tangible solutions, and providing the selected tangible solutions for use as at least part of the tangible solution to the problem.

20. The computing system of claim 15 wherein the iterative problem solution generation system is further configured to, before the interacting with the one or more humans to obtain the one or more proposed tangible solutions based on the selected problem refinements:

identify the selected problem refinements as current sub-problems to be further refined; and

perform one or more additional interactions with at least some of the multiple humans in order to obtain one or more further problem refinements based on the current sub-problems, to enable possible tangible solutions to the problem to be discerned from the further problem refinements.

21. The computing system of claim 15 wherein the interacting with the one or more humans to obtain the one or more proposed refinements to the problem includes providing an indication of the indicated problem as part of one or more questions and receiving the one or more proposed problem refinements in response, and wherein the interacting with the one or more humans to obtain the one or more proposed tangible solutions includes providing an indication of the selected problem refinements as part of one or more questions and receiving the proposed tangible solutions in response.

22. The computing system of claim 15 wherein the selecting of the at least one proposed problem refinements includes interacting with at least one of the multiple humans to obtain evaluations of the one or more proposed problem refinements and using the obtained evaluations as part of the selecting.

23. The computing system of claim 22 wherein the one or more humans from whom the proposed problem refinements are obtained are part of a first group having a plurality of the multiple humans, and wherein the at least one humans from whom the evaluations of the proposed problem refinements are obtained are a second group having a plurality of other of the multiple humans, the plurality of other humans of the second group being distinct from the plurality of humans of the first group.

24. The computing system of claim 23 wherein the one or more humans from whom the proposed tangible solutions are obtained are part of a third group having a plurality of the multiple humans, the humans of the third group not being part of the first group or of the second group, and wherein the selecting of the at least one proposed tangible solutions includes interacting with a fourth group having a plurality of the multiple humans to obtain evaluations of the proposed solutions and using the obtained evaluations as part of the selecting, the humans of the fourth group being distinct from the humans of the first, second and third groups.

25. The computing system of claim 15 wherein the iterative problem solution generation system is further configured to determine whether a tangible solution to the indicated problem is discernable from the selected problem refinements, the determining being based on at least one of input received from a solution requester user who provided the indication of the problem and of feedback received from a plurality of the multiple humans who are distinct from the one or more humans from whom the one or more proposed refinements to the problem are obtained.

26. The computing system of claim 15 wherein the iterative problem solution generation system is further configured to automatically determine whether a tangible solution to the indicated problem is discernable from the selected problem refinements, the determining being based on predetermined criteria that include at least one of a total elapsed time since receiving the indication of the problem, of a total amount of money spent as part of the interacting with the multiple humans to obtain proposed refinements to the problem and to obtain proposed tangible solutions, and of a total number of refinements that have been performed.

27. The computing system of claim 15 wherein the iterative problem solution generation system is further configured to provide tangible solutions to problems for multiple users in exchange for fees paid by the multiple users.

28. The computing system of claim 15 wherein the indication of the problem is received from a first user, and wherein the multiple humans are unaffiliated with the first user and are unaffiliated with each other.

29. The computing system of claim 15 wherein the problem is at least one of a product to be developed, of research to be performed, of an object to be designed, and of computer software to be created.

30. The computing system of claim 15 wherein the selecting of the at least one proposed problem refinements includes obtaining categorization information for the proposed refinements by interacting with at least some of the multiple humans, and wherein the selecting is based at least in part on the categorization information.

31. The computing system of claim 15 wherein the interacting with the one or more humans to obtain the proposed refinements includes obtaining categorization information for at least one of the problem and of the proposed refinements by interacting with at least some of the multiple humans, and wherein the interacting is based at least in part on the categorization information.

32. The computing system of claim 15 wherein the problem includes providing structure for an uncategorized collection of information that includes multiple information items, and wherein the tangible solution includes discovering multiple categories for the multiple information items.

33. The computing system of claim 32 wherein the interacting with the one or more humans to obtain the one or more proposed refinements to the problem includes obtaining information from each of a plurality of the multiple humans that indicates multiple groups created by the human, each created group including a distinct subset of the multiple information items, wherein the selecting of the at least one proposed problem refinement includes identifying one or more item combinations that each include two or more items grouped together in the created groups by multiple of the plurality of humans, the identified item combinations each representing a possible category that is one of the selected at least one proposed problem refinements, and wherein the selected at least one proposed tangible solutions are each a category for the multiple information items that includes at least one of the possible categories represented by the identified item combinations.

34. The computing system of claim 33 wherein the two or more items in each of the identified one or more item combinations are automatically determined based on the multiple groups created by the plurality of humans to have a likelihood of being part of a common category that exceeds a pre-determined threshold.

35. The computing system of claim 33 wherein the selecting of the at least one proposed tangible solutions includes automatically identifying an item that is a canonical example for the category of each of the selected at least one proposed tangible solutions based on the multiple groups created by the plurality of humans.

36. The computing system of claim 32 wherein the multiple information items include multiple items having a common format, the common format being at least one of images, text, and audio.