Closed loop voting feedback

• US 20060224442A1
• Filed: 03/31/2005
• Published: 10/05/2006
• Est. Priority Date: 03/31/2005
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Gadre, Aniruddha, Round, Matthew J.
• Granted Patent: US 8,566,144 B2
• Time in Patent Office: Days
• US Class Current: 705/12
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Embodiments of the present invention provide a system and method for providing reviewer feedback identifying a correlation between a reviewer'"'"'s past voting record and successful results. Based on a reviewer'"'"'s past record, the reviewer may be assigned a power rating identifying the statistical accuracy of the reviewer'"'"'s voting history. That power rating may be used to weight future votes, thereby increasing the likelihood that accurate decisions will be made.

Claims

1. A method for determining a power rating for a user, the method comprising:

receiving a reviewer'"'"'s vote with respect to a decision to be made;

following the decision, determining if the reviewer'"'"'s vote is reflective of a desired outcome of the decision; and

determining a power rating for the reviewer based at least in part on the determination of whether the reviewer'"'"'s vote is reflective of the desired outcome.

2. The method of claim 1, further comprising:

determining a timeliness of the reviewer'"'"'s vote.

3. The method of claim 2, wherein the determined timeliness provides feedback regarding the reviewer'"'"'s ability to vote in a timely manner.

4. The method of claim 2, wherein the power rating is determined for the reviewer based at least in part on the timeliness determination.

5. The method of claim 2, further comprising:

obtaining one or more timeliness determinations for prior votes of the reviewer; and

determining a voting timeliness score for the reviewer based on the determined timeliness of the reviewer'"'"'s vote and prior timeliness determinations.

6. The method of claim 1, wherein the determination of whether the reviewer'"'"'s vote is reflective of a desired outcome includes:

determining a performance review of an outcome of the decision;

comparing the performance review with the reviewer'"'"'s vote; and

determining a power rating for the reviewer based on the comparison of the performance review and the reviewer'"'"'s vote.

7. The method of claim 2, further comprising:

weighting future votes of the reviewer based on the reviewer'"'"'s power rating.

8. The method of claim 1, wherein the power rating of the reviewer provides feedback regarding the reviewer'"'"'s ability to vote in a manner that is reflective of a desired outcome.

9. The method of claim 1, further comprising:

determining a consistency of the reviewer'"'"'s vote in relation to votes of other reviewer'"'"'s with respect to the decision to be made.

10. The method of claim 9, wherein the consistency of the reviewer'"'"'s vote provides feedback regarding the reviewer'"'"'s consistency with respect to votes of other reviewers.

11. A computer-readable medium having computer-executable instructions for performing the method recited in claim 1.

12. A method for increasing the likelihood that decisions resulting in a desired outcome will be made, the method comprising:

receiving from a plurality of reviewers a vote regarding a decision to be made;

determining a power rating reflective of a likelihood of predicting outcomes on decisions for each of the reviewers from which a vote was received;

weighting the votes based on the determined power ratings; and

determining a group decision based on the weighted votes.

13. The method of claim 12, wherein the power rating of a reviewer is based on past voting history of that reviewer.

14. The method of claim 12, wherein the reviewers are ranked based on their power ratings.

15. The method of claim 12, wherein weighting the votes further comprises:

identifying a reviewer with a high power rating and increasing the weight of the identified reviewers vote.

16. The method of claim 15, wherein a high power rating identifies a reviewer that is likely to make a decision that will result in a desired outcome.

17. The method of claim 12, wherein weighting the votes further comprises:

identifying a reviewer with a low power rating and decreasing the weight of the identified reviewers vote.

18. The method of claim 17, wherein a low power rating identifies a reviewer that is likely to make a decision that will not result in a desired outcome.

19. A method for providing feedback based on voting activity of a reviewer, the method comprising:

receiving a reviewer'"'"'s vote with respect to a decision to be made;

determining a consistency of the reviewer'"'"'s vote in relation to votes of other reviewers with respect to the decision to be made, wherein the consistency of the reviewer'"'"'s vote provides feedback regarding the review'"'"'s consistency with respect to votes of other reviewers.

20. The method of claim 19, wherein the consistency of the reviewer'"'"'s vote provides feedback regarding the reviewer'"'"'s consistency with respect to votes of other reviewers.

21. The method of claim 19, further comprising obtaining one or more consistency determinations for prior votes of the reviewer and determining a voting consistency score for the reviewer based on the determined consistency of the reviewer'"'"'s vote and prior consistency determinations.

22. The method of claim 19, further comprising determining a timeliness of the reviewer'"'"'s vote and providing a power rating for the reviewer based at least in part on the determined consistency and determined timeliness of the reviewer'"'"'s vote.

23. The method of claim 22, further comprising obtaining one or more timeliness determinations for prior votes of the reviewer and determining a voting timeliness score for the reviewer based on the determined timeliness of the reviewer'"'"'s vote and prior timeliness determinations.

24. The method of claim 19, further comprising determining a group decision based on the votes of the reviewer and the other reviewers.

25. The method of claim 24, wherein the group decision is a decision to pursue.

26. The method of claim 24, wherein the group decision is a decision to hold.

27. The method of claim 24, wherein the group decision is a decision to reject.

28. The method of claim 24, further comprising weighting the reviewer'"'"'s vote based on the reviewer'"'"'s power rating prior to determining the group decision.

29. The method of claim 19, further comprising receiving a performance review of an outcome of a decision that has been made and determining a power rating of the reviewer based on a correlation between the reviewer'"'"'s vote and the performance review.

30. A computer-accessible medium having instructions stored thereon that direct a computing system to:

receive a first plurality of votes corresponding to a decision to be made with respect to an object, wherein at least one of the first plurality of votes corresponds to a first vote received from a reviewer;

based on the first plurality of received votes, determine a first action to take with respect to the object;

receive a second plurality of votes corresponding to the decision to be made with respect to the object, wherein at least one of the second plurality of votes corresponds to a second vote received from the reviewer;

based on the second plurality of received votes, determine a second action to take with respect to the object;

receive a performance review of the object; and

determine a power rating of the reviewer based on a correlation between the performance review and the first vote and second vote received from the reviewer.

31. The computer-accessible medium of claim 30, wherein the object is a job candidate.

32. The computer-accessible medium of claim 30, further comprising instructions that direct the computing system to:

determine the power rating based on a timeliness of votes received from the reviewer.

33. The computer-accessible medium of claim 30, further comprising instructions that direct the computing system to:

determine the power rating based on a voting result computed for the reviewer.

34. The computer-accessible medium of claim 30, further comprising instructions that direct the computing system to:

determine a first voting consistency score based on a correlation between the first vote received from the reviewer and a consensus of the first plurality of votes.

35. The computer-accessible medium of claim 34, further comprising instructions that direct the computing system to:

determine a second voting consistency score based on a correlation between the second vote received from the reviewer and a consensus of the second plurality of votes.

36. The computer-accessible medium of claim 35, further comprising instructions that direct the computing system to:

determine a total voting consistency score based on the determined first voting consistency score and the determined second voting consistency score.

37. A method for determining a power rating of a reviewer, the method comprising:

obtaining a performance review of an object;

determining a voting result based on a comparison of the performance review of the object and an overall voting score; and

determining the power rating of the reviewer based on the determined voting result.

38. The method of claim 37, wherein the power rating is represented graphically as an icon.

39. The method of claim 37, wherein the voting result has a greater effect on determining the power rating than a total voting consistency score.

40. The method of claim 37, further comprising:

obtaining a total voting timeliness score and a total voting consistency score; and

wherein determining the power rating is based on the voting result, the total voting accuracy score, and the total voting timeliness score.

41. The method of claim 37, wherein the voting result has a greater effect on determining the power rating than the total voting consistency score, and the total voting consistency score has a greater effect on determining the power rating than the total voting timeliness score.

42. A reviewer power rating system for rating reviewers of job candidates based on past performance, comprising:

voting data for a reviewer representative of votes provided by a reviewer for job candidates;

a power rating module for determining a power rating for the reviewer based on a performance of hired job candidates; and

a comparison module for comparing the power rating of the reviewer with a power rating of a second reviewer.

43. The system of claim 42, further comprising a timeliness module for determining a vote type timeliness score for votes cast by the reviewer for a vote type.

44. The system of claim 43, wherein the timeliness module also determines a total voting timeliness score for all votes cast by the reviewer.

45. The system of claim 42, further comprising a reviewer consistency module, for determining a vote type consistency score for votes cast by the reviewer for a vote type.

46. The system of claim 45, wherein the reviewer consistency module also determines a total voting consistency score for all votes cast by the reviewer.

47. The system of claim 42, wherein the power rating module determines a power rating based on a voting result.

48. The system of claim 47, wherein the voting result represents a correlation between the reviewer'"'"'s votes for job candidates and a performance review of the job candidate.

49. The system of claim 47, wherein the power rating module determines a power rating based on a voting result and a total voting consistency score.