Method and system for evaluating evaluators

• US 20080147742A1
• Filed: 12/07/2007
• Published: 06/19/2008
• Est. Priority Date: 12/13/2006
• Status: Abandoned Application

• Current Assignee: Chris Allen
• Original Assignee: Chris Allen
• Inventors: Allen, Chris
• Application Number:
  US11/999,826
• Publication Number:
  US 20080147742A1
• Time in Patent Office: Days
• US Class Current: 707/104.1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A system or method presents a plurality of content items to each of a plurality of users and receives an evaluation score for each item from each user. An average score for each item is calculated and then, for each user, a composite variance score is calculated. The composite variance score is the sum of the absolute value of the difference between a user'"'"'s score for an item and the population'"'"'s average score for that item. The evaluators'"'"' performance and the items'"'"' rankings can be used for additional information and purposes.

Claims

1. A rating system comprising:

a memory system storing a respective identification of a plurality of users;

a presentation component configured to present a plurality of items to each of the plurality of users;

a scoring component configured to receive a respective score from each of the plurality of users for each item;

a calculator configured to;

calculate an average score for each item based on the respective scores for that item from each user;

calculate, for each user and for each item, a respective difference between their score for each item and that item'"'"'s average score; and

calculate a respective composite variance score for each user based on the respective differences for all items for that user; and

an output component configured to identify which of the plurality of users most closely matches the averages scores for each item based on the composite variance score.

2. The system of claim 1, wherein the respective difference is considered as an absolute value.

3. The system of claim 1, wherein the composite variance score comprises a sum of the absolute values of the respective differences.

4. The system of claim 1, further comprising:

an item ranking component configured to rank the items according to the average scores.

5. The system of claim 1, further comprising:

a user ranking component configured to rank the users according to the composite variance scores.

6. The system of claim 1, wherein the items comprise one of:

news articles, political messages, songs, videos, literary works, graphical arts, advertisements, and live performances.

7. The system of claim 6, wherein the songs comprise a particular genre of music.

8. The system of claim 1, wherein a respective score is a multi-digit number.

9. The system of claim 8, wherein each digit is associated with a different characteristic appropriate to evaluating the items.

10. The system of claim 1, wherein the presentation component, the memory system, the scoring component, the calculator, the output component are part of a social networking site.

11. A method for rating items, comprising:

presenting each of a plurality of items to each of a plurality of users, wherein the users are uniquely identifiable;

receiving from each of the users a respective score for each of the items;

calculating an average score for each item based on the respective scores received from the users;

calculating, for each user and for each item, a respective difference between their score for each item and that item'"'"'s average score;

for each user calculating a composite variance score based on the respective differences for that user; and

identifying which of the plurality of users most closely matches the averages scores for each item based on the composite variance score.

12. The method of claim 11, wherein the respective difference is considered as an absolute value.

13. The method of claim 11, wherein the composite variance score comprises a sum of the absolute values of the respective differences.

14. The method of claim 11, further comprising:

ranking the items according to the average scores.

15. The method of claim 11, further comprising:

ranking the users according to the composite variance scores.

16. The method of claim 11, wherein the items comprise one of:

news articles, political messages, songs, videos, literary works, graphical arts, advertisements, and live performances.

17. The method of claim 13, further comprising:

awarding a prize to a user having a lowest composite variance score.

18. A method for holding a contest comprising the steps of:

a) registering a plurality of users to evaluate a plurality of items;

b) presenting each of the plurality of items to each of the users;

c) receiving a respective score for each item from each user;

d) determining a subset of users whose set of respective scores most closely matches a set of average scores for the items, wherein the average score for a particular item is based on the respective scores from the plurality of users for that item;

e) awarding a prize to the set of users.

19. The method of claim 18, wherein the subset of users comprise a single user.

20. The method of claim 18, further comprising:

ranking the plurality of items according to the average scores.

21. The method of claim 18, further comprising:

running a subsequent stage of the contest by repeating steps b) through d) wherein the plurality of items in the subsequent stage is limited to a subset of the plurality items in an earlier stage, the subset comprising those of the plurality of items having higher average scores.

22. The method of claim 18, further comprising:

receiving a fee from each of the users along with each set of respective scores.

23. The method of claim 18, wherein the plurality of users include a first division from which scores are received without an accompanying fee and a second division from which scores are received with an accompanying fee.

24. The method of claim 23, wherein a portion of the accompanying fees is awarded to a member of the second division whose set of respective scores most closely matches the set of average scores for the items.

25. A method for holding a contest comprising the steps of:

a) registering a plurality of users to evaluate a plurality of presented items;

b) receiving a respective score for each presented item from each user;

c) determining a subset of users whose set of respective scores most closely matches a set of average scores for the presented items, wherein the average score for a particular item is based on the respective scores from the plurality of users for that presented item;

d) awarding a prize to the set of users.

26. The method of claim 25, wherein the presented items are presented during a television show.

27. The method of claim 26, further comprising:

identifying a most popular one of the presented items based on the average scores.