System and method for manipulating success determinates in software development competitions

• US 8,909,541 B2
• Filed: 05/13/2009
• Issued: 12/09/2014
• Est. Priority Date: 01/11/2008
• Status: Active Grant

• Current Assignee: TopCoder, Inc. (Wipro Limited)
• Original Assignee: Appirio Inc. (Wipro Limited)
• Inventors: Fairfax, Ryan J., Fernandez-Ivern, Javier, Hughes, John M., Lydon, Michael, Messinger, David
• Primary Examiner(s): Pats, Justin M
• Assistant Examiner(s): Rotaru, Octavian
• Application Number:
  US12/465,369
• Publication Number:
  US 20100178978A1
• Time in Patent Office: 2,036 Days
• Field of Search: 705/7.37, 705/7.42, 705/7.25, 705/300, 463/23, 463/25
• US Class Current: 705/7.37
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/06315 Needs-based resource requirements planning or analysis
• CPC Class Codes: G06Q 10/06375 Prediction of business process outcome or impact based on a proposed change
• CPC Class Codes: G06Q 10/06398 Performance of employee with respect to a job function
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

In general, in one aspect, a method for developing an asset by competition includes specifying standards for an asset, receiving from potential contestants indicia of interest in participating in an asset development competition in which the contestants compete to develop an asset, and determining an estimated likelihood that the asset development competition will result in an asset that meets the minimum standards based on historical information about the performance of the contestants from whom the indicia of interest were received. Steps may be taken to increase the estimated likelihood of success, upon a determination that the likelihood is less than desired. Steps may be taken to prevent registration by additional participants, upon a determination that the estimated likelihood is at or above a predetermined threshold.

Claims

1. A computer implemented method for estimating a likelihood of developing and submitting an asset by contestants in a competition, the computer implemented method comprising:

specifying minimum standards for the asset;

receiving, from a number of the contestants, indicia of interest in participating in the competition, wherein the indicia of interest comprises at least a registration of each of the contestants to the competition;

receiving ratings for each of the contestants from previous submissions for the asset;

calculating, by a hardware processor, an average rating from the received ratings;

calculating, by the hardware processor, a competition factor as a square root of an addition between a summed squared volatility of the ratings for each of the contestants and a variance of the ratings of the contestants from the average rating;

estimating, by the hardware processor, a win probability from an error function used in integrating a normal distribution function;

calculating, by the hardware processor, an expected rank for each of the contestants using the win probability;

calculating, by the hardware processor, an expected performance for each of the contestants using the expected rank for each of the contestants and an inverse of the normal distribution function;

calculating, by the hardware processor, an actual performance for each of the contestants using an actual rank for each of the contestants and the inverse of the normal distribution function;

calculating, by the hardware processor, a performed as rating for each of the contestants by adding an old rating from the previous submissions to a mathematical product between the competition factor and a difference between the actual performance and the expected performance;

calculating, by the hardware processor, a weight of the competition for each of the contestants from a number of times each of the contestants have been rated before, and adjusting said weight of the competition for each of the contestants by a first percentage when the ratings for each of the contestants fall within a first interval and by a second percentage when the ratings for each of the contestants exceed the first interval;

calculating, by the hardware processor, a new updated rating for each of the contestants by multiplying the weight of the competition for each of the contestants and the performed as rating;

estimating, by the hardware processor, a likelihood of at least one of the contestants developing and submitting the asset that meets the minimum standards, based on the number of the contestants from whom the indicia of interest were received and a determination of the new updated rating for at least one of the contestants being higher than a predetermined percentile;

specifying, by the hardware processor, a completion threshold representing a minimum acceptable estimated likelihood that the competition will successfully complete; and

,

upon determining that the estimated likelihood does not meet the completion threshold, taking one or more steps to increase the estimated likelihood.

2. The computer implemented method of claim 1 wherein the one or more steps to increase the estimated likelihood comprises an increase in prize amounts.

3. The computer implemented method of claim 1 wherein the one or more steps to increase the estimated likelihood comprises a modification of an end date of the competition.

4. The computer implemented method of claim 1 further comprising, upon determining that the estimated likelihood meets the completion threshold, closing the registration.

5. The computer implemented method of claim 1 wherein the asset is at least one of:

a software program, a graphic design, an architecture design, a component design, a static prototype or a working prototype.

6. A system estimating a likelihood of developing and submitting an asset by contestants in a competition, the system comprising one or more hardware processors programmed to:

specify minimum standards for the asset;

receive, from a number of the contestants, indicia of interest in participating in the competition, wherein the indicia of interest comprises at least a registration of each of the contestants to the competition;

receive ratings for each of the contestants from previous submissions for the asset;

calculate an average rating from the received ratings;

calculate a competition factor as a square root of an addition between a summed squared volatility of the ratings for each of the contestants and a variance of the ratings of the contestants from the average rating;

estimate a win probability from an error function used in integrating a normal distribution function;

calculate an expected rank for each of the contestants using the win probability;

calculate an expected performance for each of the contestants using the expected rank for each of the contestants and an inverse of the normal distribution function;

calculate an actual performance for each of the contestants using an actual rank for each of the contestants and the inverse of the normal distribution function;

calculate a performed as rating for each of the contestants by adding an old rating from the previous submissions to a mathematical product between the competition factor and a difference between the actual performance and the expected performance;

calculate a weight of the competition for each of the contestants from a number of times each of the contestants have been rated before, and adjusting said weight of the competition for each of the contestants by a first percentage when the ratings for each of the contestants fall within a first interval and by a second percentage when the ratings for each of the contestants exceed the first interval;

calculate a new updated rating for each of the contestants by multiplying the weight of the competition for each of the contestants and the performed as rating;

estimate a likelihood of at least one of the contestants developing and submitting the asset that meets the minimum standards, based on the number of the contestants from whom the indicia of interest were received and a determination of the new updated rating for at least one of the contestants being higher than a predetermined percentile;

specify a completion threshold representing a minimum acceptable estimated likelihood that the competition will successfully complete; and

,

upon determining that the estimated likelihood does not meet the completion threshold, take one or more steps to increase the estimated likelihood.

7. The system of claim 6 wherein the one or more hardware processors are further programmed to increase the estimated likelihood by increasing prize amounts.

8. The system of claim 6 wherein the one or more hardware processors are further programmed to increase the estimated likelihood by modifying an end date of the competition.

9. The system of claim 6 wherein the one or more hardware processors are further programmed to close the registration upon determining that the estimated likelihood meets the completion threshold.

10. The system of claim 6 wherein the asset is at least one of:

a software program, a graphic design, an architecture design, a component design, a static prototype or a working prototype.

11. A non-transitory computer readable storage medium for estimating a likelihood of developing and submitting an asset by contestants in a competition, on which is recorded computer executable instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to:

specify minimum standards for the asset;

receive, from a number of the contestants, indicia of interest in participating in the competition, wherein the indicia of interest comprises at least a registration of each of the contestants to the competition;

receive ratings for each of the contestants from previous submissions for the asset;

calculate an average rating from the received ratings;

calculate a competition factor as a square root of an addition between a summed squared volatility of the ratings for each of the contestants and a variance of the ratings of the contestants from the average rating;

estimate a win probability from an error function used in integrating a normal distribution function;

calculate an expected rank for each of the contestants using the win probability;

calculate an expected performance for each of the contestants using the expected rank for each of the contestants and an inverse of the normal distribution function;

calculate an actual performance for each of the contestants using an actual rank for each of the contestants and the inverse of the normal distribution function;

calculate a performed as rating for each of the contestants by adding an old rating from the previous submissions to a mathematical product between the competition factor and a difference between the actual performance and the expected performance;

calculate a weight of the competition for each of the contestants from a number of times each of the contestants have been rated before, and adjusting said weight of the competition for each of the contestants by a first percentage when the ratings for each of the contestants fall within a first interval and by a second percentage when the ratings for each of the contestants exceed the first interval;

calculate a new updated rating for each of the contestants by multiplying the weight of the competition for each of the contestants and the performed as rating;

estimate a likelihood of at least one of the contestants developing and submitting the asset that meets the minimum standards, based on the number of the contestants from whom the indicia of interest were received and a determination of the new updated rating for at least one of the contestants being higher than a predetermined percentile;

specify a completion threshold representing a minimum acceptable estimated likelihood that the competition will successfully complete; and

,

upon determining that the estimated likelihood does not meet the completion threshold, take one or more steps to increase the estimated likelihood.

12. The non-transitory computer readable storage medium of claim 11, wherein the one or more hardware processors are caused to further increase the estimated likelihood by increasing prize amounts.

13. The non-transitory computer readable storage medium of claim 11, wherein the one or more hardware processors are caused to further increase the estimated likelihood by modifying an end date of the competition.

14. The non-transitory computer readable storage medium of claim 11, wherein the one or more hardware processors are caused to further close the registration upon determining that the estimated likelihood meets the completion threshold.

15. The non-transitory computer readable storage medium of claim 11, wherein the asset is at least one of:

a software program, a graphic design, an architecture design, a component design, a static prototype or a working prototype.