System and method for ranking a posting

• US 8,670,968 B1
• Filed: 08/31/2012
• Issued: 03/11/2014
• Est. Priority Date: 12/23/2009
• Status: Active Grant

• Current Assignee: Intuit, Inc.
• Original Assignee: Intuit, Inc.
• Inventors: Podgorny, Igor A., Morgan, Floyd J., Szydlowski, Derek
• Primary Examiner(s): Patel, Shambhavi
• Application Number:
  US13/601,732
• Time in Patent Office: 557 Days
• Field of Search: 703/13, 434/107, 434/236, 705/7.29, 705/7.32, 705/14.41, 705/14.43, 706/12, 706/14, 706/15, 706/21, 706/46, 707/713, 707/734, 707/749, 707/791, 707/776, 709/203, 709/217, 709/219, 709/246
• US Class Current: 703/13
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26

Abstract

A method for training a ranking application. The method includes ranking the help postings to create an initial ranking using initial parameter values, and storing user interactions with the help postings to obtain stored interactions. Simulations are performed using the stored interactions to generate revised parameter values for the ranking application. Performing the simulations includes calculating relevance values from the stored interactions, creating a test posting, assigning, to the test posting, an initial score and a relevance value randomly selected from the relevance values to generate a test ranking, and simulating user interactions with the test ranking to generate simulated rankings. The simulated rankings are analyzed to obtain revised parameter values. The method further includes ranking, using the revised parameter values, the help postings to generate a revised ranking, and displaying the help postings in the forum according to the revised ranking.

Claims

1. A method for training a ranking application to rank a plurality of help postings for display in a forum, comprising:

ranking the plurality of help postings to create an initial ranking, wherein the plurality of help postings in the initial ranking are ranked by the ranking application using initial parameter values;

displaying the plurality of help postings in the forum according to the initial ranking;

storing user interactions with the plurality of help postings to obtain stored interactions;

performing simulations using the stored interactions to generate revised parameter values for the ranking application, wherein performing the simulations comprises;

calculating a plurality of relevance values from the stored interactions;

creating a test posting;

assigning, to the test posting, an initial score and a relevance value randomly selected from the plurality of relevance values to generate a test ranking;

determining a probability that a user will view the plurality of help postings and the test posting in the test ranking;

generating, based on the probability, simulated rankings of the plurality of help postings and the test posting; and

analyzing the simulated rankings based on the relevance value to obtain revised parameter values; and

ranking, by the ranking application using the revised parameter values, the plurality of help postings to generate a revised ranking; and

displaying the plurality of help postings in the forum according to the revised ranking.

2. The method of claim 1, further comprising:

calculating, from the stored interactions, an average waiting time between views to obtain a viewing interval,

wherein determining the probability is performed during the viewing interval.

3. The method of claim 1, wherein the determining the probability is performed according to a cascade click model.

4. The method of claim 1, wherein analyzing the simulated rankings comprises determining whether the simulated rankings with the revised parameter values are ordered closer to relevancy than the initial rankings.

5. The method of claim 1, wherein the revised parameter values comprises an initial rank value, and wherein the initial score calculated based on the initial rank value.

6. The method of claim 1, wherein the ranking the plurality of help postings and determining the probability uses the same ranking algorithm, wherein the ranking algorithm assigns a score to each of the plurality of help postings and test posting, and wherein the score is based on a popularity value and a seed value.

7. The method of claim 6, wherein the popularity value is reduced over time using a deflation parameter value specifying a rate of reduction over time, and wherein the revised parameter values comprise the deflation parameter value.

8. A system for training a ranking application to rank a plurality of help postings for display in a forum, the system comprising:

a hardware processor;

a ranking application, executing on the hardware processor and configured to;

rank the plurality of help postings to create an initial ranking, wherein the plurality of help postings in the initial ranking is ranked by the ranking application using initial parameter values;

display the plurality of help postings in the forum according to the initial ranking;

store user interactions with the plurality of help postings to obtain stored interactions;

rank, using revised parameter values, the plurality of help postings to generate a revised ranking; and

display the plurality of help postings in the forum according to the revised ranking; and

a simulation application, executing on the hardware processor and configured to;

perform simulations using the stored interactions to generate revised parameter values for the ranking application, wherein performing the simulations comprises;

calculating a plurality of relevance values from the stored interactions;

creating a test posting;

assigning, to the test posting, an initial score and a relevance value randomly selected from the plurality of relevance values to generate a test ranking;

determining a probability that a user will view the plurality of help postings and the test posting in the test ranking;

generate, based on the probability, simulated rankings of the plurality of help postings and the test posting;

analyzing the simulated rankings based on the relevance value to obtain the revised parameter values; and

provide the revised parameter values to the ranking application.

9. The system of claim 8, wherein the simulation application is further configured to:

calculate, from the stored interactions, an average waiting time between views to obtain a viewing interval,

wherein determining the probability is performed during the viewing interval.

10. The system of claim 8, wherein the determining the probability is performed according to a cascade click model.

11. The system of claim 8, wherein analyzing the simulated rankings comprises determining whether the simulated rankings with the revised parameter values are ordered closer to relevancy than the initial rankings.

12. The system of claim 8, wherein the revised parameter values comprises an initial rank value, and wherein the initial score calculated based on the initial rank value.

13. The system of claim 8, wherein the ranking the plurality of help postings and determining the probability uses the same ranking algorithm, wherein the ranking algorithm assigns a score to each of the plurality of help postings and test posting, and wherein the score is based on a popularity value and a seed value.

14. The system of claim 13, wherein the popularity value is reduced over time using a deflation parameter value specifying a rate of reduction over time, and wherein the revised parameter values comprise the deflation parameter value.

15. A non-transitory computer readable medium comprising computer readable program code embodied therein for causing a computer system to:

rank a plurality of help postings to create an initial ranking, wherein the plurality of help postings in the initial ranking is ranked using initial parameter values;

display the plurality of help postings in the forum according to the initial ranking;

store user interactions with the plurality of help postings to obtain stored interactions;

perform simulations using the stored interactions to generate revised parameter values, wherein performing the simulations comprises;

calculating a plurality of relevance values from the stored interactions;

creating a test posting;

assigning, to the test posting, an initial score and a relevance value randomly selected from the plurality of relevance values to generate a test ranking;

determining a probability that a user will view the plurality of help postings and the test posting in the test ranking;

generating, based on the probability, simulated rankings of the plurality of help postings and the test posting; and

analyzing the simulated rankings based on the relevance value to obtain revised parameter values; and

rank, using the revised parameter values, the plurality of help postings to generate a revised ranking; and

displaying the plurality of help postings in the forum according to the revised ranking.

16. The non-transitory computer readable medium of claim 15, wherein the ranking the plurality of help postings and determining the probability uses the same ranking algorithm, wherein the ranking algorithm assigns a score to each of the plurality of help postings and test posting, and wherein the score is based on a popularity value and a seed value.

17. The non-transitory computer readable medium of claim 16, wherein the popularity value is reduced over time using a deflation parameter value specifying a rate of reduction over time, and wherein the revised parameter values comprise the deflation parameter value.