METHOD OF EVALUATING LEARNING RATE OF RECOMMENDER SYSTEMS
First Claim
Patent Images
1. A method of evaluating a recommender system, which recommender is used for recommending items of interest to subscribers of an online content service provider, the method comprising the steps of:
- (a) identifying a first set of reference items to be used in evaluating the recommender system;
(b) generating a plurality of separate recommendations for a second set of items from the recommender system;
(c) correlating said first set of reference items with said second set of items to identify an awareness level exhibited by the recommender system for said first set of reference items;
(d) generating a rating for the recommender system based on the results of step (c) based on an evaluation of said awareness level.
1 Assignment
0 Petitions
Accused Products
Abstract
A recommender system is analyzed to determine various performance characteristics, such as a learning rate for new items, or a learning rate for new subscriber tastes. Comparisons of different recommenders are presented to assist consumers and marketers in selecting appropriate e-commerce sites for purchasing, advertising, etc.
85 Citations
20 Claims
-
1. A method of evaluating a recommender system, which recommender is used for recommending items of interest to subscribers of an online content service provider, the method comprising the steps of:
-
(a) identifying a first set of reference items to be used in evaluating the recommender system; (b) generating a plurality of separate recommendations for a second set of items from the recommender system; (c) correlating said first set of reference items with said second set of items to identify an awareness level exhibited by the recommender system for said first set of reference items; (d) generating a rating for the recommender system based on the results of step (c) based on an evaluation of said awareness level. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A method of evaluating a learning rate of a recommender system for new items, which recommender is used for recommending items of interest to subscribers of an online content service provider, the method comprising the steps of:
-
(a) identifying a first set of new items to be used in evaluating the recommender system; wherein at least some of said first set of new items are characterized by relatively few explicit ratings in a recommender system database; (b) reviewing a plurality of separate recommendations for a second set of items made by the recommender system; (c) correlating said first set of reference items with said second set of items to identify an awareness level exhibited by the recommender system for said first set of reference items; (d) generating a rating for the recommender system based on the results of step (c) based on an evaluation of said awareness level. - View Dependent Claims (15, 16, 17)
-
-
18. A method of evaluating a learning rate of a recommender system for new preferences by users, which recommender is used for recommending items of interest to subscribers of an online content service provider, the method comprising the steps of:
-
(a) providing a set of proxy accounts to interact with the recommender system; (b) identifying a first set of recommendations given by the recommender system to said set of proxy accounts; (c) modifying a profile of said set of proxy accounts to create a set of modified proxy accounts, including explicit ratings for items which can be recommended by the recommender system; (d) identifying a second set of recommendations given by the recommender system to said set of modified proxy accounts. - View Dependent Claims (19, 20)
-
Specification