Implicit rating of retrieved information in an information search system

• US 6,665,655 B1
• Filed: 04/14/2000
• Issued: 12/16/2003
• Est. Priority Date: 04/14/2000
• Status: Expired due to Term

• Current Assignee: Oracle OTC Subsidiary LLC (Oracle Corporation)
• Original Assignee: RightNow Technologies Incorporated (Oracle Corporation)
• Inventors: Warner, Douglas K., Myer, Michael A.
• Primary Examiner(s): Shah, Sanjiv
• Application Number:
  US09/549,669
• Time in Patent Office: 1,341 Days
• Field of Search: 707/2, 707/3, 707/5, 707/7, 707/6, 707/10, 707/104, 709/219
• US Class Current: 1/1
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

An information retrieval system allows a user to search a database of informational items for a desired informational item, and presents the search result in the form of matching index entries in the order of relevance. The information retrieval system in accordance with the principles of the present invention assigns a relevance rating to each of the index entries without requiring an explicit input from the user with respect to the usefulness or the relevance of the retrieved information corresponding to the respective index entries. When the user selects and retrieves an informational item through a list of index entries presented by the retrieval system, as a result of a search, the relevance rating of the selected informational item is increased by a predetermined amount. The relevance rating of the selected informational item is further adjusted based on any actions the user takes subsequent to the initial selection of the informational item if the subsequent act indicates that the relevance of the selected informational item may be less than what is reflected by the rating increase by the predetermined amount. Ratings of the informational items in the database are determined from implicit suggestions from the usage of the retrieval system and the database by the user rather than from an explicit user input. In another aspect of the present invention, the ratings are allowed to decay over time to minimize the tendencies for historical usage biased rating, and to provide more temporally accurate ratings. The most recently accessed time of each of the informational items in the database is compared to a predetermined stale access time threshold, and if the most recently accessed time is older than the threshold, than the rating of the corresponding informational item is decreased to reflect the dated nature of the information contained within the item.

Claims

1. A method of implicitly assigned relevancy ratings of a plurality of informational items in an information retrieval system, comprising:

detecting an initial user selection of a selected subset of said plurality of informational items;

adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection and based upon a pattern of behavior of a set of users without requiring an explicit input from the users with respect to characteristics of said informational items; and

storing said adjusted relevancy rating for said selected subset of said plurality of information items.

2. The method of implicitly assigning relevancy ratings in accordance with claim 1, wherein said step of adjusting said relevancy rating comprises:

increasing said relevancy rating of said selected subset of said plurality of informational items by a first predetermined amount.

3. The computer readable storage medium in according to claim 1, wherein:

said instructions for adjusting said relevancy rating are adapted to increase said relevancy rating of said selected subset of said plurality of informational items by a first predetermined amount.

4. A method of implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, comprising:

detecting an initial user selection of a selected subset of said plurality of informational items;

adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection;

storing said adjusted relevancy rating for said selected subset of said plurality of informational items;

detecting a user action subsequent to said detected initial user selection;

determining a nature of said user action; and

further adjusting said relevancy rating of said selected subset of said plurality of informational items based on said detected nature of said user action,

wherein said step of determining said nature of said user action comprises;

detecting a user selection of a second selected subset of said plurality of informational items; and

wherein said step of further adjusting said relevancy rating comprises;

decreasing said adjusted relevancy rating of said first selected subset of said plurality of informational items by a second predetermined amount, said second predetermined amount being smaller than said first predetermined amount.

5. The method of implicitly assigning relevancy ratings in accordance with claim 4, further comprising:

increasing a relevancy rating of said second selected subset of said plurality of informational items by said first predetermined amount.

6. The method of implicitly assigning relevancy ratings in accordance with claim 5, further comprising:

detecting a user selection of a third selected subset of said plurality of informational items;

decreasing said relevancy rating of said second selected subset of said plurality of informational items by said second predetermined amount; and

increasing a relevancy rating of said third selected subset of said plurality of informational items by said first predetermined amount.

7. A method of implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, comprising:

detecting an initial user selection of a selected subset of said plurality of informational items;

adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection;

storing said adjusted relevancy rating for said selected subset of said plurality of informational items;

detecting a user action subsequent to said detected initial user selection;

determining a nature of said user action; and

further adjusting said relevancy rating of said selected subset of said plurality of informational items based on said detected nature of said user action,

wherein said step of determining said nature of said user action comprises;

detecting a user submission of a question; and

wherein said step of further adjusting said relevancy rating comprises;

decreasing said adjusted relevancy rating of said selected subset of said plurality of informational items by a second predetermined amount, said second predetermined amount being smaller than a first predetermined amount.

8. A method of implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, comprising:

detecting an initial user selection of a selected subset of said plurality of informational items;

adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection;

storing said adjusted relevancy rating for said selected subset of said plurality of informational items;

detecting a user action subsequent to said detected initial user selection;

determining a nature of said user action; and

further adjusting said relevancy rating of said selected subset of said plurality of informational items based on said detected nature of said user action,

wherein said step of determining said nature of said user action comprises;

detecting a user initiation of a new search; and

wherein said step of further adjusting said relevancy rating comprises;

decreasing said adjusted relevancy rating of said selected subset of said plurality of informational items by a second predetermined amount, said second predetermined amount being smaller than a first predetermined amount.

9. The method of implicitly assigning relevancy ratings in accordance with claim 8, further comprising:

increasing a relevancy rating of a fourth selected subset of said plurality of informational items by a third predetermined amount, said fourth selected subset of said plurality of informational items being found as a result of said new search, and said third predetermined amount being smaller than said first predetermined amount.

10. An apparatus for implicitly assigning relevancy of ratings of a plurality of informational items in an information retrieval system, comprising:

means for detecting an initial user selection of a selected subset of said plurality of informational items;

means for adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection and based upon a pattern of behavior of a set of users without requiring an explicit input from the users with respect to characteristics of said informational items; and

means for storing said adjusted relevancy rating for said selected subset of said plurality of informational items.

11. The apparatus for implicitly assigning relevancy ratings according to claim 10, wherein:

said means for adjusting said relevancy rating is adapted to increase said relevancy rating of said selected subset of said plurality of informational items by a first predetermined amount.

12. An apparatus for implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, comprising:

means for detecting an initial user selection of a selected subset of said plurality of informational items;

means for adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection;

means for storing said adjusted relevancy rating for said selected subset of said plurality of informational items;

means for detecting a user action subsequent to said detected initial user selection;

means for determining a nature of said user action; and

means for further adjusting said relevancy rating of said selected subset of said plurality of informational items based on said detected nature of said user action,

wherein;

said means for determining said nature of said user action is adapted to detect a user selection of a second selected subset of said plurality of informational items; and

wherein said means for further adjusting said relevancy rating is adapted to decrease said adjusted relevancy rating of said first selected subset of said plurality of informational items by a second predetermined amount, said second predetermined amount being smaller than a first predetermined amount.

13. The apparatus for implicitly assigning relevancy ratings according to claim 12, further comprising:

means for increasing a relevancy rating of said second selected subset of said plurality of informational items by said first predetermined amount.

14. The apparatus for implicitly assigning relevancy ratings according to claim 13, further comprising:

means for detecting a user selection of a third selected subset of said plurality of informational items;

means for decreasing said relevancy rating of said second selected subset of said plurality of informational items by said second predetermined amount; and

means for increasing a relevancy rating of said third selected subset of said plurality of informational items by said first predetermined amount.

15. An apparatus for implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, comprising:

means for detecting an initial user selection of a selected subset of said plurality of informational items;

means for adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection;

means for storing said adjusted relevancy rating for said selected subset of said plurality of informational items;

means for detecting a user action subsequent to said detected initial user selection;

means for determining a nature of said user action; and

means for further adjusting said relevancy rating of said selected subset of said plurality of informational items based on said detected nature of said user action,

wherein;

said means for determining said nature of said user action is adapted to detect a user submission of a question; and

wherein said means for further adjusting said relevancy rating is adapted to decrease said adjusted relevancy rating of said selected subset of said plurality of informational items by a second predetermined amount, said second predetermined amount being smaller than a first predetermined amount.

16. An apparatus for implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, comprising:

means for detecting an initial user selection of a selected subset of said plurality of informational items;

means for adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection;

means for storing said adjusted relevancy rating for said selected subset of said plurality of informational items;

means for detecting a user action subsequent to said detected initial user selection;

means for determining a nature of said user action; and

means for further adjusting said relevancy rating of said selected subset of said plurality of informational items based on said detected nature of said user action,

wherein;

said means for determining said nature of said user action is adapted to detect a user initiation of a new search; and

wherein said means for further adjusting said relevancy rating is adapted to decrease said adjusted relevancy rating of said selected first subset of said plurality of informational items by a second predetermined amount, said second predetermined amount being smaller than a first predetermined amount.

17. The apparatus for implicitly assigning relevancy ratings according to claim 16, further comprising:

means for increasing a relevancy rating of a fourth selected subset of said plurality of informational items by a third predetermined amount, said fourth selected subset of said plurality of informational items being found as a result of said new search, and said third predetermined amount being smaller than said first predetermined amount.

18. An information retrieval system, comprising:

a plurality of information items;

an adjustment means adapted to adjust a relevancy rating of a selected subset of said plurality of information items upon a detection of an initial user selection of said selected subset of said plurality of informational items and based upon a pattern of behavior of a set of users without requiring an explicit input from the users with respect to characteristics of said informational items; and

an index for storing said adjusted relevancy rating.

19. The information retrieval system according to claim 18, wherein said plurality of informational items comprise:

a plurality of world wide web pages.

20. A computer readable storage medium having stored thereon computer program for implementing a method of implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, said computer program comprising a set of instructions for:

detecting an initial user selection of a selected subset of said plurality of informational items;

adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection and based upon a pattern of behavior of a set of users without requiring an explicit input from the users with respect to characteristics of said informational items; and

storing said adjusted relevancy rating for said selected subset of said plurality of information items.

21. A computer readable storage medium having stored thereon computer program for implementing a method of implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, said computer program comprising a set of instructions for:

detecting an initial user selection of a selected subset of said plurality of informational items;

adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection;

storing said adjusted relevancy rating for said selected subset of said plurality of informational items;

detecting a user action subsequent to said detected initial user selection;

determining a nature of said user action; and

further adjusting said relevancy rating of said selected subset of said plurality of informational items based on said detected nature of said user action,

wherein;

said one or more instructions for determining said nature of said user action are adapted to detect a user selection of a second one of said plurality of informational items; and

wherein said one or more instructions for further adjusting said relevancy rating are adapted to decrease said adjusted relevancy rating of said selected subset of said plurality of informational items by a second predetermined amount, said second predetermined amount being smaller than a first predetermined amount.

22. The computer readable storage medium in according to claim 21, wherein said computer program further comprising one or more instructions for:

increasing a relevancy rating of said second selected subset of said plurality of informational items by said first predetermined amount.

23. The computer readable storage medium in according to claim 22, wherein said computer program further comprising one or more instructions for:

detecting a user selection of a third selected subset of said plurality of informational items;

decreasing said relevancy rating of said second selected subset of said plurality of informational items by said second predetermined amount; and

increasing a relevancy rating of said third selected subset of said plurality of informational items by said first predetermined amount.

24. A computer readable storage medium having stored thereon computer program for implementing a method of implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, said computer program comprising a set of instructions for:

detecting an initial user selection of a selected subset of said plurality of informational items;

adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection;

storing said adjusted relevancy rating for said selected subset of said plurality of informational items;

detecting a user action subsequent to said detected initial user selection;

determining a nature of said user action; and

further adjusting said relevancy rating of said selected subset of said plurality of informational items based on said detected nature of said user action,

wherein;

said one or more instructions for determining said nature of said user action are adapted to detect a user submission of a question; and

wherein said one or more instructions for further adjusting said relevancy rating are adapted to decrease said adjusted relevancy rating of said selected subset of said plurality of informational items by a second predetermined amount, said second predetermined amount being smaller than a first predetermined amount.

25. A computer readable storage medium having stored thereon computer program for implementing a method of implicitly assigning relevancy ratings of a plurality of informational items in an information retrieval system, said computer program comprising a set of instructions for:

detecting an initial user selection of a selected subset of said plurality of informational items;

adjusting a relevancy rating of said selected subset of said plurality of informational items in light of said detected initial user selection;

storing said adjusted relevancy rating for said selected subset of said plurality of informational items;

detecting a user action subsequent to said detected initial user selection;

determining a nature of said user action; and

further adjusting said relevancy rating of said selected subset of said plurality of informational items based on said detected nature of said user action,

wherein;

said one or more instructions for determining said nature of said user action are adapted to detect a user initiation of a new search; and

wherein said one or more instructions for further adjusting said relevancy rating are adapted to decrease said adjusted relevancy rating of said selected subset of said plurality of informational items by a second predetermined amount, said second predetermined amount being smaller than a first predetermined amount.

26. The computer readable storage medium in according to claim 25, wherein said computer program further comprising one or more instructions for:

increasing a relevancy rating of a fourth selected subset of said plurality of informational items by a third predetermined amount, said fourth selected subset of said plurality of informational items being found as a result of said new search, and said third predetermined amount being smaller than said first predetermined amount.