Unsupervised learning tool for feature correction

• US 20070043707A1
• Filed: 10/17/2005
• Published: 02/22/2007
• Est. Priority Date: 08/17/2005
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Raj, Binu, Kulkarni, Parashuram
• Granted Patent: US 7,483,903 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/95 Retrieval from the web

Abstract

Techniques for correcting miscategorized features excerpted from web pages are provided. For each of several categories and several pages on a particular web site, a separate feature may be excerpted from that page and associated with that page in relation to that category. Often, many of the “high confidence” features that have been associated with the same category are found to be associated with similar characteristics regardless of the pages from which those features were excerpted. Thus, a set of category characteristics, which are often found associated with the “high confidence” features in a particular category, may be determined. For each page, a candidate feature that is associated with the set of category characteristics may be identified in that page. If, in relation to the particular category, a feature other than the candidate feature is associated with that page, then that other feature may be replaced by the candidate feature.

Claims

1. A method of maintaining a search index, the method comprising:

identifying one or more characteristics that are associated with particular data items that were excerpted from a first plurality of documents;

selecting, from the one or more characteristics, one or more particular characteristics; and

for each particular document in a second plurality of documents, performing steps comprising;

determining whether to insert, into the search index, a candidate data item excerpted from the particular document, wherein determining whether to insert the candidate data item is based at least in part on whether the candidate data item is associated with characteristics in the one or more particular characteristics; and

inserting the candidate data item into the search index in response to a determination that the candidate data item should be inserted.

2. The method of claim 1, wherein determining whether to insert the candidate data item is based at least in part on whether the candidate data item is associated with all of the one or more particular characteristics.

3. The method of claim 1, wherein the second plurality of documents comprises the first plurality of documents.

4. The method of claim 1, wherein all of the particular data items were determined, with a degree of confidence above a specified threshold, to fit a specified category.

5. The method of claim 1, wherein determining whether to insert the candidate data item is based at least in part on whether the candidate data item is already associated, in the search index, with the particular document.

6. The method of claim 1, wherein inserting the candidate data item into the search index comprises replacing one of the particular data items, in the search index, with the candidate data item.

7. The method of claim 1, wherein selecting the one or more particular characteristics comprises selecting, from among different sets of characteristics that are associated with the particular data items, a set of one or more characteristics that is associated most often with the particular data items in the first plurality of documents.

8. The method of claim 1, wherein all of the particular data items are associated, in the search index, with the same category of data items.

9. The method of claim 1, further comprising:

in response to a determination that the candidate data item should be inserted, removing, from the search index, a data item that was excerpted from the particular document.

10. The method of claim 1, further comprising:

in response to a determination that the candidate data item should be inserted, removing, from the search index, a data item that is associated with (a) the particular document and (b) a particular category that the candidate data item fits.

11. The method of claim 1, wherein the first plurality of documents and the second plurality of documents only comprise web pages that are accessible through the same web site.

13. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

14. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

15. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

16. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

17. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

18. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

19. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

20. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

21. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

22. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

23. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

12. A method of revising a search index, the method comprising:

for each data item in the search index, (a) identifying, within a document from which the data item was excerpted, a set of one or more characteristics that are associated with the data item, and (b) adding the set of one or more characteristics to a candidate set;

selecting a particular set of one or more characteristics from the candidate set; and

for each particular document in a plurality of documents, (a) identifying, within the particular document, a candidate data item that is associated with the characteristics in the particular set of one or more characteristics, and (b) inserting the candidate data item into the search index if the search index does not contain the candidate data item.

24. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.