Semi-automatic index term augmentation in document retrieval

• US 9,275,130 B2
• Filed: 10/24/2013
• Issued: 03/01/2016
• Est. Priority Date: 03/31/1999
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Ponte, Jay Michael
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): CHEUNG, HUBERT G
• Application Number:
  US14/062,708
• Publication Number:
  US 20140136532A1
• Time in Patent Office: 859 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Disclosed are methods and systems for indexing or retrieving materials accessible through computer networks.

Claims

1. A non-transitory machine-readable medium having executable instructions to cause one or more processing units to perform a method to assign terms to a first document, the method comprising:

selecting the first document;

generating a query containing at least one term, from the selected first document;

applying the generated query to a plurality of documents to define a subset of the plurality of documents, wherein the defined subset of the plurality of documents constitutes those documents of the plurality of documents that contain the at least one term and meet a predetermined threshold of query relevance;

determining additional terms based on the defined subset of the plurality of documents, including;

determining a co-occurrence metric of the at least one term from the selected first document with each term of the defined subset of the plurality of documents,

determining a frequency score using the determined co-occurrence metric for each term of the defined subset of the plurality of documents, and

selecting a subset of terms of the defined subset of the plurality of documents based on the determined frequency score for each term;

assigning the selected subset of terms to the selected first document; and

storing the selected first document in a storage system that is remotely accessible via a network.

2. The non-transitory machine-readable medium of claim 1, wherein the at least one term includes all terms in the selected first document.

3. The non-transitory machine-readable medium of claim 1, wherein query relevance is determined using at least one of term frequency scoring and inverted document frequency scoring, and the predetermined threshold of query relevance is a maximum number of highest scoring documents.

4. The non-transitory machine-readable medium of claim 3, wherein the query relevancy relevance is determined using Robertson'"'"'s term frequency scoring.

5. The non-transitory machine-readable medium of claim 1, wherein the plurality of documents are Web pages.

6. The non-transitory machine-readable medium of claim 1, wherein the plurality of documents are Web sites.

7. The non-transitory machine-readable medium of claim 1, wherein terms do not include stop words.

8. The non-transitory machine-readable medium of claim 1, wherein selecting the subset of terms based on the determined frequency score for each term includes selecting terms having the determined frequency score above a predetermined term assignment threshold.

9. The non-transitory machine-readable medium of claim 8, wherein the predetermined term assignment threshold is a single highest scoring term.

10. The non-transitory machine-readable medium of claim 1, wherein assigning the selected subset of terms includes inserting the selected subset of terms into a portion of the selected first document.

11. The non-transitory machine-readable medium of claim 10, wherein the portion of the selected first document is a segment of the selected first document for storing terms to improve document retrieval methods.

12. The non-transitory machine-readable medium of claim 1, wherein the plurality of documents has previously had index terms assigned to the plurality of documents by another method.

13. The non-transitory machine-readable medium of claim 12, wherein weighting is applied to the index terms assigned to the plurality of documents when applying the generated query to the plurality of documents.

14. The non-transitory machine-readable medium of claim 1, wherein the network includes the Internet.

15. An apparatus for assigning terms to a first document, the apparatus comprising:

means for selecting the first document;

means for generating a query containing at least one term, from the selected first document;

means for applying the generated query to a plurality of documents to define a subset of the plurality of documents, wherein the defined subset of the plurality of documents constitutes those documents of the plurality of documents that contain the at least one term and meet a predetermined threshold of query relevance;

means for determining additional terms based on the defined subset of the plurality of documents, wherein the means for determining additional terms includes;

means for determining a co-occurrence metric of the at least one term from the selected first document with each term of the defined subset of the plurality of documents,

means for determining a frequency score using the determined co-occurrence metric for each term of the defined subset of the plurality of documents, and

means for selecting a subset of terms of the defined subset of the plurality of documents based on the determined frequency score for each term;

means for assigning the selected subset of terms to the selected first document; and

means for storing the selected first document in a storage system that is remotely accessible via a network.

16. The apparatus of claim 15, wherein the at least one term includes all terms in the selected first document.

17. The apparatus of claim 15, wherein query relevance is determined using at least one of term frequency scoring and inverted document frequency scoring, and the predetermined threshold of query relevance is a maximum number of highest scoring documents.

18. The apparatus of claim 17, wherein the query relevancy relevance is determined using Robertson'"'"'s term frequency scoring.

19. The apparatus of claim 15, wherein the plurality of documents are Web pages.

20. The apparatus of claim 15, wherein the plurality of documents are Web sites.