Query construction for semantic topic indexes derived by non-negative matrix factorization

• US 20070050356A1
• Filed: 08/22/2006
• Published: 03/01/2007
• Est. Priority Date: 08/23/2005
• Status: Abandoned Application

• Current Assignee: William Amadio
• Original Assignee: William Amadio
• Inventors: Amadio, William
• Application Number:
  US11/507,661
• Publication Number:
  US 20070050356A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification

Abstract

A method, apparatus and machine-readable medium analyze documents processed by non-negative matrix factorization in accordance with semantic topics. Users construct queries by assigning weights to semantic topics to order documents within a set. The query may be refined in accordance with the user'"'"'s evaluation of the efficacy of the query. Any document that does not result in data indicative of significant correlation with at least one semantic topic is flagged so that a user may make a manual review. The collection of semantic topics may be continually or periodically updated in response to new documents. Additionally, the collection may also be “downdated” to drop semantic factors no longer appearing in new documents received after an initial set has been analyzed. Different sets of semantic topics may be generated and each document evaluated using each set. Reports may be prepared showing results for a body of documents for each of a plurality of sets of semantic topics.

Claims

1. A method of evaluating a body of documents, comprising:

parsing the body of documents into a term-document matrix A of values a_ij, where a_ij=a function of the number of times the term i appears in document j;

factoring the matrix A into a product W*H using non-negative matrix factorization, where W represents semantic topics contained in the body of documents and wherein each column of H contains an encoding of a linear combination of the semantic topics that approximates a corresponding column of A; and

constructing queries by weighting semantic topics to order the documents in accordance with relevance to the queries.

2. A method according to claim 1, further comprising updating W in accordance with contents of successive documents.

3. A method according to claim 2, further comprising evaluating each body of documents in accordance with each of a plurality of sets of W.

4. A method according to claim 3, further comprising providing at least one input to refine values in a query in accordance with a user'"'"'s evaluation of the efficacy of the evaluation of the body of documents against the query.

5. A method according to claim 4, further comprising flagging a document having all coefficients of its linear combination of the W-basis vectors below a preselected level.

6. A method according to claim 4, further comprising downdating W to drop semantic factors no longer appearing in new documents.

7. A method according to claim 4, further comprising generating a plurality of sets of W and evaluating a body of documents using each set of W.

8. A method according to claim 7, further comprising providing reports showing results for a body of documents for each of a plurality of sets of W.

9. A machine-readable medium that provides instructions, which when executed by a processor, causes said processor to perform operations comprising:

parsing a body of documents into a term-document matrix A of values a_ij, where a_ij=a function of the number of times the term i appears in document j;

factoring the matrix A into a product W*H using non-negative matrix factorization, where W represents semantic topics contained in the body of documents and wherein each column of H contains an encoding of a linear combination of the semantic topics that approximates a corresponding column of A; and

constructing queries by weighting semantic topics to order the documents in accordance with relevance to the queries.

10. A machine-readable medium according to claim 9, further comprising instructions for updating W in accordance with contents of successive documents.

11. A machine-readable medium, according to claim 10, further comprising instructions for evaluating each body of documents in accordance with each of a plurality of sets of W.

12. A machine-readable medium, according to claim 11, further comprising instructions responding to providing at least one input to refine values in a query in accordance with a user'"'"'s evaluation of the efficacy of the evaluation of the body of documents against the query.

13. A machine-readable medium, according to claim 12, further comprising instructions for flagging a document having all coefficients of its linear combination of the W-basis vectors below a preselected level.

14. A machine-readable medium, according to claim 12, further comprising instructions responding to an input for downdating W to drop semantic factors no longer appearing in new documents.

15. A machine-readable medium, according to claim 12, further comprising instructions generating a plurality of sets of W and evaluating a body of documents using each set of W.

16. A machine-readable medium, according to claim 15, further comprising instructions providing reports showing results for a body of documents for each of a plurality of sets of W.

17. A system to evaluate a body of documents, comprising:

a reader and processor parsing the body of documents into a term-document matrix A of values a_ij, where a_ij=a function of the number of times the term i appears in document j;

said processor factoring the matrix A into a product W*H using non-negative matrix factorization, where W represents semantic topics contained in the body of documents and wherein each column of H contains an encoding of a linear combination of the semantic topics that approximates a corresponding column of A; and

said processor constructing queries by weighting semantic topics to order the documents in accordance with relevance to the queries.

18. A system according to claim 17, further comprising means for updating W in accordance with contents of successive documents.

19. A system according to claim 18, further comprising means for evaluating each body of documents in accordance with each of a plurality of sets of W.

20. A system according to claim 19, further comprising means for providing at least one input to refine values in a query in accordance with a user'"'"'s evaluation of the efficacy of the evaluation the body of documents against the query.

21. A system according to claim 20, further comprising means for flagging a document having all coefficients of its linear combination of the W-basis vectors below a preselected level.

22. A system according to claim 20, further comprising means for downdating W to drop semantic factors no longer appearing in new documents.

23. A system according to claim 20, further comprising means for generating a plurality of sets of W and evaluating a body of documents using each set of W.

24. A system according to claim 23, further comprising means for providing reports showing results for a body of documents for each of a plurality of sets of W.