ENTERPRISE RELEVANCY RANKING USING A NEURAL NETWORK
First Claim
Patent Images
1. ) A method of determining a relevancy rank ordering score for a plurality of documents comprising:
- (a) identifying a finite set of candidate documents;
(b) for each of the candidate documents;
(i) obtaining a plurality of ranking features associated with the candidate document, at least one of the ranking features selected from the group consisting of BM25, click distance, URL depth, file type, and language; and
(ii) using a neural network to calculate a relevancy score from the ranking features,
2 Assignments
0 Petitions
Accused Products
Abstract
A neural network is used to process a set of ranking features in order to determine the relevancy ranking for a set of documents or other items. The neural network calculates a predicted relevancy score for each document and the documents can then be ordered by that score. Alternate embodiments apply a set of data transformations to the ranking features before they are input to the neural network. Training can be used to adapt both the neural network and certain of the data transformations to target environments.
-
Citations
19 Claims
-
1. ) A method of determining a relevancy rank ordering score for a plurality of documents comprising:
-
(a) identifying a finite set of candidate documents; (b) for each of the candidate documents; (i) obtaining a plurality of ranking features associated with the candidate document, at least one of the ranking features selected from the group consisting of BM25, click distance, URL depth, file type, and language; and (ii) using a neural network to calculate a relevancy score from the ranking features, - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. ) A system for generating a relevancy ranking for documents comprising:
-
(a) a module which identifies a set of candidate documents and makes available a plurality of common ranking features for each of the candidate documents; (b) a data module comprising weights for each of the ranking features; and (c) a ranking module comprising at least one input transformation and a neural network, wherein the ranking module accepts the ranking features for each of the candidate documents individually, applies the input transformation to at least one ranking feature, provides the ranking features, including the transformed feature, to the neural network which applies the ranking feature weights and calculates a relevancy score for each of the candidate documents.
-
- 9. ) The relevancy ranking system of claim 9 wherein at least one of the ranking features is selected from the group consisting of BM25, click distance, URL depth, file type, and language
-
10. ) The relevancy ranking system of claim 10 wherein the BM25 feature comprises the BM25G formula which uses at least one property selected from the group consisting of body, title, author, anchor text, URL, and extracted title.
-
13. ) The relevancy ranking system of claim 13 wherein at least one of the configurable constants is adjusted during training of the neural network.
-
15. ) A method of rank ordering a plurality of documents by relevancy comprising:
-
(a) identifying a finite set of candidate documents; (b) for each of the candidate documents; (i) obtaining a plurality of ranking features associated with the candidate document, at least one of the ranking features selected from the group consisting of BM25, click distance, URL depth, file type, and language; (ii) applying a transformation to at least one of the ranking features wherein the transformation comprises a constant which is configurable; (iii) using a neural network to calculate a relevancy score from the ranking features; and (c) ordering the candidate documents by the calculated relevancy scores.
-
-
16. ) The rank ordering method of claim 16 wherein at least one of the transformations is of the form
- 17. ) The rank ordering method of claim 17 further comprising at least one ranking feature transformation comprising mapping each value of an enumerated data type to a discrete binary value and wherein the neural network accepts each discrete binary value as a separate ranking feature and applies a separate trainable weight to each of the discrete binary values.
-
19. ) The method of claim 19 wherein the BM25 feature comprises the BM25G formula which uses the properties of body, title, author, anchor text, URL, and extracted title.
Specification