MATCH FIX-UP TO REMOVE MATCHING DOCUMENTS

• US 20160378796A1
• Filed: 06/17/2016
• Published: 12/29/2016
• Est. Priority Date: 06/23/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: HOPCROFT, MICHAEL JOSEPH, GOODWIN, ROBERT LOVEJOY, WANG, FAN, JAIN, UTKARSH
• Granted Patent: US 11,281,639 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2237 Vectors, bitmaps or matrices
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

The technology described herein provides for a match fix-up stage that removes matching documents identified for a search query that don'"'"'t actually contain terms from the search query. A representation of each document (e.g., a forward index storing a list of terms for each document) is used to identify valid matching documents (i.e., documents containing terms from the search query) and invalid matching documents (i.e., documents that don'"'"'t contain terms from the search query). Any invalid matching documents are removed from further processing and ranking for the search query.

Claims

1. A computer-implemented method, carried out by at least one server having one or more processors, the method comprising:

receiving a plurality of documents found to be relevant to at least a portion of a search query, wherein the plurality of documents includes one or more invalid matching documents;

accessing a representation for each document of the plurality of documents, wherein the representation includes each term present within each document;

comparing the terms present within each document to one or more terms associated with the search query;

determining that the one or more invalid matching documents do not include the one or more terms associated with the search query; and

upon determining that the one or more invalid matching documents do not include the one or more terms associated with the search query, removing the one or more invalid matching documents from the plurality of documents found to be relevant to the at least a portion of the search query.

2. The method of claim 1, wherein the one or more invalid matching documents are documents that are not relevant to the at least a portion of the search but are included in the plurality of documents found to be relevant to the at least a portion of the search query.

3. The method of claim 1, wherein the representation includes a forward index for each document of the plurality of documents.

4. The method of claim 3, wherein the forward index includes each term included within each document of the plurality of documents.

5. The method of claim 3, wherein the forward index includes a portion of terms that is included within each document of the plurality of documents.

6. The method of claim 1, wherein, prior to removal of the one or more invalid matching documents, the plurality of documents found to be relevant to the at least a portion of the search query is associated with a false positive rate greater than 0%.

7. The method of claim 1, wherein the representation is a data structure.

8. The method of claim 1, further comprising communicating the plurality of documents found to be relevant to the at least a portion of the search query on to a ranker subsequent to removing the one or more invalid matching documents.

9. One or more computer storage media storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform a method, the method comprising:

receiving a first plurality of documents found to be relevant to at least a portion of a search query, wherein the first plurality of documents includes one or more invalid matching documents;

receiving a forward index for each document of the first plurality of documents, wherein the forward index includes one or more terms included in each document;

using the forward index for each document of the first plurality of documents, identifying one or more valid matching documents that include one or more terms associated with the search query;

using the forward index for each document of the first plurality of documents, identifying one or more invalid matching documents that do not include the one or more terms associated with the search query;

removing the one or more invalid matching documents from the first plurality of documents to create a filtered set of one or more documents found to be relevant to the at least a portion of the search query; and

communicating the filtered set of one or more documents found to be relevant to the at least a portion of the search query for ranking each document of the filtered set of one or more documents for the search query.

10. The media of claim 9, wherein the forward index for each document is associated with a data structure.

11. The media of claim 9, wherein the first plurality of documents are received from a preliminary ranker that ranked each document of the first plurality of documents for the search query.

12. The media of claim 9, wherein the forward index for each document includes each term included in each document.

13. The media of claim 9, wherein the forward index for each document includes a portion of terms included in each document.

14. The media of claim 9, wherein a false positive rate of greater than 0% is associated with the first plurality of documents.

15. A computerized system embodied on one or more computer storage media having computer-executable instructions provided thereon, the system comprising:

a preliminary ranker component to rank a first set of documents that are found to be relevant to at least a portion of a search query by a matcher component, wherein the initial set of documents includes one or more invalid matching documents;

a match fix-up component to identify when the first set of documents includes one or more invalid matching documents utilizing a forward index for each document of the initial set of documents; and

a subsequent ranker to rank a second set of documents received from the match fix-up component, wherein the second set of documents includes fewer invalid matching documents that the first set of documents.

16. The system of claim 15, wherein the match fix-up component includes the forward index for each document including each term associated with each document.

17. The system of claim 15, wherein the match fix-up component includes the forward index for each document including a portion of terms associated with each document.

18. The system of claim 15, wherein the match fix-up component includes a data structure associated with the forward index.

19. The system of claim 15, wherein the match fix-up component removes the one or more invalid matching documents from the first set of documents.

20. The system of claim 19, wherein, prior to removal of the one or more invalid matching documents, the first set of documents is associated with a false positive rate greater than 0%.