METHOD AND DEVICE OF HIERARCHICAL DOCUMENT FILTERING

• US 20170116191A1
• Filed: 10/20/2016
• Published: 04/27/2017
• Est. Priority Date: 10/21/2015
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Guangzhou SHENMA Information Technology Co. Ltd.
• Inventors: Li, Yang, Wan, Mingcheng, Zeng, Honglei
• Granted Patent: US 10,459,983 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

The present disclosure provides a method and a device of hierarchical document filtering. The method includes: selecting multiple documents from a to-be-filtered document set of a current document filtering layer according to a preset sampling strategy, and forming a first document list by using the selected documents according to an order of the selected documents in the to-be-filtered document set; calculating a quality value of each document in the first document list respectively according to a relevance calculation method for the current document filtering layer; reordering the documents in the first document list according to the quality value of each document in the first document list, to obtain a second document list; and filtering the to-be-filtered document set of the current document filtering layer according to the degree of consistency between the first document list and the second document list.

Claims

1. A method for filtering documents, comprising:

selecting multiple documents from a to-be-filtered document set of a current document filtering layer according to a preset sampling strategy, and forming a first document list by using the selected documents according to an order of the selected documents in the to-be-filtered document set, wherein documents in the to-be-filtered document set are ordered according to quality values of the documents at an upper document filtering layer;

calculating a quality value of each document in the first document list on the current document filtering layer according to a relevance calculation method for the current document filtering layer;

reordering the documents in the first document list according to the quality value of each document in the first document list on the current document filtering layer, to obtain a second document list; and

filtering the to-be-filtered document set of the current document filtering layer according to the degree of consistency between the first document list and the second document list.

2. The method according to claim 1, wherein the selecting multiple documents from a to-be-filtered document set of a current document filtering layer according to a preset sampling strategy comprises:

determining, according to the number of documents in the to-be-filtered document set of the current document filtering layer, the number of document subsets into which the to-be-filtered document set needs to be divided, and the number of documents that need to be selected from each of the document subsets;

dividing the to-be-filtered document set into the determined number of document subsets according to a document order; and

selecting the determined number of documents from each of the document subsets, to obtain the multiple documents.

3. The method according to claim 2, wherein the selecting the determined number of documents from each of the document subsets comprises:

selecting the determined number of documents from a front part of each of the document subsets;

or,

selecting the determined number of documents from each of the document subsets at a predetermined interval;

or,

selecting the determined number of documents from each of the document subsets in a random manner.

4. The method according to claim 1, wherein the selecting multiple documents from a to-be-filtered document set of a current document filtering layer according to a preset sampling strategy comprises:

determining a document sampling interval according to the number of documents in the to-be-filtered document set of the current document filtering layer; and

selecting documents from the to-be-filtered document set at the document sampling interval, to obtain the multiple documents.

5. The method according to claim 1, wherein filtering the to-be-filtered document set of the current document filtering layer according to the degree of consistency between the first document list and the second document list comprises:

determining whether the degree of consistency between the first document list and the second document list is greater than a preset consistency threshold;

if yes, filtering out documents meeting a preset filtering condition in the to-be-filtered document set of the current document filtering layer; and

if not, calculating a quality value of each document in the to-be-filtered document set according to the relevance calculation method for the current document filtering layer, reordering the documents in the to-be-filtered document set according to the calculated quality values, and filtering out documents meeting the preset filtering condition in the reordered to-be-filtered document set.

6. The method according to claim 1, wherein after the filtering the to-be-filtered document set of the current document filtering layer according to the degree of consistency between the first document list and the second document list, the method further comprises:

determining whether a preset filtering stop condition is met according to the layer number of the current document filtering layer or the number of documents in the filtered to-be-filtered document set;

if yes, determining documents in the filtered to-be-filtered document set as a search result; and

if not, using the filtered to-be-filtered document set as a new document set, and filtering the new document set by using a lower document filtering layer.

7. A device for filtering documents, comprising:

a selection module, configured to select multiple documents from a to-be-filtered document set of a current document filtering layer according to a preset sampling strategy, and form a first document list by using the selected documents according to an order of the selected documents in the to-be-filtered document set, wherein documents in the to-be-filtered document set are ordered according to quality values of the documents at an upper document filtering layer;

a calculation module, configured to calculate a quality value of each document in the first document list respectively according to a relevance calculation method for the current document filtering layer;

an ordering module, configured to reorder the documents in the first document list according to the quality value of each document in the first document list, to obtain a second document list; and

a filtering module, configured to filter the to-be-filtered document set of the current document filtering layer according to the degree of consistency between the first document list and the second document list.

8. The device according to claim 7, wherein the selection module comprises:

a first determination unit, configured to determine, according to the number of documents in the to-be-filtered document set of the current document filtering layer, the number of document subsets into which the to-be-filtered document set needs to be divided, and the number of documents that need to be selected from each of the document subsets;

a division unit, configured to divide the to-be-filtered document set into the determined number of document subsets according to a document order; and

a first selection unit, configured to select the determined number of documents from each of the document subsets respectively, to obtain the multiple documents.

9. The device according to claim 8, wherein the first selection unit comprises:

a first selection subunit, configured to select the determined number of documents from a front part of each of the document subsets respectively;

or,

a second selection subunit, configured to select the determined number of documents from each of the document subsets at a predetermined interval respectively;

or,

a third selection subunit, configured to select the determined number of documents from each of the document subsets respectively in a random manner.

10. The device according to claim 7, which is characterized in that the selection module comprises:

a second determination unit, configured to determine a document sampling interval according to the number of documents in the to-be-filtered document set of the current document filtering layer; and

a second selection unit, configured to select documents from the to-be-filtered document set at the document sampling interval, to obtain the multiple documents.

11. The device according to claim 7, wherein the filtering module comprises:

a filtering unit, configured to determine whether the degree of consistency between the first document list and the second document list is greater than a preset consistency threshold;

if yes, filter out documents meeting a preset filtering condition in the to-be-filtered document set of the current document filtering layer; and

if not, calculate a quality value of each document in the to-be-filtered document set according to the relevance calculation method for the current document filtering layer, reorder the documents in the to-be-filtered document set according to the calculated quality values, and filter out documents meeting the preset filtering condition in the reordered to-be-filtered document set.

12. The device according to claim 7, wherein the device further comprises:

a stop module, configured to determine whether a preset filtering stop condition is met according to the layer number of the current document filtering layer or the number of documents in the filtered to-be-filtered document set;

if yes, determine documents in the filtered to-be-filtered document set as a search result; and

if not, use the filtered to-be-filtered document set as a new document set, and filter the new document set by using a lower document filtering layer.