Data structure, index creation device, data search device, index creation method, data search method, and computer-readable recording medium

• US 9,600,565 B2
• Filed: 06/16/2011
• Issued: 03/21/2017
• Est. Priority Date: 10/15/2010
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kusumura, Yukitaka, Mizuguchi, Hironori, Kusui, Dai, Muraoka, Yusuke
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US13/824,740
• Publication Number:
  US 20130262470A1
• Time in Patent Office: 2,105 Days
• Field of Search: 707/742
• US Class Current: 1/1
• CPC Class Codes: G06F 16/319 Inverted lists

Abstract

In an inverted list of each node in a taxonomy, among each node, an inverted list of the highest node is a list of integer values indicating an identifier of search subject data, and an inverted list of a node other than the highest node, in place of the identifier, is a list of integer values indicating a position in an inverted list corresponding to a node that is higher by one than the node. Furthermore, a list of integer values in an inverted list of each node is divided into two or more blocks, and a differential value between an integer value and an integer value directly before the integer value in the block is converted into a bit string of a variable length integer code.

Claims

1. In a taxonomy having a tag with respect to search subject data, an index creation device configured to create an inverted list used for taking out a set of search subject data which can be reached from each node in said taxonomy, wherein

a list of integer values in an inverted list of said each node is divided into two or more blocks, and a differential value between an integer value and an integer value directly before said integer value in said block is converted into a bit string of a variable length integer code, and

said index creation device, realized by a computer, comprises:

a processor configured to execute instructions to implement;

an ancestor node determination part configured to select one ancestor node that is a higher node of said node for every node in said taxonomy, and generate data for ancestor reference indicating selected said ancestor node;

an ancestor node search part configured to generate an ancestor node list indicating one or more ancestor nodes of a tag in said taxonomy based on said data for ancestor reference; and

an ancestor number converting part configured to, upon receiving an identifier of search subject data, with respect to the highest node among each said node in said ancestor node list, adds said identifier as an element of a corresponding inverted list, and with respect to a node other than said highest node, as an element of a corresponding inverted list, in place of said identifier, adds an integer value indicating a position in an inverted list corresponding to a node that is higher by one than said node, wherein

said ancestor node determination part, upon receiving a frequency distribution indicating whether each node in said taxonomy has or may have to what extent of a frequency within a prescribed data set, for every node in said taxonomy, based on a frequency corresponding to each ancestor node of said node, calculates a data length of a corresponding inverted list in the case of selecting said each ancestor node, and among said each ancestor node, selects preferentially said ancestor node where said data length is small and said ancestor node of a higher order in said taxonomy.

2. In a taxonomy having a tag with respect to search subject data, a data search device, realized by a computer, configured to take out a set of search subject data which can be reached from a specified node specified in said taxonomy comprising:

a processor configured to execute instructions to implement;

an ancestor number inverted list storage part configured to store data for an inverted list where an inverted list of each node in said taxonomy is included, and among said each node, an inverted list of the highest node is a list of integer values indicating an identifier of said search subject data, and an inverted list of a node other than said highest node, in place of said identifier, is a list of integer values indicating a position in an inverted list corresponding to a node that is higher by one than said node, and furthermore, a list of integer values in an inverted list of said each node is divided into two or more blocks, and a differential value between an integer value and an integer value directly before said integer value in said block is converted into a bit string of a variable length integer code; and

an identifier converting part configured to, upon receiving information indicating said specified node, based on said data for an inverted list, create a list of identifiers of said search subject data corresponding to said specified node by repeating processing to take out an integer value of said inverted list corresponding to a higher node of said specified node, which corresponds to said position indicated by an integer value of said inverted list corresponding to said specified node until taking out said identifier of said inverted list corresponding to said highest node, wherein

said data search device, in the case where two or more nodes in said taxonomy are specified, takes out a set of search subject data which can be reached from any of said specified nodes, and

said identifier converting part, in the case where two or more nodes in said taxonomy are specified, acquires said inverted list corresponding to each said specified node from said ancestor number inverted list storage part, and in the case of performing processing to take out an integer value of said inverted list corresponding to a higher node of said specified node, which corresponds to said position indicated by an integer value of said inverted list corresponding to said specified node, when among said higher nodes common in a tuple of said specified nodes, taking out an integer value in said inverted list corresponding to a common ancestor node that is a higher node of the lowest order in said taxonomy, takes out said integer value common in a tuple of said specified nodes, and using taken-out said integer value, creates a list of identifiers of said search subject data corresponding to said two or more specified nodes.

3. The data search device according to claim 2, wherein

said data search device further comprises a frequency list storage part configured to store a frequency list indicating a frequency, of each node in said taxonomy, corresponding to each of search subject data, and

said identifier converting part, further, in a list of identifiers of one or more said search subject data corresponding to said specified node, sorts said identifiers based on said frequency list.

4. The data search device according to claim 2, wherein said processor is further configured to execute instructions to implement:

a child node storage part configured to store child node information indicating a set of a child node that is a lower node of each node in said taxonomy; and

a top down search part configured to, upon receiving a list of identifiers of said search subject data corresponding to said specified nodes, from among each node in said taxonomy, take out a node which has a frequency no less than a prescribed value in a set of said search subject data indicated by said list, and

wherein

said top down search part detects a position of said identifier in said inverted list corresponding to said highest node, and based on said child node information, compares detected said position with an integer value in said inverted list corresponding to a node that is lower by one than said highest node, and at least performs processing to calculate a frequency of said node based on an accordant integer value, and furthermore, with respect to a lower node of said node, compares an integer value corresponding to a position of said identifier in said inverted list corresponding to a higher node with an integer value in said inverted list corresponding to a node that is lower by one than said higher node, and based on an accordant integer value performs processing to calculate a frequency of said node that is lower by one by 0 times or more, and thereby, calculates a frequency of said each node in said search subject data.

5. In a taxonomy having a tag with respect to search subject data, an index creation method configured to create an inverted list used for taking out a set of search subject data which can be reached from each node in said taxonomy, wherein

a list of integer values in an inverted list of said each node is divided into two or more blocks, and a differential value between an integer value and an integer value directly before said integer value in said block is converted into a bit string of a variable length integer code, and

said index creation method comprises:

for every node in said taxonomy, selecting one ancestor node that is a higher node of said node, and generate data for ancestor reference indicating selected said ancestor node;

generating an ancestor node list indicating one or more ancestor nodes of a tag in said taxonomy based on said data for ancestor reference; and

upon receiving an identifier of search subject data, with respect to the highest node among each node in said ancestor node list, adding said identifier as an element of a corresponding inverted list, and with respect to a node other than the highest node, as an element of a corresponding inverted list, in place of said identifier, add an integer value indicating a position in an inverted list corresponding to a node that is higher by one than said node, wherein

in generating said data for ancestor reference, upon receiving a frequency distribution indicating whether each node in said taxonomy has or may have to what extent of a frequency within a prescribed data set, for every node in said taxonomy, based on a frequency corresponding to each ancestor node of said node, a data length of a corresponding inverted list in the case of selecting said each ancestor node is made to be calculated, and among said each ancestor node, said ancestor node where said data length is small and said ancestor node of a higher order in said taxonomy are made to be selected preferentially.

6. In a taxonomy having a tag with respect to search subject data, a data search method configured to take out a set of search subject data which can be reached from a specified node specified in said taxonomy, comprising:

acquiring data for an inverted list where an inverted list of each node in said taxonomy is included, and an inverted list of the highest node among said each node is a list of integer values indicating an identifier of said search subject data, and an inverted list of a node

other than said highest node, in place of said identifier, is a list of integer values indicating a position in an inverted list corresponding to a node that is higher by one than said node, and furthermore, a list of integer values in an inverted list of said each node is divided into two or more blocks, and a differential value between an integer value and an integer value directly before said integer value in said block is converted into a bit string of a variable length integer code; and

upon receiving information indicating said specified node, based on said data for an inverted list, creating a list of identifiers of said search subject data corresponding to said specified node by repeating processing to take out an integer value of said inverted list corresponding to a higher node of said specified node, which corresponds to said position indicated by an integer value of said inverted list corresponding to said specified node until taking out said identifier of said inverted list corresponding to said highest node, wherein

said data search method, in the case where two or more nodes in said taxonomy are specified, takes out a set of search subject data which can be reached from any of said specified nodes, and

in creating a list of said identifiers, in the case where two or more nodes in said taxonomy are specified, acquiring said inverted list corresponding to each said specified node, and in the case of performing processing to take out an integer value of said inverted list corresponding to a higher node of said specified node, which corresponds to said position indicated by an integer value of said inverted list corresponding to said specified node, when among said higher nodes common in a tuple of said specified nodes, taking out an integer value in said inverted list corresponding to a common ancestor node that is a higher node of the lowest order in said taxonomy, takes out said integer value common in a tuple of said specified nodes, and using taken-out said integer value, creates a list of identifiers of said search subject data corresponding to said two or more specified nodes.

7. The data search method according to claim 6, further comprising:

acquiring a frequency list indicating a frequency, of each node in said taxonomy, corresponding to each of search subject data; and

in a list of identifiers of one or more said search subject data corresponding to said specified node, sorting said identifiers based on said frequency list.

8. The data search method according to claim 6, further comprising:

acquiring child node information indicating a set of a child node that is a lower node of each node in said taxonomy; and

upon receiving a list of identifiers of said search subject data corresponding to said specified nodes, from among each node in said taxonomy, taking out a node which has a frequency no less than a prescribed value in a set of said search subject data indicated by said list,

wherein

in taking out said node, positions of said identifier in said inverted list corresponding to said highest node are made to be detected, and based on said child node information, detected said positions are made to be compared with integer values in said inverted list corresponding to a node that is lower by one than said highest node, and processing to calculate a frequency of said node based on accordant integer values is made to be at least performed, and furthermore, with respect to a lower node of said node, integer values corresponding to positions of said identifiers in said inverted list corresponding to a higher node is made to be compared with integer values in said inverted list corresponding to a node that is lower by one than said higher node, and based on accordant integer values, processing to calculate a frequency of said node that is lower by one is made to be performed by 0 times or more, and thereby, a frequency of said each node in said search subject data is made to be calculated.

9. In a taxonomy having a tag with respect to search subject data, a non-transitory computer-readable recording medium in which recorded is an index creation program to create an inverted list used for taking out a set of search subject data which can be reached from each node in said taxonomy,

wherein

a list of integer values in an inverted list of said each node is divided into two or more blocks, and a differential value between an integer value and an integer value directly before said integer value in said block is converted into a bit string of a variable length integer code, and

said index creation program is a program configured to make a computer execute the steps of:

selecting, for every node in said taxonomy, one ancestor node that is a higher node of said node, and generating data for ancestor reference indicating selected said ancestor node;

generating an ancestor node list indicating one or more ancestor nodes of a tag in said taxonomy based on said data for ancestor reference; and

upon receiving an identifier of said search subject data, with respect to the highest node among each said node in said ancestor node list, adding said identifier as a element of a corresponding inverted list, and with respect to a node other than said highest node, as a element of a corresponding inverted list, in place of said identifier, adding an integer value indicating a position in an inverted list corresponding to a node that is higher by one than said node, wherein

in generating said data for ancestor reference, upon receiving a frequency distribution indicating whether each node in said taxonomy has or may have to what extent of a frequency within a prescribed data set, for every node in said taxonomy, based on a frequency corresponding to each ancestor node of said node, a data length of a corresponding inverted list in the case of selecting said each ancestor node is made to be calculated, and among said each ancestor node, said ancestor node where said data length is small and said ancestor node of a higher order in said taxonomy are made to be selected preferentially.

10. In a taxonomy having a tag with respect to search subject data, a non-transitory computer-readable recording medium in which recorded is data search program to take out a set of search subject data which can be reached from a specified node specified in said taxonomy, said data search program being a program configured to make a computer execute the steps of:

acquiring data for an inverted list where an inverted list of each node in said taxonomy is included, and an inverted list of the highest node among said each node is a list of integer values indicating an identifier of said search subject data, and an inverted list of a node other than said highest node, in place of said identifier, is a list of integer values indicating a position in an inverted list corresponding to a node that is higher by one than said node, and furthermore, a list of integer values in an inverted list of said each node is divided into two or more blocks, and a differential value between an integer value and an integer value directly before said integer value in said block is converted into a bit string of a variable length integer code; and

upon receiving information indicating said specified node, based on said data for an inverted list, creating a list of identifiers of said search subject data corresponding to said specified node by repeating processing to take out an integer value of said inverted list corresponding to a higher node of said specified node, which corresponds to said position indicated by an integer value of said inverted list corresponding to said specified node until taking out said identifier of said inverted list corresponding to said highest node, wherein

said data search program, in the case where two or more nodes in said taxonomy are specified, takes out a set of search subject data which can be reached from any of said specified nodes, and

in creating a list of said identifiers, in the case where two or more nodes in said taxonomy are specified, acquiring said inverted list corresponding to each said specified node, and in the case of performing processing to take out an integer value of said inverted list corresponding to a higher node of said specified node, which corresponds to said position indicated by an integer value of said inverted list corresponding to said specified node, when among said higher nodes common in a tuple of said specified nodes, taking out an integer value in said inverted list corresponding to a common ancestor node that is a higher node of the lowest order in said taxonomy, takes out said integer value common in a tuple of said specified nodes, and using taken-out said integer value, creates a list of identifiers of said search subject data corresponding to said two or more specified nodes.

11. The non-transitory computer readable recording medium according to any of claim 10, said data search program being a program configured to make a computer further execute:

acquiring a frequency list indicating a frequency, of each node in said taxonomy, corresponding to each of search subject data; and

sorting, in a list of identifiers of one or more said search subject data corresponding to said specified node, said identifiers based on said frequency list.

12. The non-transitory computer readable recording medium according to any of claim 10, said data search program being a program configured to make a computer further execute:

acquiring child node information indicating a set of a child node that is a lower node of each node in said taxonomy; and

taking out, upon receiving a list of identifiers of said search subject data corresponding to said specified nodes, from among each node in said taxonomy, a node which has a frequency no less than a prescribed value in a set of said search subject data indicated by said list,

wherein

in taking out said node, positions of said identifier in said inverted list corresponding to said highest node are made to be detected, and based on said child node information, detected said positions are made to be compared with integer values in said inverted list corresponding to a node that is lower by one than said highest node, and processing to calculate a frequency of said node based on accordant integer values is made to be at least performed, and furthermore, with respect to a lower node of said node, integer values corresponding to positions of said identifiers in said inverted list corresponding to a higher node is made to be compared with integer values in said inverted list corresponding to a node that is lower by one than said higher node, and based on accordant integer values, processing to calculate a frequency of said node that is lower by one is made to be performed by 0 times or more, and thereby, a frequency of said each node in said search subject data is made to be calculated.