Database controller, method, and program for handling range queries

• US 9,852,182 B2
• Filed: 05/23/2014
• Issued: 12/26/2017
• Est. Priority Date: 05/29/2013
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Hu, Bo, Carvalho, Nuno
• Primary Examiner(s): Peach, Polina
• Assistant Examiner(s): Mueller, Kurt
• Application Number:
  US14/285,847
• Publication Number:
  US 20140358949A1
• Time in Patent Office: 1,313 Days
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2455 Query execution

Abstract

A database controller for a database of data items distributed across a plurality of storage units. The database controller configured to store an ordered list comprising, for each of the data items individually, the identifying information of the storage unit upon which the data item is stored. The database controller configured to maintain a frequency record, recording the values of prefix portions of the data items and the number of data items having each prefix portion. Wherein, the ordered list and the frequency record are ordered by applying the same ordering metric to the prefix portions. The database controller being configured to receive a range query specifying a search prefix portion, to use the frequency record to calculate where to find in the ordered list the identifying information of storage units storing the data items having the search prefix, and using that identifying information to retrieve query results.

Claims

1. A database controller for a database of information encoded as a set of data items distributed among a plurality of storage units distinguishable from one another by respective identifying information and each data item comprising a value of each of two or more data elements, the set of data items having a prescribed order of the two or more data elements of which the first one or two data elements are assigned as a prefix portion, the database controller comprising computing hardware including a processor and a memory coupled to the processor and storing processing instructions which when executed by the processor cause the processor to execute a process, the process comprising:

receiving a range query specifying a value of the or each of one or more data elements as a search prefix portion which each data item returned as a range query result must include in its prefix portion;

storing an ordered list comprising, for each of the data items from the set of data items individually, the identifying information of a storage unit upon which the data item is stored, the order of the identifying information being determined by an ordering metric applied to prefix portions of the respective data items;

maintaining a frequency record, recording values of the data elements assigned as prefix portions of the set of data items and the number of data items having each prefix portion, the frequency record being maintained in an order determined by applying the ordering metric to the prefix portion of the data items;

calculating, based on the frequency record, one or more list positions in the ordered list at which the identifying information of the plurality of storage units upon which data items including the search prefix portion in their prefix portion are stored, comprising calculating a start point of the one or more list positions by summing the numbers of data items having prefix portions preceding the search prefix portion in the order determined by the ordering metric, and calculating the end point of the one or more list positions by adding the number of data items having prefix portions including the search prefix portion to the start point; and

requesting the data items including the search prefix portion in their prefix portion from the plurality of storage units identified in the calculated one or more list positions, wherein the ordered list is periodically updated based on information received from the plurality of storage units, and, after each update, each occupied list position in the ordered list is succeeded by a predetermined number of vacant list positions; and

the process further comprises, when a new data item is added to the set of data items or a duplicate set, adding the identifying information of a storage unit upon which the new data item is stored to a vacant list position in the respective ordered list, the vacant list position being determined by applying the ordering metric to the prefix portion of the new data item in relation to the prefix portions of the respective occupied list positions by referring to the frequency record, such that the immediately preceding occupied list position stores the identifying information of a data item having a prefix preceding or equal to the prefix of the new data item according to the ordering metric, and the immediately following occupied list position stores the identifying information of a data item having a prefix following or equal to the prefix of the new data item according to the ordering metric.

2. A database controller according to claim 1, wherein the set of data items is duplicated one or more times, with each duplicate set storing the same database of information as the set of data items and having a different fixed order and different one or two data elements assigned as a prefix portion from that of the set of data items;

the process further comprising;

storing an ordered list for each duplicate set, the ordered list comprising, for each of the data items from the duplicate set of data items individually, the identifying information of the storage unit upon which the duplicate data item is stored, the order of the identifying information being determined by an ordering metric applied to the prefix portions of the respective duplicate data items; and

maintaining a frequency record for each duplicate set, recording the prefix portions of the duplicate set and the number of duplicate data items having each prefix.

3. A database controller according to claim 2, wherein there are one or two duplicate data item sets, the set of data items and the one or two duplicate sets have different data elements from one another in each position of the respective fixed orders of data elements.

4. A database controller according to claim 2, wherein the calculating the one or more list positions in the ordered list includes selecting the frequency record for one of the set of data items or the one or more duplicate sets such that the data elements of which values are recorded in the selected frequency record are the same data elements for which values are specified in the search prefix portion.

5. A database controller according to claim 1, wherein the process further includes storing, for each of the one or more frequency records, a prefix search trie for one of the data elements of which values are recorded included in the prefix portions of the frequency record, and for each prefix search trie, a table corresponding to each leaf;

the prefix search trie having a common root for all different values of the data element and a leaf corresponding to each different value of the data element appearing in the data item set or duplicate set;

each leaf stores a number representing the number of data items in the set of data items set or duplicate set having the corresponding value of the data element; and

the table corresponding to each leaf stores a record of values of the other data element in the prefix portion with which the value corresponding to the leaf appears, and for each value of the other data element, a co-occurrence number representing the number of data items in the set of data items or duplicate set having a prefix portion comprising the value corresponding to the leaf and the value of the other data element.

6. A database controller according to claim 5, wherein the table corresponding to each leaf is a hash table.

7. A database controller according to claim 1, wherein the process further comprises:

controlling allocation of data items to the plurality of storage units, and updating the allocation of data items to the plurality of storage units; and

updating the one or more ordered lists and the one or more frequency records to reflect the update.

8. A database controller according to claim 1, wherein the prefix portion is the first two of the three data elements, and the search prefix portion is one or two specified data elements.

9. A database controller according to claim 1, wherein the database of information is a graph database and the set of data items encoding the graph database is a set of triples each comprising a value of each of three triple elements.

10. A data storage system according to claim 1, comprising a database controller, and the plurality of storage units.

11. A method for obtaining range query results from a database of information encoded as a set of data items distributed among a plurality of storage units distinguishable from one another by respective identifying information and each data item comprising a value of each of three data elements, the set of data items having a prescribed order of the three data elements of which the first one or two data elements are assigned as a prefix portion, the plurality of storage units being connected to a database controller storing:

an ordered list comprising, for each of the data items from the set of data items individually, the identifying information of a storage unit upon which the data item is stored, the order of the identifying information being determined by an ordering metric applied to prefixes of the respective data items; and

a frequency record, recording values of the data elements assigned as prefix portions of the set of data items and the number of data items having each prefix portion, wherein the frequency record is in an order determined by applying the ordering metric to the prefix portions of the data items;

the method comprising;

receiving a range query specifying a value of the or each of one or more data elements as a search prefix portion which each data item returned as a range query result must include in its prefix portion;

calculating, based on a frequency record, the one or more list positions in the ordered list at which the storage unit IDs of the plurality of storage units upon which data items including the search prefix portion in their prefix portion are stored, comprising calculating a start point of the one or more list positions by summing the numbers of data items having prefix portions preceding the search prefix portion in the order determined by the ordering metric, and calculating the end point of the one or more list positions by adding the number of data items having prefix portions including the search prefix portion to the start point; and

requesting data items including the search prefix portion in their prefix portion from the plurality of storage units identified in the calculated one or more list positions, wherein the ordered list is periodically updated based on information received from the plurality of storage units, and, after each update, each occupied list position in the ordered list is succeeded by a predetermined number of vacant list positions; and

the method further comprises, when a new data item is added to the set of data items or a duplicate set, adding the identifying information of the storage unit upon which the new data item is stored to a vacant list position in the respective ordered list, the vacant list position being determined by applying the ordering metric to the prefix portion of the new data item in relation to the prefix portions of the respective occupied list positions by referring to the frequency record, such that the immediately preceding occupied list position stores the identifying information of a data item having a prefix preceding or equal to the prefix of the new data item according to the ordering metric, and the immediately following occupied list position stores the identifying information of a data item having a prefix following or equal to the prefix of the new data item according to the ordering metric.

12. A non-transitory storage medium storing a computer program which, when executed by one or more computing apparatuses, causes the one or more computing apparatuses to perform a method for obtaining range query results from a database of information encoded as a set of data items distributed among a plurality of storage units distinguishable from one another by respective identifying information and each data item comprising a value of each of three data elements, the set of data items having a prescribed order of the three data elements of which the first one or two data elements are assigned as a prefix portion, the plurality of storage units being connected to a database controller storing:

an ordered list comprising, for each of the data items from the set of data items individually, the identifying information of the storage unit upon which the data item is stored, the order of the identifying information being determined by an ordering metric applied to the prefixes of the respective data items; and

a frequency record, recording the values of the data elements assigned as prefix portions of the set of data items and the number of data items having each prefix portion, wherein the frequency record is in an order determined by applying the ordering metric to the prefix portions of the data items;

the method comprising;

receiving a range query specifying a value of the or each of one or more data elements as a search prefix portion which each data item returned as a range query result must include in its prefix portion;

calculating, based on a frequency record, the one or more list positions in the ordered list at which the storage unit IDs of the plurality of storage units upon which data items including the search prefix portion in their prefix portion are stored, comprising calculating a start point of the one or more list positions by summing the numbers of data items having prefix portions preceding the search prefix portion in the order determined by the ordering metric, and calculating the end point of the one or more list positions by adding the number of data items having prefix portions including the search prefix portion to the start point; and

requesting data items including the search prefix portion in their prefix portion from the plurality of storage units identified in the calculated one or more list positions, wherein the ordered list is periodically updated based on information received from the plurality of storage units, and, after each update, each occupied list position in the ordered list is succeeded by a predetermined number of vacant list positions; and

the method further comprises, when a new data item is added to the set of data items or a duplicate set, adding the identifying information of a storage unit upon which the new data item is stored to a vacant list position in the respective ordered list, the vacant list position being determined by applying the ordering metric to the prefix portion of the new data item in relation to the prefix portions of the respective occupied list positions by referring to the frequency record, such that the immediately preceding occupied list position stores the identifying information of a data item having a prefix preceding or equal to the prefix of the new data item according to the ordering metric, and the immediately following occupied list position stores the identifying information of a data item having a prefix following or equal to the prefix of the new data item according to the ordering metric.