Method and system for database storage management

• US 9,720,927 B2
• Filed: 07/12/2012
• Issued: 08/01/2017
• Est. Priority Date: 07/12/2011
• Status: Active Grant

• Current Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford University)
• Original Assignee: Board of Trustees of the Leland Stanford Junior University (Stanford University)
• Inventors: Mohapatra, Abhijeet, Genesereth, Michael
• Primary Examiner(s): Liao, Jason
• Assistant Examiner(s): Brooks, David T
• Application Number:
  US13/548,105
• Publication Number:
  US 20130097127A1
• Time in Patent Office: 1,846 Days
• Field of Search: None
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/901 Indexing; Data structures therefor; Storage structures for retrieval from the web G06F16/951

Abstract

Embodiments of the present invention relate to run-length encoded sequences and supporting efficient offset-based updates of values while allowing fast lookups. In an embodiment of the present invention, an indexing scheme is disclosed, herein called count indexes, that supports O(log n) offset-based updates and lookups on a run-length sequence with n runs. In an embodiment, count indexes of the present invention support O(log n) updates on bitmapped sequences of size n. Embodiments of the present invention can be generalize to be applied to block-oriented storage systems.

Claims

1. A computer-implemented method for performing database storage management, comprising:

encoding a first sequence of information comprising a compression of at least one successive repetition of values in the first sequence of information;

calculating a maximum number of child nodes based on a number of groups to group the encoded first sequence of information;

grouping the encoded first sequence of information into sets of leaf nodes, each set having a number of leaf nodes that are no more than the calculated maximum number of child nodes, wherein each leaf node includes an identification of a repeated value and a count of the repetition of the repeated value; and

generating at least one hierarchical node from the sets of leaf nodes wherein each of the hierarchical node at each level of hierarchy has a number of child nodes that are no more than the calculated maximum number of child nodes and maintains a sum of the count of the repetition of the repeated value of each descendant leaf node of the hierarchical node; and

generating a count index for the encoded first sequence of information based on the sets of leaf nodes and the hierarchical nodes by;

for each piece of information in the encoded sequence of information;

determining a first index location based on the number of groups, where the index location corresponds to a first hierarchical node;

setting the first hierarchical node as the parent node;

creating a pointer defining a relationship between the parent node and at least one leaf node, where the number of leaf nodes corresponds to the maximum number of child nodes for the first hierarchical node, thereby creating a subgroup of related leaf nodes within the sets of leaf nodes; and

updating the first index location to correspond to the next hierarchical node in the at least one hierarchical node in the set of leaf nodes; and

recursively generating the count index for each subgroup of related leaf nodes.

2. The method of claim 1, further comprising looking up at least one item of the first sequence of information by traversing the at least one hierarchical node.

3. The method of claim 2, wherein looking up at least one item of the first sequence of information includes the steps of

receiving position information for the at least one item of information;

determining branches of the at least one hierarchical node corresponding to the position information;

determining a node of the lowest level of hierarchy corresponding to the at least one item of the first sequence of information.

4. The method of claim 3, wherein the position information includes offset information.

5. The method of claim 3, wherein looking up the at least one item of the first sequence of information is performed in O(log n) computation steps where n is the number of items of information in the first sequence of information.

6. The method of claim 1, further comprising making at least one change to at least one leaf node.

7. The method of claim 6, wherein the at least one change includes changing at least one count of the repetition of the repeated value.

8. The method of claim 6, wherein the at least one change includes inserting at least one new leaf node.

9. The method of claim 6;

wherein the at least one change includes deleting at least one leaf node.

10. The method of claim 1, wherein the maximum number of child nodes is based on the number of sequences in the encoded first sequence of information.

11. A non-transitory computer-readable medium including instructions that, when executed by a processing unit, cause the processing unit to perform database storage management, by performing the steps of:

encoding a first sequence of information comprising a compression of at least one successive repetition of values in the first sequence of information;

calculating a maximum number of child nodes based on a number of groups to group the encoded first sequence of information;

grouping the encoded first sequence of information into sets of leaf nodes, each set having a number of leaf nodes that are no more than the calculated maximum number of child nodes, wherein each leaf node includes an identification of a repeated value and a count of the repetition of the repeated value; and

generating at least one hierarchical node from the sets of leaf nodes wherein each of the hierarchical node at each level of hierarchy has a number of child nodes that are no more than the calculated maximum number of child nodes and maintains a sum of the count of the repetition of the repeated value of each descendant leaf node of the hierarchical node; and

generating a count index for the encoded first sequence of information based on the sets of leaf nodes and the hierarchical nodes by;

for each piece of information in the encoded sequence of information;

determining a first index location based on the number of groups, where the index location corresponds to a first hierarchical node;

setting the first hierarchical node as the parent node;

creating a pointer defining a relationship between the parent node and at least one leaf node, where the number of leaf nodes corresponds to the maximum number of child nodes for the first hierarchical node, thereby creating a subgroup of related leaf nodes within the sets of leaf nodes; and

updating the first index location to correspond to the next hierarchical node in the at least one hierarchical node in the set of leaf nodes; and

recursively generating the count index for each subgroup of related leaf nodes.

12. The non-transitory computer-readable medium of claim 11, further comprising looking up at least one item of the first sequence of information by traversing the at least one hierarchical node.

13. The non-transitory computer-readable medium of claim 12, wherein looking up at least one item of the first sequence of information includes the steps of

receiving position information for the at least one item of information;

determining branches of the at least one hierarchical node corresponding to the position information;

determining a node of the lowest level of hierarchy corresponding to the at least one item of the first sequence of information.

14. The non-transitory computer-readable medium of claim 13, wherein the position information includes offset information.

15. The non-transitory computer-readable medium of claim 13, wherein looking up the at least one item of the first sequence of information is performed in O(log n) computation steps where n is the number of items of information in the first sequence of information.

16. The non-transitory computer-readable medium of claim 11, further comprising making at least one change to at least one leaf node.

17. The non-transitory computer-readable medium of claim 16, wherein the at least one change includes changing at least one count of the repetition of the repeated value.

18. The non-transitory computer-readable medium of claim 16, wherein the at least one change includes inserting at least one new leaf node.

19. The non-transitory computer-readable medium of claim 16, wherein the at least one change includes deleting at least one leaf node.

20. A computing device comprising:

a data bus;

a memory unit coupled to the data bus;

a processing unit coupled to the data bus and directed to;

encode a first sequence of information comprising a compression of at least one successive repetition of values in the first sequence of information;

calculate a maximum number of child nodes based on a number of groups to group the encoded first sequence of information;

group the encoded first sequence of information into sets of leaf nodes, each set having a number of leaf nodes that are no more than the calculated maximum number of child nodes, wherein each leaf node includes an identification of a repeated value and a count of the repetition of the repeated value; and

generate at least one hierarchical node from the sets of leaf nodes wherein each of the hierarchical node at each level of hierarchy has a number of child nodes that are no more than the calculated maximum number of child nodes and maintains a sum of the count of the repetition of the repeated value of each descendant leaf node of the hierarchical node; and

generate a count index for the encoded first sequence of information based on the sets of leaf nodes and the hierarchical nodes by;

for each piece of information in the encoded sequence of information;

determining a first index location based on the number of groups, where the index location corresponds to a first hierarchical node;

setting the first hierarchical node as the parent node;

creating a pointer defining a relationship between the parent node and at least one leaf node, where the number of leaf nodes corresponds to the maximum number of child nodes for the first hierarchical node, thereby creating a subgroup of related leaf nodes within the sets of leaf nodes; and

updating the first index location to correspond to the next hierarchical node in the at least one hierarchical node in the set of leaf nodes; and

recursively generating the count index for each subgroup of related leaf nodes.