Method of indexed storage and retrieval of multidimensional information

• US 7,882,106 B2
• Filed: 08/10/2007
• Issued: 02/01/2011
• Est. Priority Date: 09/28/1999
• Status: Expired due to Fees

• Current Assignee: University of Tennessee Research Foundation (University of Tennessee)
• Original Assignee: University of Tennessee Research Foundation (University of Tennessee)
• Inventors: Birdwell, John D., Wang, Tse-Wei, Horn, Roger D., Yadav, Puneet, Icove, David J.
• Primary Examiner(s): Wong, Don
• Assistant Examiner(s): Nguyen, Merilyn P
• Application Number:
  US11/836,836
• Publication Number:
  US 20080172402A1
• Time in Patent Office: 1,271 Days
• Field of Search: 707/736, 707/737, 707/740, 707/748, 707/752
• US Class Current: 707/736
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/2264 Multidimensional index structures
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G16B 40/00 ICT specially adapted for biostatistics; ICT specially adapted for bioinformatics-related machine learning or data mining, e.g. knowledge discovery or pattern finding
• CPC Class Codes: G16B 40/30 Unsupervised data analysis
• CPC Class Codes: G16B 50/00 ICT programming tools or database systems specially adapted for bioinformatics
• CPC Class Codes: G16B 50/20 Heterogeneous data integration
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A tree-structured index to multidimensional data is created using naturally occurring patterns and clusters within the data which permit efficient search and retrieval strategies in a database of DNA profiles. A search engine utilizes hierarchical decomposition of the database by identifying clusters of similar DNA profiles and maps to parallel computer architecture, allowing scale up past previously feasible limits. Key benefits of the new method are logarithmic scale up and parallelization. These benefits are achieved by identification and utilization of naturally occurring patterns and clusters within stored data. The patterns and clusters enable the stored data to be partitioned into subsets of roughly equal size. The method can be applied recursively, resulting in a database tree that is balanced, meaning that all paths or branches through the tree have roughly the same length. The method achieves high performance by exploiting the natural structure of the data in a manner that maintains balanced trees. Implementation of the method maps naturally to parallel computer architectures, allowing scale up to very large databases.

Claims

1. A computer-implemented method of partitioning data records of a multi-dimensional database into groups comprising:

defining a function of a distribution of values of a designated variable associated with the data records, the data records being stored in computer processor memory, wherein the function comprises a combination of measures of entropy and adjacency utilized by a computer processor associated with said computer processor memory, adjacency being weighted by a weighting factor and

partitioning the values of the designated variable into two or more groups by the computer processor according to the function.

2. A computer-implemented method as recited in claim 1 wherein a value of said function is determined by applying an optimization procedure.

3. A computer-implemented method as recited in claim 1 wherein the partitioning of the values into two or more groups is determined responsive to an optimization procedure.

4. A computer-implemented method as recited in claim 1 further comprising assigning a data record to a group according to the values of the designated variable.

5. A computer-implemented method of partitioning data records of a multi-dimensional database of a computer processor memory into groups of approximately equal size, comprising the steps of:

(a) defining a function of a distribution of values of a designated variable associated with the data records, wherein the function comprises a combination of measures of entropy and adjacency utilized by a first and a second computer processor, adjacency being weighted by a weighting factor;

(b) partitioning the values of the designated variable into two or more groups by one of said first and second computer processors, wherein a value of the function is determined by applying an optimization procedure; and

(c) assigning a data record to a group in said computer processor memory according to the values of the designated variable.

6. A computer-implemented method as recited in claim 5 further comprising selecting a partition from a set of computed solutions yielding acceptable performance.

7. A computer-implemented method as recited in claim 5 wherein said optimization procedure results in an optimal assignment.

8. A computer-implemented method as recited in claim 5 wherein said combination is linear.

9. A computer-implemented method as recited in claim 5 wherein the designated variable simultaneously comprises a plurality of values.

10. The computer-implemented method of claim 5 wherein said first and second computer processor comprise a parallel data processor.

11. The computer-implemented method of claim 5 wherein said first computer processor comprises a computer processor of a first server and said second computer processor comprises a computer processor of a second server.

12. The computer-implemented method of claim 5 wherein said first computer processor communicates with said second communications processor.

13. The computer-implemented method of claim 5 wherein said first computer processor balances load with said second communications processor.

14. A parallel data processing system comprising first and second computer processors for implementing a method of partitioning data records of a multi-dimensional database into groups comprising:

defining a function of a distribution of values of a designated variable associated with the data records, wherein the function comprises a combination of measures of entropy and adjacency, adjacency being weighted by a weighting factor and

partitioning the values of the designated variable into two or more groups for storage in computer processor memory.

15. A parallel data processing system as recited in claim 14 comprising first and second panels each comprising first and second computer processors, each of said first and second panels having a bus control module, the bus control module of said first and second panels being linked to a bus control module of a control panel, the control panel having at least one control host computer processor linked to said bus control module of said control panel.

16. The parallel data processing system as recited in claim 14 wherein said first and second computer processor comprise a parallel data processor.

17. The parallel data processing system as recited in claim 14 wherein said first computer processor comprises a computer processor of a first server and said second computer processor comprises a computer processor of a second server.

18. The parallel data processing system as recited in claim 14 wherein said first computer processor communicates with said second communications processor.

19. The parallel data processing system as recited in claim 14 wherein said first computer processor balances load with said second communications processor.

20. The parallel data processing system as recited in claim 14 where said first computer processor comprises a computer processor of a server and said second computer processor comprises a client of said server.