PARTITIONED DATABASE MODEL TO INCREASE THE SCALABILITY OF AN INFORMATION SYSTEM

• US 20120303628A1
• Filed: 05/24/2011
• Published: 11/29/2012
• Est. Priority Date: 05/24/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Silvola, Brian
• Granted Patent: US 9,507,816 B2
• Time in Patent Office: Days
• US Class Current: 707/741
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof

Abstract

A database includes data tables and indexes that are partitioned. Searches against the data table are performed in parallel over the multiple partitions. The indexes on each partition maintain indexes associated with the data on the given partition. Data tables storing string data include a string data file and index files for each word stored in the string data file.

Claims

1. A computer implemented method for managing a database that is configured for use with a processing system, the processing system performing the method comprising:

maintaining a structured organization of data over a plurality of partitions within the database, each one of the plurality of partitions having a size limit;

inserting new data into the structured organization of data;

automatically adding a new partition when the inserted new data results in a size of the one of the plurality of partitions meeting or exceeding the respective size limit; and

indexing the data within each one of the plurality of partitions.

2. The method of claim 1, wherein the processing system includes a number of processors or cores of processors that is at least equal to a number of the plurality of partitions, each one of the plurality of partitions associated with a respective core or processor of the processing system.

3. The method of claim 1, wherein the automatically added new partition is associated with a new processor or core.

4. The method of claim 1, further comprising executing a search criteria against the structured organization of data, wherein the executed search includes executing at least one search task against each of the partitions, in parallel with each other, based on the search criteria.

5. The method of claim 4, further comprising aggregating the results of each one of the executed search tasks.

6. The method of claim 1, wherein the data of the structured organization is string data, the structured organization further including,

for each partition,

a string file configured to include plurality of strings, the string file further including a table referencing each one of the strings within the respective string file, and

a plurality of index files, each one of the plurality of index files related to a word from the plurality of strings in the string file, each one of the plurality of index files storing index information that references the associated word within the string file.

7. The method of claim 6, further comprising:

executing a string search criteria against the structured organization of string data, wherein the executed search includes executing at least one string search task against each of the partitions, in parallel, based on the string search criteria, the at least one string search task including;

     1) locating at least one of the plurality of index files relating to the search criteria; and

     2) reading index information from the located index file.

8. The method of claim 7, further comprising aggregating the index information from each one of the executed string search tasks.

9. The method of claim 7, further comprising retrieving a word from the string file based on the read index information, the index information including a recordID and a position reference to the word within the string.

10. The method of claim 7, further comprising maintaining an open file stream to the string file over multiple accesses to the string file.

11. The method of claim 7, dividing each one of the plurality of index files into multiple subindex files when a number of index information for the associated word in the index file is greater than a predetermined amount, each one of the multiple subindexes storing a subset of the index information.

12. The method of claim 1, wherein at least a subset of the data in the structured organization is stored in a cache that has a determined size.

13. The method of claim 12, wherein a size of the cache is configurable by a user.

14. The method of claim 12, further comprising adding new cache data to the cache when the new cache data does not already exist in the cache.

15. The method of claim 14, further comprising dequeing already cached data from the cache before the new cache data is added to the queue when the size of the cache meets or exceeds a determined size.

16. The method of claim 15, wherein the determined size is based on exceeding an amount of memory associated with processing system that is allocated to the cache.

17. A database-management system for managing a database, the database-management system comprising:

a processing system configured to;

maintain a structured organization of data over a plurality of partitions within the database, each one of the plurality of partitions having a size limit;

insert new data into the structured organization of data;

automatically add a new partition when the inserted new data results in a size of the one of the plurality of partitions meeting or exceeding the respective size limit; and

maintain an index on each of the partitions for the data located on the associated partition.

18. The system of claim 17, wherein the data of the structured organization is string data, the structured organization further including,

for each partition,

a string file configured to include plurality of strings, the string file further including a table pointing to each one of the strings within the respective string file, and

wherein the index includes a plurality of index files, each one of the plurality of index files relating to a word in the string file, each one of the plurality of index files storing a record index and a position index of the associated word within the string file.

19. The system of claim 18, wherein the processing system is further configured to execute a string search criteria against the structured organization of string data, wherein the string search criteria includes at least one string search task configured to be executed in association with each of the partitions, in parallel, based on the string search criteria, each string search task further configured to:

locate at least one of the plurality of index files based on the related word of the at least one of the plurality of index files and the search criteria;

identify at least one record index and at least one position index within the at least one of the plurality of index files; and

retrieve at least one string from the string file based on the identified at least one record index and identified at least one position index.

20. A non-transitory computer readable storage medium storing computer-readable instructions for performing a string search against a database system storing string data over a plurality of partitions, each one of the partitions including:

1) a string file configured to include at least some of the string data, and a table referencing to each one of the strings within the respective string file,

     2) a plurality of index files, each one of the plurality of index files associated with a word or words within the string file of the respective partition, each one of the plurality of index files storing a reference an instance of the associated word, the database system including at least one processor, the stored instructions comprising instructions configured to;

execute a search, in parallel, over the multiple partitions, that is related to at least one word that is within the string data;

locate at least one of the plurality of index files related to the at least one word;

read a reference from the located at least one of the plurality of index files; and

retrieve at least one string and/or word from the string file based on the read reference.