Localized data affinity system and hybrid method

• US 10,509,807 B2
• Filed: 03/31/2017
• Issued: 12/17/2019
• Est. Priority Date: 05/17/2010
• Status: Active Grant

• Current Assignee: United States Postal Service (Government of the United States of America)
• Original Assignee: United States Postal Service (Government of the United States of America)
• Inventors: Atkins, C. Scot, Conway, Joseph
• Primary Examiner(s): Casanova, Jorge A
• Application Number:
  US15/476,105
• Publication Number:
  US 20170206246A1
• Time in Patent Office: 991 Days
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/24552 Database cache management
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 2209/5012 Processor sets
• CPC Class Codes: G06F 9/5033 considering data affinity

Abstract

A method, system, and computer program for processing records is disclosed. In some aspects, a method includes associating, on at least one of the plurality of processors, each record with a record set of a plurality of record sets. Each record set is assigned to a sub-database based on the record set. A cache is associated with each sub-database, and each sub-database and its associated cache is associated with a processor set. An affinity is created between each database cache and the associated processor set, and records are processed with the processor sets according to the associations.

Claims

1. A method of processing records in a database on a plurality of processors grouped into a plurality of physically separate processor sets, the method comprising:

associating, on at least one of the plurality of processors, each record with a record set of a plurality of record sets;

assigning each record set to a sub-database based on the record set;

associating a cache with each sub-database;

associating each sub-database and its associated cache with a processor set;

creating an affinity between each database cache and the associated processor set; and

processing the record sets with the processor sets according to the associations between the record sets, sub-databases, and the processor sets.

2. The method of claim 1, further comprising:

creating a logging cache for each processor set of the plurality of processor sets or a single logging cache for the plurality of processors;

creating an affinity between each logging cache and an associated processor set; and

assigning a single processor in each processor set to log database transactions,

wherein each processor set comprises a plurality of processors.

3. The method of claim 1, further comprising:

creating a plurality of server processes, wherein each server process is associated with a processor set of the plurality of processor sets;

creating an affinity between each server process and the associated processor set; and

associating each server process with a record set,

wherein each server process processes records in the associated record set.

4. The method of claim 1, wherein routing records comprises routing records to processing sets based on a static routing table.

5. The method of claim 1, wherein associating records comprises associating records with record sets such that there is no data dependency between record sets when processing records.

6. An apparatus comprising a plurality of processors grouped into a plurality of physically separate processor sets configured to process records in a database, wherein at least one of the physically separate processor sets is configured to:

associate each record with a record set of a plurality of record sets;

assign each record set to a sub-database based on the record set;

associate a cache with each sub-database;

associate each sub-database and its associated cache with a processor set;

create an affinity between each database cache and the associated processor set; and

process the records with the processor sets according to the associations between the record sets, sub-databases, and the processor sets.

7. The apparatus of claim 6, wherein the apparatus is further configured to:

create a logging cache for each processor set of the plurality of processor sets or a single logging cache for the plurality of processors;

create an affinity between each logging cache and an associated processor set; and

assign a single processor in each processor set to log database transactions,

wherein each processor set comprises a plurality of processors.

8. The apparatus of claim 6, wherein the apparatus is further configured to:

create a plurality of server processes, wherein each server process is associated with a processor set of the plurality of processor sets;

create an affinity between each server process and the associated processor set; and

associate each server process with a record set,

wherein each server process processes records in the associated record set.

9. The apparatus of claim 6, wherein the apparatus is further configured to route records to processing sets based on a static routing table.

10. The apparatus of claim 6, wherein the apparatus is further configured to associate records with record sets such that there is no data dependency between record sets when processing records.

11. A computer-readable non-transitory storage medium comprising code capable of causing a computer to:

associating, on at least one of a plurality of processors, each record with a record set of a plurality of record sets;

assigning each record set to a sub-database based on the record set;

associating a cache with each sub-database;

associating each sub-database and its associated cache with a processor set;

creating an affinity between each database cache and the associated processor set; and

processing the records with the processor sets according to the associations between the record sets, sub-databases, and the processor sets.

12. The computer-readable non-transitory storage medium of claim 11, further comprising code capable of causing a computer to:

create a logging cache for each processor set of the plurality of processor sets or a single logging cache for the plurality of processors;

create an affinity between each logging cache and an associated processor set; and

assign a single processor in each processor set to log database transactions,

wherein each processor set comprises a plurality of processors.

13. The computer-readable non-transitory storage medium of claim 11, further comprising code capable of causing a computer to:

create a plurality of server processes, wherein each server process is associated with a processor set of the plurality of processor sets;

create an affinity between each server process and the associated processor set; and

associate each server process with a record set,

wherein each server process processes records in the associated record set.

14. The computer-readable non-transitory storage medium of claim 11, further comprising code capable of causing a computer to route records to processing sets based on a static routing table.

15. The computer-readable non-transitory storage medium of claim 11, further comprising code capable of causing a computer to associate records with record sets such that there is no data dependency between record sets when processing records.