BATCHING REQUESTS FOR ACCESSING DIFFERENTIAL DATA STORES

• US 20100281077A1
• Filed: 04/30/2009
• Published: 11/04/2010
• Est. Priority Date: 04/30/2009
• Status: Abandoned Application

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Deolalikar, Vinay, Lillibridge, Mark David, Eshghi, Kave
• Application Number:
  US12/432,804
• Publication Number:
  US 20100281077A1
• Time in Patent Office: Days
• US Class Current: 707/812
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Data objects are selectively stored across a plurality of differential data stores, where selection of the differential data stores for storing respective data objects is according to a criterion relating to compression of the data objects in each of the data stores, and where the differential data stores are stored in persistent storage media. Plural requests for accessing the differential data stores are batched, and one of the differential data stores is selected to page into temporary storage from the persistent storage media. The batched plural requests for accessing the selected differential data store that has been paged into the temporary storage are executed.

Claims

1. A method, comprising:

selectively storing data objects across a plurality of differential data stores, wherein selection of the differential data stores for storing respective data objects is according to a criterion relating to compression of the data objects in each of the data stores, and wherein the differential data stores are stored in persistent storage media;

batching plural requests for accessing at least one of the differential data stores;

selecting one of the differential data stores to page into temporary storage from the persistent storage media; and

executing the batched plural requests for accessing the selected differential data store that has been paged into the temporary storage.

2. The method of claim 1, further comprising storing the plurality of differential data stores in the persistent storage media of one physical storage system.

3. The method of claim 1, further comprising providing multiple physical storage systems, wherein the persistent storage media are located in the multiple physical storage systems, wherein at least one of the physical storage systems includes multiple ones of the plurality of differential data stores.

4. The method of claim 3, further comprising:

routing the plural requests to selected ones of the multiple physical storage systems using a routing algorithm to enhance compression of data objects in the data stores; and

performing routing, in a particular one of the selected ones of the multiple physical storage systems, of requests received by the particular physical storage system to selected ones of the multiple differential data stores stored in the persistent storage media of the particular physical storage system to enhance compression of data objects stored in the data stores stored in the persistent storage media of the particular physical storage system.

5. The method of claim 1, wherein at least some of the plural requests comprise write requests, the method further comprising:

writing data objects associated with the write requests for accessing the selected differential data store into the selected differential data store that has been paged into the temporary storage.

6. The method of claim 1, wherein at least some of the plural requests comprise read requests, the method further comprising:

sorting the read requests according to differential data stores to be accessed by the respective read requests; and

paging a first set of the differential data stores from the persistent storage media into the temporary storage, wherein the sorted read requests for accessing the differential data stores in the first set are satisfied from the first set of the differential data stores while they are in the temporary storage.

7. The method of claim 6, further comprising replacing in the temporary storage the first set of the differential data stores with a second set of differential data stores, wherein the sorted read requests for accessing the differential data stores in the second set are satisfied from the second set of differential data stores while they are in the temporary storage.

8. The method of claim 6, wherein the read requests are received from one or more clients, the method further comprising:

responding to the received read requests with responsive data objects read from the differential data stores in an order different from an order in which the read requests are received.

9. The method of claim 6, wherein each read request includes an identifier of a file, the method further comprising:

translating the identifier of the file into one or more retrieval identifiers, wherein each of the retrieval identifiers identifies a respective data store.

10. The method of claim 9, wherein the differential data stores use chunk-based deduplication.

11. A storage system comprising:

a temporary storage;

a persistent storage to store a plurality of differential data stores;

a processor to;

receive requests to access differential data stores;

route the requests using a routing algorithm to the differential data stores, wherein the routing algorithm is to enhance compression of data objects stored in each of the differential data stores;

collect at least one group of the received requests;

select one of the differential data stores to page from the persistent storage into the temporary storage; and

execute the group of requests for accessing the selected differential data store that has been paged into the temporary storage.

12. The storage system of claim 11, wherein the processor is to further perform chunk-based deduplication in the differential data stores.

13. The storage system of claim 11, wherein the received requests include write requests, the storage system further comprising inbox buffers associated with corresponding ones of the differential data stores,

wherein the processor is to further:

route the write requests into respective inbox buffers to collect respective batches of the write requests; and

execute write requests in a corresponding one of the inbox buffers against the differential data store paged into the temporary storage.

14. The storage system of claim 13, wherein the received requests further include delete requests, and

wherein the processor is to further:

route the delete requests into respective inbox buffers; and

execute the delete requests in a corresponding one of the inbox buffers against the selected differential store paged into the temporary storage.

15. The storage system of claim 11, wherein the received requests include read requests, and

wherein the processor is to further:

map a file identifier in each of the read requests into corresponding retrieval identifiers that each identify a corresponding differential data store; and

sort the retrieval identifiers according to respective differential data stores,

wherein the sorted read requests are collected into the at least one group.

16. A system comprising:

a plurality of storage systems, wherein each of the storage systems includes persistent storage, wherein the persistent storage of at least a first one of the storage systems is to store a plurality of differential data stores, wherein the storage systems are to receive requests issued by one or more client computers, and wherein the requests are routed to corresponding one or more of the storage systems,

wherein the first storage system includes a temporary storage and a processor to collect requests received by the first storage system into one or more groups, and the processor is to select the differential data stores according to a scheduling algorithm to page into the temporary storage, and to execute one of the one or more groups of requests against a first differential data store paged into the temporary storage.

17. The system of claim 16, further comprising a portal that includes a routing module to perform a compression-enhancing routing algorithm when routing requests to the storage systems to enhance compression of data stored in the respective storage systems.

18. The system of claim 16, wherein the processor of the first storage system employs a routing algorithm when routing requests to corresponding differential data stores to enhance compression of data stored in the differential data stores contained in the first storage system.