Copying a differential data store into temporary storage media in response to a request

• US 9,141,621 B2
• Filed: 04/30/2009
• Issued: 09/22/2015
• Est. Priority Date: 04/30/2009
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Lillibridge, Mark David, Eshghi, Kave, Bhagwat, Deepavali Mahendra, Deolalikar, Vinay
• Primary Examiner(s): LE, UYEN T
• Application Number:
  US12/432,807
• Publication Number:
  US 20100280997A1
• Time in Patent Office: 2,336 Days
• Field of Search: 707/652, 707/681, 707/736, 707/748, 707/749, 707/756, 707/770, 707/792, 707/E17.001, 707/E17.002, 707/E17.032, 707/966, 707/999.1, 707/999.101, 707/999.2
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 3/064 Management of blocks

Abstract

A plurality of differential data stores are stored in persistent storage media. In response to receiving a first request to store a particular data object, one of the differential data stores that are stored in the persistent storage media is selected, wherein selecting the one differential data store is according to a criterion relating to compression of data objects in the differential data stores. The selected differential data store is copied into temporary storage media, where the copying is not delayed after receiving the first request to await receipt of more requests. The particular data object is inserted into the copy of the selected differential data store in the temporary storage media, where the inserting is performed without having to retrieve more data from the selected differential store in the persistent storage media. The selected differential data store in the persistent storage media is replaced with the copy of the selected differential data store in the temporary storage media that has been modified.

Claims

1. A method comprising:

storing a plurality of differential data stores in persistent storage media;

in response to receiving a first request to store a particular data object, selecting one of the differential data stores that are stored in the persistent storage media, wherein selecting the one differential data store is according to a criterion relating to compression of data objects in the differential data stores;

copying the selected differential data store into temporary storage media, wherein the copying is not delayed after receiving the first request to await receipt of more requests;

inserting the particular data object into the copy of the selected differential data store in the temporary storage media, wherein the inserting is performed without having to retrieve more data from the selected differential store in the persistent storage media; and

replacing the selected differential data store in the persistent storage media with the copy of the selected differential data store in the temporary storage media that has been modified.

2. The method of claim 1, further comprising storing the differential data stores in multiple storage systems.

3. The method of claim 1, wherein copying the selected differential data store into the temporary storage media comprises copying both a temporary storage part and a persistent storage part of the selected differential data store into the temporary storage media.

4. The method of claim 3, wherein the persistent storage part contains data objects, and the temporary storage part contains metadata associated with the data objects.

5. The method of claim 1, wherein selecting the one differential data store according to the criterion relating to compression of data objects in the differential data stores comprises selecting the one differential data store using a compression-enhancing routing algorithm, and wherein the algorithm maps data objects to virtual bins.

6. The method of claim 5, further comprising:

receiving a second request to store a second data object;

mapping the second data object using the algorithm to a particular virtual bin of the virtual bins;

determining whether a differential data store that corresponds to the particular virtual bin exists; and

if the differential data store corresponding to the particular virtual bin does not exist, creating a new differential data store that will correspond to the particular virtual bin and storing the new differential data store on the persistent storage media.

7. The method of claim 1, further comprising returning an identifier of the particular data object and an identifier of the one differential data store to an entity that sent or relayed the request.

8. The method of claim 1, further comprising caching a number of differential data stores in the temporary storage, wherein the cached differential data stores are available for use to process subsequently received requests.

9. The method of claim 1, further comprising:

receiving a retrieve request to retrieve a second data object;

copying a second differential data store containing the second data object specified by the retrieve request into the temporary storage media; and

performing the retrieve request with respect to the copied second differential data store.

10. The method of claim 1, further comprising:

sampling a subset of the differential data stores;

identifying chunks that occur in more than a threshold number of the sampled differential data stores; and

creating a list of the identified chunks.

11. The method of claim 10, further comprising:

after copying any given differential data store into the temporary storage media, determining whether the given differential data store has a chunk that is identified in the list;

replacing each chunk in the given differential data store that is identified in the list with a pointer to a copy of that chunk located outside of the given differential data store.

12. A system comprising:

temporary storage media;

a processor to;

receive a request to store a particular data object;

map the particular data object to a particular differential data store, wherein the mapping is according to a criterion relating to compression of data objects in a plurality of differential data stores;

copy the particular differential data store from persistent storage media to the temporary storage media, wherein copying the particular differential data store is performed without having to wait for receipt of additional requests, and wherein the copy of the particular differential object includes both data objects of the particular differential data store and metadata of the particular differential data store;

insert the particular data object into the copy of the particular differential data store in the temporary storage media; and

write the copy of the particular differential data store that has been modified back to the persistent storage media to replace the particular differential data store in the persistent storage media.

13. The system of claim 12, wherein the processor is to further:

receive a second request to store a second data object;

discover that a differential data store corresponding to the second data object has not yet been allocated;

in response to determining that the differential data store corresponding to the second data object has not yet been allocated, create the differential data store corresponding to the second data object.

14. The system of claim 12, comprising a computer within an environment of multiple computers that have shared access to the persistent storage media.

15. The system of claim 12, wherein the processor is configured to further:

receive a second request to access data;

map the second request to a second differential data store; and

copy the second differential data store from the persistent storage media to the temporary storage media.

16. The system of claim 12, wherein the processor is configured to further:

cache, in the temporary storage media, copies of differential data stores; and

execute subsequently received requests with respect to the cached differential data stores.

17. An article comprising at least one computer-readable storage medium containing instructions that upon execution cause a processor in a system to:

receive a request to store a data object;

select a differential data store from among plural differential data stores to copy from persistent storage media to temporary storage media, wherein selecting the one differential data store is according to a criterion relating to compression of data objects in the differential data stores;

copy the selected differential data store into the temporary storage media without first performing batching of the received request with one or more other requests and without performing reordering of the received request with respect to one or more other requests; and

execute the request with respect to the copy of the selected differential data store.

18. The article of claim 17, wherein the instructions upon execution cause the processor in the system to further:

cache, in the temporary storage media, copies of a subset of the differential data stores; and

execute subsequently received requests associated with the cached differential data stores with respect to the copies of the cached differential data stores in the temporary storage media.

19. The article of claim 17, wherein the instructions upon execution cause the processor in the system to further:

sample a subset of the plural differential data stores;

populate a very-frequent-chunk list having a maximum number of entries with chunks from the sampled differential data stores in order of most frequently occurring until the entries in the very-frequent-chunk list are depleted.

20. The article of claim 19, wherein the instructions upon execution cause the processor in the system to further:

after copying any given differential data store into the temporary storage, determine whether the given differential data store has a chunk that is identified in the list;

replace each chunk in the given differential data store that is identified in the list with a pointer to a copy of that chunk located outside of the given differential data store.