Data processing apparatus and method of processing data

• US 8,150,851 B2
• Filed: 10/27/2008
• Issued: 04/03/2012
• Est. Priority Date: 10/25/2007
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Jones, Kevin Lloyd
• Primary Examiner(s): Lee, Wilson
• Application Number:
  US12/258,638
• Publication Number:
  US 20090112946A1
• Time in Patent Office: 1,254 Days
• Field of Search: 707737-741, 707758-761, 707769-780, 717/140, 717/141, 717145-150
• US Class Current: 707/737
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 16/1756 based on delta files

Abstract

Data processing apparatus comprising: a chunk store partitioned into a plurality of chunk sections, at least one section storing specimen data chunks, the processing apparatus being operable to: process input data into one or more input data chunks; identify a chunk section already containing a specimen data chunk corresponding to at least one input data chunk; and store the at least one input data chunk in another chunk section as a specimen data chunk if the identified chunk section has a predetermined characteristic.

Claims

1. Data processing apparatus comprising:

a chunk store partitioned into a plurality of chunk sections, at least one of the chunk sections storing specimen data chunks,

a processor to;

process input data into input data chunks;

identify a chunk section already containing a specimen data chunk corresponding to a particular one of the input data chunks; and

store the particular input data chunk in another chunk section as a specimen data chunk if the identified chunk section has a predetermined characteristic, wherein the predetermined characteristic is selected from the group consisting of;

a characteristic that specimen data chunks corresponding to other input data chunks of the input data being processed are stored in more than a predetermined number of chunk sections,

a characteristic that specimen data chunks in the identified chunk section have been referenced by fewer than a predetermined number of the most recent M manifests to be compiled,

a characteristic that fewer than a predetermined number of specimen data chunks in the identified chunk section correspond to input data chunks of the input data being processed, and

a characteristic that fewer than a predetermined number of manifests in a manifest store contain a reference to the said corresponding specimen chunk in the identified chunk section.

2. Data processing apparatus comprising:

a chunk store partitioned into a plurality of chunk sections, at least one of the chunk sections storing specimen data chunks,

a processor to;

process input data into input data chunks;

identify a chunk section already containing a specimen data chunk corresponding to a particular one of the input data chunks; and

store the particular input data chunk in another chunk section as a specimen data chunk if the identified chunk section has a predetermined characteristic; and

if the identified chunk section has said predetermined characteristic, and a subsequent input data chunk is found to correspond to said specimen data chunk in said another chunk section, compile a manifest with a reference to said specimen data chunk stored in said another chunk section.

3. Data processing apparatus comprising:

a chunk store partitioned into a plurality of chunk sections, at least one of the chunk sections storing specimen data chunks,

a processor to;

process input data into input data chunks;

identify a chunk section already containing a specimen data chunk corresponding to a particular one of the input data chunks; and

store the particular input data chunk in another chunk section as a specimen data chunk if the identified chunk section has a predetermined characteristic,

if the identified chunk section has said predetermined characteristic, decide not to compile subsequent manifests with references to the said identified chunk section.

4. Data processing apparatus comprising:

a chunk store partitioned into a plurality of chunk sections, at least one of the chunk sections storing specimen data chunks,

a processor to;

process input data into input data chunks;

identify a chunk section already containing a specimen data chunk corresponding to a particular one of the input data chunks; and

store the particular input data chunk in another chunk section as a specimen data chunk if the identified chunk section has a predetermined characteristic,

if the identified chunk section has said predetermined characteristic, store at least one further specimen data chunk of the identified chunk section in said another chunk section.

5. Data processing apparatus comprising:

a chunk store partitioned into a plurality of chunk sections, at least one of the chunk sections storing specimen data chunks,

a processor to;

process input data into input data chunks;

identify a chunk section already containing a specimen data chunk corresponding to a particular one of the input data chunks;

store the particular input data chunk in another chunk section as a specimen data chunk if the identified chunk section has a predetermined characteristic; and

if the identified chunk section does not have the predetermined characteristic, compile a manifest with a reference to the specimen data chunk in the identified chunk section.

6. Data processing apparatus according to claim 5, further comprising a manifest store, containing the manifest that represents at least a part of a data set.

7. Data processing apparatus according to claim 5, further comprising a chunk index containing information on at least one specimen data chunk, the processor to use the chunk index to identify said chunk section already containing a specimen data chunk corresponding to the particular input data chunk.

8. Data processing apparatus according to claim 5, wherein the predetermined characteristic is that the identified chunk section is fragmented to at least a predetermined extent.

9. Data processing apparatus comprising:

a chunk store partitioned into a plurality of chunk sections, at least one of the chunk sections storing specimen data chunks,

a processor to;

process input data into input data chunks;

identify a chunk section already containing a specimen data chunk corresponding to a particular one of the input data chunks; and

store the particular input data chunk in another chunk section as a specimen data chunk if the identified chunk section has a predetermined characteristic, wherein the predetermined characteristic is that a predetermined number of specimen data chunks stored in the identified chunk section are non-contiguous with one another.

10. A computer-implemented method of processing data, using:

a chunk store partitioned into a plurality of chunk sections, at least one of the chunk sections storing specimen data chunks,

a manifest store, containing at least one manifest that represents at least a part of a data set and that comprises at least one reference to at least one specimen data chunk stored in the chunk store,

the method comprising;

processing input data into one or more input data chunks;

identifying a chunk section already containing a specimen data chunk corresponding to at least one input data chunk;

storing the at least one input data chunk in another chunk section as a specimen data chunk if the identified chunk section has a predetermined characteristic; and

compiling a manifest with a reference to said another chunk section if the identified chunk section has the predetermined characteristic.

11. A method of processing data according to claim 10, further using a chunk index containing information on at least one specimen data chunk, the method further comprising using the chunk index to identify said chunk section already containing a specimen data chunk corresponding to at least one input data chunk.

12. A method of processing data according to claim 10, wherein subsequent manifests are not compiled with references to the identified chunk section if the identified chunk section has the predetermined characteristic.