DATA PROCESSING APPARATUS AND METHOD OF PROCESSING DATA

• US 20100198832A1
• Filed: 10/25/2007
• Published: 08/05/2010
• Est. Priority Date: 10/25/2007
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Inventors: Jones, Kevin Loyd
• Granted Patent: US 9,372,941 B2
• Time in Patent Office: Days
• US Class Current: 707/741
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 16/90339 by using parallel associative memories or content-addressable memories

Abstract

Data processing apparatus comprising: a chunk store configured for containing specimen data chunks, a discrete manifest store, configured for containing at least one manifest that represents at least a part of a data set and that comprises at least one reference to at least one of said specimen data chunks, the processing apparatus being operable to process input data into input data chunks and to identify at least one manifest in the manifest store that includes at least one said reference to a said specimen data chunk corresponding to at least one of said input data chunks.

Claims

1. Data processing apparatus comprising:

a chunk store configured for containing specimen data chunks,

a discrete manifest store, configured for containing at least one manifest that represents at least a part of a data set and that comprises at least one reference to at least one of said specimen data chunks,

the processing apparatus being operable to process input data into input data chunks and to identify at least one manifest in the manifest store that includes at least one said reference to a said specimen data chunk corresponding to at least one of said input data chunks.

2. Data processing apparatus according to claim 1, operable to analyse said identified at least one manifest and identify at least one said reference to a said specimen data chunk corresponding to at least one further input data chunk.

3. Data processing apparatus according to claim 1, operable to store at least one of said input data chunks in the chunk store as a specimen data chunk, should said at least one input data chunk, in use, be found not to correspond to at least one of said specimen data chunks in the chunk store.

4. Data processing apparatus according to claim 1, configured to comprise a chunk index containing information relating to said specimen data chunks contained in the chunk store.

5. Data processing apparatus according to claim 1, configured to comprise a chunk index containing information relating only to some of said specimen data chunks contained in the chunk store.

6. Data processing apparatus according to claim 4, wherein the apparatus is operable to identify said corresponding specimen data chunk using said information contained in the chunk index.

7. Data processing apparatus according to claim 4, wherein said information contained in the chunk index includes details of at least one said manifest in the manifest store comprising a reference to at least one of said specimen data chunks.

8. Data processing apparatus according to claim 4, wherein said information contained in the chunk index includes a chunk identifier of at least one of said specimen data chunks.

9. Data processing apparatus according to claim 8, wherein the chunk identifier is a partial chunk identifier corresponding to one or more of said specimen data chunks.

10. Data processing apparatus according to claim 8, wherein the apparatus is operable to generate a chunk identifier of a said input data chunk to be processed.

11. Data processing apparatus according to claim 10, wherein the apparatus is operable to identify said corresponding specimen chunk by comparing the chunk identifier of said specimen chunk with the chunk identifier of the said input data chunk.

12. Data processing apparatus according to claim 1, wherein the manifest store is operable to contain information relating to said at least one manifest stored therein.

13. Data processing apparatus according to claim 12, wherein the manifest store contains a chunk identifier of said the at least one specimen data chunk referenced by said at least one manifest.

14. Data processing apparatus according to claim 13, operable to analyse said identified at least one manifest and identify at least one said reference to a said specimen data chunk corresponding to at least one further input data chunk, by comparing a chunk identifier of a said input data chunk with the chunk identifier of said at least one specimen data chunk 6 stored in the manifest store.

15. Data processing apparatus according to claim 1, wherein a corresponding specimen data chunk has a chunk identifier which is at least partially similar to a chunk identifier of a said input data chunk.

16. Data processing apparatus comprising:

a chunk store containing specimen data chunks of previously processed data sets,

a chunk index containing a chunk identifier for at least one of said specimen data chunks,

a discrete manifest store, containing at least one manifest that represents a previously processed data set and that comprises at least one reference to at least one of said specimen data chunks,

the processing apparatus being configured to divide input data to be processed into input data chunks;

generate a chunk identifier for at least one of said input data chunks, identify a corresponding chunk identifier in the chunk index; and

identify therefrom at least one of said manifests in the manifest store comprising at least one reference to a specimen data chunk identical to said input data chunk.

17. A data compactor comprising:

a chunk store configured for containing specimen data chunks,

a discrete manifest store, configured for containing at least one manifest that represents a data set and that comprises at least one reference to at least one of said specimen data chunks,

the compactor being configured to divide input data into input data chunks, select one of said input data chunks and identify at least one manifest having at least one reference to a specimen data chunk corresponding to the said selected input data chunk and to explore the at least one identified manifest for references to other specimen data chunks corresponding to other input data chunks of the data set.

18. A method of processing input data, using:

a chunk store containing specimen data chunks,

a discrete manifest store, containing at least one manifest that represents a data set and that comprises at least one reference to at least one of said specimen data chunks,

the method comprising processing said input data into input data chunks and identifying at least one said manifest in the manifest store that includes at least one said reference to a said specimen data chunk corresponding to a said input data chunk.

19. A method according to 18, comprising reviewing the identified at least one manifest to find at least one further specimen data chunk corresponding to at least one further said input data chunk of said input data.

20. A method of processing input data, using:

a chunk store containing specimen data chunks of previously processed data sets,

a chunk index containing a chunk identifier for at least one of said specimen data chunks,

a discrete manifest store, containing at least one manifest that represents a previously processed data set and that comprises at least one reference to at least one of said specimen data chunks,

the method comprising,

dividing the input data into input data chunks,

generating a chunk identifier for one of said input data chunks,

identifying a corresponding chunk identifier in said chunk index and identifying therefrom at least one manifest in said manifest database containing a reference to a specimen data chunk identical to said input data chunk.