Data processing apparatus and method of processing data
First Claim
1. A data processing apparatus comprising:
- a chunk store configured for containing specimen data chunks,a manifest store configured for containing at least one manifest that represents at least a part of a data set and that comprises at least one reference to at least one of said specimen data chunks,a sparse chunk index configured for containing information on only those specimen data chunks having a predetermined characteristic, wherein said information of said sparse chunk index refers to at least one manifest in the manifest store comprising a reference to at least one of said specimen data chunks having said predetermined characteristic,at least one processor to;
process input data into input data chunks, and use the sparse chunk index to identify at least one of said at least one manifest that includes at least one reference to one of said specimen data chunks that corresponds to one of said input data chunks having the predetermined characteristic.
2 Assignments
0 Petitions
Accused Products
Abstract
Data processing apparatus comprising: a chunk store containing specimen data chunks, a manifest store containing at least one manifest that represents at least a part of a data set and that comprises at least one reference to at least one of said specimen data chunks, a sparse chunk index containing information on only those specimen data chunks having a predetermined characteristic, the processing apparatus being operable to process input data into input data chunks and to use the sparse chunk index to identify at least one of said at least one manifest that includes at least one reference to one of said specimen data chunks that corresponds to one of said input data chunks having the predetermined characteristic.
-
Citations
18 Claims
-
1. A data processing apparatus comprising:
-
a chunk store configured for containing specimen data chunks, a manifest store configured for containing at least one manifest that represents at least a part of a data set and that comprises at least one reference to at least one of said specimen data chunks, a sparse chunk index configured for containing information on only those specimen data chunks having a predetermined characteristic, wherein said information of said sparse chunk index refers to at least one manifest in the manifest store comprising a reference to at least one of said specimen data chunks having said predetermined characteristic, at least one processor to; process input data into input data chunks, and use the sparse chunk index to identify at least one of said at least one manifest that includes at least one reference to one of said specimen data chunks that corresponds to one of said input data chunks having the predetermined characteristic. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A data processing apparatus comprising:
-
a chunk store configured for containing specimen data chunks, a manifest store configured for containing at least one manifest that represents at least a part of a data set and that comprises at least one reference to at least one of said specimen data chunks, a sparse chunk index configured for containing information on only some specimen data chunks having a predetermined characteristic, at least one processor to; process input data into input data chunks; identify candidate data chunks from the input data chunks, where the candidate data chunks have the predetermined characteristic; select less than all of candidate data chunks according to at least one selection criterion to populate the sparse chunk index with information relating to the selected candidate data chunks, where the at least one selection criterion selects a particular one of the candidate data chunks to populate the sparse chunk index based on a position of the particular candidate data chunk to another one of the candidate data chunks that has been selected to populate the sparse chunk index; and use the sparse chunk index to identify at least one manifest that includes at least one reference to one of said specimen data chunks corresponding to an input data chunk having the predetermined characteristic.
-
-
12. A data processing apparatus comprising:
-
a chunk store configured for containing specimen data chunks, a sparse chunk index, configured for containing information on only some specimen data chunks, at least one processor to; process input data into input data chunks; identify at least one candidate data chunk from the input data chunks having a predetermined characteristic; and select at least one candidate data chunk according to at least one selection criterion to populate the sparse chunk index with information relating to at least one of the specimen data chunks corresponding to the selected at least one candidate data chunk, wherein the selection criterion is that there are at least a predetermined number of input data chunks between selected candidate data chunks. - View Dependent Claims (13, 14)
-
-
15. A data processing apparatus comprising:
-
a chunk store configured for containing specimen data chunks, a sparse chunk index, configured for containing information on only some specimen data chunks, at least one processor to; process input data into input data chunks; identify at least one candidate data chunk from the input data chunks having a predetermined characteristic; and select at least one candidate data chunk according to at least one selection criterion to populate the sparse chunk index with information relating to at least one of the specimen data chunks corresponding to the selected at least one candidate data chunk, wherein the selection criterion is that there are no more than a predetermined number of input data chunks between selected candidate data chunks.
-
-
16. A method of processing data comprising:
-
providing a chunk store containing specimen data chunks; providing a manifest store containing at least one manifest that represents at least a part of a data set and that comprises at least one reference to at least one of said specimen data chunks; providing a sparse chunk index containing information on only those specimen data chunks having a chunk identifier having a predetermined characteristic, wherein said information contained in said sparse chunk index refers to at least one manifest in the manifest store comprising a reference to at least one of said specimen data chunks having said predetermined characteristic; processing input data into input data chunks; and using the sparse chunk index to identify at least one manifest in the manifest store that includes at least one reference to a said specimen data chunk corresponding to at least one of said input data chunks having a chunk identifier having the predetermined characteristic. - View Dependent Claims (17)
-
-
18. A method of processing data comprising:
-
providing a chunk store configured for containing specimen data chunks; providing a sparse chunk index, configured for containing information on only some specimen data chunks; processing input data into input data chunks; generating a chunk identifier for each input data chunk; identifying at least one candidate data chunk from the input data chunks having a chunk identifier having a predetermined characteristic; and selecting at least one candidate data chunk according to at least one selection criterion to populate the sparse chunk index with information relating to a specimen data chunk corresponding to one of the selected at least one candidate data chunk, wherein the selection criterion is selected from the group consisting of;
(1) a selecting criterion specifying that there are at least a first predetermined number of input data chunks between selected candidate data chunks, and (2) a selection criterion specifying that there are no more than a second predetermined number of input data chunks between selected candidate data chunks.
-
Specification