Encoded data processing

• US 8,832,046 B2
• Filed: 09/14/2012
• Issued: 09/09/2014
• Est. Priority Date: 07/10/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Franke, Hubertus, Iyer, Balakrishna R., Min, Hong, Purcell, Terence P., Zhang, Guogen
• Primary Examiner(s): Hu, Jensen
• Application Number:
  US13/617,614
• Publication Number:
  US 20140019422A1
• Time in Patent Office: 725 Days
• Field of Search: 707/769, 707/693
• US Class Current: 707/693
• CPC Class Codes: G06F 16/116 Details of conversion of file system types or formats
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Techniques are provided for encoded data processing which allows for continuous data processing as encoded data changes. Data is decomposed into one or more blocks with each block containing at least one data record. At least one data record within a given block is encoded with a first encoding process selected from one or more encoding processes. The first encoding process is associated with the given data block. Techniques evaluate whether or not to implement an encoding change for a given block when updating a given data record in the given block. Responsive to the evaluation, the given block is re-encoded with a second encoding process. Responsive to the re-encoding, the association of the given block is updated. A map is formed to convert the given data record encoded with the first encoding process to the second encoding process so as to preserve comparative relationships of the given data record.

Claims

1. An apparatus, comprising:

a memory; and

a processor device operatively coupled to the memory and configured to;

decompose data into one or more blocks, each block containing at least one data record;

encode the at least one data record within a given one of the one or more blocks with a first encoding process selected from one or more encoding processes;

associate the first encoding process with the given block;

evaluate whether or not to implement an encoding change for the given block containing a given data record when updating the given data record, wherein updating the given data record comprises at least one of inserting data into, updating data in, and deleting data from the given data record;

re-encode the given block containing the given data record with a second encoding process responsive to said evaluating step;

update the association of the given block responsive to said re-encoding step; and

form a map to convert the given data record encoded with the first encoding process to the second encoding process;

wherein the given data record encoded with the first encoding process has one or more comparative relationships with one or more other data records encoded with the first encoding process; and

wherein the map is formed so as to preserve, for the given data record encoded with the second encoding process, the one or more comparative relationships of the given data record encoded with the first encoding process.

2. The apparatus of claim 1, wherein the given block is transaction-locked while the processor device re-encodes and updates the given block.

3. The apparatus of claim 1, wherein the first encoding process is discarded when the data records encoded with the first encoding process have been re-encoded with the second encoding process.

4. The apparatus of claim 1, wherein blocks associated with a first encoding process not already re-encoded with the second encoding process are re-encoded with the second encoding process in a background operation responsive to the re-encoding.

5. The apparatus of claim 1, wherein the second encoding process is created response to the evaluation.

6. The apparatus of claim 1, wherein the given block is split when the given block does not provide storage under the second encoding process.

7. The apparatus of claim 1, wherein the processor device is further configured to:

receive a request for a search;

dynamically generate a search code for each encoding process;

select the search code for a block based on the encoding process associated with the block; and

process a search on the block using the selected search code.

8. The apparatus of claim 7, wherein a search code is generated for a given encoding process when at least one data record is encoded with the given encoding process.

9. The apparatus of claim 1, wherein the map is formed by applying at least one of a logical operation, a bit-wise operation and an arithmetical operation on the given data record.

10. An article of manufacture comprising a computer readable storage medium for storing computer readable program code which, when executed, causes a computer to:

decompose data into one or more blocks, each block containing at least one data record;

encode the at least one data record within a given one of the one or more blocks with a first encoding process selected from one or more encoding processes;

associate the first encoding process with the given block;

evaluate whether or not to implement an encoding change for the given block containing a given data record when updating the given data record, wherein updating the given data record comprises at least one of inserting data into, updating data in, and deleting data from the given data record;

re-encode the given block containing the given data record with a second encoding process responsive to said evaluating step;

update the association of the given block responsive to said re-encoding step; and

form a map to convert the given data record encoded with the first encoding process to the second encoding process;

wherein the given data record encoded with the first encoding process has one or more comparative relationships with one or more other data records encoded with the first encoding process; and

wherein the map is formed so as to preserve, for the given data record encoded with the second encoding process, the one or more comparative relationships of the given data record encoded with the first encoding process.

11. The article of manufacture of claim 10, wherein the given block is transaction-locked during said re-encoding and updating steps.

12. The article of manufacture of claim 10, wherein the first encoding process is discarded when the data records encoded with the first encoding process have been re-encoded with the second encoding process.

13. The article of manufacture of claim 10, wherein blocks associated with a first encoding process not already re-encoded with the second encoding process are re-encoded with the second encoding process in a background operation responsive to said re-encoding step.

14. The article of manufacture of claim 10, wherein a given one of the one or more encoding processes is used to encode data within the same spatial locality region.

15. The article of manufacture of claim 10, wherein the computer readable program code, when executed, further causes a computer to utilize one or more search operations associated with a block based on the encoding process associated with the block.

16. The article of manufacture of claim 10, wherein a given one of the one or more encoding processes is used on more than one block.

17. The article of manufacture of claim 10, wherein the given block is split when the given block does not provide storage under the second encoding process.

18. The article of manufacture of claim 10, wherein data records in the given block are encoded with the same encoding process.

19. The article of manufacture of claim 10, wherein the second encoding process is created responsive to said evaluating step.

20. The article of manufacture of claim 10, wherein the map is formed by applying at least one of a logical operation, a bit-wise operation and an arithmetical operation on the given data record.