Encoded data processing

• US 8,756,208 B2
• Filed: 07/10/2012
• Issued: 06/17/2014
• Est. Priority Date: 07/10/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Franke, Hubertus, Iyer, Balakrishna R., Min, Hong, Purcell, Terence P., Zhang, Guogen
• Primary Examiner(s): Hu, Jensen
• Application Number:
  US13/545,522
• Publication Number:
  US 20140019472A1
• Time in Patent Office: 707 Days
• Field of Search: 707/769, 707/693
• US Class Current: 707/693
• CPC Class Codes: G06F 16/116 Details of conversion of file system types or formats
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Techniques for encoded data processing which allow for continuous data processing as encoded data changes. Data is decomposed into one or more blocks with each block containing at least one data record. At least one data record within a given block is encoded with a first encoding process selected from one or more encoding processes. The first encoding process is associated with the given data block. Techniques evaluate whether or not to implement an encoding change for a given block when updating a given data record in the given block. Responsive to the evaluation, the given block is re-encoded with a second encoding process. Responsive to the re-encoding, the association of the given block is updated. A map is formed to convert the given data record encoded with the first encoding process to the second encoding process so as to preserve comparative relationships of the given data record.

Claims

1. A method, comprising:

decomposing data into one or more blocks, each block containing at least one data record;

encoding the at least one data record within a given one of the one or more blocks with a first encoding process selected from one or more encoding processes;

associating the first encoding process with the given block;

evaluating whether or not to implement an encoding change for the given block containing a given data record when updating the given data record, wherein updating the given data record comprises at least one of inserting data into, updating data in, and deleting data from the given data record;

re-encoding the given block containing the given data record with a second encoding process responsive to said evaluating step;

updating the association of the given block responsive to said re-encoding step; and

forming a map to convert the given data record encoded with the first encoding process to the second encoding process;

wherein the given data record encoded with the first encoding process has one or more comparative relationships with one or more other data records encoded with the first encoding process; and

wherein the map is formed so as to preserve, for the given data record encoded with the second encoding process, the one or more comparative relationships of the given data record encoded with the first encoding process.

2. The method of claim 1, wherein the given block is transaction-locked during said re-encoding and updating steps.

3. The method of claim 1, wherein the first encoding process is discarded when the data records encoded with the first encoding process have been re-encoded with the second encoding process.

4. The method of claim 1, wherein blocks associated with a first encoding process not already re-encoded with the second encoding process are re-encoded with the second encoding process in a background operation responsive to said re-encoding step.

5. The method of claim 1, wherein a given one of the one or more encoding processes is used to encode data within the same spatial locality region.

6. The method of claim 1, further comprising the step of utilizing one or more search operations associated with a block based on the encoding process associated with the block.

7. The method of claim 1, wherein a given one of the one or more encoding processes is used on more than one block.

8. The method of claim 1, wherein the given block is split when the given block does not provide storage under the second encoding process.

9. The method of claim 1, wherein said evaluating step is based on whether the first encoding process has enough encoding bits to represent an updated data record.

10. The method of claim 1, wherein data records in the given block are encoded with the same encoding process.

11. The method of claim 1, wherein the second encoding process is created responsive to said evaluating step.

12. The method of claim 1, further comprising the steps of:

receiving a request for a search;

dynamically generating a search code for each encoding process;

selecting the search code for a block based on the encoding process associated with the block; and

processing the search on the block using the selected search code.

13. The method of claim 12, wherein the search can be processed without decoding data records.

14. The method of claim 12, wherein a search code is generated for a given encoding process when at least one data record is encoded with the given encoding process.

15. The method of claim 1, wherein the map is formed by applying at least one of a logical operation, a bit-wise operation and an arithmetical operation on the given data record.