Method and System for Implementing Parallel Transformations of Records

• US 20100036861A1
• Filed: 08/08/2008
• Published: 02/11/2010
• Est. Priority Date: 08/08/2008
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Lee, Jonghyun, Srihari, Vinay H., Hansen, Roger, Shi, Jia
• Granted Patent: US 9,104,662 B2
• Time in Patent Office: Days
• US Class Current: N/A
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor

Abstract

An improved approach is described for implementing transformations of data records in high concurrency environments. Each transformation is performed in parallel at the source when the data record is first generated. According to one approach for data integrity validation, record generators compute an integrity checksum for a newly generated record before copying into a data unit in shared memory. Subsequent generators may aggregate integrity checksums for data records into checksums for data units incrementally. This approach achieves end-to-end protection of data records against corruption using an efficient method of maintaining verifiable data integrity. In another approach, compression and encryption data transformations may be performed by themselves, or in combination with an integrity checksum transformation.

Claims

1. A method for transforming data records, comprising:

identifying a data unit;

generating a first record to place into the data unit;

generating a second record to place into the data unit, wherein the first record and the second records are concurrently generated;

generating a first transformation corresponding to the first record; and

generating a second transformation corresponding to the second record, in which the first and second transformations are generated after generation of the first and second records but before placement of the record in the data unit, and wherein the first and second transformations are concurrently generated.

2. The method of claim 1 in which the data records comprise database log records.

3. The method of claim 1 in which the first and second transformations are additive or commutative.

4. The method of claim 1 in which the first and second transformations are data integrity checksum transformations.

5. The method of claim 1 in which the first and second transformations are compression or encryption transformations.

6. The method of claim 1 in which a data record undergoes multiple transformations, comprising at least two of encryption, compression, and checksum transformations.

7. The method of claim 1 in which the first transformation is copied to the transformation value for the data unit by itself if the transformation value does not have a pre-existing value.

8. The method of claim 1 in which the first transformation is aggregated with a pre-existing transformation value for the data unit.

9. The method of claim 1 in which the transformation value for the data unit is stored within a header for the data unit.

10. The method of claim 1 in which the first and second transformations are aggregated.

11. The method of claim 10 in which a later record generator aggregates the first and second transformation values.

12. The method of claim 10 in which the first transformation is generated by a first record generator, the second transformation is generated by a second record generator, and the act of aggregating the first and second transformations is performed by another entity.

13. The method of claim 1 in which the first and second transformations are retained in un-aggregated states.

14. A system for transforming data records, comprising:

means for identifying a data unit;

means for generating a first record to place into the data unit;

means for generating a second record to place into the data unit, wherein the first record and the second records are concurrently generated; and

means for generating a first transformation corresponding to the first record;

means for generating a second transformation corresponding to the second record, in which the first and second transformations are generated after generation of the first and second records but before placement of the record in the data unit, and wherein the first and second transformations are concurrently generated.

15. The system of claim 14 in which the data records comprise database log records.

16. The system of claim 14 in which the first and second transformations are additive or commutative.

17. The system of claim 14 in which the first and second transformations are data integrity checksum transformations.

18. The system of claim 14 in which the first and second transformations are compression or encryption transformations.

19. The system of claim 14 in which a data record undergoes multiple transformations, comprising at least two of encryption, compression, and checksum transformations.

20. The system of claim 14 in which the first transformation is copied to the transformation value for the data unit by itself if the transformation value does not have a pre-existing value.

21. The system of claim 14 in which the first transformation is aggregated with a preexisting transformation value for the data unit.

22. The system of claim 14 in which the transformation value for the data unit is stored within a header for the data unit.

23. The system of claim 14 in which the first and second transformations are aggregated.

24. The system of claim 23 in which a later record generator aggregates the first and second transformation values.

25. The system of claim 23 in which the first transformation is generated by a first record generator, the second transformation is generated by a second record generator, and the act of aggregating the first and second transformations is performed by another entity.

26. The system of claim 14 in which the first and second transformations are retained in un-aggregated states.

27. A computer program product that includes a computer readable medium, the computer readable medium comprising a plurality of computer instructions which, when executed by a processor, cause the processor to transform data records, the process comprising:

identifying a data unit;

generating a first record to place into the data unit;

generating a second record to place into the data unit, wherein the first record and the second records are concurrently generated;

generating a first transformation corresponding to the first record; and

generating a second transformation corresponding to the second record, in which the first and second transformations are generated after generation of the first and second records but before placement of the record in the data unit, and wherein the first and second transformations are concurrently generated.

28. The computer program product of claim 27 in which the data records comprise database log records.

29. The method of claim 27 in which the first and second transformations are additive or commutative.

30. The computer program product of claim 27 in which the first and second transformations are data integrity checksum transformations.

31. The computer program product of claim 27 in which the first and second transformations are compression or encryption transformations.

32. The computer program product of claim 27 in which a data record undergoes multiple transformations, comprising at least two of encryption, compression, and checksum transformations.

33. The computer program product of claim 27 in which the first transformation is copied to the transformation value for the data unit by itself if the transformation value does not have a pre-existing value.

34. The computer program product of claim 27 in which the first transformation is aggregated with a preexisting transformation value for the data unit.

35. The computer program product of claim 27 in which the transformation value for the data unit is stored within a header for the data unit.

36. The computer program product of claim 27 in which the first and second transformations are aggregated.

37. The computer program product of claim 36 in which a later record generator aggregates the first and second transformation values.

38. The computer program product of claim 36 in which the first transformation is generated by a first record generator, the second transformation is generated by a second record generator, and the act of aggregating the first and second transformations is performed by another entity.

39. The computer program product of claim 27 in which the first and second transformations are retained in un-aggregated states.