Systems and methods for arbitrary data transformations

• US 8,521,752 B2
• Filed: 06/03/2005
• Issued: 08/27/2013
• Est. Priority Date: 06/03/2005
• Status: Expired due to Fees

• Current Assignee: Open Invention Network LLC
• Original Assignee: OSR Open Systems Resources, Inc.
• Inventors: Noone, Scott J., Mason, W. Anthony, Viscarola, Peter G., Cariddi, Mark J.
• Primary Examiner(s): Uddin, Mohammed R
• Application Number:
  US11/145,433
• Publication Number:
  US 20060277153A1
• Time in Patent Office: 3,007 Days
• Field of Search: 707/100, 707101-104, 707 1- 3, 707/756, 707/809, 707/602, 707/999.107
• US Class Current: 707/756
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database

Abstract

Methods for organizing data. The methods may comprise the step of receiving a write request comprising a data unit. The methods may also comprise the steps of organizing the data unit into a sub-file and incorporating the sub-file into a data file according to a log-structured organization system. In addition, the methods may include the step of writing the data file to a data storage according to a second organization system.

Claims

1. A method for organizing data in a system comprising a data storage device and a host device, wherein the host device is programmed to execute an application;

an operating system, a first file system for logically and physically organizing a plurality of data files on the data storage device, and a data transformation module logically positioned between the operating system and the first file system, the method comprising;

the application sending to the operating system a first write request indicating a logical grouping of at least one data unit;

the data transformation module receiving the first write request from the operating system;

the data transformation module organizing the logical grouping of the at least one data unit into a sub-file;

the data transformation module organizing the sub-file into a data file of the first file system according to a log-structured file system, wherein the log structured file system manages a logical and a physical organization of the sub-file within the data file; and

the data transformation module transmitting a second write request to the first file system, the second write request requesting storage of the data file on the data storage device according to the first file system;

the data transformation module receiving a first read request identifying a second sub-file;

the data transformation module, in response to the first read request, identifying a second data file selected from the plurality of data files that comprises the second sub-file;

the data transformation directing a second read request to the first file system, wherein the second read request indicates the second data file;

the data transformation module receiving the second data file from the first file system in response to the second read request;

the data transformation module extracting the second sub-file from the second data file utilizing the log-structured file system; and

the data transformation module returning the second sub-file in response to the first read request.

2. The method of claim 1, further comprising the transformation module performing an operation on the logical grouping of at least one data unit, wherein the operation changes the size of the at least one data unit.

3. The method of claim 2, wherein the operation is at least one of compression, encryption, translation from a first language to a second language, transformation from a first data format to a second data format, and translation from one encoding to a different encoding.

4. The method of claim 1, further comprising maintaining a previous version of the at least one data unit.

5. The method of claim 1, wherein the first file system is at least one of File Allocation Table 16 (FAT16), File Allocation Table 32 (FAT32), NTFS, High Performance File System (HPFS), UNIX file system (UFS), XFS, journaled file system (JFS), Universal Data Format File System (UDFS), CD-ROM File System (CDFS), Enhanced File System (EFS), SGI XFS, Clustered XFS (CXFS), HFS, VxFS, Raw File System (RawFS), and Local File System (DCE/LFS).

6. The method of claim 1, further comprising decrypting the second sub-file.

7. The method of claim 1, further comprising decompressing the second sub-file.

8. The method of claim 1, wherein the at least one data unit comprises a plurality of data units.

9. The method of claim 6, wherein the at least one data unit includes at least one of an encryption header, a thumbnail image, a checksum, and information indexing the data unit.

10. A system for storing data, the system comprising:

a data storage device;

a host device comprising at least one processor and associated memory, wherein the host device is in communication with the data storage device, and wherein the host device is programmed to execute;

an application programmed to send a first write request indicating a logical grouping of at least one data unit;

an operating system;

a first file system for logically and physically organizing a plurality of data files on the data storage device; and

a data transformation module logically positioned between the operating system and the first file system, wherein the data transformation module is programmed to implement within the first data file a log-structured file system for logically and physically organizing a plurality of sub-files within the first data file, and wherein the data transformation module is further programmed to;

receive the first write request;

organize the logical grouping of at least one data unit into a sub-file;

organize the sub-file into a second data file of the first file system according to the log-structured file system, wherein the log-structured file system manages a logical and a physical organization of the sub-file within the second data file;

transmit a second write request to the first file system for storage of the second data file on the data storage device according to the first file system;

receive a first read request specifying a first sub-file selected from the plurality of sub-files;

in response to the first read request, derive an indication of a first data file selected from the plurality of data files that comprises the first sub-file;

direct a second read request to the first file system, wherein the second read request indicates the first data file;

receive the first data file from the first file system in response to the second read request;

extract the first sub-file from the first data file utilizing the log-structured file system; and

return the first sub-file in response to the first read request.

11. The system of claim 10, wherein the data transformation is further programmed to, before organizing the sub-file into the second data file, perform a transformation on the sub-file that changes the size of the sub-file.

12. The system of claim 11, wherein the transformation comprises encryption.

13. The system of claim 11, wherein the transformation comprises compression.

14. The system of claim 11, wherein the transformation comprises a translation of the sub-file from a first human language to a second human language.

15. The system of claim 10, wherein the data storage device comprises the plurality of data files organized according to the first file system, wherein the plurality of data files comprises the first data file, and wherein the plurality of sub-files includes the first sub-file logically and physically organized within the first data file according to the log-structured file system.

16. The system of claim 10, wherein first file system defines a plurality of directories distinct from the plurality of data files, wherein the plurality of directories are for logically organizing the plurality of data files.

17. The system of claim 10, wherein the log-structured file system defines a second plurality of directories distinct from the plurality of sub-files, wherein the second plurality of directories are for logically organizing the plurality of sub-files within the first data file.

18. The system of claim 16, wherein the first file system defines a physical position of each of the plurality of data files on the storage device, and wherein the log-structured file system defines a physical position of each of the plurality of sub-files within the first data file.

19. The system of claim 10, wherein the first file system is a native file system of the operating system.

20. The system of claim 10, wherein the first file system is a native file system of the operating system.