Systems and methods for arbitrary data transformations
First Claim
Patent Images
1. A computer system comprising:
- at least one processor;
non-transitory memory operatively associated with the at least one processor; and
a data storage device in communication with the at least one processor, wherein the data storage device comprises stored thereon a plurality of data files logically and physically organized according to a first file system, and wherein the at least one processor is programmed to;
execute an application;
execute a data transformation module, wherein the data transformation module is further programmed to;
receive a first read request originating from the application and specifying a first sub-file, wherein the first sub-file is selected from a plurality of sub-files logically and physically organized within a first data file of the plurality of data files according to a log-structured file system;
in response to the first read request, derive an indication of the first data file;
direct a second read request to the data storage device, wherein the second read request indicates the first data file;
receive the first data file from the data storage device in response to the second read request;
extract the first sub-file from the first data file utilizing the log-structured file system;
return the first sub-file in response to the first read request;
receive a first write request originating from the application, wherein the first write request indicates a logical grouping of at least one data unit;
organize the logical grouping of at least one data unit into a second sub-file;
perform a transformation on the second sub-file that changes the size of the second sub-file;
after performing the transformation on the second sub-file, organize the second sub-file into a second data file of the first file system according to the log-structured file system; and
transmit a second write request to the data storage device for storage of the second data file on the data storage device according to the first file system.
3 Assignments
0 Petitions
Accused Products
Abstract
Methods for organizing data. The methods may comprise the step of receiving a write request comprising a data unit. The methods may also comprise the steps of organizing the data unit into a sub-file and incorporating the sub-file into a data file according to a log-structured organization system. In addition, the methods may include the step of writing the data file to a data storage according to a second organization system.
157 Citations
22 Claims
-
1. A computer system comprising:
- at least one processor;
non-transitory memory operatively associated with the at least one processor; and
a data storage device in communication with the at least one processor, wherein the data storage device comprises stored thereon a plurality of data files logically and physically organized according to a first file system, and wherein the at least one processor is programmed to;execute an application; execute a data transformation module, wherein the data transformation module is further programmed to; receive a first read request originating from the application and specifying a first sub-file, wherein the first sub-file is selected from a plurality of sub-files logically and physically organized within a first data file of the plurality of data files according to a log-structured file system; in response to the first read request, derive an indication of the first data file; direct a second read request to the data storage device, wherein the second read request indicates the first data file; receive the first data file from the data storage device in response to the second read request; extract the first sub-file from the first data file utilizing the log-structured file system; return the first sub-file in response to the first read request; receive a first write request originating from the application, wherein the first write request indicates a logical grouping of at least one data unit; organize the logical grouping of at least one data unit into a second sub-file; perform a transformation on the second sub-file that changes the size of the second sub-file; after performing the transformation on the second sub-file, organize the second sub-file into a second data file of the first file system according to the log-structured file system; and transmit a second write request to the data storage device for storage of the second data file on the data storage device according to the first file system. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
- at least one processor;
-
12. A method for organizing data in a system comprising:
- at least one processor;
non-transitory memory operatively associated with the at least one processor; and
a data storage device in communication with the at least one processor, wherein the data storage device comprises stored thereon a plurality of data files logically and physically organized according to a first file system, wherein the at least one processor is programmed to execute an application and a data transformation module, the method comprising;the data transformation module receiving a first read request originating from the application and identifying a first sub-file selected from a plurality of sub-files logically and physically organized with a first data file of the plurality of data files according to a log-structured file system; the data transformation module, in response to the first read request, identifying the first data file; the data transformation module directing a second read request to the data storage device, wherein the second read request indicates the first data file; the data transformation module receiving the first data file from the data storage device in response to the second read request; the data transformation module extracting the first sub-file from the first data file utilizing the log-structured file system; and the data transformation module returning the first sub-file in response to the first read request; the data transformation module receiving a first write request originating from the application, wherein the first write request indicates a logical grouping of at least one data unit; the data transformation module performing an operation on the logical grouping of at least one data unit, wherein the operation changes the size of the at least one data unit; the data transformation module organizing the logical grouping of the at least one data unit into a second sub-file; the data transformation module organizing the second sub-file into a second data file of the first file system according to a log-structured file system, wherein the log structured file system manages a logical and a physical organization of the second sub-file within the second data file; and the data transformation module transmitting a second write request to the data storage device, the second write request requesting storage of the second data file on the data storage device according to the first file system. - View Dependent Claims (13, 14, 15, 16, 17, 18, 19, 20, 21, 22)
- at least one processor;
Specification