Systems and methods for transforming data in buffer memory without unnecessarily copying data to additional memory locations
First Claim
Patent Images
1. A method for transforming data comprising:
- extracting data comprising a plurality of rows wherein each row comprises at least one column from at least one external data source;
storing the data in a buffer;
establishing a first set of pointers to the data, the first set of pointers representative of a first transform to the at least one column in the plurality of rows;
establishing a second set of pointers to the data, the second set of pointers representative of a second transform to the at least one column in the plurality of rows, wherein the second transform is based on the first transform;
passing the first set of pointers and the second set of pointers to at least one component in order for the at least one component to apply the first transform and the second transform to the at least one column in the plurality of rows directly in the buffer without moving the data within the buffer or copying the data to another location within the buffer or to another storage medium; and
loading the data transformed by the first transform and second transform to at least one database table.
2 Assignments
0 Petitions
Accused Products
Abstract
Various embodiments of the present invention are directed to a unique memory management scheme utilized by the DTP whereby data extracted from an external source is placed in a memory buffer and is then manipulated by the components without the need for copying the data to any other location in memory. While logically the data moves from component to component in the DFE for the various embodiments described herein, the data does not in fact change locations but, instead, the data resides in the buffer and is operated upon by a series of components that, in turn, access the data in the buffer via pointers and manipulate same.
-
Citations
25 Claims
-
1. A method for transforming data comprising:
-
extracting data comprising a plurality of rows wherein each row comprises at least one column from at least one external data source; storing the data in a buffer; establishing a first set of pointers to the data, the first set of pointers representative of a first transform to the at least one column in the plurality of rows; establishing a second set of pointers to the data, the second set of pointers representative of a second transform to the at least one column in the plurality of rows, wherein the second transform is based on the first transform; passing the first set of pointers and the second set of pointers to at least one component in order for the at least one component to apply the first transform and the second transform to the at least one column in the plurality of rows directly in the buffer without moving the data within the buffer or copying the data to another location within the buffer or to another storage medium; and loading the data transformed by the first transform and second transform to at least one database table. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A method for transforming data comprising:
-
extracting data from a source, said data comprising a plurality of rows; writing the data to a buffer; creating a plurality of pointers wherein each pointer uniquely points to a single row of data from among the plurality of rows of data in the buffer; passing the plurality of pointers to a first transformation objects in a path, wherein the first transformation object applies a first transformation to the data in series, said transformation objects directly accessing the data in the buffer via the pointers, wherein the passing the plurality of pointers is done without moving the data within the buffer or copying the data to another location within the buffer or to another storage medium; passing the plurality of pointers to a second transformation object when there remains transformations objects unexecuted in the path, wherein the second transformation object applies a first transformation to the data in series, wherein the second transform is based on the first transform; reading the data from the buffer; and loading the data transformed by the first transform and second transform to a destination. - View Dependent Claims (10, 11, 12, 13)
-
-
14. A tangible computer-readable medium bearing computer-readable instructions for:
-
extracting data comprising a plurality of rows wherein each row comprises at least one column from at least one external data source; storing the data in a buffer; establishing a first set of pointers to the data, the first set of pointers representative of a first transform to the at least one column in the plurality of rows; establishing a second set of pointers to the data, the second set of pointers representative of a second transform to the at least one column in the plurality of rows, wherein the second transform is based on the first transform; passing the first set of pointers and the second set of pointers to at least one component in order for the at least one component to apply the first transform and the second transform to the at least one column in the plurality of rows directly in the buffer without moving the data within the buffer or copying the data to another location within the buffer or to another storage medium; loading the data transformed by the first transform and second transform to at least one database table. - View Dependent Claims (15, 16, 17, 18, 19)
-
-
20. A system comprising a processor, memory, and instructions, wherein the instructions, if executed by the processor cause the system to:
-
input a graph describing the data flow among a plurality of components; extract data comprising a plurality of rows from at least one external data source; store the data in a buffer; pass a first set of pointers and a second set of pointers to the data in the buffer to at least one component in the plurality of components in order for the at least one component to transform the data directly in the buffer without moving the data within the buffer or copying the data to another location within the buffer or to another storage medium; and loading the data from the buffer to at least one external data destination; wherein the second set of pointers is based on the first set of pointers. - View Dependent Claims (21, 22, 23, 24, 25)
-
Specification