Database tuple-encoding-aware data partitioning in a direct memory access engine
First Claim
1. A method for partitioning columns of rows among co-processors, comprising:
- for each data descriptor of a first set of data descriptors, copying a respective column of said columns that is at a source memory to an intermediate memory;
wherein each data descriptor of said data descriptors specifiesa width of the respective column each data descriptor;
a number of rows;
a respective source memory location for said respective column;
a destination memory location within said intermediate memory;
for a second descriptor that specifies a particular algorithm, generating, according to the particular algorithm, a column of core processor identifiers that are each indexed to a respective row of said rows and that identify a respective core processor of said core processors;
for each core partitioning descriptor of a set of core partitioning descriptors, copying each row of a respective column of said columns from said intermediate memory to a scratch pad memory of the core processor identified by the respective core processor identifier indexed to said each row, said respective core processor being indexed to said each row in said column of core processor identifiers;
wherein each core partitioning descriptor of said set of core partitioning descriptors specifiesa width of the respective column of said each core partitioning descriptor;
a number of rows;
a respective source memory location in said intermediate memory for the respective column of each core partitioning descriptor;
a destination memory location.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques provide for hardware accelerated data movement between main memory and an on-chip data movement system that comprises multiple core processors that operate on the tabular data. The tabular data is moved to or from the scratch pad memories of the core processors. While the data is in-flight, the data may be manipulated by data manipulation operations. The data movement system includes multiple data movement engines, each dedicated to moving and transforming tabular data from main memory data to a subset of the core processors. Each data movement engine is coupled to an internal memory that stores data (e.g. a bit vector) that dictates how data manipulation operations are performed on tabular data moved from a main memory to the memories of a core processor, or to and from other memories. The internal memory of each data movement engine is private to the data movement engine. Tabular data is efficiently copied between internal memories of the data movement system via a copy ring that is coupled to the internal memories of the data movement system and/or is coupled to a data movement engine. Also, a data movement engine internally broadcasts data to other data movement engines, which then transfer the data to respective core processors. Partitioning may also be performed by the hardware of the data movement system. Techniques are used to partition data “in flight”. The data movement system also generates a column of row identifiers (RIDs). A row identifier is a number treated as identifying a row or element'"'"'s position within a column. Row identifiers each identifying a row in column are also generated.
161 Citations
17 Claims
-
1. A method for partitioning columns of rows among co-processors, comprising:
-
for each data descriptor of a first set of data descriptors, copying a respective column of said columns that is at a source memory to an intermediate memory; wherein each data descriptor of said data descriptors specifies a width of the respective column each data descriptor; a number of rows; a respective source memory location for said respective column; a destination memory location within said intermediate memory; for a second descriptor that specifies a particular algorithm, generating, according to the particular algorithm, a column of core processor identifiers that are each indexed to a respective row of said rows and that identify a respective core processor of said core processors; for each core partitioning descriptor of a set of core partitioning descriptors, copying each row of a respective column of said columns from said intermediate memory to a scratch pad memory of the core processor identified by the respective core processor identifier indexed to said each row, said respective core processor being indexed to said each row in said column of core processor identifiers; wherein each core partitioning descriptor of said set of core partitioning descriptors specifies a width of the respective column of said each core partitioning descriptor; a number of rows; a respective source memory location in said intermediate memory for the respective column of each core partitioning descriptor; a destination memory location. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A method for partitioning columns of a first set of rows among co-processors that reside on a same chip, comprising:
-
for each data descriptor of a first set of data descriptors, copying a respective column of said columns that is at a source memory to an intermediate memory in response to a memory address of said each data descriptor being pushed onto a register of first electronic circuitry; wherein each data descriptor of said data descriptors specifies a width of the respective column of said each data descriptor; a number of rows; a respective source memory location for of said each data descriptor; a destination memory location within said intermediate memory; wherein said first electronic circuitry resides on said same chip and is coupled to each core processor of said core processors and a respective scratch pad memory of said each core processor, said respective scratch pad memory residing on said same chip; for a second descriptor that specifies a particular algorithm, generating, according to the particular algorithm, a column of core processor identifiers that are each indexed to a respective row of said rows and that identify a respective core processor of said core processors in response to a memory address of said second descriptor being pushed onto a register of said first electronic circuitry; for each core partitioning descriptor of a set of core partitioning descriptors, copying each row of a respective column of said columns from said intermediate memory to the respective scratch pad memory of the core processor identified by the respective core processor identifier indexed to said each row, in response to a memory address of said each core partitioning descriptor being pushed onto a register of said first electronic circuitry, said respective core processor being indexed to said each row in said column of core processor identifiers; wherein each core partitioning descriptor of said core partitioning descriptors specifies; a width of the respective column of said each core partitioning descriptor; a number of rows; a respective source memory location in said intermediate memory for said respective column of said each core partitioning descriptor; a destination memory location. - View Dependent Claims (16, 17)
-
Specification