Asymmetric data streaming architecture having autonomous and asynchronous job processing unit
First Claim
1. An asymmetric data processing system comprising:
- a first group of one or more host computers, each comprising a memory, a network interface and one or more Central Processing Units (CPUs), each host computer accepting and responding to requests to process data;
a second group of two or more Job Processing Units (JPUs), operating autonomously and asynchronously from one another, each JPU comprising a memory, a network interface, a data interface with exclusive access to one or more sources of data, and one or more general purpose CPUs, each JPU in the second group being responsive to requests received from a host computer to execute jobs and return results independent of execution at other JPUs, the jobs containing instructions for the processing of a particular subset of data under the JPU'"'"'s exclusive control; and
a network connecting the network interfaces within each group and between the two groups;
wherein the sources of data comprise one or more storage devices which are directly accessed by no other JPU in the second group and by none of the host computers in first group.
8 Assignments
0 Petitions
Accused Products
Abstract
In a preferred embodiment, the present invention is an asymmetric data processing system having two or more groups of processors that have attributes that are optimized for their assigned functions. A first processor group, which may be SMP machines, are responsible for interfacing with applications and/or end users to obtain queries, and for planning query execution. A second processor group consists of many streaming record-oriented processors called Job Processing Units (JPUs), typically arranged as an MPP structure. The JPUs carry out the bulk of the data processing required to implement the logic of a query, running autonomously and asynchronously from other processors in the system. The JPUs preferably use a multi-tasking operating system that permits multiple tasks to run at a given instant in time, in either an absolute-priority-based or a weighted-priority-based demand scheduling environment.
177 Citations
37 Claims
-
1. An asymmetric data processing system comprising:
-
a first group of one or more host computers, each comprising a memory, a network interface and one or more Central Processing Units (CPUs), each host computer accepting and responding to requests to process data; a second group of two or more Job Processing Units (JPUs), operating autonomously and asynchronously from one another, each JPU comprising a memory, a network interface, a data interface with exclusive access to one or more sources of data, and one or more general purpose CPUs, each JPU in the second group being responsive to requests received from a host computer to execute jobs and return results independent of execution at other JPUs, the jobs containing instructions for the processing of a particular subset of data under the JPU'"'"'s exclusive control; and a network connecting the network interfaces within each group and between the two groups; wherein the sources of data comprise one or more storage devices which are directly accessed by no other JPU in the second group and by none of the host computers in first group. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35)
-
-
36. An asymmetric data processing system comprising:
-
a first group of one or more host computers, each comprising a memory, a network interface and one or more Central Processing Units (CPUs), each host computer accepting and responding to requests to process data; a second group of two or more Job Processing Units (JPUs), operating autonomously and asynchronously from one another, each JPU comprising a memory, a network interface, a data interface with exclusive access to one or more sources of data, and one or more general purpose CPUs. each JPU in the second group being responsive to requests received from a host computer to execute jobs and return results independent of execution at other JPUs, the jobs containing instructions for the processing of a particular subset of data under the JPU'"'"'s exclusive control; and a network connecting the network interfaces within each group and between the two groups; wherein the sources of data comprise an external source of streaming data, such that the streaming data is directly accessed by no other JPU in the second group and by none of the host computers in the first group.
-
-
37. An asymmetric data processing system comprising:
-
a first group of one or more host computers, each comprising a memory, a network interface and one or more Central Processing Units (CPUs), each host computer accepting and responding to requests to process data; a second group of two or more Job Processing Units (JPUs), operating autonomously and asynchronously from one another, each JPU comprising a memory, a network interface, a data interface with exclusive access to one or more sources of data, and one or more general purpose CPUs, each JPU in the second group being responsive to requests received from a host computer to execute jobs and return results independent of execution at other JPUs, the jobs containing instructions for the processing of a particular subset of data under the JPU'"'"'s exclusive control; and a network connecting the network interfaces within each group and between the two groups; wherein JPUs in the second tier are embedded components that are not directly accessible to applications that present data Processing requests to the hosts; and wherein the hosts in the first group are exclusively responsible for interfacing to external applications, thereby supporting the use of JPUs having different processing capabilities, without requiring changes to be made to the applications making requests.
-
Specification