Data engine integration and data refinement
First Claim
1. A computer-implemented method comprising:
- receiving, by an in-memory engine, a data request;
transmitting, by the in-memory engine and to a metadata storage unit, data that is associated with the request;
receiving, by the in-memory engine and from the metadata storage unit, metadata;
based on the metadata, generating a query tree that includes query tasks for processing;
determining, by the in-memory engine, a first portion of the data to be processed by the in-memory engine according to a first group of the query tasks and a second portion of the data to be processed by a data engine according to a second group of the query tasks to balance a processing load of the in-memory engine and a processing load of the data engine;
transmitting, by the in-memory engine and to the data engine, a request (i) for the first portion of the data and (ii) to process the second portion of the data according to the second group of the query tasks;
receiving, by the in-memory engine and from the data engine, the first portion of the data and the second portion of the data that is processed by the data engine;
processing, by the in-memory engine, the first portion of the data according to the first group of the query tasks; and
in response to the data request, providing, by the in-memory engine, the first portion of the data that is processed by the in-memory engine and the second portion of the data that is processed by the data engine.
2 Assignments
0 Petitions
Accused Products
Abstract
Data engine integration and data refinement are described. The actions include receiving, by an in-memory engine, a request for data. The actions include transmitting, by the in-memory engine and to a metadata storage unit, data that is associated with the request. The actions include receiving metadata. The actions include determining a first amount of processing to be performed by the in-memory engine and a second amount of processing to be performed by a data engine. The actions include transmitting a request (i) for a first portion of unprocessed data and (ii) to perform a second amount of processing on a second portion of unprocessed data. The actions include receiving the first portion of unprocessed data and the second portion of processed data. The actions include processing the first portion of unprocessed data. The actions include, providing the first portion of processed data and the second portion of processed data.
-
Citations
14 Claims
-
1. A computer-implemented method comprising:
-
receiving, by an in-memory engine, a data request; transmitting, by the in-memory engine and to a metadata storage unit, data that is associated with the request; receiving, by the in-memory engine and from the metadata storage unit, metadata; based on the metadata, generating a query tree that includes query tasks for processing; determining, by the in-memory engine, a first portion of the data to be processed by the in-memory engine according to a first group of the query tasks and a second portion of the data to be processed by a data engine according to a second group of the query tasks to balance a processing load of the in-memory engine and a processing load of the data engine; transmitting, by the in-memory engine and to the data engine, a request (i) for the first portion of the data and (ii) to process the second portion of the data according to the second group of the query tasks; receiving, by the in-memory engine and from the data engine, the first portion of the data and the second portion of the data that is processed by the data engine; processing, by the in-memory engine, the first portion of the data according to the first group of the query tasks; and in response to the data request, providing, by the in-memory engine, the first portion of the data that is processed by the in-memory engine and the second portion of the data that is processed by the data engine. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A system comprising:
-
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising; receiving, by an in-memory engine, a data request; transmitting, by the in-memory engine and to a metadata storage unit, data that is associated with the request; receiving, by the in-memory engine and from the metadata storage unit, metadata; based on the metadata, generating a query tree that includes query tasks for processing; determining, by the in-memory engine, a first portion of the data to be processed by the in-memory engine according to a first group of the query tasks and a second portion of the data to be processed by a data engine according to a second group of the query tasks to balance a processing load of the in-memory engine and a processing load of the data engine; transmitting, by the in-memory engine and to the data engine, a request (i) for the first portion of the data and (ii) to process the second portion of the data according to the second group of the query tasks; receiving, by the in-memory engine and from the data engine, the first portion of the data and the second portion of the data that is processed by the data engine; processing, by the in-memory engine, the first portion of the data according to the first group of the query tasks; and in response to the data request, providing, by the in-memory engine, the first portion of the data that is processed by the in-memory engine and the second portion of the data that is processed by the data engine. - View Dependent Claims (7, 8, 9, 10)
-
-
11. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
-
receiving, by an in-memory engine, a data request; transmitting, by the in-memory engine and to a metadata storage unit, data that is associated with the request; receiving, by the in-memory engine and from the metadata storage unit, metadata; based on the metadata, generating a query tree that includes query tasks for processing; determining, by the in-memory engine, a first portion of the data to be processed by the in-memory engine according to a first group of the query tasks and a second portion of the data to be processed by a data engine according to a second group of the query tasks to balance a processing load of the in-memory engine and a processing load of the data engine; transmitting, by the in-memory engine and to the data engine, a request (i) for the first portion of the data and (ii) to process the second portion of the data according to the second group of the query tasks; receiving, by the in-memory engine and from the data engine, the first portion of the data and the second portion of the data that is processed by the data engine; processing, by the in-memory engine, the first portion of the data according to the first group of the query tasks; and in response to the data request, providing, by the in-memory engine, the first portion of the data that is processed by the in-memory engine and the second portion of the data that is processed by the data engine. - View Dependent Claims (12, 13, 14)
-
Specification