Method and system for distributed analysis

• US 10,541,936 B1
• Filed: 10/31/2017
• Issued: 01/21/2020
• Est. Priority Date: 04/06/2015
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Florissi, Patricia Gomes Soares
• Primary Examiner(s): Tokuta, Shean
• Application Number:
  US15/799,314
• Time in Patent Office: 812 Days
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

A computing device of a data zone includes a persistent storage and a processor. The persistent storage includes a data source. The processor obtains a global computation request, instantiates a global computation based on the global computation request, and instantiates an intermediate computation in a second data zone based on the instantiated global computation.

Claims

1. A computing device of a first data zone, comprising:

a persistent storage comprising at least a first data source; and

a processor programmed;

to obtain a first global computation request;

to instantiate a first global computation based at least in part on the first global computation request;

to instantiate a first intermediate computation on first data stored in the first data source based at least in part on the instantiated first global computation;

to instantiate at least a second intermediate computation on second data stored in a second data source in a second data zone based at least in part on the instantiated first global computation;

to obtain, from the instantiated first intermediate computation, a first intermediate computation result, the first intermediate computation result being based at least in part on the first data stored in the first data source;

to obtain, from the instantiated second intermediate computation, a second intermediate computation result without transferring the second data stored in the second data source from the second data zone to the first data zone, the second intermediate computation result being based at least in part on the second data stored in the second data source; and

to generate a first global computation result specified by the first global computation request, the first global computation result being based at least in part on the first intermediate computation result and the second intermediate computation results;

wherein instantiating the first global computation based at least in part on the first global computation request comprises;

determining a type of computation result specified by the first global computation request;

matching the determined type of computation result to a template; and

generating executable code for the first global computation based at least in part on a global computation prototype of the matched template.

2. The computing device of claim 1, wherein the global computation request specifies the first data source in the first data zone and the second data source in the second data zone.

3. The computing device of claim 1, wherein the matched template comprises:

the global computation prototype; and

an intermediate computation prototype.

4. The computing device of claim 3, wherein instantiating the second intermediate computation in the second data zone based at least in part on the instantiated first global computation comprises:

identifying the second data source in the second data zone based at least in part on an input of the instantiated first global computation; and

generating executable code for the second intermediate computation based at least in part on the intermediate computation prototype of the matched template.

5. The computing device of claim 1, wherein instantiating the second intermediate computation in a second data zone based at least in part on the instantiated first global computation comprises:

instantiating a third intermediate computation on third data stored in at least a third data source in a third data zone based at least in part on the instantiated second intermediate computation in the second data zone.

6. The computing device of claim 1, wherein the first global computation request specifies:

a requesting entity;

at least one computation to be performed; and

grouping criteria for identifying a plurality of data sources, including at least the first data source and the second data source, on which the at least one computation is to be performed.

7. The computing device of claim 1, wherein instantiating the second intermediate computation comprises instantiating a second global computation on a computing device of the second data zone, and wherein the second global computation:

instantiates one or more additional intermediate computations on data stored in one or more additional data sources in one or more additional data zones; and

generates a second global computation result based at least in part on one or more additional intermediate computation results, obtained from the one or more additional intermediate computations, without transferring the data stored in the one or more additional data sources from the one or more additional data zones to the second data zone.

8. The computing device of claim 7, wherein the second intermediate computation result is based at least in part on the second global computation result.

9. The computing device of claim 1, wherein the first data stored in the first data source comprises first sensor data collected from a first set of sensors, the second data stored in the second data source comprises second sensor data collected from a second set of sensors, the first intermediate computation result anonymizes the first sensor data collected from the first set of sensors, and the second intermediate computation result anonymizes the second sensor data collected from the second set of sensors.

10. The computing device of claim 1, wherein:

the first data in the first data source comprises dynamically changing first sensor data streamed from a first set of sensors to the first data zone;

the second data in the second data source comprises dynamically changing second sensor data streamed from a second set of sensors to the second data zone;

the first intermediate computation result is associated with a first intermediate computation result identifier, the first intermediate computation result identifier specifying a first time stamp identifying the dynamically changing first sensor data at a time of generation of the first intermediate computation result; and

the second intermediate computation result is associated with a second intermediate computation result identifier, the second intermediate computation result identifier specifying a second time stamp identifying the dynamically changing second sensor data at a time of generation of the second intermediate computation result.

11. The computing device of claim 10, wherein the processor is further programmed to associate a first global computation result identifier associated with the first global computation result, the first global computation result identifier specifying a third time stamp identifying a time associated with generation of the first global computation result.

12. A method of operating a computing device of a first data zone, comprising:

obtaining, by the computing device, a first global computation request;

instantiating, by the computing device, a first global computation based at least in part on the first global computation request;

instantiating, by the computing device, a first intermediate computation on first data stored in the first data source based at least in part on the instantiated first global computation;

instantiating, by the computing device, at least a second intermediate computation on second data stored in a second data zone based at least in part on the instantiated first global computation;

obtaining, from the instantiated first intermediate computation, a first intermediate computation result, the first intermediate computation result being based at least in part on the first data stored in the first data source;

obtaining, from the instantiated second intermediate computation, a second intermediate computation result without transferring the second data stored in the second data zone to the first data zone, the second intermediate computation result being based at least in part on the second data stored in the second data source; and

generating a first global computation result specified by the first global computation request, the first global computation result being based at least in part on the first intermediate computation result and the second intermediate computation result;

wherein instantiating the first global computation based at least in part on the first global computation request comprises;

determining a type of computation result specified by the first global computation request;

matching the determined type of computation result to a template; and

generating executable code for the first global computation based at least in part on a global computation prototype of the matched template.

13. The method of claim 12, wherein

instantiating the second intermediate computation in the second data zone based at least in part on the instantiated first global computation comprises:

identifying the second data source in the second data zone based at least in part on an input of the instantiated first global computation; and

generating executable code for the second intermediate computation based at least in part on an intermediate computation prototype of the matched template.

14. The method of claim 12, wherein instantiating the second intermediate computation comprises instantiating a second global computation on a computing device of the second data zone, and wherein the second global computation:

instantiates one or more additional intermediate computations on data stored in one or more additional data sources in one or more additional data zones; and

generates a second global computation result based at least in part on one or more additional intermediate computation results, obtained from the one or more additional intermediate computations without transferring the data stored in the one or more additional data sources from the one or more additional data zones to the second data zone.

15. The method of claim 12, wherein the first data stored in the first data source comprises first sensor data collected from a first set of sensors, the second data stored in the second data source comprises second sensor data collected from a second set of sensors, the first intermediate computation result anonymizes the first sensor data collected from the first set of sensors, and the second intermediate computation result anonymizes the second sensor data collected from the second set of sensors.

16. A non-transitory computer readable medium comprising computer readable program code, which when executed by a computer processor enables the computer processor to perform a method for operating a computing device of a data zone, the method comprising:

obtaining, by the computing device, a first global computation request;

instantiating, by the computing device, a first global computation based at least in part on the first global computation request;

instantiating, by the computing device, a first intermediate computation on first data stored in the first data source based at least in part on the instantiated first global computation;

instantiating, by the computing device, at least a second intermediate computation on second data stored in a second data zone based at least in part on the instantiated first global computation;

obtaining, from the instantiated first intermediate computation, a first intermediate computation result, the first intermediate computation result being based at least in part on the first data stored in the first data source;

obtaining, from the instantiated second intermediate computation, a second intermediate computation result without transferring the second data stored in the second data zone to the first data zone, the second intermediate computation result being based at least in part on the second data stored in the second data source; and

generating a first global computation result specified by the first global computation request, the first global computation result being based at least in part on the first intermediate computation result and the second intermediate computation result;

wherein instantiating the first global computation based at least in part on the first global computation request comprises;

determining a type of computation result specified by the first global computation request;

matching the determined type of computation result to a template; and

generating executable code for the first global computation based at least in part on a global computation prototype of the matched template.

17. The non-transitory computer readable medium of claim 16, wherein

instantiating the second intermediate computation in the second data zone based at least in part on the instantiated first global computation comprises:

identifying the second data source in the second data zone based at least in part on an input of the instantiated first global computation; and

generating executable code for the second intermediate computation based at least in part on an intermediate computation prototype of the matched template.

18. The non-transitory computer readable medium of claim 16, wherein instantiating the second intermediate computation comprises instantiating a second global computation on a computing device of the second data zone, and wherein the second global computation:

instantiates one or more additional intermediate computations on data stored in one or more additional data sources in one or more additional data zones; and

generates a second global computation result based at least in part on one or more additional intermediate computation results, obtained from the one or more additional intermediate computations without transferring the data stored in the one or more additional data sources from the one or more additional data zones to the second data zone.

19. The non-transitory computer readable medium of claim 16, wherein the first data stored in the first data source comprises first sensor data collected from a first set of sensors, the second data stored in the second data source comprises second sensor data collected from a second set of sensors, the first intermediate computation result anonymizes the first sensor data collected from the first set of sensors, and the second intermediate computation result anonymizes the second sensor data collected from the second set of sensors.

20. The non-transitory computer readable medium of claim 16, wherein:

the first data in the first data source comprises dynamically changing first sensor data streamed from a first set of sensors to the first data zone;

the second data in the second data source comprises dynamically changing second sensor data streamed from a second set of sensors to the second data zone;

the first intermediate computation result is associated with a first intermediate computation result identifier, the first intermediate computation result identifier specifying a first time stamp identifying the dynamically changing first sensor data at a time of generation of the first intermediate computation result; and

the second intermediate computation result is associated with a second intermediate computation result identifier, the second intermediate computation result identifier specifying a second time stamp identifying the dynamically changing second sensor data at a time of generation of the second intermediate computation result.