EXPLOITING PARALLELISM IN EXPONENTIAL SMOOTHING OF LARGE-SCALE DISCRETE DATASETS

• US 20150234686A1
• Filed: 02/19/2015
• Published: 08/20/2015
• Est. Priority Date: 02/20/2014
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: MUKHIN, Denis B., ZHANG, Lei, ABOYOUN, Patrick Edward
• Granted Patent: US 9,798,589 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system

Abstract

A system, and computer program product for large-scale data transformations. Embodiments include a smoothing engine within an R environment to configure at least one master task and at least two worker tasks. A chunk calculator receives a series of data values and divides the series of data values into portions of data values which are in turn assigned as workloads to at least two worker tasks. The worker tasks serve to calculate a first state value of a first one of the portions of data values, and calculate a second state value of a second one of the portions of data values. The workloads are selected such that calculating a second state value does not depend on the first state value. The results of the workload calculations are used to calculate a smoothing factor used to predict a trend.

Claims

1. A system comprising:

one or more processors corresponding to a plurality of processing entities, wherein the plurality of processing entities comprises at least one master entity and at least a first and second worker entities;

one or more memory segments being concurrently accessible by at least some of the plurality of processing entities, wherein a series of data values is received into the one or more memory segments and divided into a plurality of portions of data values; and

the plurality of processing entities configurable to process the plurality of portions of data values in the one or more memory segments for the first worker entity and the second worker entity to calculate a first state value by the first worker entity and a second state value by the second worker entity to perform a smoothing function, and

wherein the second state value does not depend on the first state value such that the first worker entity is operable in parallel with the second worker entity to determine results of the smoothing function.

2. The system of claim 1, wherein the first worker entity calculates a smoothing factor using a sum of errors determined based at least in part on the first state value and the second state value.

3. The system of claim 2, wherein the smoothing factor is calculated using an exponential smoothing factor objective function.

4. The system of claim 2, wherein the smoothing factor is used to predict a trend.

5. The system of claim 1, wherein the data values comprise a time series of observations.

6. The system of claim 1, wherein the first worker entity performs a first workload on a first node, and the second worker entity performs a second workload on a second node.

7. The system of claim 1, wherein the portions of data values are load balanced over at least two worker entities running on at least two hardware computing nodes, and wherein the master entity performs the load balancing.

8. The system of claim 1, wherein the first worker entity is run on database server, and the second worker entity is run on an external data appliance.

9. The system of claim 8, wherein the external data appliance comprises a database query processor.

10. The system of claim 8, wherein the external data appliance uses Hadoop to perform query processing.

11. A method comprising:

configuring one or more processors corresponding to a plurality of processing entities, wherein the plurality of processing entities comprises at least one master entity and at least a first and second worker entities;

populating one or more memory segments being concurrently accessible by at least some of the plurality of processing entities, wherein a series of data values is received into the one or more memory segments and divided into a plurality of portions of data values; and

invoking at least some of the plurality of processing entities to process the plurality of portions of data values in the one or more memory segments for the first worker entity and the second worker entity to calculate a first state value by the first worker entity and a second state value by the second worker entity to perform a smoothing function, and

wherein the second state value does not depend on the first state value such that the first worker entity is operable in parallel with the second worker entity to determine results of the smoothing function.

12. The method of claim 11, wherein the first worker entity calculates a smoothing factor using a sum of errors determined based at least in part on the first state value and the second state value.

13. The method of claim 12, wherein the smoothing factor is calculated using an exponential smoothing factor objective function.

14. The method of claim 12, wherein the smoothing factor is used to predict a trend.

15. The method of claim 11, wherein the data values comprise a time series of observations.

16. The method of claim 11, wherein the first worker entity performs a first workload on a first node, and the second worker entity performs a second workload on a second node.

17. The method of claim 11, wherein the portions of data values are load balanced over at least two worker entities running on at least two hardware computing nodes, and wherein the master entity performs the load balancing.

18. The method of claim 11, wherein the first worker entity is run on database server, and the second worker entity is run on an external data appliance.

19. A computer program product, embodied in a non-transitory computer readable medium, the computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a process, the computer program product comprising:

program code for configuring one or more processors corresponding to a plurality of processing entities, wherein the plurality of processing entities comprises at least one master entity and at least a first and second worker entities;

program code for populating one or more memory segments being concurrently accessible by at least some of the plurality of processing entities, wherein a series of data values is received into the one or more memory segments and divided into a plurality of portions of data values; and

program code for invoking at least some of the plurality of processing entities to process the plurality of portions of data values in the one or more memory segments for the first worker entity and the second worker entity to calculate a first state value by the first worker entity and a second state value by the second worker entity to perform a smoothing function, and

wherein the second state value does not depend on the first state value such that the first worker entity is operable in parallel with the second worker entity to determine results of the smoothing function.

20. The computer program product of claim 19, wherein the first worker entity calculates a smoothing factor using a sum of errors determined based at least in part on the first state value and the second state value.