SCALABLE CLOUD-BASED TIME SERIES ANALYSIS

• US 20200110602A1
• Filed: 11/27/2019
• Published: 04/09/2020
• Est. Priority Date: 11/16/2017
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Leonard, Michael James, Quirino, Thiago Santos, Blair, Edward Tilden, Beeman, Jennifer Leigh Sloan, Elsheimer, David Bruce, Delgado, Javier
• Granted Patent: US 10,642,610 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 2201/835 Timestamp
• CPC Class Codes: G06F 8/60 Software deployment
• CPC Class Codes: G06F 8/76 Adapting program code to run in a different environment; Porting
• CPC Class Codes: G06F 9/5072 Grid computing

Abstract

In some examples, computing devices can partition timestamped data into groups. The computing devices can then distribute the timestamped data based on the groups. The computing devices can also obtain copies of a script configured to process the timestamped data, such that each computing device receives a copy of the script. The computing devices can determine one or more code segments associated with the groups based on content of the script. The one or more code segments can be in one or more programming languages that are different than a programming language of the script. The computing devices can then run the copies of the script to process the timestamped data within the groups. This may involve interacting with one or more job servers configured to run the one or more code segments associated with the groups.

Claims

1. A system comprising:

one or more data processors associated with a plurality of computing devices; and

a non-transitory computer-readable storage medium comprising instructions that are executable by the one or more data processors for causing the plurality of computing devices to;

read timestamped data and partition the timestamped data into a plurality of groups according to a criterion;

distribute the timestamped data among the plurality of computing devices based on the plurality of groups;

obtain copies of a script configured to process the timestamped data, each computing device among the plurality of computing devices receiving a copy of the script, the script comprising instructions for analyzing the timestamped data;

obtain one or more code segments associated with the plurality of groups based on content of the script, the one or more code segments being in one or more programming languages that are different than a programming language of the script, the one or more code segments being referenceable to a string variable, an external file or an external table, and the script including one or more commands for facilitating execution of the one or more code segments in the different programming language; and

run the copy of the script in each computing device to process the timestamped data within the plurality of groups, each copy of the script being run on the timestamped data in a respective group among the plurality of groups at least in part by interacting with a job server configured to run a respective code segment associated with the respective group, the respective code segment comprising a reference to the string variable, the external file or the external table, the job server being compatible with a particular programming language of the respective code segment and being configured to;

generate a modified code segment from the respective code segment in the programming language that is different from the script; and

run the modified code segment to generate respective model values associated with the respective group.

2. The system of claim 1, wherein the content of the script includes references to:

an identifier of a code segment to run with the job server in relation to the group of timestamped data; and

a programming language identifier that indicates the programming language of the code segment to run with the job server.

3. The system of claim 1, wherein the content of the script includes references to one or more external tables for logging output data associated with the one or more code segments.

4. The system of claim 1, wherein the content of the script is configured to dynamically generate the one or more code segments from a combination of one or more conditional statements,

wherein the one or more conditional statements yields one or more code portions from the string variable, from the external file, or from the external table, and

wherein a combination of the one or more code portions defines the one or more code segments.

5. The system of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions that are executable by the one or more data processors for causing the plurality of computing devices to:

determine that the script includes one or more references to one or more files containing the one or more code segments, the one or more files being external to the script;

or

determine that the script includes a reference to at least one table entry in a table that is external to the script;

determine that the reference to the at least one file or the at least one table entry includes a variable value that is changeable based on the timestamped data in the respective group; and

determine the one or more code segments associated with the timestamped data in the respective group based on the variable value.

6. The system of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions that are executable by the one or more data processors for causing the plurality of computing devices to:

compile at least one of the copies of the script into an executable file;

select the job server from a plurality of job servers based on the job server being compatible with the particular programming language to run the one or more code segments to produce output values;

run the executable file on the timestamped data within the respective group among the plurality of groups to produce the output values; and

return the produced output values to the executable file.

7. The system of claim 1, wherein each copy of the script is precompiled into an executable file prior to each computing device among the plurality of computing devices receiving the copy of the script.

8. The system of claim 1, wherein the job server is compatible with an interpretative programming language, and

wherein the job server includes an interpreter for running the respective code segment with the interpretative programming language.

9. The system of claim 1, wherein the job server is compatible with a compiled programming language, and

wherein the job server includes a compiler for running the respective code segment with the compiled programming language.

10. The system of claim 1, wherein the job server is configured to:

receive a request from each computing device among the plurality of computing devices associated with the one or more code segments;

generate header code and footer code associated with the one or more code segments, the modified code segment incorporating the generated header code and the generated footer code, the header code or the footer code facilitating integration between the one or more code segments and the script; and

transmit to the requesting computing device the generated respective model values associated with the respective group.

11. The system of claim 1, wherein running the copy of the script in each computing device to process the timestamped data within the plurality of groups comprises:

running a first copy of the script in a first computing device with a first operating system; and

running a second copy of the script in second computing device with a second operation system, the second operating system being different than the first operating system.

12. A computer-implemented method comprising:

reading timestamped data and partitioning the timestamped data into a plurality of groups according to a criterion;

distributing the timestamped data among a plurality of computing devices based on the plurality of groups;

obtaining copies of a script configured to process the timestamped data, each computing device among the plurality of computing devices receiving a copy of the script, the script comprising instructions for analyzing the timestamped data;

obtaining one or more code segments associated with the plurality of groups based on content of the script, the one or more code segments being in one or more programming languages that are different than a programming language of the script, the one or more code segments being referenceable to a string variable, an external file or an external table, and the script including one or more commands for facilitating execution of the one or more code segments in the different programming language; and

running the copy of the script in each computing device to process the timestamped data within the plurality of groups, each copy of the script being run on the timestamped data in a respective group among the plurality of groups at least in part by interacting with a job server configured to run a respective code segment associated with the respective group, the respective code segment comprising a reference to the string variable, the external file or the external table, the job server being compatible with a particular programming language of the respective code segment and being configured to;

generating a modified code segment from the respective code segment in the programming language that is different from the script; and

running the modified code segment to generate respective model values associated with the respective group.

13. The computer-implemented method of claim 12, wherein the content of the script includes references to:

an identifier of a code segment to run with the job server in relation to the group of timestamped data; and

a programming language identifier that indicates the programming language of the code segment to run with the job server.

14. The computer-implemented method of claim 12, wherein the content of the script includes references to one or more external tables for logging output data associated with the one or more code segments.

15. The computer-implemented method of claim 12, wherein the content of the script is configured to dynamically generate the one or more code segments from a combination of one or more conditional statements,

wherein the one or more conditional statements yields one or more code portions from the string variable, from the external file, or from the external table, and

wherein a combination of the one or more code portions defines the one or more code segments.

16. The computer-implemented method of claim 12, further comprising:

determining that the script includes one or more references to one or more files containing the one or more code segments, the one or more files being external to the script;

or determining that the script includes a reference to at least one table entry in a table that is external to the script;

determining that the reference to the at least one file or the at least one table entry includes a variable value that is changeable based on the timestamped data in the respective group; and

determining the one or more code segments associated with the timestamped data in the respective group based on the variable value.

17. The computer-implemented method of claim 12, further comprising:

compiling at least one of the copies of the script into an executable file;

selecting the job server from a plurality of job servers based on the job server being compatible with the particular programming language to run the one or more code segments to produce output values;

running the executable file on the timestamped data within the respective group among the plurality of groups to produce the output values; and

returning the produced output values to the executable file.

18. The computer-implemented method of claim 12, wherein each copy of the script is precompiled into an executable file prior to each computing device among the plurality of computing devices receiving the copy of the script.

19. The computer-implemented method of claim 12, wherein the job server is compatible with an interpretative programming language, and

wherein the job server includes an interpreter for running the respective code segment with the interpretative programming language.

20. The computer-implemented method of claim 12, wherein the job server is compatible with a compiled programming language, and

wherein the job server includes a compiler for running the respective code segment with the compiled programming language.

21. The computer-implemented method of claim 12, wherein the job server is configured to:

receive a request from each computing device among the plurality of computing devices associated with the one or more code segments;

generate header code and footer code associated with the one or more code segments, the modified code segment incorporating the generated header code and the generated footer code, the header code or the footer code facilitating integration between the one or more code segments and the script; and

transmit to the requesting computing device the generated respective model values associated with the respective group.

22. A non-transitory computer-readable medium comprising program code that is executable by one or more data processors associated with a plurality of computing devices for causing the plurality of computing devices to:

read timestamped data and partition the timestamped data into a plurality of groups according to a criterion;

distribute the timestamped data among the plurality of computing devices based on the plurality of groups;

obtain copies of a script configured to process the timestamped data, each computing device among the plurality of computing devices receiving a copy of the script, the script comprising instructions for analyzing the timestamped data;

obtain one or more code segments associated with the plurality of groups based on content of the script, the one or more code segments being in one or more programming languages that are different than a programming language of the script, the one or more code segments being referenceable to a string variable, an external file or an external table, and the script including one or more commands for facilitating execution of the one or more code segments in the different programming language; and

run the copy of the script in each computing device to process the timestamped data within the plurality of groups, each copy of the script being run on the timestamped data in a respective group among the plurality of groups at least in part by interacting with a job server configured to run a respective code segment associated with the respective group, the respective code segment comprising a reference to the string variable, the external file or the external table, the job server being compatible with a particular programming language of the respective code segment and being configured to;

generate a modified code segment from the respective code segment in the programming language that is different from the script; and

run the modified code segment to generate respective model values associated with the respective group.

23. The non-transitory computer-readable medium of claim 22, wherein the content of the script includes references to:

an identifier of a code segment to run with the job server in relation to the group of timestamped data; and

a programming language identifier that indicates the programming language of the code segment to run with the job server.

24. The non-transitory computer-readable medium of claim 22, wherein the content of the script includes references to one or more external tables for logging output data associated with the one or more code segments.

25. The non-transitory computer-readable medium of claim 22, wherein the content of the script is configured to dynamically generate the one or more code segments from a combination of one or more conditional statements,

wherein the one or more conditional statements yields one or more code portions from the string variable, from the external file, or from the external table, and

wherein a combination of the one or more code portions defines the one or more code segments.

26. The non-transitory computer-readable medium of claim 22, further comprising program code that is executable by the one or more data processors for causing the plurality of computing devices to:

determine that the script includes one or more references to one or more files containing the one or more code segments, the one or more files being external to the script;

or

determine that the script includes a reference to at least one table entry in a table that is external to the script;

determine that the reference to the at least one file or the at least one table entry includes a variable value that is changeable based on the timestamped data in the respective group; and

determine the one or more code segments associated with the timestamped data in the respective group based on the variable value.

27. The non-transitory computer-readable medium of claim 22, further comprising program code that is executable by the one or more data processors for causing the plurality of computing devices to:

compile at least one of the copies of the script into an executable file;

select the job server from a plurality of job servers based on the job server being compatible with the particular programming language to run the one or more code segments to produce output values;

run the executable file on the timestamped data within the respective group among the plurality of groups to produce the output values; and

return the produced output values to the executable file.

28. The non-transitory computer-readable medium of claim 22, wherein each copy of the script is precompiled into an executable file prior to each computing device among the plurality of computing devices receiving the copy of the script.

29. The non-transitory computer-readable medium of claim 22, wherein the job server is compatible with an interpretative programming language, and

wherein the job server includes an interpreter for running the respective code segment with the interpretative programming language.

30. The non-transitory computer-readable medium of claim 22, wherein the job server is compatible with a compiled programming language, and

wherein the job server includes a compiler for running the respective code segment with the compiled programming language.