Computer-implemented systems and methods for time series exploration

• US 10,037,305 B2
• Filed: 02/06/2018
• Issued: 07/31/2018
• Est. Priority Date: 07/13/2012
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Leonard, Michael James, Blair, Edward Tilden, Brzezicki, Jerzy Michal, Sglavo, Udo V., Pothireddy, Sujatha
• Primary Examiner(s): Titcomb, William
• Application Number:
  US15/890,019
• Publication Number:
  US 20180157620A1
• Time in Patent Office: 175 Days
• Field of Search: 715804
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 17/10 Complex mathematical operations function generation by table look-up G06F1/03; evaluation of elementary functions by calculation G06F7/544
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06Q 10/04 Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" market predictions or forecasting for commercial activities G06Q30/0202
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

Systems and methods are provided for analyzing unstructured time stamped data. A distribution of time-stamped data is analyzed to identify a plurality of potential time series data hierarchies for structuring the data. An analysis of a potential time series data hierarchy may be performed. The analysis of the potential time series data hierarchies may include determining an optimal time series frequency and a data sufficiency metric for each of the potential time series data hierarchies. One of the potential time series data hierarchies may be selected based on a comparison of the data sufficiency metrics. Multiple time series may be derived in a single-read pass according to the selected time series data hierarchy. A time series forecast corresponding to at least one of the derived time series may be generated.

Claims

1. A system comprising:

one or more processors;

one or more computer-readable storage mediums containing instructions configured to cause the one or more processors to perform operations including;

deriving, in a single-read pass, multiple structured time series from a distribution of time-stamped unstructured data, wherein the multiple structured time series are in longitudinal form, and wherein the multiple structured time series include a structured time series with a longitudinal dimension associated with a set of attributes;

identifying, in the single-read pass, a reduction function from a functions repository;

applying, in the single-read pass, the reduction function to the structured time series, wherein applying the reduction function includes mapping data from the longitudinal dimension of the structured time series to data in a coordinate dimension of a time series in coordinate form;

identifying, in the single-read pass, a reduced set of attributes from the structured time series using the reduction function, wherein the reduced set of attributes is a subset of attributes extracted from the longitudinal dimension of the structured time series, and wherein the reduced set of attributes includes data that describes the structured time series as a whole;

generating, in the single-read pass, a reduced time series using the structured time series and the reduced set of attributes, wherein the reduced time series is in coordinate form, wherein the reduced time series is a subset of the structured time series, and wherein the reduced time series captures the reduced set of attributes; and

generating a time series forecast corresponding to the reduced time series.

2. The system of claim 1, wherein identifying the reduction function comprises identifying a user-specified reduction function from the functions repository, wherein identifying the user-specified reduction function from the functions repository includes using a function specification, wherein the function specification describes the contents of the functions repository.

3. The system of claim 1, wherein the reduced time series is a summarized version of the structured time series.

4. The system of claim 1, wherein the operations further comprise:

applying, in the single-read pass, the reduction function to other structured time series of the multiple structured time series;

identifying, in the single-read pass, a reduced set of attributes from each of the multiple structured time series using the reduction function;

generating, in the single-read pass, a reduced time series associated with each of the multiple structured time series; and

generating a time series forecast corresponding using the reduced time series associated with each of the multiple structured time series.

5. The system of claim 4, wherein the operations further comprise:

generating the time series forecast using parallelization, wherein the reduced time series associated with each of the multiple structured time series are processed separately on separate computational threads.

6. The system of claim 1, wherein the reduced time series in coordinate form includes a fixed coordinate dimension.

7. The system of claim 1, wherein identifying the reduced set of attributes from the structured time series using the reduction function includes using a combination of transactional and time series analysis techniques.

8. The system of claim 1, wherein the operations further comprise:

analyzing, in the single-read pass, the distribution of time-stamped unstructured data to identify a plurality of potential time series data hierarchies for structuring the unstructured data; and

selecting, in the single-read pass, a time series data hierarchy from the plurality of potential time series data hierarchies.

9. The system of claim 8, wherein analyzing the distribution of time-stamped unstructured data includes identifying a plurality of time series frequencies, wherein deriving the multiple structured time series from the unstructured data includes using an optimal time series frequency from the plurality of time series frequencies, and wherein the optimal time series frequency is associated with the time series data hierarchy.

10. The system of claim 1, wherein the function specification maps each function in the functions repository to attribute variables in the time-stamped unstructured data for re-use with other data sets after the time-stamped unstructured data has been structured.

11. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause a data processing apparatus to perform operations including:

deriving, in a single-read pass, multiple structured time series from a distribution of time-stamped unstructured data, wherein the multiple structured time series are in longitudinal form, and wherein the multiple structured time series include a structured time series with a longitudinal dimension associated with a set of attributes;

identifying, in the single-read pass, a reduction function from a functions repository;

applying, in the single-read pass, the reduction function to the structured time series, wherein applying the reduction function includes mapping data from the longitudinal dimension of the structured time series to data in a coordinate dimension of a time series in coordinate form;

identifying, in the single-read pass, a reduced set of attributes from the structured time series using the reduction function, wherein the reduced set of attributes is a subset of attributes extracted from the longitudinal dimension of the structured time series, and wherein the reduced set of attributes includes data that describes the structured time series as a whole;

generating, in the single-read pass, a reduced time series using the structured time series and the reduced set of attributes, wherein the reduced time series is in coordinate form, wherein the reduced time series is a subset of the structured time series, and wherein the reduced time series captures the reduced set of attributes; and

generating a time series forecast corresponding to the reduced time series.

12. The computer-program product of claim 11, wherein identifying the reduction function comprises identifying a user-specified reduction function from the functions repository, wherein identifying the user-specified reduction function from the functions repository includes using a function specification, wherein the function specification describes the contents of the functions repository.

13. The computer-program product of claim 11, wherein the reduced time series is a summarized version of the structured time series.

14. The computer-program product of claim 11, wherein the operations further comprise:

applying, in the single-read pass, the reduction function to other structured time series of the multiple structured time series;

identifying, in the single-read pass, a reduced set of attributes from each of the multiple structured time series using the reduction function;

generating, in the single-read pass, a reduced time series associated with each of the multiple structured time series; and

generating a time series forecast corresponding using the reduced time series associated with each of the multiple structured time series.

15. The computer-program product of claim 14, wherein the operations further comprise:

generating the time series forecast using parallelization, wherein the reduced time series associated with each of the multiple structured time series are processed separately on separate computational threads.

16. The computer-program product of claim 11, wherein the reduced time series in coordinate form includes a fixed coordinate dimension.

17. The computer-program product of claim 11, wherein identifying the reduced set of attributes from the structured time series using the reduction function includes using a combination of transactional and time series analysis techniques.

18. The computer-program product of claim 11, wherein the operations further comprise:

analyzing, in the single-read pass, the distribution of time-stamped unstructured data to identify a plurality of potential time series data hierarchies for structuring the unstructured data; and

selecting, in the single-read pass, a time series data hierarchy from the plurality of potential time series data hierarchies.

19. The computer-program product of claim 18, wherein analyzing the distribution of time-stamped unstructured data includes identifying a plurality of time series frequencies, wherein deriving the multiple structured time series from the unstructured data includes using an optimal time series frequency from the plurality of time series frequencies, and wherein the optimal time series frequency is associated with the time series data hierarchy.

20. The computer-program product of claim 11, wherein the function specification maps each function in the functions repository to attribute variables in the time-stamped unstructured data for re-use with other data sets after the time-stamped unstructured data has been structured.

21. A computer-implemented method, the method comprising:

deriving, in a single-read pass, multiple structured time series from a distribution of time-stamped unstructured data, wherein the multiple structured time series are in longitudinal form, and wherein the multiple structured time series include a structured time series with a longitudinal dimension associated with a set of attributes;

identifying, in the single-read pass, a reduction function from a functions repository;

applying, in the single-read pass, the reduction function to the structured time series, wherein applying the reduction function includes mapping data from the longitudinal dimension of the structured time series to data in a coordinate dimension of a time series in coordinate form;

identifying, in the single-read pass, a reduced set of attributes from the structured time series using the reduction function, wherein the reduced set of attributes is a subset of attributes extracted from the longitudinal dimension of the structured time series, and wherein the reduced set of attributes includes data that describes the structured time series as a whole;

generating, in the single-read pass, a reduced time series using the structured time series and the reduced set of attributes, wherein the reduced time series is in coordinate form, wherein the reduced time series is a subset of the structured time series, and wherein the reduced time series captures the reduced set of attributes; and

generating a time series forecast corresponding to the reduced time series.

22. The method of claim 21, wherein identifying the reduction function comprises identifying a user-specified reduction function from the functions repository, wherein identifying the user-specified reduction function from the functions repository includes using a function specification, wherein the function specification describes the contents of the functions repository.

23. The method of claim 21, wherein the reduced time series is a summarized version of the structured time series.

24. The method of claim 21, further comprising:

applying, in the single-read pass, the reduction function to other structured time series of the multiple structured time series;

identifying, in the single-read pass, a reduced set of attributes from each of the multiple structured time series using the reduction function;

generating, in the single-read pass, a reduced time series associated with each of the multiple structured time series; and

generating a time series forecast corresponding using the reduced time series associated with each of the multiple structured time series.

25. The method of claim 24, further comprising:

generating the time series forecast using parallelization, wherein the reduced time series associated with each of the multiple structured time series are processed separately on separate computational threads.

26. The method of claim 21, wherein the reduced time series in coordinate form includes a fixed coordinate dimension.

27. The method of claim 21, wherein identifying the reduced set of attributes from the structured time series using the reduction function includes using a combination of transactional and time series analysis techniques.

28. The method of claim 21, further comprising:

analyzing, in the single-read pass, the distribution of time-stamped unstructured data to identify a plurality of potential time series data hierarchies for structuring the unstructured data; and

selecting, in the single-read pass, a time series data hierarchy from the plurality of potential time series data hierarchies.

29. The method of claim 28, wherein analyzing the distribution of time-stamped unstructured data includes identifying a plurality of time series frequencies, wherein deriving the multiple structured time series from the unstructured data includes using an optimal time series frequency from the plurality of time series frequencies, and wherein the optimal time series frequency is associated with the time series data hierarchy.

30. The method of claim 21, wherein the function specification maps each function in the functions repository to attribute variables in the time-stamped unstructured data for re-use with other data sets after the time-stamped unstructured data has been structured.