Method and system to identify discrete trends in time series

• US 7,818,224 B2
• Filed: 03/22/2001
• Issued: 10/19/2010
• Est. Priority Date: 03/22/2001
• Status: Expired due to Fees

• Current Assignee: Sean T. Boerner
• Original Assignee: Sean T. Boerner
• Inventors: Boerner, Sean T.
• Primary Examiner(s): Nguyen; Nga B.
• Application Number:
  US09/815,360
• Publication Number:
  US 20030009399A1
• Time in Patent Office: 3,498 Days
• Field of Search: 705/35, 705/36, 705/37
• US Class Current: 705/35
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange

Abstract

A signal processing system and method for breaking a time series into piece-wise discrete trends and determining whether new data represents the continuation of a trend. The method identifies and utilizes at least one set of trend determination parameters, which have favorable trend fit characteristics relative to other possible sets of parameters. In a semi-automated embodiment, the error and trend length characteristics are cross-plotted for multiple sets of possible parameters, and one or more of the parameter sets is selected from the graph. In an automated embodiment, an objective function is formulated from the characteristics, and an optimization technique is applied to identify one or more good parameter sets.

Claims

1. A computer system for breaking a time series into a plurality of discontinuous trends, the system comprising:

an input means for inputting time series data to a computer, the time series comprising a plurality of data elements, at least a portion of which represents a trend which is generally increasing or decreasing;

a memory means for storing the time series data;

a processing means for

selecting a plurality of sets of trend determination parameters for the time series, each set of trend determination parameters comprising at least one window size, such that the window size defines a number of adjacent data elements from the time series to be used to generate trends;

selecting a useful group of sets of trend determination parameters for the time series from the plurality of sets of trend determination parameters, such that the useful group of sets includes at least one member;

processing the time series with each member of the useful group of sets of trend determination parameters to generate a set of trends and trend attributes for each member;

evaluating the trend attributes for each member;

selecting at least one set of trends; and

an output means for outputting the set of trends from the computer.

2. The system of claim 1 wherein the processing means further comprises

deciding, based on the composite results of a plurality of the members of the useful group of sets of trend determination parameters, whether the newest data element of the time series represents a continuation of a trend, such that the trend is increasing, decreasing, or flat.

3. The system of claim 1 wherein inputting time series data to a computer further comprises

inputting a plurality of time series data sets to the computer;

and

selecting a particular time series from a plurality of time series data sets.

4. The system of claim 1 wherein inputting time series data to a computer further comprises

inputting a plurality of vector datasets to the computer;

and

selecting a particular vector data set from a plurality of time series data sets.

5. The system of claim 3 wherein selecting a particular time series from a plurality of time series data sets further comprises

for each of the plurality of time series data sets:

selecting at least a portion of the elements in the data set to create a selected data subset;

normalizing the selected data subset to generate a normalized subset for the time series;

storing the normalized subset on the computer;

calculating, on a processor, the slope of a best-fit polynomial regression through the normalized subset; and

selecting a particular time series that has a large absolute slope and a large correlation coefficient between the trend and the data elements.

6. The system of claim 1 wherein the processing means further comprises

specifying a range of values for each of a plurality of trend determination parameters;

and

generating the sets of trend determination parameters by selecting unique combinations of trend determination parameter values, such that the values are within the range of values for each of the plurality of trend determination parameters.

7. The system of claim 6 wherein specifying a range of values for each of a plurality of trend determination parameters further comprises

specifying a minimum value for a first trend determination parameter of initial data window size;

specifying a maximum value for a first trend determination parameter of initial data window size;

specifying a minimum value for a second trend determination parameter of deviation limit; and

specifying a maximum value for a second trend determination parameter of deviation limit.

8. The system of claim 1 wherein the processing means further comprises

specifying a range of potential values for each of a plurality of trend determination parameters;

creating an objective function from at least one indications of trend results, such that the objective function generates a resultant value for a set of trend determination parameters; and

selecting a useful group of sets of trend determination parameters by applying an optimization procedure to the objective function and the range of potential values for each of the plurality of trend determination parameters.

9. The system of claim 1 wherein selecting a useful group of sets of trend determination parameters for the time series from the plurality of sets of time series parameters further comprises

for each trend determination parameter set:

applying the trend determination parameters to at least a portion of the time series data elements, thereby generating at least one trend and at least one trend attribute parameter;

identifying the dynamic trend and at least one dynamic trend attribute parameter;

calculating at least one indication of trend results between the time series and the trend set; and

storing at least one indication of trend results on the computer; and

selecting at least one set of trend determination parameters based on at least one indication of trend results for each of the sets.

10. The system of claim 9 wherein applying the trend determination parameters to at least a portion of the time series data elements thereby generating at least one trend and at least one trend attribute parameter further comprises

assigning each of the data elements to at least one trend by generating a first trend with at least a portion of the data elements;

identifying the first trend as the current trend;

evaluating each subsequent data element to determine whether the data element is a continuation of the current trend, and assigning the data element to the current trend if it is a continuation of the current trend, and assigning the data element to a new trend if it is not continuation of the current trend, and identifying the new trend as the current trend; and

determining at least one trend attribute for each trend.

11. The system of claim 9 wherein calculating at least one indication of trend results between the time series and trend set further comprises

calculating at least one measure from the group consisting of:

the number of trends in the subset of the time series, the RMS Error between the input data values and trend values, the average trend length;

the average trend length divided by the minimum number of data points needed to define a trend (window size parameter), the average percent return of the trends, the summed cumulative percent return of the trends;

the fraction of correct predictions, the fraction of incorrect predictions, the quotient of the root mean square error and the average length of the trends divided by the minimum number of data points needed to define a trend, and the RMS error*L_s the efficiency of the trends, where efficiency is defined as the average return of the trends divided by the average length of the trends, and compounded return of the trends.

12. The system of claim 9 wherein selecting at least one set of trend determination parameters based on at least one indication of trend results for each of the sets further comprises

plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters and its associated trend results, where a first axis represents a first measure of trend results and a second axis represents a second measure of trend results;

and

selecting from the graph at least one data point that represents a trend determination parameters set that has desirable trend results.

13. The system of claim 12 wherein plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters and their associated trend results and selecting from the graph at least one data point that represents a parameters set that has desirable trend results further comprises

plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters, where the y-axis represents the product of the root mean square error and the deviation limit for the set of trend determination parameters, and the x-axis represents the average trend length for the set of trend determination parameters;

and

selecting at least one set of trend determination parameters associated with a point from the graph that has a minimum value for the product of the root mean square error and the deviation limit for a given average trend length.

14. The system of claim 12 wherein plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters and their associated trend results and selecting from the graph at least one data point that represents a parameters set that has desirable trend results further comprises

plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters, where the y-axis represents the root mean square error for the set of trend determination parameters, and x-axis represents the average trend length for the set of trend determination parameters;

and

selecting at least one set of trend determination parameters associated with a point from the graph that has a minimum value for the root mean square error for a given average trend length.

15. The system of claim 12 wherein plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters and their associated trend results and selecting from the graph at least one data point that represents a parameters set that has desirable trend results further comprises

plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters, where the y-axis represents the summed cumulative percent return of the trends for the set of trend determination parameters, and x-axis represents the average percent return for the set of trend determination parameters;

and

selecting at least one set of trend determination parameters associated with a point from the graph that has a maximum value for both the summed cumulative percent return and the average percent return.

16. The system of claim 12 wherein plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters and their associated trend results and selecting from the graph at least one data point that represents a parameters set that has desirable trend results further comprises

plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters, where the y-axis represents the summed cumulative percent return of the trends for the set of trend determination parameters, and x-axis represents the fraction of correct predictions for the set of trend determination parameters;

and

selecting at least one set of trend determination parameters associated with a point from the graph that has a maximum value for both the summed cumulative percent return and the fraction of correct predictions.

17. The system of claim 12 wherein plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters and their associated trend results and selecting from the graph at least one data point that represents a parameters set that has desirable trend results further comprises

plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters, where the y-axis represents the average percent return of the trends for the set of trend determination parameters, and x-axis represents the fraction of correct predictions for the set of trend determination parameters;

and

selecting at least one set of trend determination parameters associated with a point from the graph that has a maximum value for both the average percent return and the fraction of correct predictions.

18. The system of claim 12 wherein plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters and their associated trend results and selecting from the graph at least one data point that represents a parameters set that has desirable trend results further comprises

plotting on a graph a representative data point for each of the plurality of sets of trend determination parameters, where the y-axis represents the average percent return for the trends for the set of trend determination parameters, and x-axis represents the average trend length for the set of trend determination parameters;

and

selecting at least one set of trend determination parameters associated with a point from the graph that has a maximum value for the average percent return for a given average trend length.

19. The system of claim 9 wherein selecting at least one set of trend determination parameters based on at least one indication of trend results for each of the sets further comprises

specifying an objective function that incorporates at least one measure of trend results, such that minimization of the objective function produces desirable trend results;

applying an optimization technique to the objective function such that the optimization technique minimizes the objective function; and

selecting at least one set of trend determination parameters as a result of minimizing the objective function.

20. The system of claim 1 wherein processing the time series with each member of the useful group of sets of trend determination parameters to generate a set of trends and trend attributes for each member further comprises

for each member applying the trend determination parameters to at least a portion of the time series data elements, thereby generating at least one trend and at least one trend attribute parameter;

and

identifying the dynamic trend and at least one dynamic trend attribute parameter; and

storing at least one dynamic trend attribute parameter on the computer.

21. The system of claim 20 wherein applying the trend determination parameters to at least a portion of the time series data elements thereby generating at least one trend and at least one trend attribute parameter further comprises

assigning each of the data elements to at least one trend by generating a first trend with at least a portion of the data elements;

identifying the first trend as the current trend;

evaluating each subsequent data element to determine whether the data element is a continuation of the current trend, and assigning the data element to the current trend if it is a continuation of the current trend, and assigning the data element to a new trend if it is not continuation of the current trend, and identifying the new trend as the current trend; and

determining at least one trend attribute for each trend.

22. The system of claim 21 wherein generating a first trend with at least a portion of the data elements further comprises

recalling the first trend determination parameter, the data window size, from the set of trend determination parameters;

forming a proposed trend data set by selecting a number of data elements from the time series, such that the number of data elements selected is at least as large as the value of the data window size trend determination parameter;

calculating a first best-fit curve through the proposed trend data set; and

identifying the best fit curve as the first trend.

23. The system of claim 21 wherein evaluating each subsequent data element to determine whether the data element is a continuation of the current trend further comprises

forming a proposed trend data set from the selected data elements;

calculating a new best-fit curve through the proposed trend data set;

calculating at least one measure of predictive error for the new best-fit curve with respect to the values of the data elements in the proposed trend data set;

projecting the best-fit curve to the location of the subsequent data element;

evaluating the deviation of the subsequent data element from the projected best-fit curve value at the new location; and

applying at least one acceptance criteria to the measure of predictive error, and if the acceptance criteria is met, setting the subsequent data element to the proposed trend data set, identifying the new best-fit curve as the current trend, and if the acceptance criteria is not met, setting the subsequent data element to a new trend set, determining a new trend, and identifying the new trend as the current trend.

24. The system of claim 23 wherein calculating at least one measure of predictive error for the new best-fit curve with respect to the values of the data elements in the proposed data set further comprises

evaluating the derivative of the best-fit curve;

obtaining an estimated value of the best-fit curve at the new element;

calculating the residuals between the proposed trend data set and the new best-fit curve;

normalizing the residuals;

quantifying the spread of the distribution of the normalized residuals;

calculating the deviation of predicted trend value at the new element from the actual value of the new element; and

normalizing the deviation of predicted trend value at the new element from the actual value of the new element using the spread of the distribution of normalized residuals.

25. The system of claim 23 wherein applying at least one acceptance criteria to the measure of predictive error further comprises

designating a first test criterion as true if the sign of the derivative of the trend curve generated through the proposed trend dataset changes compared to the sign of the derivative of the trend curve generated previously;

designating a second test criterion as true if the absolute value of normalized deviation exceeds the deviation limit parameter;

designating a third test criterion as true if the absolute value of normalized deviation exceeds the deviation limit parameter and the normalized deviation is in the opposite direction as the direction of the trend as designated by the derivative of the trend curve through the proposed trend dataset;

designating a fourth test criterion as true if the correlation coefficient between the proposed trend curve and the proposed trend dataset decreases below the correlation limit parameter;

designating a fifth test criterion as true if the number of times that the absolute value of normalized deviation exceeds the deviation limit parameter; and

the normalized deviation is in the opposite direction as the direction of the trend as designated by the derivative of the trend curve through the proposed trend dataset;

exceeds the number of values parameter;

designating a sixth test criterion as true if the absolute value of the normalized deviation from a flat trend exceeds the flat trend deviation limit; and

determining whether at least one of the first, second, third, fourth, fifth, and sixth test criteria is true.