Detecting instabilities in time series forecasting

• US 20070150077A1
• Filed: 12/28/2005
• Published: 06/28/2007
• Est. Priority Date: 12/28/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Heckerman, David, Chickering, David, Bocharov, Alexei
• Granted Patent: US 7,617,010 B2
• Time in Patent Office: Days
• US Class Current: 700/29
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases

Abstract

A predictive model analysis system comprises a receiver component that receives predictive samples created by way of forward sampling. An analysis component analyzes a plurality of the received predictive samples and automatically determines whether a predictive model is reliable at a time range associated with the plurality of predictive sample, wherein the determination is made based at least in part upon an estimated norm associated with a forward sampling operator.

Claims

1. A data monitoring system that comprises the following computer-executable components:

a receiver component that receives predictive samples created by way of forward sampling; and

an analysis component that analyzes a plurality of the received predictive samples and automatically determines whether a predictive model is reliable at a time range associated with the plurality of predictive samples, the determination is based at least in part upon an estimated norm of a forward sampling operator (FS) associated with the predictive model.

2. The system of claim 1, the analysis component comprises a measuring component that determines the forward sampling operator (FS).

3. The system of claim 2, the analysis component further comprises a derivative component that creates a derivative of FS (FS′

).

4. The system of claim 3, the analysis component derives the estimated norm in closed form.

5. The system of claim 4, the analysis component further comprises an estimation component that estimates a rate of change of divergence by determining an expectation of |FS′

|, where

$E<br><br>\left(<br><br>{\mathrm{FS}}^{\mathrm{\prime <br><br>}}<br><br>\right)=\frac{\sum <br><br>{<br><br>{\mathrm{FS}}^{\mathrm{\prime <br><br>}}<br><br>}_s}{N},$

where E(|FS′

|) is an expectation of |FS′

| and N is a number of samples.

6. The system of claim 5, the estimation component creates a sequence of norm estimates through creation of a vector of norm estimates, {right arrow over (v)}[j], where {right arrow over (v)}[j]=E(|FS(j)′

|), 1≦

j<

k, where j relates to a time associated with a prediction and k is a constant.

7. The system of claim 6, further comprising an evaluation component that determines whether entries in the vector of norm estimates are less than a predefined threshold.

8. The system of claim 7, the evaluation component determines that an entry in the vector of norm estimates is greater than the threshold and monitors entries following the entry that is greater than the threshold to ensure that a pre-defined threshold number of entries subsequent the entry are below the pre-defined threshold.

9. The system of claim 8, the evaluation component deems a forward sampling prediction unstable and halts subsequent predictions if the number of acceptable vector entries subsequent the entry are below the predefined threshold number of entries.

10. The system of claim 1, the predictive model is an autoregressive predictive model.

11. The system of claim 1, further comprising a selection component that selects a number of predictive samples to be provided to the analysis component.

12. The system of claim 11, further comprising a calculator component that generates a joint distribution estimate with respect to time-series data at a future point in time, the predictive model comprises the calculator component.

13. The system of claim 1, further comprising a data store that includes time-series data, the predictive model accesses the time-series data and utilizes forward sampling to create the predictive samples.

14. The system of claim 1, a server comprises the receiver component and the analysis component.

15. The system of claim 1, a client comprises the receiver component and the analysis component.

16. A computer-implemented method for determining operability of a predictive model with respect to predicted values of variables at future instances in time, comprising the following computer-executable acts:

receiving predictive samples from a predictive model that employs forward sampling; and

automatically determining a distance in time in the future that the predictive model can reliably predict values of at least one variable based at least in part upon the received predictive samples.

17. The method of claim 16, further comprising measuring a norm of a forward sampling operator that relates to a subset of the received predictive samples, the automatic determination of the distance in time in the future is based at least in part upon the measured norm.

18. The method of claim 17, further comprising estimating a rate of change of divergence based at least in part upon the measured norm.

19. The method of claim 18, further comprising ensuring that a particular rate of change of divergence lies within a predefined threshold.

20. A system that facilitates monitoring of a predictive model, comprising:

computer-implemented means for receiving predicted values for at least one variable, the predicted values obtained by way of forward sampling;

computer-implemented means for estimating a rate of change of divergence based at least in part upon the received predicted values; and

computer-implemented means for determining how far in the future the predictive model can accurately generate predictions based at least in part upon the estimated rate of change of divergence.