System and method for continuous diagnosis of data streams

• US 20060010093A1
• Filed: 06/30/2004
• Published: 01/12/2006
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Yu, Philip S., Fan, Wei, Wang, Haixun
• Granted Patent: US 7,464,068 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/906 Clustering; Classification
• CPC Class Codes: G06F 2216/03 Data mining
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

In connection with the mining of time-evolving data streams, a general framework that mines changes and reconstructs models from a data stream with unlabeled instances or a limited number of labeled instances. In particular, there are defined herein statistical profiling methods that extend a classification tree in order to guess the percentage of drifts in the data stream without any labelled data. Exact error can be estimated by actively sampling a small number of true labels. If the estimated error is significantly higher than empirical expectations, there preferably re-sampled a small number of true labels to reconstruct the decision tree from the leaf node level.

Claims

1. An apparatus for facilitating the mining of time-evolving data streams, said apparatus comprising:

an input arrangement for accepting a data stream comprising unlabeled data; and

an arrangement for determining an amount of drifts in the data stream comprising unlabeled data;

said determining arrangement being adapted to employ a signature profile of an inductive model in determining an amount of drifts in the data stream.

2. The apparatus according to claim 1, wherein said determining arrangement is adapted to determine a percentage of drifts in the data stream.

3. The apparatus according to claim 1, wherein said determining arrangement is adapted to employ a signature profile in reconstructing an inductive model via minor model replacement.

4. The apparatus according to claim 1, wherein said determining arrangement is adapted to employ statistical measures to define the profile of an inductive model.

5. The apparatus according to claim 1, wherein said determining arrangement is adapted to employ statistical measures to estimate the error rate of an inductive model.

6. A method of facilitating the mining of time-evolving data streams, said method comprising the steps of:

accepting a data stream comprising unlabeled data; and

determining an amount of drifts in the data stream comprising unlabeled data;

said determining step comprising employing a signature profile of an inductive model in determining an amount of drifts in the data stream.

7. The method according to claim 6, wherein said determining step comprises determining a percentage of drifts in the data stream.

8. The method according to claim 6, wherein said employing step comprises employing a signature profile in reconstructing an inductive model via minor model replacement.

9. The method according to claim 6, wherein said determining arrangement step comprises employing statistical measures to define the profile of an inductive model.

10. The method according to claim 6, wherein said determining step comprises employing statistical measures to estimate the error rate of an inductive model.

11. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for facilitating the mining of time-evolving data streams, said method comprising the steps of:

accepting a data stream comprising unlabeled data; and

determining an amount of drifts in the data stream comprising unlabeled data;

said determining step comprising employing a signature profile of an inductive model in determining an amount of drifts in the data stream.