Method and apparatus for detection and management of data streams

• US 6,986,019 B1
• Filed: 04/21/2003
• Issued: 01/10/2006
• Est. Priority Date: 04/21/2003
• Status: Active Grant

• Current Assignee: Maxtor Corporation (Seagate Technology Holdings Plc)
• Original Assignee: Maxtor Corporation (Seagate Technology Holdings Plc)
• Inventors: Schlumberger, Maurice, Bagashev, Yuri, Ruster, Jean-Pierre
• Primary Examiner(s): Ellis, Kevin L.
• Application Number:
  US10/419,402
• Time in Patent Office: 995 Days
• Field of Search: 711/113, 711/137, 711/213, 711/217
• US Class Current: 711/217
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache

Abstract

A system for the detection and support of data streams is disclosed. The system determines whether new commands comprise a data stream. If a new data stream is detected, the system next determines whether adequate resources are available to launch the new data stream. If the system determines that the data stream can be launched, system resources, particularly cache memory space, are assigned to the data stream to provide the data stream with the necessary amount of data throughput needed to support the data stream efficiently. The data stream'"'"'s throughput is the amount of data that the stream requires per unit time. The system monitors all supported data streams to determine when a particular data stream has terminated, at which time resources dedicated to the data stream are released and become available to support other data streams. The cache for each supported data stream is maintained at as full a level as possible, with the cache for the “least full” data stream given priority for refresh. Allocating resources by throughput allows for more efficient resource utilization.

Claims

1. A method for supporting data stream detection and processing in a computing system, the method comprising the steps of:

detecting data streams in the data traffic between a mass storage device and a processor device;

launching newly detected data streams by allocating memory storage for the detected data streams;

managing memory storage to maintain at least a predefined time interval'"'"'s worth of data in memory storage for each launched data stream; and

releasing the memory storage for the launched data streams when the launched data streams terminate.

2. The method of claim 1 wherein the step of detecting data streams further comprises the steps of detecting if a new command forms part of a data stream that has already been launched.

3. The method of claim 2 wherein the step of detecting data streams further comprises the step of detecting if a new command relates to at least one of a plurality of potential data streams being tracked by the computing system.

4. The method of claim 3 wherein the step of detecting data streams further comprises the step of placing each new command that is not part of a data stream that has already been launched and which is not associated with any of the potential data streams being tracked by the computing system into a list of commands which list tracks commands that may form potential data streams.

5. The method of claim 1 wherein memory storage is allocated to launched data streams based on the amount of data needed to support the launched data stream for a predetermined amount of time.

6. The method of claim 1 wherein the step of launching newly detected data streams further comprise the steps of:

determining if the system can support launching a newly detected data stream without interrupting support for already launched data streams; and

launching the newly detected data stream by allocating memory storage to it and filling the memory storage with sufficient data for the newly launched data stream to support the newly launched data stream for enough time to resume managing memory storage for the already launched data streams, if the newly detected data stream can be launched without interrupting support for already launched data streams.

7. The method of claim 6 wherein the step of launching further comprises the step of storing newly detected data streams in a list of data streams which will be launched when resources are available, if the system has determined that it cannot launch the newly detected data stream without interrupting support for already launched data streams.

8. The method of claim 7 wherein the step of launching further comprises prioritizing the list of newly detected data streams that cannot be launched based on the throughput of each newly detected data stream.

9. The method of claim 1 wherein releasing the memory storage occurs when a launched data stream drops below a minimum predefined level of data throughput.

10. A system for detecting and processing data streams, the system comprising:

a data stream detector;

a data stream manager that determines whether a data stream detected by the data stream detector can be launched and allocates an amount of cache memory to each data stream it determines can be launched, the amount of cache memory determined by the throughput of the detected and launched data stream; and

a data stream terminator.

11. The system of claim 10 wherein the data stream detector detects if a command for data comprises part of a data stream already launched and managed by the data stream manager.

12. The system of claim 11 wherein the data stream detector can detect if a command for data comprises part of a potential data stream already being tracked for recognition by the data stream detector.

13. The system of claim 12 wherein the data stream detector can detect if a command for data does not belong to either a data stream already launched and managed or a potential data stream already being tracked and is therefore a potential command of a new data stream that should be tracked.

14. The system of claim 10 wherein the allocated cache memory is continually readjusted based on the variable throughput of the launched data stream.

15. The system of claim 10 wherein the data stream detector can detect a data stream that exhibits one of seven predefined data stream patterns.

16. A data stream processing method comprising the steps of:

detecting at least a first data stream in the data exchanged with a first mass storage device;

launching and supporting the at least first detected data stream, and further comprising determining the throughput of the at least first data stream and allocating cache memory to the at least first data stream based on the throughput so determined; and

detecting that the at least first data stream has ended and terminating support to the at least first data stream.

17. The data stream processing method of claim 16 wherein the step of detecting at least a first data stream further comprises determining whether every new command for data belongs to the at least first data stream that has already been launched and supported.

18. The data stream processing method of claim 17 wherein the step of detecting at least a first data stream further comprises determining whether each new command for data comprises part of other data streams which have been detected, launched and supported.

19. The data stream processing method of claim 18 wherein the step of detecting at least a first data stream further comprises maintaining a list of a plurality of potential data streams, each new command for data being checked to determine if the new command, when combined with one of the plurality of potential data streams now comprises a data stream that should be launched and supported.

20. The data stream processing method of claim 19 wherein the step of detecting at least a first data stream further comprises the step of creating a new potential data stream candidate for the list of potential data streams and adding a new command for data to it, if the new command for data does not form part of an already detected data stream and does not belong to any one of the potential data streams already being tracked.

21. The data stream processing method of claim 16 wherein the allocated cache memory can be adjusted depending on variations in the throughput of the supported data streams.

22. In a data processing system having at least a first microprocessor for requesting and processing data streams, at least a first mass storage device for storing data streams, at least a first cache memory for temporarily storing data retrieved from the mass storage device before transmitting the retrieved data for processing, a data stream controller coupled to the at least first mass storage device, the at least first microprocessor and the at least first cache memory, the data stream controller comprising:

a data stream detector for monitoring the data traffic to and from the at least first mass storage device and for detecting the presence of data streams in the data traffic; and

cache memory controller for launching, supporting and terminating the data streams detected by the detector, wherein the cache memory controller allocates cache memory to each launched and supported data stream based on the data throughput of each data stream.

23. The data stream controller of claim 22, wherein the data stream detector detects whether every new command in the data traffic is associated with a data stream which the cache memory controller has already launched and is supporting.

24. The data stream controller of claim 23, wherein the data stream detector maintains lists of potential data streams and checks whether each new command in the data traffic is associated with one of the potential data streams already being tracked by the detector.

25. The data stream controller of claim 24, wherein the data stream detector creates a new entry in the list of potential data streams if the new command does not belong to either a launched and supported data stream or a potential data stream already being tracked in the list of potential data streams.

26. The data stream controller of claim 22, wherein the amount of cache memory allocated to a data stream can vary over time as the throughput of the data stream varies.

27. The data stream controller of claim 22 wherein the data stream detector can detect a data stream exhibiting at least one of at least seven predefined data stream patterns.

28. A system for detecting and processing data streams, the system comprising:

a data stream detector;

a data stream manager that does not launch a data stream detected by the data stream detector until the system can launch and support the newly detected data stream without affecting the support of previously launched and non-terminated data streams; and

a data stream terminator.

29. The system of claim 28 wherein the data stream detector detects if a command for data comprises part of a data stream already launched and managed by the data stream manager.

30. The system of claim 29 wherein the data stream detector can detect if a command for data comprises part of a potential data stream already being tracked for recognition by the data stream detector.

31. The system of claim 30 wherein the data stream detector can detect if a command for data does not belong to either a data stream already launched and managed or a potential data stream already being tracked and is therefore a potential command of a new data stream that should be tracked.

32. The system of claim 28 wherein the data stream manager prioritizes all data streams that have been detected but which it determined could not be launched, the data stream with the highest priority being launched first after the data stream manager determines that another data stream can be launched.

33. The system of claim 28 wherein the data stream detector can detect a data stream that exhibits one of seven predefined data stream patterns.

34. A system for detecting and processing data streams, the system comprising:

a data stream detector;

a data stream manager; and

a data stream terminator that determines the end of a data stream when a supported data stream'"'"'s throughput drops below a predefined minimum throughput.

35. The system of claim 34 wherein the data stream detector detects if a command for data comprises part of a data stream already launched and managed by the data stream manager.

36. The system of claim 35 wherein the data stream detector can detect if a command for data comprises part of a potential data stream already being tracked for recognition by the data stream detector.

37. The system of claim 36 wherein the data stream detector can detect if a command for data does not belong to either a data stream already launched and managed or a potential data stream already being tracked and is therefore a potential command of a new data stream that should be tracked.

38. The system of claim 34 wherein the data stream detector can detect a data stream that exhibits one of seven predefined data stream patterns.

39. A data stream processing method comprising the steps of:

detecting at least a first data stream in the data exchanged with a first mass storage device;

launching and supporting the at least first detected data stream; and

detecting that the at least first data stream has ended and terminating support to the at least first data stream;

wherein the steps of detecting, launching and supporting further comprise the step of launching a newly detected data stream only if the newly detected data stream can be launched without affecting the support of earlier launched and supported data streams.

40. The data stream processing method of claim 39 wherein the step of detecting at least a first data stream further comprises determining whether every new command for data belongs to the at least first data stream that has already been launched and supported.

41. The data stream processing method of claim 40 wherein the step of detecting at least a first data stream further comprises determining whether each new command for data comprises part of other data streams which have been detected, launched and supported.

42. The data stream processing method of claim 41 wherein the step of detecting at least a first data stream further comprises maintaining a list of a plurality of potential data streams, each new command for data being checked to determine if the new command, when combined with one of the plurality of potential data streams now comprises a data stream that should be launched and supported.

43. The data stream processing method of claim 42 wherein the step of detecting at least a first data stream further comprises the step of creating a new potential data stream candidate for the list of potential data streams and adding a new command for data to it, if the new command for data does not form part of an already detected data stream and does not belong to any one of the potential data streams already being tracked.

44. The data stream processing method of claim 39 wherein a newly detected data stream and is stored in a list of newly detected data streams and held in that list until the data stream can be launched and supported without affecting the support of earlier launched and supported data streams.

45. A data stream processing method comprising the steps of:

detecting at least a first data stream in the data exchanged with a first mass storage device;

launching and supporting the at least first detected data stream; and

detecting that the at least first data stream has ended and terminating support to the at least first data stream, and further comprising monitoring the at least first data stream for a period of time that is based on the throughput of the data stream, so that the terminating a data stream occurs on a variable time interval based on the throughput of each supported data stream.

46. The data stream processing method of claim 45 wherein the step of detecting at least a first data stream further comprises determining whether every new command for data belongs to the at least first data stream that has already been launched and supported.

47. The data stream processing method of claim 46 wherein the step of detecting at least a first data stream further comprises determining whether each new command for data comprises part of other data streams which have been detected, launched and supported.

48. The data stream processing method of claim 47 wherein the step of detecting at least a first data stream further comprises maintaining a list of a plurality of potential data streams, each new command for data being checked to determine if the new command, when combined with one of the plurality of potential data streams now comprises a data stream that should be launched and supported.

49. The data stream processing method of claim 48 wherein the step of detecting at least a first data stream further comprises the step of creating a new potential data stream candidate for the list of potential data streams and adding a new command for data to it, if the new command for data does not form part of an already detected data stream and does not belong to any one of the potential data streams already being tracked.

50. In a data processing system having at least a first microprocessor for requesting and processing data streams, at least a first mass storage device for storing data streams, at least a first cache memory for temporarily storing data retrieved from the mass storage device before transmitting the retrieved data for processing, a data stream controller coupled to the at least first mass storage device, the at least first microprocessor and the at least first cache memory, the data stream controller comprising:

a data stream detector for monitoring the data traffic to and from the at least first mass storage device and for detecting the presence of data streams in the data traffic; and

cache memory controller for launching, supporting and terminating the data streams detected by the detector, wherein the cache memory controller does not launch a newly detected data stream if the launch will impact earlier launched and supported data streams negatively.

51. The data stream controller of claim 50 wherein the data stream detector detects whether every new command in the data traffic is associated with a data stream which the cache memory controller has already launched and is supporting.

52. The data stream controller of claim 51 wherein the data stream detector maintains lists of potential data streams and checks whether each new command in the data traffic is associated with one of the potential data streams already being tracked by the detector.

53. The data stream controller of claim 52 wherein the data stream detector creates a new entry in the list of potential data streams if the new command does not belong to either a launched and supported data stream or a potential data stream already being tracked in the list of potential data streams.

54. The data stream controller of claim 50 wherein the cache memory controller generates a priority number for each newly detected data stream if the newly detected data stream cannot be launched immediately and wherein the cache memory controller launches and supports the newly detected data streams according to their priority number as it becomes possible to launch newly detected data streams without impacting earlier launched and supported data streams negatively.

55. The data stream controller of claim 50 wherein the data stream detector can detect a data stream exhibiting at least one of at least seven predefined data stream patterns.

56. In a data processing system having at least a first microprocessor for requesting and processing data streams, at least a first mass storage device for storing data streams, at least a first cache memory for temporarily storing data retrieved from the mass storage device before transmitting the retrieved data for processing, a data stream controller coupled to the at least first mass storage device, the at least first microprocessor and the at least first cache memory, the data stream controller comprising:

a data stream detector for monitoring the data traffic to and from the at least first mass storage device and for detecting the presence of data streams in the data traffic; and

cache memory controller for launching, supporting and terminating the data streams detected by the detector, wherein the cache memory controller continually checks to determine if each supported data stream'"'"'s throughput has dropped below a predefined minimum, the cache memory controller terminating support to each data stream whose throughput falls below the predefined minimum.

57. The data stream controller of claim 56 wherein the data stream detector detects whether every new command in the data traffic is associated with a data stream which the cache memory controller has already launched and is supporting.

58. The data stream controller of claim 57 wherein the data stream detector maintains lists of potential data streams and checks whether each new command in the data traffic is associated with one of the potential data streams already being tracked by the detector.

59. The data stream controller of claim 58 wherein the data stream detector creates a new entry in the list of potential data streams if the new command does not belong to either a launched and supported data stream or a potential data stream already being tracked in the list of potential data streams.

60. The data stream controller of claim 56 wherein the data stream detector can detect a data stream exhibiting at least one of at least seven predefined data stream patterns.

61. The data stream controller of claim 56, wherein the cache memory controller provides support to each data stream by supplying each data stream with enough data to maintain that data stream at its established throughput for at least an additional predefined minimum period of time.