File server system and method for scheduling data streams according to a distributed scheduling policy

• US 7,146,424 B2
• Filed: 04/12/2002
• Issued: 12/05/2006
• Est. Priority Date: 03/10/1999
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bolosky, William J., Douceur, John R.
• Primary Examiner(s): Najjar, Saleh
• Assistant Examiner(s): DUONG, OANH L
• Application Number:
  US10/121,813
• Publication Number:
  US 20020143979A1
• Time in Patent Office: 1,698 Days
• Field of Search: 709/202, 709/226, 709/231, 709/232, 709/201, 709/203, 709/217, 709/219, 709/224, 709/243, 707/10
• US Class Current: 709/231
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04N 21/2326 Scheduling disk or memory reading operations
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

A distributed file server system has multiple data servers connected to stream data files continuously over a network to multiple clients. Data files are distributed across the data servers so that data blocks of the data files are stored at each of the data servers. A scheduler is located at each of the data servers to manage distributed portions of a schedule. Each data server sees a different portion of the schedule, but no one data server sees the whole schedule. The scheduler facilitates service of requested data streams from its corresponding data server according to a schedule portion that is available to the data server. The scheduler determines whether to insert a new data stream into the current slot it presently owns in its schedule portion, or to wait for a subsequent slot in the schedule based on a thrifty policy.

Claims

1. In a file server system having multiple data servers connected to distribute data streams over a network, each data server supporting at least one storage device, wherein data files are distributed across the data servers and stored on each of the storage devices, the system maintaining a schedule for serving requested ones of the data streams, the schedule being segmented into slots to which data streams are assigned for coordinating service of the requested data streams, a method comprising the following steps:

distributing the schedule among the data servers so that individual data servers view different portions of the schedule;

receiving multiple requests to insert new data streams into the schedule, the schedule formed at least in part based on historical and future slots capable of accommodating data streams;

queuing the request in a queue;

examining the schedule to determine whether all of the queued requests can be inserted into the schedule under a policy that attempts to maximize distances between consecutively occupied slots and minimize contiguously occupieds slots; and

wherein the receiving, queuing, and examining steps are performed locally at individual ones of the data servers.

2. A method as recited in claim 1, wherein the examining step comprises determining whether all of the queued requests can be inserted into the schedule under the policy without exceeding a preset slip period measured as a time between when a request is received and when a corresponding data stream is served.

3. A method as recited in claim 1, further comprising the step of maintaining a central schedule which is used to coordinate service of the is requested data streams, wherein the receiving, queuing, and examining steps are performed centrally.

4. A method as recited in claim 1, wherein the examining step comprises the step of computing the distances between the consecutively occupied slots.

5. A method as recited in claim 1, wherein the examining step comprises the step of deriving a highest number of the contiguously occupied slots.

6. A computer programmed to perform the steps recited in the method of claim 1.

7. A computer-readable medium having computer-executable instructions for performing the steps of the method as recited in claim 1.

8. A file server system, comprising:

multiple data servers, each data server supporting at least one storage device, wherein data files are distributed across the data servers so that data blocks of the data files are stored on each of the storage devices;

a scheduler to coordinate service of requested data streams from the data servers according to a schedule formed at least in part based on historical and future slots capable of accommodating data streams;

a queue to hold multiple requests to insert new data streams into the schedule; and

the scheduler being configured to examine the schedule to determine whether all of the queued requests can be inserted into the schedule under a policy that attempts to maximize distances between consecutively occupied slots and minimize contiguously occupied slots,

wherein the schedule is distributed among the multiple data servers so that individual data servers view different portions of the schedule; and

the scheduler and the queue are located locally at individual ones of the data servers.

9. A file server system as recited in claim 8, further comprising a controller coupled to the data servers, wherein the scheduler resides at the controller.

10. A file server system as recited in claim 8, wherein the scheduler resides at each of the data servers.

11. A file server system as recited in claim 8, wherein the scheduler determines whether all of the queued requests can be inserted into the schedule under the policy without exceeding a preset slip period measured as a time between when a request is received and when a corresponding data stream is served.

12. In a file server system having multiple data servers connected to distribute data streams over a network, each data server supporting at least one storage device, wherein data files are distributed across the data servers and stored on each of the storage devices, the system maintaining a schedule for serving requested ones of the data streams, the schedule being segmented into slots to which data streams are assigned for coordinating service of the requested data streams, a method comprising the following steps:

distributing the schedule among the data servers so that individual data servers view different portions of the schedule;

receiving multiple requests to insert new data streams into the schedule, the schedule formed at least in part based on historical and future slots capable of accommodating data streams;

queuing the requests in a queue;

examining the schedule to determine whether all of the queued requests can be inserted into the schedule, without changing where existing data streams are slotted in the schedule, under a policy that attempts to maximize distances between consecutively occupied slots and minimize contiguously occupied slots;

and

wherein the receiving, queuing, and examining steps are performed locally at individual ones of the data servers.

13. A method as recited in claim 12, wherein the examining step comprises determining whether all of the queued requests can be inserted into the schedule under the policy without exceeding a preset slip period measured as a time between when a request is received and when a corresponding data stream is served.

14. A method as recited in claim 12, further comprising the step of maintaining a central schedule which is used to coordinate service of the requested data streams, wherein the receiving, queuing, and examining steps are performed centrally.

15. A method as recited in claim 12, wherein the examining step comprises the step of computing the distances between the consecutively occupied slots.

16. A method as recited in claim 12, wherein the examining step comprises the step of deriving a highest number of the contiguously occupied slots.

17. A computer programmed to perform the steps recited in the method of claim 12.

18. A computer-readable medium having computer-executable instructions for performing the steps of the method as recited in claim 12.

19. A file server system, comprising:

multiple data servers, each data server supporting at least one storage device, wherein data files are distributed across the data servers so that data blocks of the data files are stored on each of the storage devices;

a scheduler to coordinate service of requested data streams from the data servers according to a schedule formed at least in part based on historical and future slots capable of accommodating data streams;

a queue to hold multiple requests to insert new data streams into the schedule; and

the scheduler being configured to examine the schedule to determine whether all of the queued requests can be inserted into the schedule, without modifying where existing data streams are scheduled, under a policy that attempts to maximize distances between consecutively occupied slots and minimize contiguously occupied slots,

wherein the schedule is distributed among the multiple data servers so that individual data servers view different portions of the schedule; and

the scheduler and the queue are located locally at individual ones of the data server.

20. A file server system as recited in claim 19, further comprising a controller coupled to the data servers, wherein the scheduler resides at the controller.

21. A file server system as recited in claim 19, wherein the scheduler resides at each of the data servers.

22. A file server system as recited in claim 19, wherein the scheduler determines whether all of the queued requests can be inserted into the schedule under the policy without exceeding a preset slip period measured as a time between when a request is received and when a corresponding data stream is served.

23. A continuous media file server system, comprising:

multiple data servers, each data server supporting at least one storage device, wherein data files are distributed across the data servers so that data blocks of the data files are stored on each of the storage devices;

multiple schedulers located at corresponding ones of the data servers, each scheduler facilitating service of requested data streams from its corresponding data server according to a portion of a schedule that is available to the scheduler and that is formed at least in part based on historical and future slots capable of accommodating data streams, the schedule portion having slots which are assigned to the requested data streams;

one or more queues to hold multiple requests to insert new data streams into the schedule; and

each scheduler being configured to examine their portion of the schedule to determine whether the queued requests can be inserted into the portion of the schedule, without modifying where existing data streams are scheduled, under a policy that attempts to maximize distances between consecutively occupied slots and minimize contiguously occupied slots.

24. A continuous media file server system as recited in claim 23, wherein said each scheduler is configured to assume all future slots following the schedule portion are occupied.

25. A continuous media file server system as recited in claim 23, wherein said each scheduler is configured to assume all past slots preceding the schedule portion have an occupancy density approximately equal to that of the schedule portion.

26. A continuous media file server system as recited in claim 23, wherein said each scheduler is configured to derive a bound representing a maximum number of past slots preceding the schedule portion on which said assumptions will be made.

27. A continuous media file server system as recited in claim 23, wherein said each scheduler is configured to track a number of contiguously vacant slots preceding the schedule portion.

28. A continuous media file server system as recited in claim 23, wherein said each scheduler is configured to compute the distances between the consecutively occupied slots.

29. A continuous media file server system as recited in claim 23, wherein said each scheduler is configured to derive a highest number of the contiguously occupied slots.

30. A scheduler embodied as a computer program on a computer-readable medium, the scheduler being implemented in a distributed file server system having multiple data servers connected to distribute data streams over a network, each data server supporting at least one storage device, wherein data files are distributed across the data servers and stored on each of the storage devices, wherein a schedule for serving requested ones of the data streams is distributed among the data servers so that that individual data servers view different portions of the schedule, the schedule being segmented into slots to which data streams are assigned for coordinating service of the requested data streams, the scheduler comprising:

code means for distributing the schedule among the data servers so that individual data servers view different portions of the schedule;

code means for receiving a portion of the schedule, the schedule formed at least in part based on historical and future slots capable of accommodating data streams;

code means for making assumptions as to whether slots preceding the schedule portion and following the schedule portion are vacant or occupied; and

code means for determining whether multiple new data streams can be inserted into the schedule portion, without modifying where existing data streams are scheduled, based upon a policy that attempts to maximize distances between consecutively occupied slots and minimize contiguously occupied slots,

wherein the code means for the receiving, making assumptions and determining are executed locally at individual ones of the data server.