Method and system for load balancing by replicating a portion of a file being read by a first stream onto second device and reading portion with a second stream capable of accessing

• US 6,223,206 B1
• Filed: 08/28/1998
• Issued: 04/24/2001
• Est. Priority Date: 05/11/1994
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dan, Asit, Sitaram, Dinkar, Kienzle, Martin Gerhard, Yu, Philip Shi-lung
• Primary Examiner(s): Lee, Thomas
• Assistant Examiner(s): Park, Ilwoo
• Application Number:
  US09/143,001
• Time in Patent Office: 970 Days
• Field of Search: 709/105, 709/203, 709/231, 709/235, 709/223, 710/33, 711/150, 711/168
• US Class Current: 718/105
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04N 21/2182 involving memory arrays, e.g. RAID disk arrays RAID arrays per se G06F3/0689; use of parity to protect data in RAID systems G06F11/1008

Abstract

A method and system for improving load balancing in a file server is disclosed. The method includes determining the existence of an overload condition on a storage device, the storage device having a plurality of retrieval streams accessing at least one file thereon; selecting a first retrieval stream reading a file; replicating a portion of said file being read by said first retrieval stream onto a second storage device; and reading the replicated portion of said file on said second storage device with a retrieval stream capable of accessing the replicated portion of said file. The method and system enables the dynamic replication of data objects to respond to fluctuating user demand. The method and system is particularly useful in file servers such as multimedia servers delivering continuously in real time large multimedia files such as movies.

Claims

1. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for improving load balancing in a file server, comprising:

code means for determining the existence of an overload condition on a first storage device, said first storage device having a plurality of retrieval streams accessing at least one file thereon;

code means for selecting a first retrieval stream reading a file;

code means for replicating a portion of said file while being read by said first retrieval stream onto a second storage device; and

code means for reading said replicated portion of said file on said second storage device with a retrieval stream capable of accessing said replicated portion of said file.

2. The article of manufacture of claim 1 wherein said determining code means determines the existence of an overload condition on the first storage device by detecting a threshold level on said storage device.

3. The article of manufacture of claim 1 wherein said first retrieval stream is selectively chosen from a plurality of retrieval streams reading said file.

4. The article of manufacture of claim 1 wherein said second storage device is chosen from a plurality of available storage devices, said second storage device having the largest available amount of storage space.

5. The article of manufacture of claim 1 wherein said code means for replicating a portion of said file being read by said first retrieval stream onto said second storage device includes code means for initially creating a switching point entry on a file information table so that said portion of said file can be indexed for ease of switching between said second storage device and another storage device.

7. The article of manufacture of claim 1 wherein said file comprises a plurality of blocks.

8. The article of manufacture of claim 7 wherein said file comprises a plurality of segments, each of said plurality of segments comprising a plurality of blocks.

9. The article of manufacture of claim 1 wherein said file comprises a plurality of segments and said portion of the file being replicated comprises one or more segments.

10. The article of manufacture of claim 1 wherein said file includes at least one primary segment.

11. The article of manufacture of claim 10 wherein said file includes a plurality of primary segments.

12. The system of claim 11 wherein said primary segments are permanent segments.

20. The article of manufacture of claim 1 wherein said file server comprises a multimedia file server.

21. The article of manufacture of claim 1 wherein said storage device comprises a disk.

28. The article of manufacture of claim 1 wherein said file server comprises a video server, and wherein said file comprises a video file.

6. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for improving load balancing in a file server comprising:

code means for determining the existence of an overload condition on a first storage device, said first storage device having a plurality of retrieval streams accessing at least one file thereon;

code means for selecting a first retrieval stream reading a file;

code means for replicating a portion of said file being read by said first retrieval stream onto a second storage device; and

code means for reading said replicated portion of said file on said second storage device with a retrieval stream capable of accessing said replicated portion of said file;

wherein said code means for replicating a portion of said file being read by said first retrieval stream onto a second storage device comprises;

code means for reading said file using said first retrieval stream from said first storage device; and

code means for writing said portion of said file using a retrieval stream capable of accessing said replicated portion of said file onto said second storage device.

13. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for improving load balancing in a file server, comprising:

code means for determining the existence of an overload condition on a first storage device, said first storage device having a plurality of retrieval streams accessing at least one file thereon;

code means for selecting a first retrieval stream reading a file;

code means for replicating a portion of said file being read by said first retrieval stream onto a second storage device; and

code means for reading said replicated portion of said file on said second storage device with a retrieval stream capable of accessing said replicated portion of said file;

wherein said file includes at least one permanent, primary segment; and

wherein said replicated portion of said file comprises at least one secondary segment.

14. The article of manufacture claim 13 wherein said secondary segment is a temporary segment.

15. The article of manufacture of claim 14 wherein said secondary segment may be deleted.

16. The article of manufacture of claim 15 further comprising code means for converting one of said at least one primary segment to a secondary segment.

17. The article of manufacture of claim 15 further comprising code means for converting one of said at least one secondary segment to a primary segment.

18. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for improving load balancing in a file server comprising:

code means for determining the existence of an overload condition on a first storage device, said first storage device having a plurality of retrieval streams accessing at least one file thereon;

code means for selecting a first retrieval stream reading a file;

code means for replicating a portion of said file being read by said first retrieval stream onto a second storage device; and

code means for reading said replicated portion of said file on said second storage device with a retrieval stream capable of accessing said replicated portion of said file;

wherein said file comprises a plurality of blocks; and

wherein said code means for replicating said portion of said file being read by said first retrieval stream onto said second storage device comprises;

code means for sequentially reading said plurality of blocks from said first retrieval stream; and

code means for writing one or more of said plurality of blocks onto said second storage device.

19. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for improving load balancing in a file server comprising:

code means for determining the existence of an overload condition on a first storage device, said first storage device having a plurality of retrieval streams accessing at least one file thereon;

code means for selecting a first retrieval stream reading a file;

code means for replicating a portion of said file being read by said first retrieval stream onto a second storage device; and

code means for reading said replicated portion of said file on said second storage device with a retrieval stream capable of accessing said replicated portion of said file;

wherein said file comprises a plurality of segments, each of said plurality of segments comprising a plurality of blocks; and

wherein said code means for replicating said portion of said file being read by said first retrieval stream onto said second storage device comprises;

code means for sequentially reading at least one of said segments from said first retrieval stream; and

code means for writing at least one of said segments on to said second storage device.

22. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for improving load balancing in a file server comprising:

code means for determining an excess number of permanently stored primary copies of a segment of a file;

code means for converting one of said permanently stored primary copies of said segment, not being accessed, to a temporary secondary copy of said segment, wherein said secondary copy of said segment is capable of being overwritten; and

code means for overwriting said secondary copy of said segment for improving the load balancing in said file server.

23. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for improving load balancing in a file server comprising:

code means for determining an excess number of permanently stored primary segments of a file;

code means for estimating a required number of primary segments based on load history;

code means for converting one of said permanently stored primary segments, not being accessed, to a temporary secondary segment, wherein said secondary segment is capable of being overwritten; and

code means for overwriting said secondary segment for improving the load balancing in said file server.

24. The article of manufacture of claim 23 further comprising code means for converting all permanently stored primary segments greater than said required number of primary segments.

25. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for improving load balancing in a file server comprising:

code means for determining a load on a temporarily stored secondary copy of a segment of a file;

code means for determining if said load on said secondary copy of said segment exceeds a threshold value; and

code means for converting said temporarily stored secondary copy of said segment to a permanently stored primary copy of said segment of said file, wherein said primary copy of said segment is prevented from being overwritten.

26. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for improving load balancing in a file server comprising:

code means for determining a load on a temporarily stored secondary segment of a file including means for determining a variable load history for said secondary segment;

code means for determining if said load on said secondary segment exceeds a threshold value; and

code means for converting said temporarily stored secondary segment to a permanently stored primary segment of said file, wherein said primary segment is prevented from being overwritten.

27. The article of manufacture of claim 26 further comprising code means for analyzing said variable load history for determining if said threshold value is exceeded.

29. A method for balancing a load in a file server having a plurality of retrieval streams reading at least one file therein, comprising:

copying a portion of a file of said at least one file from a first storage area to a second storage area in said file server, for future access by at least one other retrieval stream, while the same data of the portion of the same file is being read by a given retrieval stream of the plurality of retrieval streams, wherein said copying includes using the given retrieval stream to read the data of the portion of the file from the first storage area.

30. A method for balancing a load in a file server having a plurality of retrieval streams reading at least one file therein, comprising:

copying a portion of a file of said at least one file from a first storage area to a second storage area in said file server while the same data of the portion of the same file is being read by a given retrieval stream of the plurality of retrieval streams, wherein said copying includes using the given retrieval stream to read the data of the portion of the file from the first storage area, and using another retrieval stream to write the data of the portion of the file to the second storage area.

31. A system for balancing a load in a file server having a plurality of retrieval streams reading at least one file therein, comprising:

means for copying a portion of a file of said at least one file from a first storage area to a second storage area in said file server, for future access by at least one other retrieval stream, while the same data of the portion of the same file is being read by a given retrieval stream of the plurality of retrieval streams, wherein said means for copying includes means for using the given retrieval stream to read the data of the portion of the file from the first storage area.

32. A system for balancing a load in a file server having a plurality of retrieval streams reading at least one file therein, comprising:

means for copying a portion of a file of said at least one file from a first storage area to a second storage area in said file server while the same data of the portion of the same file is being read by a given retrieval stream of the plurality of retrieval streams, wherein said means for copying includes means for using the given retrieval stream to read the data of the portion of the file from the first storage area, and using another retrieval stream to write the data of the portion of the file to the second storage area.

33. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for balancing a load in a file server having a plurality of retrieval streams reading at least one file therein, comprising:

code means for copying a portion of a file of said at least one file from a first storage area to a second storage area in said file server, for future access by at least one other retrieval stream, while the same data of the portion of the same file is being read by a given retrieval stream of the plurality of retrieval streams, wherein said code means for copying includes code means for using the given retrieval stream to read the data of the portion of the file from the first storage area.

34. An article of manufacture comprising a computer useable medium having computer readable program code means embodied therein for balancing a load in a file server having a plurality of retrieval streams reading at least one file therein, comprising:

code means for copying a portion of a file of said at least one file from a first storage area to a second storage area in said file server while the same data of the portion of the same file is being read by a given retrieval stream of the plurality of retrieval streams, wherein said code means for copying includes code means for using the given retrieval stream to read the data of the portion of the file from the first storage area, and code means for using another retrieval stream to write the data of the portion of the file to the second storage area.