Web server content replication

• US 7,035,943 B2
• Filed: 08/19/1999
• Issued: 04/25/2006
• Est. Priority Date: 05/29/1998
• Status: Expired due to Term

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Mateos, Arthur, DeBettencourt, Jason, Lara, Marco, Yamane, Stanley, Shah, Swapnil, Weng, Keith
• Primary Examiner(s): SALAD, ABDULLAHI ELMI
• Application Number:
  US09/376,017
• Publication Number:
  US 20030041093A1
• Time in Patent Office: 2,441 Days
• Field of Search: 709/200, 709/201, 709/205, 709/217, 709/218, 709/241, 709/246, 709/223, 709/224, 709/203, 709/248, 709/226, 709/202, 707/8, 707/10, 707/201, 707/203, 714/48
• US Class Current: 709/248
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: G06F 2201/88 Monitoring involving counting
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 41/5083 wherein the managed service relates to web hosting
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/1031 Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/563 Data redirection of data network streams
• CPC Class Codes: H04L 67/564 Enhancement of application control based on intercepted application data
• CPC Class Codes: H04L 67/5682 : Policies or rules for updating, deleting or replacing the stored data

Abstract

Changes to files in a master source file set on a master computer are identified. Local host addresses of the files are converted to uniform resource locators. The identified changes are stored in a modification list comprising the uniform resource locators specifying changed files of the master source file set. The modification list is transmitted to one or more computers, notifying the one or more computers that the master source file set has changed. A response is received from at least one of the one or more computers, indicating that the identified changes are installed. An alert is issued indicating that a copying of the changes by an agent in at least one of the one or more computers has failed, to cause a traffic manager that redirects traffic to decrease requests made to a web server on the computer having the agent.

Claims

1. A method for notifying a computer of changes to a master source file set, comprising the steps of:

(a) identifying changes to files in the master source file set on a master computer;

(b) converting local host addresses of the files to uniform resource locators;

(c) storing the identified changes in a modification list comprising the uniform resource locators specifying changed files of the master source file set;

(d) transmitting the modification list to one or more computers, thereby notifying the one or more computers that the master source file set has changed;

(e) receiving a response from at least one of the one or more computers indicating that the identified changes are installed; and

(f) issuing an alert that an installing of the changes by an agent in at least one of the one or more computers has failed, to cause a traffic manager that redirects traffic to decrease requests made to a web server on the computer having the agent.

2. The method of claim 1 wherein the one or more computers comprises a web cache server.

3. The method of claim 1 wherein the identifying step comprises the steps of:

inspecting a current version of the master source file set; and

comparing the current master source file set to a previous version of the master source file set on the master computer.

4. The method of claim 3 wherein the comparing step comprises comparing a file attribute of a file of the master source file set to a file attribute of a file of the previous version of the master source file set.

5. The method of claim 4 wherein the file attribute comprises at least one attribute chosen from the group consisting of file size, file permissions, file ownership, modification time, and a hash of the file.

6. The method of claim 4 wherein the file attribute is stored in a list of file attributes.

7. The method of claim 1 wherein the identifying step comprises the steps of:

installing a device driver to perform file operations, and monitor changes to the master source file set; and

recording, by the device driver, changes to the master source file set.

8. The method of claim 1 wherein the identifying step comprises;

receiving a manifest describing changes to the master source file set.

9. The method of claim 1, further comprising, before the identifying step, the step of calling a script of user configurable instructions by calling an operating system function.

10. The method of claim 1, further comprising, before the transmitting step, the step of calling a script of user configurable instructions by calling an operating system function.

11. The method of claim 1, further comprising, after the transmitting step, the step of calling a script of user configurable instructions by calling an operating system function.

12. The method of claim 1, wherein a prefix of a source directory of the local host addresses corresponds to a particular prefix of possible URLs, and characters which are not allowed in URLs, such as spaces, are encoded according to a specified rule.

13. The method of claim 1, wherein, following the alert, the traffic manager directs requests away from the web server on the computer having the agent onto other servers.

14. The method of claim 13, wherein, following the alert, the traffic manager does not direct web page requests from another web server to the web server on the computer having the agent.

15. The method of claim 1, wherein step (d) is performed when a predetermined number of servers running in the one or more computers have indicate they are ready to accept updated content.

16. The method of claim 1, wherein step (d) is performed when a predetermined percentage of servers running in the one or more computers have indicate they are ready to accept updated content.

17. A method for replicating changes in a master source file set on a destination file system and for notifying one or more computers of the changes, comprising the steps of:

(a) identifying changes in the master source file set on a master computer;

(b) storing the changes in a first modification list;

(c) transmitting the first modification list to an agent having access to the destination file system;

(d) converting local host addresses of the files having the changes to uniform resource locators;

(e) storing the uniform resource locators in a second modification list specifying a changed file of the master source file set;

(f) transmitting the second modification list to the one or more computers, thereby notifying the one or more computers that the master source file set has changed;

(g) receiving a response indicating that the identified changes are installed on at least one of the one or more computers; and

(g) issuing an alert that an installing of the changes by an agent in at least one of the one or more computers has failed, to cause a traffic manager that redirects traffic to decrease requests made to a web server on the computer having the agent.

18. The method of claim 17 wherein the one or more computers comprises a web cache server.

19. The method of claim 17 further comprising, before step (d), the step of converting the first modification list into the second modification list.

20. A method for replicating changes in a master source file set on a destination file system and for notifying one or more computers of the changes, comprising the steps of:

(a) identifying changes to files in the master source file set on a master computer;

(b) converting local host addresses of the files to uniform resource locators;

(c) storing the changes in a modification list comprising the uniform resource locators specifying a changed file of the master source file list;

(d) transmitting the modification list to one or more agents having access to the destination file system;

(e) transmitting the modification list to the one or more computers, thereby notifying the one or more computers the master source file set has changed; and

(f) receiving a response indicating the identified changes are installed on at least one of the one or more computers; and

(g) issuing an alert that an installing of the changes by an agent in at least one of the one or more computers has failed, to cause a traffic manager that redirects traffic to decrease requests made to a web server on the computer having the agent.

21. A web service system, comprising:

a manager for managing the web service system;

a host comprising a web server for receiving web page requests and an agent in communication with the manager and the web server;

a web cache; and

a content distributor in communication with the web cache, the host, and the manager, the content distributor converting local host addresses of changed file in a master source file list to uniform resource locators, providing notification of changes to a master source file set on a master computer to the host in a modification list comprising the uniform resource locators, and receiving a response indicating that the identified changes are installed, wherein;

either the content distributor or the agent issues an alert to the manager indicating that an installing of the changes by the agent has failed; and

the manager instructs the traffic manager to redirect traffic to decrease requests made to the web server on the host having the agent and the web server.

22. The system of claim 21, further comprising a traffic manager in communication with the manager, the traffic manager automatically directing web page requests to the web server.

23. The system of claim 21, wherein the content distributor comprises:

(a) an identification module for identifying changes in the master source file set on a master computer;

(b) means for converting local host addresses of the files to uniform resource locators;

(d) a modification list for storing identified changes to the master source file set;

(c) a transmitter for transmitting the modification list to the agent, the agent having access to a destination file system; and

(e) a receiver for receiving a response indicating that the identified changes are installed.

24. The system of claim 21, wherein:

the content distributor generates and transmits a catch-up update to the agent, to provide the notification of changes to the web server.

25. The system of claim 24, wherein, following the alert, the traffic manager directs requests away from the web server on the computer having the agent onto other servers.

26. The system of claim 25, wherein, following the alert, the traffic manager does not direct web page requests from another web server to the web server on the computer having the agent.

27. The system of claim 21 wherein the content distributor provides the notification of changes when a predetermined number of servers running in one or more computers have indicate they are ready to accept updated content.

28. The system of claim 21 wherein the content distributor provides the notification of changes when a predetermined percentage of servers running in the one or more computers have indicate they are ready to accept updated content.

29. A content distributor, comprising:

(a) an identification module for identifying changes in a master source file set on a master computer, the identifying module installing a pass-through device driver that monitors and passes data between an operating system of a computer having the master source file set and a storage device driver through which the operating system communicates with a storage device in which the master source file set is stored;

(b) a modification list for storing the identified changes to the master source file set;

(c) means for converting local host addresses of the changes to uniform resource locators;

(d) a transmitter for transmitting the modification list of uniform resource locators to one or more agents having access to a destination file system;

(e) a transmitter for transmitting a copy of a changed file of the master source file set to a web cache server; and

(f) a receiver for receiving a response indicating that the web cache server received and installed the copy of the changed file of the master source file set.

30. A method for notifying a computer of changes to a master source file set, comprising the steps of:

(a) identifying changes to files in the master source file set on a master computer, wherein the identifying step installs a device driver to perform file operations, and monitor changes to the master source file set, and the device driver records changes to the master source file set;

(b) converting local host addresses of the files to uniform resource locators;

(c) storing the identified changes in a modification list comprising the uniform resource locators specifying changed files of the master source file set;

(d) transmitting the modification list to one or more computers, thereby notifying the one or more computers that the master source file set has changed; and

(e) receiving a response from the one or more computers indicating that the identified changes are installed,

wherein the device driver is a pass-through device driver that monitors and passes data between an operating system of a computer having the master source file set and a storage device driver through which the operating system communicates with a storage device in which the master source file set is stored.

31. A web service system, comprising:

a manager for managing the web service system;

a host comprising a web server for receiving web page requests and an agent in communication with the manager and the web server;

a web cache; and

a content distributor in communication with the web cache, the host, and the manager, the content distributor converting local host addresses of changed file in a master source file list to uniform resource locators, providing notification of changes to a master source file set on a master computer to the host in a modification list comprising the uniform resource locators, and receiving a response indicating that the identified changes are installed; and

a pass-through device driver that monitors and passes data between an operating system of a computer having the master source file set and a storage device driver through which the operating system communicates with a storage device in which the master source file set is stored.