MIRRORED FILE SYSTEM

• US 20100287526A1
• Filed: 07/22/2010
• Published: 11/11/2010
• Est. Priority Date: 04/14/2006
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bumgarner, William M., Gross, Andrew R., Swift, Adam C.
• Granted Patent: US 8,286,127 B2
• Time in Patent Office: Days
• US Class Current: 717/103
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 2209/5017 Task decomposition
• CPC Class Codes: G06F 2209/509 Offload
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes

Abstract

Mounting and populating a mirrored version of at least a portion of a file system of a recruiter machine at a volunteer machine performing one or more data processing tasks for the recruiter machine is disclosed. In some embodiments, an input stored in such a dedicated file system for the recruiter machine at the volunteer machine may be locally retrieved and reused, if still consistent with a corresponding input stored in a file system at the recruiter machine, when performing a data processing task for the recruiter machine at the volunteer machine. In some embodiments, if an input required by the volunteer machine to perform a data processing task for the recruiter machine is not cached in such a dedicated file system for the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine, the input is obtained from the recruiter machine.

Claims

1. A method for accomplishing a data processing task to obtain output data, comprising:

receiving at a recruiter machine an indication that an input that is required by a volunteer machine to perform a data processing task for the recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine, wherein the recruiter machine distributed the data processing task to the volunteer machine, and the data processing task is associated with uncompiled code; and

providing the input to the volunteer machine in a state that is consistent with the corresponding input at the recruiter machine.

2. A method as recited in claim 1, wherein receiving at a recruiter machine an indication that an input that is required by a volunteer machine to perform a data processing task for the recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine comprises receiving a request for the input from the volunteer machine.

3. A method as recited in claim 1, wherein receiving at a recruiter machine an indication that an input that is required by a volunteer machine to perform a data processing task for the recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that one or more parameters associated with a version of the input as cached in the dedicated file system associated with the recruiter machine at the volunteer machine are not consistent with one or more corresponding parameters of the corresponding input as stored in a file system at the recruiter machine.

4. A method as recited in claim 1, wherein receiving at a recruiter machine an indication that an input that is required by a volunteer machine to perform a data processing task for the recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that a hash of a version of the input as cached in the dedicated file system associated with the recruiter machine at the volunteer machine is not consistent with a hash of the corresponding input as stored in a file system at the recruiter machine.

5. A method as recited in claim 1, further comprising sending from the recruiter machine to the volunteer machine a request to perform the data processing task.

6. A method as recited in claim 1, wherein the data processing task is part of a set of tasks that comprise a data processing job that needs to be performed at the recruiter machine.

7. A method for performing a data processing task, comprising:

determining at a volunteer machine that an input that is required by the volunteer machine to perform a data processing task for a recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine, wherein the recruiter machine distributed the data processing task to the volunteer machine, and the data processing task is associated with uncompiled code; and

obtaining the input from the recruiter machine in a state that is consistent with the corresponding input at the recruiter machine.

8. A method as recited in claim 7, further comprising receiving from the recruiter machine a request to perform the data processing task.

9. A method as recited in claim 8, wherein receiving the request includes configuring at the volunteer machine in response to the request a process to intercept and direct file system calls associated with the data processing task to the dedicated file system.

10. A method as recited in claim 7, wherein determining at a volunteer machine that an input that is required by the volunteer machine to perform a data processing task for a recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that the input does not exist in the dedicated file system associated with the recruiter machine at the volunteer machine.

11. A method as recited in claim 7, wherein determining at a volunteer machine that an input that is required by the volunteer machine to perform a data processing task for a recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that a version of the input cached in the dedicated file system associated with the recruiter machine at the volunteer machine is marked invalid.

12. A method as recited in claim 7, further comprising receiving from the recruiter machine an invalid list that includes files that have been previously provided to the volunteer machine but have since been updated at the recruiter machine and are no longer valid at the volunteer machine.

13. A system for performing a data processing task, comprising:

a processor configured to;

determine that an input that is required to perform a data processing task for a recruiter machine is not locally cached in a dedicated file system associated with the recruiter machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine, wherein the recruiter machine distributed the data processing task to the system, and the data processing task is associated with uncompiled code; and

obtain the input from the recruiter machine in a state that is consistent with the corresponding input at the recruiter machine; and

a memory coupled to the processor and configured to provide instructions to the processor.

14. A system as recited in claim 13, wherein the processor is further configured to receive from the recruiter machine a request to perform the data processing task.

15. A system as recited in claim 13, wherein to determine that an input that is required to perform a data processing task for a recruiter machine is not locally cached in a dedicated file system associated with the recruiter machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that one or more parameters associated with a version of the input as locally cached in the dedicated file system associated with the recruiter machine are not consistent with one or more corresponding parameters of the corresponding input as stored in a file system at the recruiter machine.

16. A system as recited in claim 13, wherein to determine that an input that is required to perform a data processing task for a recruiter machine is not locally cached in a dedicated file system associated with the recruiter machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that a hash of a version of the input as locally cached in the dedicated file system associated with the recruiter machine is not consistent with a hash of the corresponding input as stored in a file system at the recruiter machine.

17. A system as recited in claim 13, wherein to determine that an input that is required to perform a data processing task for a recruiter machine is not locally cached in a dedicated file system associated with the recruiter machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that the input does not locally exist in the dedicated file system associated with the recruiter machine.

18. A system as recited in claim 13, wherein to determine that an input that is required to perform a data processing task for a recruiter machine is not locally cached in a dedicated file system associated with the recruiter machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that a version of the input locally cached in the dedicated file system associated with the recruiter machine is marked invalid.

19. A computer program product for performing a data processing task, the computer program product being embodied in a computer readable storage medium and comprising computer instructions for:

determining at a volunteer machine that an input that is required by the volunteer machine to perform a data processing task for a recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine, wherein the recruiter machine distributed the data processing task to the volunteer machine, and the data processing task is associated with uncompiled code; and

obtaining the input from the recruiter machine in a state that is consistent with the corresponding input at the recruiter machine.

20. A computer program product as recited in claim 19, wherein receiving at a recruiter machine an indication that an input that is required by a volunteer machine to perform a data processing task for the recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine comprises receiving a request for the input from the volunteer machine.

21. A computer program product as recited in claim 19, wherein receiving at a recruiter machine an indication that an input that is required by a volunteer machine to perform a data processing task for the recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that one or more parameters associated with a version of the input as cached in the dedicated file system associated with the recruiter machine at the volunteer machine are not consistent with one or more corresponding parameters of the corresponding input as stored in a file system at the recruiter machine.

22. A computer program product as recited in claim 19, wherein receiving at a recruiter machine an indication that an input that is required by a volunteer machine to perform a data processing task for the recruiter machine is not cached in a dedicated file system associated with the recruiter machine at the volunteer machine in a state that is consistent with a corresponding input stored in a file system at the recruiter machine includes determining that a hash of a version of the input as cached in the dedicated file system associated with the recruiter machine at the volunteer machine is not consistent with a hash of the corresponding input as stored in a file system at the recruiter machine.