Multi-threaded optimization for data upload

• US 8,473,585 B1
• Filed: 06/26/2012
• Issued: 06/25/2013
• Est. Priority Date: 06/26/2012
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Inventors: Smith, Joshua Earl, Mills, Robert Griffin
• Primary Examiner(s): MIRZA, ADNAN M
• Application Number:
  US13/533,507
• Time in Patent Office: 364 Days
• Field of Search: 709/218, 709/219, 709/230
• US Class Current: 709/219
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method may include receiving, at a first computing device, a command, from a second computing device, to upload a file to a storage region; negotiating, with a third computing device, a destination for the file, where the destination correlates to a storage location at the storage region; monitoring upload of at least a first chunk of the file and a second chunk of the file to the destination, where the first chunk is logically contiguous with the second chunk; determining that a threshold portion of the second chunk has been uploaded prior to upload of an entirety of the first chunk; initiating suspension of upload of the second chunk; determining upload of the entirety of the first chunk has completed; initiating continuation of upload of the second chunk; and reporting success of upload of the first chunk to the second computing device.

Claims

1. A method comprising:

receiving, at a first computing device, a command, from a second computing device, to upload a file to a storage region;

negotiating, with a third computing device, a destination for the file, wherein the destination correlates to a storage location at the storage region;

monitoring, by a processor of the first computing device, upload of a plurality of chunks of the file to ensure in-order receipt of the plurality of chunks by the destination, wherein ensuring in-order receipt of the plurality of chunks comprises;

determining, by the processor, that a threshold portion of a second chunk of the plurality of chunks has been transmitted to the destination prior to upload of an entirety of a first chunk of the plurality of chunks, wherein

the first chunk comprises a beginning portion of the file,

the second chunk comprises a portion of the file logically contiguous with the beginning portion of the file, and

at least a portion of the first chunk is transmitted to the destination in parallel with at least a portion of the second chunk,

responsive to determining that the threshold portion of the second chunk has been transmitted, initiating suspension of upload of the second chunk,

determining upload of the entirety of the first chunk has completed, and

responsive to determining upload of the entirety of the first chunk has completed,

(a) initiating continuation of upload of the second chunk, and

(b) reporting success of upload of the first chunk to the second computing device.

2. The method of claim 1, further comprising, responsive to determining upload of the entirety of the first chunk has completed, initiating upload of a third chunk of the file, wherein the third chunk is logically contiguous with the second chunk.

3. The method of claim 2, wherein initiating upload of the third chunk comprises issuing an upload command to the second computing device.

4. The method of claim 1, wherein the threshold portion comprises a threshold number of bytes of data.

5. The method of claim 1, further comprising:

monitoring, by the processor of the first computing device, upload of a third chunk of the file and a fourth chunk of the file, wherein;

the third chunk is logically contiguous with the second chunk,

the fourth chunk is logically contiguous with the third chunk, and

at least a portion of the third chunk and at least a portion of the fourth chunk are transmitted in parallel with at least a portion of the first chunk and at least a portion of the second chunk.

6. The method of claim 5, further comprising:

determining, by the processor, that a threshold portion of the third chunk has been transmitted prior to upload of the entirety of the first chunk; and

responsive to determining that a threshold portion of the third chunk has been transmitted prior to upload of the entirety of the first chunk,

initiating suspension of upload of the third chunk,

determining upload of the entirety of the second chunk has completed, and

responsive to determining both upload of the entirety of the first chunk has completed and upload of the entirety of the second chunk has completed, initiating continuation of upload of the third chunk.

7. The method of claim 6, wherein the threshold portion of the third chunk is equal to the threshold portion of the second chunk.

8. The method of claim 1, further comprising:

determining, at the processor, a failure of upload of the first chunk;

responsive to determining the failure of upload of the first chunk,

initiating suspension of upload of the second chunk, and

initiating re-upload of the first chunk.

9. A system comprising:

a processor; and

a memory storing instructions thereon, wherein the instructions when executed cause the processor to;

receive, at a first computing device, a command, from a second computing device, to upload a file to a storage region;

negotiate, with a third computing device, a destination for the file, wherein the destination correlates to a storage location at the storage region;

monitor upload of a plurality of chunks of the file to ensure in-order receipt of the plurality of chunks by the destination, wherein ensuring in-order receipt of the plurality of chunks comprises;

determining that a threshold portion of a second chunk of the plurality of chunks has been transmitted to the destination prior to upload of an entirety of a first chunk of the plurality of chunks, wherein

the first chunk comprises a beginning portion of the file,

the second chunk comprises a portion of the file logically contiguous with the beginning portion of the file, and

at least a portion of the first chunk is transmitted to the destination in parallel with at least a portion of the second chunk,

after determining that the threshold portion of the second chunk has been transmitted, initiating suspension of upload of the second chunk,

determining upload of the entirety of the first chunk has completed, and

after determining upload of the entirety of the first chunk has completed,

(a) initiating continuation of upload of the second chunk, and

(b) reporting success of upload of the first chunk to the second computing device.

10. The system of claim 9, wherein the system receives the command from the second computing device through a virtual machine application interface.

11. The system of claim 9, wherein the first computing device is in communication with a plurality of networked storage devices.

12. The system of claim 9, wherein the system comprises a Web server configured to communicate with the second computing device through a browser-based file sharing service.

13. The system of claim 9, wherein the file is uploaded using Hypertext Transfer Protocol Secure.

14. The system of claim 9, wherein the instructions when executed further cause the processor to:

responsive to completion of upload of the file, create a data base entry regarding the file, wherein the data base entry comprises an address for accessing the file through the first computing device.

15. A non-transient computer readable medium having instructions stored thereon that, when executed, cause a processor to:

receive, at a first computing device, a request, from a second computing device, to upload a file;

monitor upload of a plurality of chunks of the file to ensure in-order receipt of the plurality of chunks by a third computing device, wherein ensuring in-order receipt of the plurality of chunks comprises;

determining that a threshold portion of a second chunk of the plurality of chunks has been transmitted to the third computing device prior to upload of an entirety of a first chunk of the plurality of chunks, wherein

the first chunk comprises a beginning portion of the file,

the second chunk comprises a portion of the file logically contiguous with the beginning portion of the file, and

at least a portion of the first chunk is transmitted to the destination in parallel with at least a portion of the second chunk,

responsive to determining that a threshold portion of the second chunk has been transmitted prior to upload of an entirety of the first chunk, initiating suspension of upload of the second chunk;

determine determining receipt of the entirety of the first chunk by the third computing device, and

responsive to determining receipt of the entirety of the first chunk,

continuing upload of the second chunk, and

initiating upload of a third chunk of the plurality of chunks, wherein the third chunk is logically contiguous with the second chunk.

16. The computer readable medium of claim 15, wherein the third computing device comprises a storage server in communication with a plurality of networked storage devices.

17. The computer readable medium of claim 16, wherein the instructions, when executed, further cause the processor to negotiate, with the storage server, a destination for the file, wherein the destination correlates to a storage location on at least one of the plurality of networked storage devices.

18. The computer readable medium of claim 15, wherein the instructions, when executed, further cause the processor to, responsive to determining receipt of the entirety of the first chunk, report success of upload of the first chunk to the second computing device.

19. The computer readable medium of claim 15, wherein the instructions, when executed, further cause the processor to, responsive to completion of upload of an entirety of the file, store an address, wherein the address provides access to the file.

20. The computer readable medium of claim 15, further comprising, prior to monitoring upload of the plurality of chunks:

initiating a first execution thread for transmission of the first chunk; and

initiating a second execution thread for transmission of the second chunk, wherein monitoring the upload of the plurality of chunks comprises monitoring the first execution thread and the second execution thread.