Communicating chunks between devices

• US 8,140,637 B2
• Filed: 09/26/2008
• Issued: 03/20/2012
• Est. Priority Date: 10/25/2007
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Slater, Alastair, Pelly, Simon, Camble, Peter Thomas
• Primary Examiner(s): Boutah, Alina N.
• Application Number:
  US12/239,263
• Publication Number:
  US 20090113145A1
• Time in Patent Office: 1,271 Days
• Field of Search: 709/213, 711/154, 711/E12.001
• US Class Current: 709/213
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments

Abstract

An apparatus, for connection to a storage device, comprising: a module for communication with a software application and operable to receive instructions to copy data to the storage device; a chunking and identifier generation module operable to receive the data, to process the data into chunks, to generate a first chunk identifier, representative of the identity of a first of the chunks of data and, upon processing of the data, to initiate the issuance of a confirmation signal to the software application indicating that the data has been copied to the storage device; and an interface for communication with the storage device, wherein the interface is operable to send the first chunk identifier to the storage device, and to send the first chunk of data to the storage device upon receipt of a transfer instruction from the storage device.

Claims

1. A communication apparatus, for connection to a remote storage device, comprising:

a computer;

a chunking and identifier generation module operable in the computer to receive data, to process the data into chunks, and to generate a first chunk identifier representative of the identity of a first of the chunks of data;

a first interface in the computer for communication with the remote storage device, wherein the first interface is operable to send the first chunk identifier to the remote storage device, and to send the first chunk of data to the remote storage device upon receipt of a transfer instruction from the remote storage device; and

a second interface in the computer for communication with the remote storage device and operable to send data to and receive data from the remote storage device independently of the chunking and identifier generation module.

2. The communication apparatus of claim 1, wherein the receipt of the transfer instruction is responsive to a determination of the remote storage device that the first chunk of data is unknown at the remote storage device.

3. The communication apparatus of claim 1, wherein the chunking and identifier generation module is to generate chunk identifiers for corresponding ones of the chunks, and the first interface is to:

send the chunk identifiers without associated chunks to the remote storage device,

send selected ones of the chunks to the remote storage device in response to receiving indications from the remote storage device identifying which of the chunks are unknown at the remote storage device, the indications responsive to the chunk identifiers, and

decline to send to the remote storage device remaining ones of the chunks that are known at the remote storage device.

4. The communication apparatus of claim 1, further comprising at least one processor to:

selectively use one of a plurality of techniques to communicate with the remote storage device, based on at least one factor,

wherein a first of the plurality of techniques includes sending particular data to the remote storage device through the second interface without performing chunking on the particular data being sent, and

wherein a second of the plurality of techniques includes the chunking and identifier generation module processing the particular data into chunks and generating respective chunk identifiers, and sending the chunk identifiers and at least some of the chunks to the remote storage device rather than the particular data.

5. The communication apparatus of claim 4, wherein the at least one factor is selected from among:

capacities of first and second networks connected respectively to the first and second interfaces, levels of usage of the first and second networks, time period since last backup operation, and amount of data or chunks stored on the remote storage device.

6. The communication apparatus of claim 1, further comprising at least one processor to:

receive, from the remote storage device, a plurality of chunk identifiers,

determine whether a chunk corresponding to each of the plurality of chunk identifiers is known to the communication apparatus;

in response to determining that a particular chunk corresponding to a particular one of the plurality of chunk identifiers is known to the communication apparatus, send a first indication to the remote storage device to cause the remote storage device not to send the particular chunk to the communication apparatus; and

in response to determining that the particular chunk corresponding to the particular chunk identifier is unknown to the communication apparatus, send a second indication to the remote storage device to cause the remote storage device to send the particular chunk to the communication apparatus.

7. The communication apparatus of claim 6, wherein the particular chunk is determined to be known if a copy of the particular chunk is already in the communication apparatus.

8. The communication apparatus of claim 6, wherein the receiving of the chunk identifiers and the determining are part of a data restore operation to restore data from the remote storage device.

9. A storage device for connection to a remote communication apparatus, the storage device comprising:

a non-transitory storage medium;

a first interface for coupling to the remote communication apparatus, the first interface being operable to receive a first identifier representative of the identity of a first data chunk from the remote communication apparatus, to issue a transfer instruction to the remote communication apparatus if the first identifier is an unknown identifier, and receive and store on the storage medium the first data chunk in response to the issuance of the transfer instruction to the remote communication apparatus; and

a second interface operable to receive data and process the data into one or more data chunks for storage on the storage medium.

10. The storage device of claim 9, wherein the first interface is operable to further receive a second identifier representative of the identity of a second data chunk from the remote communication apparatus, to issue an indication to the remote communication apparatus if the second identifier is a known identifier, where the indication is to cause the remote communication apparatus not to send the second data chunk.

11. The storage device of claim 9, further comprising at least one processor to:

selectively use one of a plurality of techniques to communicate with the remote communication apparatus, based on at least one factor,

wherein a first of the plurality of techniques includes sending particular data to the remote communication apparatus, rather than chunks corresponding to the particular data, through the second interface, and

wherein a second of the plurality of techniques includes determining which chunks corresponding to the particular data are known and unknown at the remote communication apparatus, and selectively sending chunks to the remote communication apparatus that are unknown at the remote communication apparatus.

12. The storage device of claim 9, further comprising at least one processor to:

receive, from the remote communication apparatus, a plurality of chunk identifiers;

determine whether a chunk corresponding to each of the plurality of chunk identifiers is known to the storage device;

in response to determining that a particular chunk corresponding to a particular one of the plurality of chunk identifiers is known to the storage device, send a first indication to the remote communication apparatus to cause the remote communication apparatus not to send the particular chunk to the storage device; and

in response to determining that the particular chunk corresponding to the particular chunk identifier is unknown to the storage device, send a second indication to the remote communication apparatus to cause the remote communication apparatus to send the particular chunk to the storage device.

13. The storage device of claim 12, wherein the particular chunk is determined to be known if a copy of the particular chunk is already in the storage device.

14. A communication apparatus, for connection to a remote storage device, comprising:

a computer;

a module for communication with a software application and operable in the computer to receive instructions to copy data to the remote storage device;

a chunking and identifier generation module operable in the computer to receive the data, to process the data into chunks, to generate a first chunk identifier representative of the identity of a first of the chunks of data and, upon processing of the data, to issue a confirmation signal to the software application indicating that the data has been copied to the remote storage device even though at least some of the chunks have not yet been sent to the remote storage device; and

an interface in the computer for communication with the remote storage device, wherein the interface is operable to send the first chunk identifier to the remote storage device, and to send the first chunk of data to the remote storage device upon receipt of a transfer instruction from the remote storage device.

15. The apparatus of claim 14, further comprising a second interface for communication with the remote storage device, wherein the second interface is operable to send data to the remote storage device independently of the chunking and identifier generation module.

16. The communication apparatus of claim 14, further comprising at least one processor to:

receive, from the remote storage device, a plurality of chunk identifiers,

determine whether a chunk corresponding to each of the plurality of chunk identifiers is known to the communication apparatus;

in response to determining that a particular chunk corresponding to a particular one of the plurality of chunk identifiers is known to the communication apparatus, send a first indication to the remote storage device to cause the remote storage device not to send the particular chunk to the communication apparatus; and

in response to determining that the particular chunk corresponding to the particular chunk identifier is unknown to the communication apparatus, send a second indication to the remote storage device to cause the remote storage device to send the particular chunk to the communication apparatus.

17. The communication apparatus of claim 16, wherein the particular chunk is determined to be known if a copy of the particular chunk is already in the communication apparatus.

18. A data storage system having a first processing unit and a second processing unit remote from the first processing unit, the system comprising:

a module for receiving an instruction from a software application for copying of data from the first to the second processing unit;

at least one hardware processor;

a chunking and identifier generation module executable on the at least one hardware processor to process the data to generate data chunks and identifiers each representative of the identity of a corresponding data chunk and, upon processing the data, to issue a confirmation signal to the software application that the data has been copied to the second processing unit even though at least some of the data chunks have not yet been sent to the second processing unit; and

a first interface to link the second processing unit to the first processing unit and the first and second processing units are operable to communicate with each other across the first interface using the identifiers in order to copy the data chunks which are not known to the second processing unit.

19. The data storage system according to claim 18, further comprising a second interface which allows data to be copied between the first and second processing units independently of the chunking and identifier generation module.