Providing integrity for data sets backed-up from client systems to objects in a network storage

• US 10,528,431 B2
• Filed: 02/04/2016
• Issued: 01/07/2020
• Est. Priority Date: 02/04/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Figueroa, Ernesto E., Gensler, Jr., Robert S., Suarez, Jeffrey R.
• Primary Examiner(s): Truong, Dennis
• Application Number:
  US15/015,598
• Publication Number:
  US 20170228293A1
• Time in Patent Office: 1,433 Days
• Field of Search: 707661, 707649, 707667
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Provided are a computer program product, system, and method for providing integrity for data sets backed-up from client systems to objects in a network storage. Data set objects include data sets at a client storage and an inventory object has information on each of the data set objects, which are part of a backup object group to backup to the network storage. The data set and inventory objects in the backup object group are transferred to the network storage to store. The inventory object from the network storage is retrieved to select at least one of the data set objects in the network storage, which is requested. Notification is received that the backup object group has a data integrity error in response to one of the requested data set objects not located in the network storage.

Claims

1. A computer program product to backup data sets to a network storage over a network, wherein the computer program product comprises a computer readable storage medium having computer readable program instructions executed by a processor to perform operations, the operations comprising:

generating data set objects including data sets at a client storage;

generating an inventory object having information on each of the data set objects, wherein the data set objects and the inventory object are part of a backup object group to backup to the network storage;

transferring the data set objects and the inventory object in the backup object group to the network storage, wherein the data set objects are transferred in an order of the data set objects indicated in the inventory object, to store using a block level interface to perform a block level storage transfer to transfer blocks of the data objects to write to a sequential file to store sequentially in the sequential file in the order indicated in the inventory object, wherein the data set objects are written sequentially to the sequential file in a sequential manner used for transferring data to sequential access media, wherein the data set objects are stored in non-sequential access storage in the network storage;

retrieving the inventory object from the network storage to select at least one of the data set objects in the network storage;

requesting at least one data set object indicated in the retrieved inventory object; and

receiving from the network storage the requested at least one data set object transferred using the block level storage transfer, to retrieve data stored in the sequential manner in the sequential file in the network storage, in response to the requested at least one data set object located in the network storage.

2. The computer program product of claim 1, wherein the operations further comprise:

issuing a GET command to perform the retrieving of the inventory object from the network storage, wherein the retrieved inventory object is processed to determine the data set objects in the network storage; and

issuing a GET command to perform the requesting of the at least one of the data set objects.

3. The computer program product of claim 2, wherein the operations further comprise:

issuing PUT commands to perform the transferring of the data set objects and the inventory object to the network storage for storage.

4. The computer program product of claim 3, wherein the PUT and GET commands comprise Hypertext Transport Protocol (“

HTTP”

) PUT and GET commands.

5. The computer program product of claim 3, wherein the transferring the data set objects further comprises initiating a DUMP command to copy the data sets to objects in the network storage, wherein the DUMP command initiates the PUT command to perform the transferring of the data sets that are indicated in the DUMP command, and wherein the requesting the at least one data set object in the network storage comprises issuing a RESTORE command to restore at least one of the data sets in the objects from the network, wherein the RESTORE command initiates the GET command to retrieve at least one of the objects having the data sets to restore.

6. The computer program product of claim 1, wherein the operations further comprise:

receiving notification that the backup object group has a data integrity error in response to at least one calculated identifier code of the at least one of the data set objects subject to the requesting does not match at least one identifier code stored for the at least one of the data set objects stored in the network storage, wherein the requested at least one data set object is received from the network storage using the block level storage transfer in response to the requested at least one data set object located in the network storage and the at least one calculated identifier code matching the at least one identifier code stored for the at least one of the data set objects.

7. The computer program product of claim 1, wherein the operations further comprise:

receiving notification that the backup object group has a data integrity error in response to the at least one of the data set objects subject to the requesting is not located in the network storage.

8. A computer program product to backup data sets in a network storage that are received from a plurality of client systems over a network, wherein the computer program product comprises a computer readable storage medium having computer readable program instructions executed by a processor to perform operations, the operations comprising:

receiving, from the plurality of client systems over the network, data set objects and inventory objects, wherein each inventory object has information on each of the data set objects in a backup object group, wherein the client systems transfer the data set objects and the inventory objects, in an order of the data set objects indicated in the inventory object, using a block level interface to perform a block level storage transfer to transfer blocks to a sequential file to store sequentially in the sequential file in the order indicated in the inventory object, wherein the data set objects are written sequentially to the sequential file in a sequential manner used for transferring data to sequential access media;

storing the received data set objects in the network storage, wherein the data set objects are stored to non-sequential access storage in the network storage;

indicating the stored data set objects in an inventory object;

receiving a request for a data set object comprising one of the data set objects from a client system of the client systems;

determining from the inventory object whether the requested data set object is indicated in the inventory object for the backup object group is stored in the network storage; and

returning from the network storage the requested data set object to the client system using the block level storage transfer, to retrieve data stored in the sequential manner in the sequential file in the network storage, in response to the requested data set object located in the network storage.

9. The computer program product of claim 8, wherein the inventory object is read-only and cannot be modified, wherein the determining that the requested data set object indicated in the inventory object is stored in the network storage comprises:

generating a data integrity error for the backup object group including the requested data set object in response to determining at least one that the requested data set object is not located in the network storage and a data integrity error with the data set objects stored in the network storage; and

returning a message with indication of the data integrity error to the client system requesting the data set object in response to generating the data integrity error.

10. The computer program product of claim 8, wherein the network storage comprises a cloud based storage service for the client systems to backup object groups at the network storage.

11. The computer program product of claim 8, wherein the operations further comprise:

generating a data integrity error in response to determining that a calculated identifier code of the requested data set object does not match an identifier code stored for the requested data set object, wherein the requested data set object is returned to the client system in response to calculated identifier code matching the identifier code stored for the requested data set object.

12. A system coupled to a network storage and a plurality of client systems over a network, comprising:

a processor;

a computer readable storage medium including computer readable program instructions that when executed by a processor causes operations, the operations comprising;

receiving, from the plurality of client systems over the network, data set objects and inventory objects, wherein each inventory object has information on each of the data set objects in a backup object group, wherein the client systems transfer the data set objects and the inventory objects, in an order of the data set objects indicated in the inventory object, using a block level interface to perform a block level storage transfer to transfer blocks to a sequential file to store sequentially in the sequential file in the order indicated in the inventory object, wherein the data set objects are written sequentially to the sequential file in a sequential manner used for transferring data to sequential access media;

storing the received data set objects in the network storage, wherein the data set objects are stored to non-sequential access storage in the network storage;

indicating the stored data set objects in an inventory object;

receiving a request for a data set object comprising one of the data set objects from a client system of the client systems;

determining from the inventory object whether the requested data set object is indicated in the inventory object for the backup object group is stored in the network storage; and

returning from the network storage the requested data set object to the client system using the block level storage transfer, to retrieve data stored in the sequential manner in the sequential file in the network storage, in response to the requested data set object located in the network storage.

13. The system of claim 12, wherein the inventory object is read-only and cannot be modified, wherein the determining that the requested data set object indicated in the inventory object is not stored in the network storage comprises;

generating a data integrity error for the backup object group including the requested data set object in response to determining at least one that the requested data set object is not located in the network storage and a data integrity error with the data set objects stored in the network storage; and

returning a message with indication of the data integrity error to the client system requesting the data set object in response to generating the data integrity error.

14. The system of claim 12, wherein the network storage comprises a cloud based storage service for the client systems to backup object groups at the network storage.

15. The system of claim 12, wherein the operations further comprise:

generating a data integrity error in response to determining that a calculated identifier code of the requested data set object does not match an identifier code stored for the requested data set object, wherein the requested data set object is returned to the client system in response to calculated identifier code matching the identifier code stored for the requested data set object.

16. A method implemented in a computer system to backup data sets to a network storage over a network, comprising:

generating data set objects including data sets at a client storage;

generating an inventory object having information on each of the data set objects, wherein the data set objects and the inventory object are part of a backup object group to backup to the network storage;

transferring the data set objects and the inventory object in the backup object group to the network storage, wherein the data set objects are transferred in an order of the data set objects indicated in the inventory object, to store using a block level interface to perform a block level storage transfer to transfer blocks of the data objects to write to a sequential file to store sequentially in the sequential file in the order indicated in the inventory object, wherein the data set objects are written sequentially to the sequential file in a sequential manner used for transferring data to sequential access media, wherein the data set objects are stored in non-sequential access storage in the network storage;

retrieving the inventory object from the network storage to select at least one of the data set objects in the network storage;

requesting at least one data set object indicated in the retrieved inventory object;

and

receiving from the network storage the requested at least one data set object transferred using the block level storage transfer, to retrieve data stored in the sequential manner in the sequential file in the network storage, in response to the requested at least one data set object located in the network storage.

17. The method of claim 16, further comprising:

issuing a GET command to perform the retrieving of the inventory object from the network storage, wherein the retrieved inventory object is processed to determine the data set objects in the network storage; and

issuing a GET command to perform the requesting of the at least one of the data set objects.

18. The method of claim 17, further comprising:

issuing PUT commands to perform the transferring of the data set objects and the inventory object to the network storage for storage.

19. The method of claim 18, wherein the transferring the data set objects further comprises initiating a DUMP command to copy the data sets to objects in the network storage, wherein the DUMP command initiates the PUT command to perform the transferring of the data sets that are indicated in the DUMP command, and wherein the requesting the at least one data set object in the network storage comprises issuing a RESTORE command to restore at least one of the data sets in the objects from the network, wherein the RESTORE command initiates the GET command to retrieve at least one of the objects having the data sets to restore.

20. The method of claim 16, further comprising:

receiving notification that the backup object group has a data integrity error in response to at least one calculated identifier code of the at least one of the data set objects subject to the requesting does not match at least one identifier code stored for the at least one of the data set objects stored in the network storage, wherein the requested at least one data set object is received from the network storage using the block level storage transfer in response to the requested at least one data set object located in the network storage and the at least one calculated identifier code matching the at least one identifier code stored for the at least one of the data set objects.

21. The method of claim 16, further comprising:

receiving notification that the backup object group has a data integrity error in response to the at least one of the data set objects subject to the requesting is not located in the network storage.

22. A method implemented in a computer system to backup data sets in a network storage that are received from a plurality of client systems over a network, comprising:

receiving, from the plurality of client systems over the network, data set objects and inventory objects, wherein each inventory object has information on each of the data set objects in a backup object group, wherein the client systems transfer the data set objects and the inventory objects, in an order of the data set objects indicated in the inventory object, using a block level interface to perform a block level storage transfer to transfer blocks to a sequential file to store sequentially in the sequential file in the order indicated in the inventory object, wherein the data set objects are written sequentially to the sequential file in a sequential manner used for transferring data to sequential access media;

storing the received data set objects in the network storage, wherein the data set objects are stored to non-sequential access storage in the network storage;

indicating the stored data set objects in an inventory object;

receiving a request for a data set object comprising one of the data set objects from a client system of the client systems;

determining from the inventory object whether the requested data set object is indicated in the inventory object for the backup object group is stored in the network storage;

and

returning from the network storage the requested data set object to the client system using the block level storage transfer, to retrieve data stored in the sequential manner in the sequential file in the network storage, in response to the requested data set object located in the network storage.

23. The method of claim 22, wherein the inventory object is read-only and cannot be modified, wherein the determining that the requested data set object indicated in the inventory object is stored in the network storage comprises:

generating a data integrity error for the backup object group including the requested data set object in response to determining at least one that the requested data set object is not located in the network storage and a data integrity error with the data set objects stored in the network storage; and

returning a message with indication of the data integrity error to the client system requesting the data set object in response to generating the data integrity error.

24. The method of claim 22, wherein the network storage comprises a cloud based storage service for the client systems to backup object groups at the network storage.