DATA PROCESSING METHOD, APPARATUS, AND SYSTEM

• US 20170295239A1
• Filed: 06/27/2017
• Published: 10/12/2017
• Est. Priority Date: 12/27/2014
• Status: Abandoned Application

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Fang, Xin
• Application Number:
  US15/634,774
• Publication Number:
  US 20170295239A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Embodiments of the present invention provide a data management technology. An OSD receives a strip write request sent by a client server, where the strip write request carries a to-be-written strip, a version number of the to-be-written strip, an offset of the to-be-written strip, and an object ID of the to-be-written strip; and the OSD writes the to-be-written strip into a storage location determined by using the object ID, the version number of the to-be-written strip, and the offset of the to-be-written strip. By applying the present invention, a quantity of object IDs can be reduced.

Claims

1. A data storage method, wherein the method comprises:

receiving, by an object storage device (OSD), a strip write request sent by a client server, wherein the strip write request carries a to-be-written strip, a version number of the to-be-written strip, an offset of the to-be-written strip, and an object ID of the to-be-written strip, the version number of the to-be-written strip is corresponding to a snapshot ID of a latest snapshot of a file or a volume to which the to-be-written strip belongs, the offset of the to-be-written strip describes a location of the to-be-written strip in an object to which the to-be-written strip belongs, and the object ID of the to-be-written strip is an ID of the object to which the to-be-written strip belongs; and

writing, by the OSD, the to-be-written strip into a storage location determined by using the object ID, the version number of the to-be-written strip, and the offset of the to-be-written strip.

2. The method according to claim 1, wherein before receiving a strip write request sent by a client server, the method further comprises:

taking, by the client server, a snapshot of the file or the volume to which the to-be-written strip belongs, and generating the snapshot ID of the latest snapshot; and

generating the version number of the to-be-written strip according to the snapshot ID of the latest snapshot.

3. The method according to claim 2, wherein the method further comprises:

updating, by the client server, the version number of the to-be-written strip to metadata of the file or the volume.

4. The method according to claim 1, wherein before the receiving, by an OSD, a strip write request, the method further comprises:

receiving, by the client server, a file write request, wherein the file write request carries to-be-written data, an offset of the to-be-written data, and a file name, and the to-be-written data is a part of the file;

obtaining, by the client server, a file identifier FID according to the file name, performing a query on the metadata of the file according to the FID to obtain a version number of the file, and using the version number of the file as the version number of the to-be-written strip, wherein the version number of the file is corresponding to the snapshot ID of the latest snapshot of the file to which the to-be-written strip belongs;

splitting, by the client server according to the offset of the to-be-written data and a size of the to-be-written data, the to-be-written data into multiple strips that comprise the to-be-written strip, determining the ID of the object to which the to-be-written strip belongs, and obtaining the offset of the to-be-written strip; and

creating the strip write request.

5. The method according to claim 1, wherein before the receiving, by an OSD, a strip write request, the method further comprises:

receiving, by the client server, a volume write request, wherein the volume write request carries to-be-written data, an offset of the to-be-written data, and a volume identifier ID, and the to-be-written data is a part of the volume;

performing, by the client server, a query on the metadata of the volume according to the volume ID to obtain a version number of the volume, and using the version number of the volume as the version number of the to-be-written strip, wherein the version number of the volume is corresponding to the snapshot ID of the latest snapshot of the volume to which the to-be-written strip belongs;

splitting, by the client server according to the offset of the to-be-written data and a size of the to-be-written data, the to-be-written data into multiple strips that comprise the to-be-written strip, determining the ID of the object to which the to-be-written strip belongs, and obtaining the offset of the to-be-written strip; and

creating the strip write request.

6. A data storage method, wherein the method comprises:

receiving, by an object storage device (OSD), a strip write request sent by a client server, wherein the strip write request carries a to-be-written strip, a version number of the to-be-written strip, an offset of the to-be-written strip, and an object ID of the to-be-written strip, the version number of the to-be-written strip is corresponding to a snapshot ID of a latest snapshot of a file or a volume to which the to-be-written strip belongs, the offset of the to-be-written strip describes a location of the to-be-written strip in an object to which the to-be-written strip belongs, and the object ID of the to-be-written strip is an ID of the object to which the to-be-written strip belongs;

determining, by the OSD, whether an object determined by using the version number of the to-be-written strip and the object ID is backed up; and

if the object is backed up, writing, by the OSD, the to-be-written strip into a storage location determined by using the object ID, the version number of the to-be-written strip, and the offset of the to-be-written strip;

or

if the object is not backed up, creating, by the OSD, a spliced object by using the to-be-written strip, and then writing the spliced object into a storage location determined by using the version number of the to-be-written strip and the object ID.

7. The method according to claim 6, wherein the creating, by the OSD, a spliced object by using the to-be-written strip specifically comprises:

selecting, by the OSD, an object with a latest snapshot time from a backed-up object that belongs to an object set of the object ID of the to-be-written strip, to obtain a strip whose offset is different from the offset of the to-be-written strip, and jointly constituting the spliced object by using the to-be-written strip and the strip whose offset is different from the offset of the to-be-written strip, wherein

a set of an object that is stored in the OSD and whose object ID is the same as the object ID of the to-be-written strip and version number is different from the version number of the to-be-written strip is referred to as the object set of the object ID of the to-be-written strip.

8. The method according to claim 6, wherein before the receiving a strip write request sent by a client server, the method further comprises:

taking, by the client server, a snapshot of the file or the volume to which the to-be-written strip belongs, and generating the snapshot ID of the latest snapshot;

generating the version number of the to-be-written strip according to the snapshot ID of the latest snapshot; and

updating, by the client server, the version number of the to-be-written strip to metadata of the file or the volume.

9. An object storage device, comprising a processor, and a storage medium and an interface that are connected to the processor, wherein:

the interface is configured to receive a strip write request sent by a client server, wherein the strip write request carries a to-be-written strip, a version number of the to-be-written strip, an offset of the to-be-written strip, and an object ID of the to-be-written strip, the version number of the to-be-written strip is corresponding to a snapshot ID of a latest snapshot of a file or a volume to which the to-be-written strip belongs, the offset of the to-be-written strip describes a location of the to-be-written strip in an object to which the to-be-written strip belongs, and the object ID of the to-be-written strip is an ID of the object to which the to-be-written strip belongs;

the storage medium stores a computer program; and

by running the computer program, the processor is configured to;

write the to-be-written strip into a storage location determined by using the object ID, the version number of the to-be-written strip, and the offset of the to-be-written strip.

10. An object storage device, comprising a processor, and a storage medium and an interface that are connected to the processor, wherein:

the interface is configured to receive a strip write request sent by a client server, wherein the strip write request carries a to-be-written strip, a version number of the to-be-written strip, an offset of the to-be-written strip, and an object ID of the to-be-written strip, the version number of the to-be-written strip is corresponding to a snapshot ID of a latest snapshot of a file or a volume to which the to-be-written strip belongs, the offset of the to-be-written strip describes a location of the to-be-written strip in an object to which the to-be-written strip belongs, and the object ID of the to-be-written strip is an ID of the object to which the to-be-written strip belongs;

the storage medium stores a computer program; and

by running the computer program, the processor is configured to;

determine whether an object determined by using the version number of the to-be-written strip and the object ID is backed up; and

if the object is backed up, the processor is further configured to write the to-be-written strip into a storage location determined by using the object ID, the version number of the to-be-written strip, and the offset of the to-be-written strip;

or

if the object is not backed up, the processor is further configured to create a spliced object by using the to-be-written strip, and then write the spliced object into a storage location determined by using the version number of the to-be-written strip and the object ID.

11. The object storage device according to claim 10, wherein that the processor is configured to create a spliced object by using the to-be-written strip specifically comprises:

the processor is configured to;

select an object with a latest snapshot time from a backed-up object that belongs to an object set of the object ID of the to-be-written strip, to obtain a strip whose offset is different from the offset of the to-be-written strip, and jointly constitute the spliced object by using the to-be-written strip and the strip whose offset is different from the offset of the to-be-written strip, wherein;

a set of an object that is stored in the object storage device and whose object ID is the same as the object ID of the to-be-written strip and version number is different from the version number of the to-be-written strip is referred to as the object set of the object ID of the to-be-written strip.