Object store architecture for distributed data processing system
First Claim
Patent Images
1. A method comprising:
- mapping each compute node of a plurality of compute nodes to a different volume of a plurality of volumes that defines a logical arrangement of storage space of a storage system, wherein each compute node has read-write access to the volume to which the compute node is mapped and read-only access to those of the plurality of volumes not mapped to the compute node; and
in response to receiving an identifier for a data object at a first compute node of the plurality of compute nodes,identifying an object descriptor associated with the identifier and a plurality of chunk descriptors referenced by the object descriptor, wherein the object descriptor and the plurality of chunk descriptors are arranged on the first compute node according to an incore layout;
utilizing the plurality of chunk descriptors to retrieve, via a network from the plurality of volumes, blocks of data stored according to an on-disk layout of the plurality of volumes, wherein the blocks of data include a plurality of chunks described by the plurality of chunk descriptors;
the first compute node assembling the plurality of chunks described by the plurality of chunk descriptors into the data object; and
returning the assembled data object.
1 Assignment
0 Petitions
Accused Products
Abstract
Embodiments described herein provide an object store that efficiently manages and services objects for use by clients of a distributed data processing system. Illustratively, the object store may be embodied as a quasi-shared storage system that interacts with nodes of the distributed data processing system to service the objects as blocks of data stored on a plurality of storage devices, such as disks, of the storage system. To that end, an architecture of the object store may include an on-disk layout, e.g., of the storage system, and an incore layout, e.g., of the nodes, that cooperate to illustratively convert the blocks to objects for access by the clients.
-
Citations
20 Claims
-
1. A method comprising:
-
mapping each compute node of a plurality of compute nodes to a different volume of a plurality of volumes that defines a logical arrangement of storage space of a storage system, wherein each compute node has read-write access to the volume to which the compute node is mapped and read-only access to those of the plurality of volumes not mapped to the compute node; and in response to receiving an identifier for a data object at a first compute node of the plurality of compute nodes, identifying an object descriptor associated with the identifier and a plurality of chunk descriptors referenced by the object descriptor, wherein the object descriptor and the plurality of chunk descriptors are arranged on the first compute node according to an incore layout; utilizing the plurality of chunk descriptors to retrieve, via a network from the plurality of volumes, blocks of data stored according to an on-disk layout of the plurality of volumes, wherein the blocks of data include a plurality of chunks described by the plurality of chunk descriptors; the first compute node assembling the plurality of chunks described by the plurality of chunk descriptors into the data object; and returning the assembled data object. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A computer readable medium having computer executable instructions stored therein, the instructions to:
-
map each compute node of a plurality of compute nodes to a different volume of a plurality of volumes that defines a logical arrangement of storage space of a storage system, wherein each compute node has read-write access to the volume to which the compute node is mapped and read-only access to those of the plurality of volumes not mapped to the compute node; and in response to receipt of an identifier for a data object at a first compute node of the plurality of compute nodes, identify an object descriptor associated with the identifier and a plurality of chunk descriptors referenced by the object descriptor, wherein the object descriptor and the plurality of chunk descriptors are arranged on the first compute node according to an incore layout; utilize the plurality of chunk descriptors to retrieve blocks of data from the plurality of volumes, wherein the blocks of data are stored according to an on-disk layout of the plurality of volumes, wherein the blocks of data include a plurality of chunks described by the plurality of chunk descriptors; assemble the plurality of chunks described by the plurality of chunk descriptors into the data object; and return the assembled data object. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
-
15. An apparatus comprising:
-
a processor; and a machine-readable medium having program code executable by the processor to cause the apparatus to, map each compute node of a plurality of compute nodes to a different volume of a plurality of volumes that defines a logical arrangement of storage space of a storage system, wherein each compute node has read-write access to the volume to which the compute node is mapped and read-only access to those of the plurality of volumes not mapped to the compute node; and in response to receipt of an identifier for a data object at a first compute node of the plurality of compute nodes, identify an object descriptor associated with the identifier and a plurality of chunk descriptors referenced by the object descriptor, wherein the object descriptor and the plurality of chunk descriptors are arranged on the first compute node according to an incore layout; utilize the plurality of chunk descriptors to retrieve, from the plurality of volumes via a network, blocks of data stored according to an on-disk layout of the plurality of volumes, wherein the blocks of data include a plurality of chunks described by the plurality of chunk descriptors; the first compute node assemble the plurality of chunks described by the plurality of chunk descriptors into the data object; and return the assembled data object. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification