OBJECT STORE ARCHITECTURE FOR DISTRIBUTED DATA PROCESSING SYSTEM
First Claim
1. A method comprising:
- coupling a plurality of compute nodes to a storage system, the storage system interacting with the compute nodes to embody an object store adapted to service objects stored on one or more disks of the storage system, the object store having an on-disk layout that includes a plurality of volumes of the storage system and an incore layout that includes incore data structures of the compute nodes, wherein each volume is a container of the objects, the on-disk layout cooperating with the incore layout to service the objects;
assembling the compute nodes and the volumes into a compute group;
mapping the volumes into each compute node of the compute group, each compute node accessing the objects of the object store via the volumes, the mapping enabling each compute node read-write (RW) access to a specified volume of the compute group mapped into the compute node and read-only (RO) access to remaining volumes of the compute group mapped into the compute node; and
configuring the storage system as a quasi-shared storage system by relaxing a restriction of private storage of the compute nodes to allow shared storage access of the object store within the compute group.
1 Assignment
0 Petitions
Accused Products
Abstract
Embodiments described herein provide an object store that efficiently manages and services objects for use by clients of a distributed data processing system. Illustratively, the object store may be embodied as a quasi-shared storage system that interacts with nodes of the distributed data processing system to service the objects as blocks of data stored on a plurality of storage devices, such as disks, of the storage system. To that end, an architecture of the object store may include an on-disk layout, e.g., of the storage system, and an incore layout, e.g., of the nodes, that cooperate to illustratively convert the blocks to objects for access by the clients.
25 Citations
5 Claims
-
1. A method comprising:
-
coupling a plurality of compute nodes to a storage system, the storage system interacting with the compute nodes to embody an object store adapted to service objects stored on one or more disks of the storage system, the object store having an on-disk layout that includes a plurality of volumes of the storage system and an incore layout that includes incore data structures of the compute nodes, wherein each volume is a container of the objects, the on-disk layout cooperating with the incore layout to service the objects; assembling the compute nodes and the volumes into a compute group; mapping the volumes into each compute node of the compute group, each compute node accessing the objects of the object store via the volumes, the mapping enabling each compute node read-write (RW) access to a specified volume of the compute group mapped into the compute node and read-only (RO) access to remaining volumes of the compute group mapped into the compute node; and configuring the storage system as a quasi-shared storage system by relaxing a restriction of private storage of the compute nodes to allow shared storage access of the object store within the compute group. - View Dependent Claims (2, 3, 4)
-
-
5. A computer readable medium encoded with software comprising computer executable instructions and when the software is executed operable to:
-
embody an object store as a storage system configured to interact with nodes to service an object as a block of data stored on one or more disks of the storage system, the object store having an on-disk layout that includes volumes of the storage system, the nodes including a job distribution node and compute nodes; assemble the compute nodes and the volumes into a compute group connected to the job distribution node; map the volumes into each compute node of the compute group, the compute node configured to access the objects of the object store via the volumes; enable the compute node read-write (RW) access to a specified volume of the compute group mapped into the compute node and read-only (RO) access to remaining volumes of the compute group mapped into the compute node; and maintain a virtual replica map at the job distribution node to identify the RW access and RO access of the compute node to each volume of the compute group to thereby enable load balancing and redirecting of read and write traffic across the compute nodes.
-
Specification