Decoupling partitioning for scalability
First Claim
1. A method for performing operations on a client defined namespace in a distributed computing environment, comprising:
- receiving a namespace at a system, from a client application, the namespace comprising a plurality of key values that span the namespace;
receiving one or more application-defined interfaces, at the system, wherein application-defined interfaces are comprised of computations, operations, or functions defined by the client application;
dividing, by one or more fixed system interfaces, the received namespace into a plurality of partitions, wherein fixed interfaces are comprised of computations, operation, or functions integrated into the system;
assigning, by the one or more fixed interfaces, the plurality of partitions to one or more partition servers that implement at least one of the received application-defined interfaces, wherein the fixed interfaces and application-defined interfaces operationally separate distributed computing environment operations for the one or more partition servers, from application-specific operations for the one or more partition servers;
creating, by the one or more fixed interfaces, a storage object for each of the one or more partition servers, the storage object comprising information related to partitions assigned to the one or more partition servers;
receiving a client request to perform a computation, the request including a key value from the namespace;
identifying a partition server corresponding to the key value based on the assigned one or more partitions; and
returning, in response to the received request, an address for the identified partition server.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems and methods are provided that enable a general framework for partitioning application-defined computations (e.g., jobs) in a scalable environment. The general framework decouples partitioning of a computation from the other aspects of the computation. As a result, the effort required to define an application-defined job is reduced or minimized, as the user is not required to provide a partitioning algorithm. A user can optionally take advantage of a partitioning framework by providing application-defined interfaces to perform the desired job. Optionally, a user can provide additional information to allow for modification of how partitions are assigned.
-
Citations
20 Claims
-
1. A method for performing operations on a client defined namespace in a distributed computing environment, comprising:
-
receiving a namespace at a system, from a client application, the namespace comprising a plurality of key values that span the namespace; receiving one or more application-defined interfaces, at the system, wherein application-defined interfaces are comprised of computations, operations, or functions defined by the client application; dividing, by one or more fixed system interfaces, the received namespace into a plurality of partitions, wherein fixed interfaces are comprised of computations, operation, or functions integrated into the system; assigning, by the one or more fixed interfaces, the plurality of partitions to one or more partition servers that implement at least one of the received application-defined interfaces, wherein the fixed interfaces and application-defined interfaces operationally separate distributed computing environment operations for the one or more partition servers, from application-specific operations for the one or more partition servers; creating, by the one or more fixed interfaces, a storage object for each of the one or more partition servers, the storage object comprising information related to partitions assigned to the one or more partition servers; receiving a client request to perform a computation, the request including a key value from the namespace; identifying a partition server corresponding to the key value based on the assigned one or more partitions; and returning, in response to the received request, an address for the identified partition server. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. One or more hardware computer-storage media storing computer-useable instructions that, when executed, perform a method for performing partitioning on a client defined namespace in a distributed computing environment, comprising:
-
receiving a namespace at a system, from a client application comprising a plurality of key values that span the namespace; accessing one or more application-defined interfaces, at the system, wherein application-defined interfaces are comprised of computations, operations, or functions defined by the client application; providing a partition table comprising assignments of a plurality of partitions based on the received namespace to one or more partition servers that implement at least one of the received application-defined partitioning system interfaces; modifying, by one or more fixed interfaces of a master role instance, one or more partition assignments in the partition table, wherein fixed interfaces are comprised of computations, operations, or functions integrated into the system, wherein the fixed interfaces and application-defined interfaces operationally separate distributed computing environment operations for the one or more partition servers, from application-specific operations for the one or more partition servers; updating, by the one or more fixed interfaces of a partition server, a storage object for at least one partition server, the storage object comprising information related to partitions assigned to the partition server, the update corresponding to one or more of the modified partition assignments; receiving a client request to perform a computation, the request including a key value from the namespace; identifying a partition server corresponding to the key value based on the modified one or more partition assignments in the partition table; and returning, in response to the received request, an address for the identified partition server. - View Dependent Claims (11, 12, 13, 14)
-
-
15. One or more hardware computer-storage media storing computer-useable instructions that, when executed, perform a method for performing load balancing in a distributed computing environment, comprising:
-
receiving a namespace at a system, from an application, the namespace comprising a plurality of key values that span the namespace; assigning, by one or more fixed interfaces, a plurality of partitions based on the namespace to one or more partition servers, the plurality of partitions corresponding to key ranges that span the namespace, the partition servers implementing one or more application-defined interfaces, wherein fixed interfaces are comprised of computations, operations, or functions integrated into the system; receiving, by one or more application-defined interfaces, an application-defined load balancing condition and a filter condition, the load balancing condition comprising a condition for performing a load balancing action, the condition being based on an expression including at least one metric received from the partition servers, wherein application-defined interfaces are comprised of computations, operations, or functions defined by the application, wherein the fixed interfaces and application-defined interfaces operationally separate distributed computing environment operations for the one or more partition servers, from application-specific operations for the one or more partition servers; determining that the load balancing condition and the filter condition are satisfied; and modifying, by the one or more fixed interfaces, a load balancing action corresponding to the load balancing condition based on the filter condition being satisfied; and performing, by the one or more fixed interfaces, the modified load balancing action. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification