Computational storage for distributed computing
First Claim
1. A computer-implemented method comprising:
- setting up, by a computational storage server, a computational storage media in the computational storage server, wherein the computational storage media is set up to comprise a first memory area for storing the input data received from multiple clients, a second memory area designated for storing computation functions that are executed by the computational storage server using the input data received from the multiple clients, a client specific management metadata area in the computational storage media for storing metadata related to computations performed in the computational storage server for specific clients, and a persistent storage area designated for storing checkpoints for aggregating computations performed by the computation functions;
detecting, by the computational storage server, an indication from one or more clients from the multiple clients indicating a probe-ready status for the computational storage media;
in response to one or more clients from the multiple clients indicating the probe-ready status for the computational storage media, exposing, by the computational storage server, the computational storage media to the multiple clients as an available storage media;
receiving, by the computational storage server, input data from the multiple clients;
executing, within the computational storage server, one or more computation functions using the received input data as inputs;
aggregating, by the computational storage server, outputs of the one or more computation functions that used the received input data as inputs; and
returning, from the computational storage server to the multiple clients, aggregated outputs of the one or more computation functions that used the received input data as inputs.
1 Assignment
0 Petitions
Accused Products
Abstract
Computational storage techniques for distribute computing are disclosed. The computational storage server receives input from multiple clients, which is used by the server when executing one or more computation functions. The computational storage server can aggregate multiple client inputs before applying one or more computation functions. The computational storage server sets up: a first memory area for storing input received from multiple clients; a second memory area designated for storing the computation functions to be executed by the computational storage server using the input data received from the multiple clients; a client specific memory management area for storing metadata related to computations performed by the computational storage server for specific clients; and a persistent storage area for storing checkpoints associated with aggregating computations performed by the computation functions.
14 Citations
18 Claims
-
1. A computer-implemented method comprising:
-
setting up, by a computational storage server, a computational storage media in the computational storage server, wherein the computational storage media is set up to comprise a first memory area for storing the input data received from multiple clients, a second memory area designated for storing computation functions that are executed by the computational storage server using the input data received from the multiple clients, a client specific management metadata area in the computational storage media for storing metadata related to computations performed in the computational storage server for specific clients, and a persistent storage area designated for storing checkpoints for aggregating computations performed by the computation functions; detecting, by the computational storage server, an indication from one or more clients from the multiple clients indicating a probe-ready status for the computational storage media; in response to one or more clients from the multiple clients indicating the probe-ready status for the computational storage media, exposing, by the computational storage server, the computational storage media to the multiple clients as an available storage media; receiving, by the computational storage server, input data from the multiple clients; executing, within the computational storage server, one or more computation functions using the received input data as inputs; aggregating, by the computational storage server, outputs of the one or more computation functions that used the received input data as inputs; and returning, from the computational storage server to the multiple clients, aggregated outputs of the one or more computation functions that used the received input data as inputs. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A computer program product for aggregating computation operations and their outputs from a computational storage server, wherein the computational storage server is a management computational storage server, the computer program product comprising a computer readable storage device having program instructions embodied therewith, the program instructions readable and executable by a computer to cause:
-
receiving input data from multiple clients; executing one or more computation functions using the received input data as inputs, wherein the one or more computation functions are a matrix of multiple different computation functions that execute using different computer languages; aggregating, by the computational storage server, outputs of the one or more computation functions that used the received input data as inputs; returning, to the multiple clients, aggregated outputs of the one or more computation functions that used the received input data as inputs; splitting, by the management computational storage server, the matrix into multiple sub-matrixes of computation functions, wherein each of the multiple sub-matrixes is written in a different computer language than other sub-matrixes from the multiple sub-matrixes; storing, by the management computational storage server, each of the multiple sub-matrixes into a different computational storage server, wherein each different computational storage server is specific for a particular computer language; receiving, by the management computational storage server, first data from a first client that is written in a first computer language; receiving, by the management computational storage server, second data from a second client that is written in a second computer language; transmitting, by the management computational storage server, the first data to a first computational storage server than contains a first sub-matrix computational function that runs under the first computer language; transmitting, by the management computational storage server, the second data to a second computational storage server than contains a second sub-matrix computational function that runs under the second computer language; receiving, by the management computational storage server, a first result from the first computational storage server and a second result from the second computational storage server, wherein the first result is an output of the first sub-matrix computational function using the first data, and wherein the second result is an output of the second sub-matrix computational function using the second data; consolidating, by the management computational storage server, the first result and the second result into a consolidated result; sending, from the management computational storage server, the consolidated result to the first client and the second client; receiving, by the first computational storage server, a set of global parameters from a global parameter server, wherein the set of global parameters comprises data that weights one or more operands used in the first sub-matrix computational function; and executing, by multiple learner processors in the first computational storage server, the first sub-matrix computational function using the set of global parameters. - View Dependent Claims (8, 9, 10, 11)
-
-
12. A computer system comprising:
-
one or more processors; one or more computer readable memories, operably coupled to the one or more processors, wherein the one or more computer readable memories store program instructions for execution by at least one of the one or more processors, the stored program instructions comprising; program instructions to set up a computational storage media in a computational storage server, wherein the computational storage media is set up to comprise a first memory area for storing input data received from multiple clients, a second memory area designated for storing computation functions executed by the computational storage server using the input data received from the multiple clients, a client specific management metadata area in the computational storage media for storing metadata related to computations performed in the computational storage server for specific clients, and a persistent storage area designated for storing checkpoints for aggregating computations performed by the computation functions; program instructions to detect an indication from one or more clients from the multiple clients indicating a probe-ready status for the computational storage media; program instructions to, in response to one or more clients from the multiple clients indicating the probe-ready status for the computational storage media, expose the computational storage media to the multiple clients as an available storage media; program instructions to receive input data from the multiple clients; program instructions to execute one or more of the computation functions using the received input data as inputs; program instructions to aggregate outputs of the one or more computation functions that used the received input data as inputs; and program instructions to return, to the multiple clients, aggregated outputs of the one or more computation functions that used the received input data as inputs. - View Dependent Claims (13, 14, 15, 16, 17, 18)
-
Specification