Systems and methods of improving parallel functional processing
First Claim
1. A method of improving parallel functional processing, the method including:
- performing data transformations in a functional processing pipeline running on multiple processors using at least one instance or composition combining each of categorical functions in a group including PairMaker, FreeMonoidReduce, PairABtoAFMB, ReducePairs, and MonoidReduce, wherein;
the PairMaker categorical function transforms a free monoid over strings into a free monoid over tuples including at least one key-value pair in each tuple;
the FreeMonoidReduce categorical function merges a nested free monoid over tuples into one free monoid over tuples, thereby reducing a nesting depth of the nested free monoid over tuples;
the PairABtoAFMB categorical function transforms one element in each tuple of the one free monoid over tuples into list of one element in a free monoid over tuples with an embedded list element;
the ReducePairs categorical function merges consecutive tuples of the free monoid over tuples with the embedded list element; and
the MonoidReduce categorical function transforms a plurality of list values into a single value based on a parameterized operation.
1 Assignment
0 Petitions
Accused Products
Abstract
The technology disclosed relates to improving parallel functional processing using abstractions and methods defined based on category theory. In particular, the technology disclosed provides a range of useful categorical functions for processing large data sets in parallel. These categorical functions manage all phases of distributed computing, including dividing a data set into subsets of approximately equal size and combining the results of the subset calculations into a final result, while hiding many of the low-level programming details. These categorical functions are extraordinarily well-ordered and have a sophisticated type system and type inference, which allows for generating maps and reducing them in an elegant and succinct way using concise and expressive programs that can significantly efficientize a whole software development process.
158 Citations
20 Claims
-
1. A method of improving parallel functional processing, the method including:
performing data transformations in a functional processing pipeline running on multiple processors using at least one instance or composition combining each of categorical functions in a group including PairMaker, FreeMonoidReduce, PairABtoAFMB, ReducePairs, and MonoidReduce, wherein; the PairMaker categorical function transforms a free monoid over strings into a free monoid over tuples including at least one key-value pair in each tuple; the FreeMonoidReduce categorical function merges a nested free monoid over tuples into one free monoid over tuples, thereby reducing a nesting depth of the nested free monoid over tuples; the PairABtoAFMB categorical function transforms one element in each tuple of the one free monoid over tuples into list of one element in a free monoid over tuples with an embedded list element; the ReducePairs categorical function merges consecutive tuples of the free monoid over tuples with the embedded list element; and the MonoidReduce categorical function transforms a plurality of list values into a single value based on a parameterized operation. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
15. A non-transitory computer readable storage medium impressed with computer program instructions for programming one or more processors to improve parallel functional processing, when executed on the processors, implementing actions including:
performing data transformations in a functional processing pipeline running on multiple processors using at least one instance or composition combining each of categorical functions in a group including PairMaker, FreeMonoidReduce, PairABtoAFMB, ReducePairs, and MonoidReduce, wherein; the PairMaker categorical function transforms a free monoid over strings into a free monoid over tuples including at least one key-value pair in each tuple; the FreeMonoidReduce categorical function merges a nested free monoid over tuples into one free monoid over tuples, thereby reducing a nesting depth of the nested free monoid over tuples; the PairABtoAFMB categorical function transforms one element in each tuple of the one free monoid over tuples into list of one element in a free monoid over tuples with an embedded list element; the ReducePairs categorical function merges consecutive tuples of the free monoid over tuples with the embedded list element; and the MonoidReduce categorical function transforms a plurality of list values into a single value based on a parameterized operation. - View Dependent Claims (16, 17)
-
18. A system including multiple coupled to memory, the memory loaded with computer instructions for programming multiple processors to improve parallel functional processing, when executed on the processors, implementing actions including:
performing data transformations in a functional processing pipeline running on multiple processors using at least one instance or composition combining each of categorical functions in a group including PairMaker, FreeMonoidReduce, PairABtoAFMB, ReducePairs, and MonoidReduce, wherein; the PairMaker categorical function transforms a free monoid over strings into a free monoid over tuples including at least one key-value pair in each tuple; the FreeMonoidReduce categorical function merges a nested free monoid over tuples into one free monoid over tuples, thereby reducing a nesting depth of the nested free monoid over tuples; the PairABtoAFMB categorical function transforms one element in each tuple of the one free monoid over tuples into list of one element in a free monoid over tuples with an embedded list element; the ReducePairs categorical function merges consecutive tuples of the free monoid over tuples with the embedded list element; and the MonoidReduce categorical function transforms a plurality of list values into a single value based on a parameterized operation. - View Dependent Claims (19, 20)
Specification