Custom partitioning of a data stream
First Claim
Patent Images
1. A system, comprising:
- a memory storing computer-executable instructions; and
a processor configured to access the memory and execute the computer-executable instructions to at least;
identify a data stream associated with a user;
receive a query for processing at least a portion of the data stream;
identify a rate of events of the data stream, the rate of events comprising a number of events received by the processor per time period;
determine that the rate of events is above a threshold; and
when the rate of events is above the threshold;
provide, to the user, a software development kit for generating a java class;
enable the user to create a partition class in java using the software development kit, the partition class configured to;
receive each event of the data stream at runtime; and
partition the data stream into a plurality of sub-stream portions;
receive, from the user, an attribute of the data stream;
receive, from the user, the partition class generated by the user for partitioning the data stream, the partition class is based at least in part on the software development kit and the attribute of the data stream;
associate, by the computer system, the partition class with the query;
process each event of the data stream with the partition class at runtime to partition the data stream based at least in part on the association of the partition class with the query;
the process comprising;
partitioning the data stream into the plurality of sub-stream portions based at least in part on the attribute;
provide at least one sub-stream portion of the plurality of sub-stream portions to the query;
process the at least one sub-stream portion, using the query, in parallel with at least a second sub-stream portion of the plurality of sub-stream portions that is received as an output from the partition class; and
provide results of the query based at least in part on the at least one sub-stream portion and the second sub-stream portion.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques for partitioning data streams are provided. In some examples, a query for processing at least a portion of a data stream may be identified. The data stream may be associated with a user. Additionally, in some examples, code identifying an attribute of the identified stream may be received. The code may be capable of configuring the query based at least in part on the attribute. Further, in some aspects, the code may be configured to partition the data stream into at least a sub-stream based at least in part on the attribute.
-
Citations
14 Claims
-
1. A system, comprising:
-
a memory storing computer-executable instructions; and a processor configured to access the memory and execute the computer-executable instructions to at least; identify a data stream associated with a user; receive a query for processing at least a portion of the data stream; identify a rate of events of the data stream, the rate of events comprising a number of events received by the processor per time period; determine that the rate of events is above a threshold; and when the rate of events is above the threshold; provide, to the user, a software development kit for generating a java class; enable the user to create a partition class in java using the software development kit, the partition class configured to; receive each event of the data stream at runtime; and partition the data stream into a plurality of sub-stream portions; receive, from the user, an attribute of the data stream; receive, from the user, the partition class generated by the user for partitioning the data stream, the partition class is based at least in part on the software development kit and the attribute of the data stream; associate, by the computer system, the partition class with the query; process each event of the data stream with the partition class at runtime to partition the data stream based at least in part on the association of the partition class with the query;
the process comprising;partitioning the data stream into the plurality of sub-stream portions based at least in part on the attribute; provide at least one sub-stream portion of the plurality of sub-stream portions to the query; process the at least one sub-stream portion, using the query, in parallel with at least a second sub-stream portion of the plurality of sub-stream portions that is received as an output from the partition class; and provide results of the query based at least in part on the at least one sub-stream portion and the second sub-stream portion. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A non-transitory computer-readable storage memory storing a plurality of instructions executable by one or more processors, the plurality of instructions comprising:
-
instructions to identify a query for processing at least a portion of a data stream associated with a user; instructions to identify the data stream associated with the user; instructions to identify a rate of events of the data stream, the rate of events comprising a number of events received per time period; instructions to determine that the rate of events is above a threshold; and when the rate of events is above the threshold, the plurality of instructions further comprise; instructions to provide a software development kit to the user; instructions to enable the user to create a partition class in java using the software development kit, the partition class is configured to; receive each event of the data stream at runtime;
anpartition the data stream into a plurality of sub-stream portions; instructions to receive, from the user, an attribute of the data stream; instructions to receive, from the user, the partition class generated by the user for partitioning the data stream, the partition class is identifying the attribute of the data stream and is generated based at least in part on the software development kit; instructions to process each event of the data stream with the partition class at runtime to partition the data stream based at least in part on an association of the partition class with the query, the processing of each event comprising; partitioning the data stream into the plurality of sub-stream portions based at least in part on the attribute;
aninstructions to configure the query to process at least one sub stream portion of the plurality of sub-stream portions in parallel with at least a second sub-stream portion of the plurality of sub-stream portions that is received as an output from the partition class; and instructions to provide results of the query based at least in part on the at least one sub-stream portion and the at least one second sub-stream portion. - View Dependent Claims (8, 9, 10, 11)
-
-
12. A computer-implemented method, comprising:
-
identifying, by a computing system, a data stream associated with a user, receiving a query for processing at least a portion of the data stream; identifying a rate of events of the data stream, the rate of events comprising a number of events received by the computing system per time period; determining that the rate of events is above a threshold; and when the rate of events is above the threshold; receiving, from the user, a partition class in java identifying an attribute of the data stream, the partition class is based at least in part on a software development kit received by the user, and the partition class is configured to; receive each event of the data stream; and partition the data stream into a plurality of sub-stream portions based at least in part on the attribute; executing the partition class at runtime to partition the data stream into at least one sub-stream portion of the plurality of sub-stream portions based at least in part on the attribute and an association of the partition class with the query; processing the at least one sub-stream using the query in parallel with at least a second sub-stream portion of the plurality of sub-stream portions that is received as an output from the partition class; and providing results of the query based at least in part on the at least one sub-stream portion and the second sub-stream portion. - View Dependent Claims (13, 14)
-
Specification