High-performance data streaming

• US 10,721,323 B2
• Filed: 12/19/2018
• Issued: 07/21/2020
• Est. Priority Date: 06/19/2018
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: White, Seth John, Le Stum, Guillaume, Shaw, Benjamin
• Primary Examiner(s): Miah, Razu A
• Application Number:
  US16/225,897
• Publication Number:
  US 20190387068A1
• Time in Patent Office: 580 Days
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1017 based on a round robin mechanism
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/288 Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level

Abstract

High performance data streaming is provided. An intermediate iterator receives a request to process streaming data in a parent request buffer from a parent iterator. The intermediate iterator sends a request to provide a batch of the streaming data in a request child buffer to a child iterator. The intermediate iterator receives the batch of the streaming data in a child response buffer from the child iterator. The intermediate iterator processes the batch of the streaming data. The intermediate iterator sends the processed batch of the streaming data in a response parent buffer to the parent iterator.

Claims

1. A system comprising:

one or more processors; and

a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to;

receive, from a parent iterator, a request to process streaming data in a parent request buffer;

cause a request to provide a batch of the streaming data in a request child buffer to be sent to a child iterator;

cause a request to provide another batch of the streaming data in another request child buffer to be sent to another child iterator;

receive, from the child iterator, the batch of the streaming data in a child response buffer;

receive, from the other child iterator, the other batch of the streaming data in another child response buffer;

process a combination of the batch of the streaming data and the other batch of the streaming data; and

cause a combination of the processed batch of the streaming data and the other processed batch of the streaming data stored in a response parent buffer to be sent to the parent iterator.

2. The system of claim 1, wherein the parent request buffer comprises the response parent buffer, and the parent request buffer is one of different from both the request child buffer and the child response buffer, and same as both the request child buffer and the child response buffer.

3. The system of claim 2, wherein the request child buffer is one of different from the child response buffer and same as the child response buffer.

4. The system of claim 1, comprising further instructions, which when executed, cause the one or more processors to:

convey, from the parent iterator to the child iterator, a request to allocate the parent request buffer; and

convey, from the child iterator to the parent iterator, a reference to the parent request buffer.

5. The system of claim 1, comprising further instructions, which when executed, cause the one or more processors to:

allocate the parent request buffer;

convey, to the parent iterator, a reference to the parent request buffer;

cause a request to allocate the request child buffer to be sent to the child iterator; and

receive, from the child iterator, a reference to the request child buffer.

6. The system of claim 1, comprising further instructions, which when executed, cause the one or more processors to:

cause a request to allocate the other request child buffer to be sent to the other child iterator; and

receive, from the other child iterator, a reference to the other request child buffer.

7. A computer program product comprising a non-transitory computer-readable medium having computer-readable program code embodied therein to be executed by one or more processors, the program code including instructions to:

receive, from a parent iterator, a request to process streaming data in a parent request buffer;

cause a request to provide a batch of the streaming data in a request child buffer to be sent to a child iterator;

cause a request to provide another batch of the streaming data in another request child buffer to be sent to another child iterator;

receive, from the child iterator, the batch of the streaming data in a child response buffer;

receive, from the other child iterator, the other batch of the streaming data in another child response buffer;

process a combination of the batch of the streaming data and the other batch of the streaming data; and

cause a combination of the processed batch of the streaming data and the other processed batch of the streaming data stored in a response parent buffer to be sent to the parent iterator.

8. The computer program product of claim 7, wherein the parent request buffer comprises the response parent buffer, and the parent request buffer is one of different from both the request child buffer and the child response buffer, and same as both the request child buffer and the child response buffer, and the request child buffer is one of different from the child response buffer and same as the child response buffer.

9. The computer program product of claim 7, wherein the program code comprises further instructions to:

convey, from the parent iterator to the child iterator, a request to allocate the parent request buffer; and

convey, from the child iterator to the parent iterator, a reference to the parent request buffer.

10. The computer program product of claim 7, wherein the program code comprises further instructions to:

allocate the parent request buffer;

convey, to the parent iterator, a reference to the parent request buffer;

cause a request to allocate the request child buffer to be sent to the child iterator; and

receive, from the child iterator, a reference to the request child buffer.

11. The computer program product of claim 7, wherein the program code comprises further instructions to:

cause a request to allocate the other request child buffer to be sent to the other child iterator; and

receive, from the other child iterator, a reference to the other request child buffer.

12. A method comprising:

receiving, from a parent iterator, a request to process streaming data in a parent request buffer;

causing a request to provide a batch of the streaming data in a request child buffer to be sent to a child iterator;

causing a request to provide another batch of the streaming data in another request child buffer to be sent to another child iterator;

receiving, from the child iterator, the batch of the streaming data in a child response buffer;

receiving, from the other child iterator, the other batch of the streaming data in another child response buffer;

processing a combination of the batch of the streaming data and the other batch of the streaming data; and

causing a combination of the processed batch of the streaming data and the other processed batch of the streaming data stored in a response parent buffer to be sent to the parent iterator.

13. The method of claim 12, wherein the parent request buffer comprises the response parent buffer, and the parent request buffer is one of different from both the request child buffer and the child response buffer, and same as both the request child buffer and the child response buffer.

14. The method of claim 13, wherein the request child buffer is one of different from the child response buffer and same as the child response buffer.

15. The method of claim 12, the method further comprising:

conveying, from the parent iterator to the child iterator, a request to allocate the parent request buffer; and

conveying, from the child iterator to the parent iterator, a reference to the parent request buffer.

16. The method of claim 12, the method further comprising:

allocating the parent request buffer;

conveying, to the parent iterator, a reference to the parent request buffer;

causing a request to allocate the request child buffer to be sent to the child iterator; and

receiving, from the child iterator, a reference to the request child buffer.

17. The method of claim 12, the method further comprising:

causing a request to allocate the other request child buffer to the other child iterator; and

receiving, from the other child iterator, a reference to the other request child buffer.