System and method for providing scatter/gather data processing in a middleware environment

• US 9,864,759 B2
• Filed: 06/28/2011
• Issued: 01/09/2018
• Est. Priority Date: 09/15/2010
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Bihani, Ballav, Torstensson, Patrik, Messinger, Adam, Revanuru, Naresh
• Primary Examiner(s): Mejia, Anthony
• Assistant Examiner(s): GOODWIN, SCHQUITA D
• Application Number:
  US13/170,490
• Publication Number:
  US 20120066460A1
• Time in Patent Office: 2,387 Days
• Field of Search: 709223, 709250, 709232, 709216, 709206, 709227, 710 22, 326 39, 7071041, 714 50, 370338, 370466, 370335, 385 24, 719328
• CPC Class Codes: G06F 15/17318 Parallel communications techniques, e.g. gather, scatter, reduce, roadcast, multicast, all to all
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning
• CPC Class Codes: G06F 9/45504 Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
• CPC Class Codes: H04L 41/042 comprising distributed management centres cooperatively managing the network
• CPC Class Codes: H04L 41/0686 Additional information in the notification, e.g. enhancement of specific meta-data
• CPC Class Codes: H04L 49/358 Infiniband Switches
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1004 Server selection for load balancing
• CPC Class Codes: H04L 67/1027 Persistence of sessions during load balancing
• CPC Class Codes: H04L 67/1034 Reaction to server failures by a load balancer
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/14 Session management for real-time applications in data packet communications networks H04L65/1066
• CPC Class Codes: H04L 67/142 Managing session states for stateless protocols; Signalling session states; State transitions; Keeping-state mechanisms
• CPC Class Codes: H04L 67/146 Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
• CPC Class Codes: H04L 69/14 Multichannel or multilink protocols
• CPC Class Codes: Y10T 409/308904 : Multiple row dividing head
• CPC Class Codes: Y10T 409/309072 : including cutter limited to rotary motion
• CPC Class Codes: Y10T 409/309128 : with means to adjust work support vertically
• CPC Class Codes: Y10T 409/309352 : Cutter spindle or spindle support
• CPC Class Codes: Y10T 409/309408 : with cutter holder

Abstract

Systems and methods are provided for providing scatter/gather data processing. In accordance with an embodiment, a such a system can include a cluster of one or more high performance computing systems, each including one or more processors and a high performance memory. The cluster communicates over an InfiniBand network. The system can also include a middleware environment, executing on the cluster, that includes one or more application server instances. The system can further include a plurality of muxers. Each application server instance includes at least one muxer, and each muxer is operable to collect data from a plurality of locations in the high performance memory, and transfer the data in bulk.

Claims

1. A system for providing scatter/gather data processing, comprising:

a plurality of microprocessors including a first microprocessor and a second microprocessor;

a cluster of a plurality of application server instances executing on the plurality of microprocessors,

wherein a first application server instance of the plurality of application server instances executes on the first microprocessor, and a second application server instance of the plurality of application server instances executes on the second microprocessor;

wherein the first application server instance and the second application server instance communicate through a switch fabric network;

wherein each application server instance includes a software muxer;

a plurality of parallel logical channels between the first application server instance and the second application server instance,

wherein each parallel logical channel is created based on a replication channel template configured on a cluster management bean, and

wherein the number of the plurality of parallel logical channels is specified by an attribute in a server management bean;

wherein the first microprocessor executes the software muxer on the first application server instance to

obtain a plurality of pointers, wherein each pointer points to one or more of a plurality of pieces of data, wherein the plurality of pieces of data are stored in different locations in a memory allocated to the first application server instance,

configure a plurality of threads to execute the plurality of parallel logical channels,

collect the plurality of pieces of data from the different locations in the memory using the plurality of pointers, and

transfer, in a single bulk write and using a single context switch of the first microprocessor, the plurality of pieces of data collected using the plurality of pointers concurrently over the plurality of parallel logical channels from the first application server instance to the second application server instance.

2. The system of claim 1 wherein each muxer is a New I/O (NIO) muxer.

3. The system of claim 1, further comprising a utility function configured to determine whether the plurality of parallel logical channels have been configured, and cache a result of the determination for subsequent use.

4. The system of claim 1, wherein the first application server instance, at a kernel level, operates to recognize an increased bandwidth and transfers the data without imposing a flow control.

5. The system of claim 1, wherein the first muxer is a parallel muxer.

6. The system of claim 1, wherein the plurality of pieces of data are related to one another.

7. The system of claim 1, wherein the memory allocated to the first application server instance is a high performance memory.

8. A method for providing scatter/gather data processing, comprising:

configuring a cluster of a plurality of application server instances executing on a plurality of microprocessors including a first microprocessor and a second microprocessor, wherein each application server instance includes a software muxer, wherein a first application server instance of the plurality of application server instances executes on the first microprocessor, and a second application server instance of the plurality of application server instances executes on the second microprocessor;

wherein the first application server instance and the second application server instance communicate through a switch fabric network;

configuring a plurality of parallel logical channels between the first application server instance and the second application server instance,

wherein each parallel logical channel is created based on a replication channel template configured on a cluster management bean, and

wherein the number of the plurality of parallel logical channels is specified by an attribute in a server management bean;

executing, by the first microprocessor the software muxer on the first application server instance to perform the steps comprising

obtaining a plurality of pointers, wherein each pointer points to one or more of a plurality of pieces of data, wherein the plurality of pieces of data are stored in different locations in a memory allocated to the first application server instance,

configuring a plurality of threads to execute the plurality of parallel logical channels,

collecting the plurality of pieces of data from the different locations in the memory using the plurality of pointers,

transferring, in a single bulk write and using a single context switch of the first microprocessor, the plurality of pieces of data collected using the plurality of pointers concurrently over the plurality of parallel logical channels from the first application server instance to the second application server instance.

9. The system of claim 8 wherein each muxer is a New I/O (NIO) muxer.

10. The method of claim 8, further comprising determining, via a utility function, whether the plurality of parallel logical channels have been configured, wherein a result of the determination is cached for subsequent use.

11. The method of claim 8, wherein the first application server instance, at a kernel level, operates to recognize an increased bandwidth and transfers the data without imposing a flow control.

12. The method of claim 8, wherein the first muxer is a parallel muxer.

13. The method of claim 8, wherein the plurality of pieces of data are related to one another.

14. The method of claim 8, wherein the memory allocated to the first application server instance is a high performance memory.

15. A non-transitory computer readable storage medium including instructions stored thereon which, when executed by a computer, cause the computer to perform the steps comprising:

configuring a cluster of a plurality of application server instances executing on a plurality of microprocessors including a first microprocessor and a second microprocessor, wherein each application server instance includes a software muxer, wherein a first application server instance of the plurality of application server instances executes on the first microprocessor, and a second application server instance of the plurality of application server instances executes on the second microprocessor;

wherein the first application server instance and the second application server instance communicate through a switch fabric network;

configuring a plurality of parallel logical channels between the first application server instance and the second application server instance,

wherein each parallel logical channel is created based on a replication channel template configured on a cluster management bean, and

wherein the number of the plurality of parallel logical channels is specified by an attribute in a server management bean;

executing, by the first microprocessor the software muxer on the first application server instance to perform the steps comprising

obtaining a plurality of pointers, wherein each pointer points to one or more of a plurality of pieces of data, wherein the plurality of pieces of data are stored in different locations in a memory allocated to the first application server instance,

configuring a plurality of threads to execute the plurality of parallel logical channels,

collecting the plurality of pieces of data from the different locations in the memory using the plurality of pointers,

transferring, in a single bulk write and using a single context switch of the first microprocessor, the plurality of pieces of data collected using the plurality of pointers concurrently over the plurality of parallel logical channels from the first application server instance to the second application server instance.

16. The non-transitory computer readable storage medium of claim 15 wherein each muxer is a New I/O (NIO) muxer.

17. The non-transitory computer readable storage medium of claim 15, further comprising determining, via a utility function, whether the plurality of parallel logical channels have been configured, wherein a result of the determination is cached for subsequent use.

18. The non-transitory computer readable storage medium of claim 15, wherein the first application server instance, at a kernel level, operates to recognize an increased bandwidth and transfers the data without imposing a flow control.

19. The non-transitory computer readable storage medium of claim 15, wherein the first muxer is a parallel muxer.

20. The non-transitory computer readable storage medium of claim 15, wherein the plurality of pieces of data are related to one another.

21. The non-transitory computer readable storage medium of claim 15, wherein the memory allocated to the first application server instance is a high performance memory.