Inter-layer communication of events between TCP layer and higher layers when implemented in different concurrent execution entities

• US 9,929,968 B2
• Filed: 03/19/2015
• Issued: 03/27/2018
• Est. Priority Date: 03/19/2015
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Narasimhamurthy, Giridhar
• Primary Examiner(s): Coby, Frantz
• Application Number:
  US14/662,270
• Publication Number:
  US 20160277478A1
• Time in Patent Office: 1,104 Days
• Field of Search: 709207, 709213, 709202, 709227-229, 709217, 370328, 370392, 370231, 370235
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
• CPC Class Codes: H04L 69/162 involving adaptations of sockets based mechanisms secure socket layer H04L63/168

Abstract

Inter-layer communication of events between different concurrent execution entities, where a Transmission Control Protocol (TCP) layer instance executes in a first execution entity and a higher layer instance executes in a second execution entity. In an example transaction, the first execution entity receives a TCP packet, and the TCP layer instance in the first execution entity generates an event message by processing the received TCP packet. The TCP layer instance thereafter stores the event message in an event queue at the second execution entity. The second execution entity identifies the presence of the event message in the event queue (e.g., by polling) and updates a first data structure, subsequent to the processing of the event message by the higher layer instance in the second execution entity. In an embodiment, the updated first data structure corresponds to application control block (ACB) maintained for each TCP connection.

Claims

1. A method of communicating events between different concurrent execution entities in a server system addressable by an Internet Protocol (IP) address on a network, said method comprising:

executing a first Transmission Control Protocol (TCP) layer instance in a first execution entity and a higher layer instance in a second execution entity;

wherein a second TCP layer instance is executed in said second execution entity, and wherein each of said first TCP layer instance and said second TCP layer instance is designed to receive packets on said network with a destination IP field set to said IP address,

wherein both of said first execution entity and second execution entity are executed on said server system addressable by said IP address on said network;

receiving, by said first execution entity, a first TCP packet on said network with said IP address in said destination IP field;

generating, by said first TCP layer instance, a first event message by processing of said TCP packet;

storing said first event message in an event queue provided in said server system;

identifying, by said second execution entity, the presence of said event message in said event queue; and

updating, by said second execution entity, a first data structure corresponding to processing of said event message by said higher layer instance in said second execution entity.

2. The method of claim 1, wherein said server system is a multi-processor system containing a plurality of processors, wherein said first execution entity is provided in a first processor, said second execution entity is provided in a second processor and a third execution entity is provided in a third processor, said third execution entity implementing a third TCP layer instance, wherein said first processor, said second processor and said third processor are contained in said plurality of processors,

wherein said first TCP packet is received on a first connection of a first plurality of TCP connections, wherein a third TCP packet of a third connection of a third plurality of TCP connections is received and processed by said third TCP layer instance, wherein said first plurality of TCP connections and said third plurality of TCP connections terminate at applications supported by said higher layer instance and executing in said second processor,

said method further comprises:

queuing in said event queue, a third event message generated by said third TCP layer instance by processing of said third packet,

wherein said identifying further comprises polling said event queue for incoming event messages, wherein said polling identifies presence of said first event message and said third event message,

wherein said updating updates said first data structure in response to said polling identifying said first event message, said first data structure corresponding to said first connection,

wherein said updating updates a third data structure corresponding to said third connection.

3. The method of claim 2, wherein said higher layer instance corresponds to a socket layer supporting said applications executing in said second processor, said method further comprising:

maintaining, by said socket layer, an application control block (ACB) corresponding to each TCP connection of said plurality of TCP connections, wherein said first data structure and said third data structure are a first ACB and a third ACB corresponding to said first TCP connection and said third TCP connection respectively,

wherein said updating updates the first ACB in processing said first event message in said event queue without having to implement a lock, and updates the third ACB in processing said third event message in said event queue without having to implement a lock.

4. The method of claim 3, said method further comprising:

maintaining a transmission control block (TCB) corresponding to each TCP connection of said first plurality of TCP connections and said third plurality of TCP connection, wherein the TCBs corresponding to said first plurality of TCP connections are maintained by said first TCP layer instance in said first execution entity, and the TCBs corresponding to said third plurality of TCP connections are maintained by said third TCP layer instance in said third execution entity,

wherein each TCB contains an ACB identifier of a corresponding ACB of the same TCP connection, an application identifier of a corresponding application instance at which the TCP connection terminates,

wherein each of said first event message and said third event message contain the ACB identifier and the application identifier retrieved from the corresponding TCB,

wherein the application identifier is used to deliver the data contained in said first TCP packet and said third TCP packet to corresponding destination application instances.

5. The method of claim 4, wherein each execution entity is assigned a corresponding Receive Side Scaling (RSS) queue number in said multi-processor system, said method further comprising:

examining each TCP packet to determine an associated TCB corresponding to the TCP connection on which the TCP packet is received;

retrieving a first Receive Side Scaling (RSS) queue number from the determined TCB;

comparing, in said first execution entity, said first RSS queue number with a RSS queue number assigned to said first execution entity;

if said first RSS queue number equals the RSS queue number of said first execution entity, invoking an event function call to pass the TCP packet to the socket layer in said first execution entity,

if said first RSS queue number does not equal the RSS queue number of said first execution entity, passing the TCP packet as a corresponding event message to the execution entity having a RSS queue number matching said first RSS queue number, wherein said first TCP packet is passed as said first event message for storing in said event queue corresponding to said second execution entity.

6. The method of claim 5, wherein each of said execution entities is implemented in a run-to-completion model in which each TCP packet is processed by said first TCP layer and the socket layer in a single execution run such that the data is delivered to the destination application in said single execution run,

wherein each execution entity is also designed to poll for event messages in the corresponding event queue and process each event message also to completion such that the execution entity is implemented without requiring interrupts or process blocking within the processor for processing TCP packets.

7. The method of claim 4, wherein a second application instance in said second execution entity requests sending of a data sequence to an external system via a second socket call, said method further comprising:

identifying a second ACB corresponding to the socket of said second socket call, wherein said second ACB contains a second RSS queue number;

comparing, in said second execution entity, said second RSS queue number with a RSS queue number assigned to said second execution entity;

if said second RSS queue number equals the RSS queue number of said second execution entity, invoking an event function call to pass said data sequence to the TCP layer instance in said second execution entity,

if said second RSS queue number does not equal the RSS queue number of said second execution entity, passing said data sequence as a corresponding event message to the execution entity having a RSS queue number matching said second RSS queue number.

8. A non-transitory machine readable medium storing one or more sequences of instructions for enabling a server system to support communication of events between different execution entities in a server system addressable by an Internet Protocol (IP) address on a network, wherein execution of said one or more instructions by one or more processors contained in said server system enables said server system to perform the actions of:

executing a first Transmission Control Protocol (TCP) layer instance in a first execution entity and a higher layer instance in a second execution entity;

wherein a second TCP layer instance is executed in said second execution entity, and wherein each of said first TCP layer instance and said second TCP layer instance is designed to receive packets on said network with a destination IP field set to said IP address,

wherein both of said first execution entity and second execution entity are executed on said server system addressable by said IP address on said network;

receiving, by said first execution entity, a first TCP packet on said network with said IP address in said destination IP field;

generating, by said first TCP layer instance, a first event message by processing of said TCP packet;

storing said first event message in an event queue provided in said server system;

identifying, by said second execution entity, the presence of said event message in said event queue; and

updating, by said second execution entity, a first data structure corresponding to processing of said event message by said higher layer instance in said second execution entity.

9. The non-transitory machine readable medium of claim 8, wherein said server system is a multi-processor system containing a plurality of processors, wherein said first execution entity is provided in a first processor, said second execution entity is provided in a second processor and a third execution entity is provided in a third processor, said third execution entity implementing a third TCP layer instance, wherein said first processor, said second processor and said third processor are contained in said plurality of processors,

wherein said first TCP packet is received on a first connection of a first plurality of TCP connections, wherein a third TCP packet of a third connection of a third plurality of TCP connections is received and processed by said third TCP layer instance, wherein said first plurality of TCP connections and said third plurality of TCP connections terminate at applications supported by said higher layer instance and executing in said second processor,

further comprising:

queuing in said event queue, a third event message generated by said third TCP layer instance by processing of said third packet,

wherein said identifying further comprises polling said event queue for incoming event messages, wherein said polling identifies presence of said first event message and said third event message,

wherein said updating updates said first data structure in response to said polling identifying said first event message, said first data structure corresponding to said first connection,

wherein said updating updates a third data structure corresponding to said third connection.

10. The non-transitory machine readable medium of claim 9, wherein said higher layer instance corresponds to a socket layer supporting said applications executing in said second processor, further comprising:

maintaining, by said socket layer, an application control block (ACB) corresponding to each TCP connection of said plurality of TCP connections, wherein said first data structure and said third data structure are a first ACB and a third ACB corresponding to said first TCP connection and said third TCP connection respectively,

wherein said updating updates the first ACB in processing said first event message in said event queue without having to implement a lock, and updates the third ACB in processing said third event message in said event queue without having to implement a lock.

11. The non-transitory machine readable medium of claim 10, further comprising:

maintaining a transmission control block (TCB) corresponding to each TCP connection of said first plurality of TCP connections and said third plurality of TCP connection, wherein the TCBs corresponding to said first plurality of TCP connections are maintained by said first TCP layer instance in said first execution entity, and the TCBs corresponding to said third plurality of TCP connections are maintained by said third TCP layer instance in said third execution entity,

wherein each TCB contains an ACB identifier of a corresponding ACB of the same TCP connection, an application identifier of a corresponding application instance at which the TCP connection terminates,

wherein each of said first event message and said third event message contain the ACB identifier and the application identifier retrieved from the corresponding TCB,

wherein the application identifier is used to deliver the data contained in said first TCP packet and said third TCP packet to corresponding destination application instances.

12. The non-transitory machine readable medium of claim 11, wherein each execution entity is assigned a corresponding Receive Side Scaling (RSS) queue number in said multi-processor system, further comprising:

examining each TCP packet to determine an associated TCB corresponding to the TCP connection on which the TCP packet is received;

retrieving a first Receive Side Scaling (RSS) queue number from the determined TCB;

comparing, in said first execution entity, said first RSS queue number with a RSS queue number assigned to said first execution entity;

if said first RSS queue number equals the RSS queue number of said first execution entity, invoking an event function call to pass the TCP packet to the socket layer in said first execution entity,

if said first RSS queue number does not equal the RSS queue number of said first execution entity, passing the TCP packet as a corresponding event message to the execution entity having a RSS queue number matching said first RSS queue number, wherein said first TCP packet is passed as said first event message for storing in said event queue corresponding to said second execution entity.

13. The non-transitory machine readable medium of claim 11, wherein a second application instance in said second execution entity requests sending of a data sequence to an external system via a second socket call, further comprising:

identifying a second ACB corresponding to the socket of said second socket call, wherein said second ACB contains a second RSS queue number;

comparing, in said second execution entity, said second RSS queue number with a RSS queue number assigned to said second execution entity;

if said second RSS queue number equals the RSS queue number of said second execution entity, invoking an event function call to pass said data sequence to the TCP layer instance in said second execution entity,

if said second RSS queue number does not equal the RSS queue number of said second execution entity, passing said data sequence as a corresponding event message to the execution entity having a RSS queue number matching said second RSS queue number.

14. A server system addressable by an Internet Protocol (IP) address on a network, said server system comprising:

one or more processing units; and

a random access memory (RAM) to store instructions,

wherein said one or more processing units retrieve said instructions and execute said instructions, wherein execution of said instructions causes said server system to perform the actions of;

executing a first Transmission Control Protocol (TCP) layer instance in a first execution entity and a higher layer instance in a second execution entity;

wherein a second TCP layer instance is executed in said second execution entity, and wherein each of said first TCP layer instance and said second TCP layer instance is designed to receive packets on said network with a destination IP field set to said IP address,

wherein both of said first execution entity and second execution entity are executed on said server system addressable by said IP address on said network;

receiving, by said first execution entity, a first TCP packet on said network with said IP address in said destination IP field;

generating, by said first TCP layer instance, a first event message by processing of said TCP packet;

storing said first event message in an event queue provided in said server system;

identifying, by said second execution entity, the presence of said event message in said event queue; and

updating, by said second execution entity, a first data structure corresponding to processing of said event message by said higher layer instance in said second execution entity.

15. The server system of claim 14, wherein said one or more processing units correspond to a plurality of processors, wherein said first execution entity is provided in a first processor, said second execution entity is provided in a second processor and a third execution entity is provided in a third processor, said third execution entity implementing a third TCP layer instance, wherein said first processor, said second processor and said third processor are contained in said plurality of processors,

wherein said first TCP packet is received on a first connection of a first plurality of TCP connections, wherein a third TCP packet of a third connection of a third plurality of TCP connections is received and processed by said third TCP layer instance, wherein said first plurality of TCP connections and said third plurality of TCP connections terminate at applications supported by said higher layer instance and executing in said second processor,

wherein said actions further comprise:

queuing in said event queue, a third event message generated by said third TCP layer instance by processing of said third packet,

wherein said identifying further comprises polling said event queue for incoming event messages, wherein said polling identifies presence of said first event message and said third event message,

wherein said updating updates said first data structure in response to said polling identifying said first event message, said first data structure corresponding to said first connection,

wherein said updating updates a third data structure corresponding to said third connection.

16. The server system of claim 15, wherein said higher layer instance corresponds to a socket layer supporting said applications executing in said second processor, wherein said actions further comprise:

maintaining, by said socket layer, an application control block (ACB) corresponding to each TCP connection of said plurality of TCP connections, wherein said first data structure and said third data structure are a first ACB and a third ACB corresponding to said first TCP connection and said third TCP connection respectively,

wherein said updating updates the first ACB in processing said first event message in said event queue without having to implement a lock, and updates the third ACB in processing said third event message in said event queue without having to implement a lock.

17. The server system of claim 16, wherein said actions further comprise:

maintaining a transmission control block (TCB) corresponding to each TCP connection of said first plurality of TCP connections and said third plurality of TCP connection, wherein the TCBs corresponding to said first plurality of TCP connections are maintained by said first TCP layer instance in said first execution entity, and the TCBs corresponding to said third plurality of TCP connections are maintained by said third TCP layer instance in said third execution entity,

wherein each TCB contains an ACB identifier of a corresponding ACB of the same TCP connection, an application identifier of a corresponding application instance at which the TCP connection terminates,

wherein each of said first event message and said third event message contain the ACB identifier and the application identifier retrieved from the corresponding TCB,

wherein the application identifier is used to deliver the data contained in said first TCP packet and said third TCP packet to corresponding destination application instances.

18. The server system of claim 17, wherein each execution entity is assigned a corresponding Receive Side Scaling (RSS) queue number, wherein said actions further comprise:

examining each TCP packet to determine an associated TCB corresponding to the TCP connection on which the TCP packet is received;

retrieving a first Receive Side Scaling (RSS) queue number from the determined TCB;

comparing, in said first execution entity, said first RSS queue number with a RSS queue number assigned to said first execution entity;

if said first RSS queue number equals the RSS queue number of said first execution entity, invoking an event function call to pass the TCP packet to the socket layer in said first execution entity,

if said first RSS queue number does not equal the RSS queue number of said first execution entity, passing the TCP packet as a corresponding event message to the execution entity having a RSS queue number matching said first RSS queue number, wherein said first TCP packet is passed as said first event message for storing in said event queue corresponding to said second execution entity.

19. The server system of claim 18, wherein each of said execution entities is implemented in a run-to-completion model in which each TCP packet is processed by said first TCP layer and the socket layer in a single execution run such that the data is delivered to the destination application in said single execution run,

wherein each execution entity is also designed to poll for event messages in the corresponding event queue and process each event message also to completion such that the execution entity is implemented without requiring interrupts or process blocking within the processor for processing TCP packets.

20. The server system of claim 17, wherein a second application instance in said second execution entity requests sending of a data sequence to an external system via a second socket call, wherein said actions further comprise:

identifying a second ACB corresponding to the socket of said second socket call, wherein said second ACB contains a second RSS queue number;

comparing, in said second execution entity, said second RSS queue number with a RSS queue number assigned to said second execution entity;

if said second RSS queue number equals the RSS queue number of said second execution entity, invoking an event function call to pass said data sequence to the TCP layer instance in said second execution entity,

if said second RSS queue number does not equal the RSS queue number of said second execution entity, passing said data sequence as a corresponding event message to the execution entity having a RSS queue number matching said second RSS queue number.