Method and system for dynamic repurposing of payload storage as a trace buffer

• US 9,148,352 B2
• Filed: 12/20/2012
• Issued: 09/29/2015
• Est. Priority Date: 12/20/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Manula, Brian Edward, Schanke, Morten, Tvete, Knut Hallvard
• Primary Examiner(s): KHIRODHAR, MAHARISHI V
• Application Number:
  US13/722,083
• Publication Number:
  US 20140177633A1
• Time in Patent Office: 1,013 Days
• US Class Current: 1/1
• CPC Class Codes: H04L 43/028 by filtering
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation

Abstract

A method for debugging network activity involving receiving, by HCA, a packet stream comprising multiple packets, comparing a packet header of each of the packets to a trigger condition to determine whether the trigger condition has been met, after the trigger condition has been met, comparing each packet header of the packets to one or more trace filters stored in the HCA to identify matching packets, duplicating one or more portions of the matching packets and storing the duplicated portions of the matching packets in a trace buffer, where the trace buffer is located in the HCA and is dynamically repurposed from a payload RAM to the trace buffer when a corresponding port of the HCA for transmitting or receiving the packet stream is set to trace mode, and stopping the trace and copying the one or more portions of packets from the trace buffer to host memory.

Claims

1. A method for debugging network activity, comprising:

receiving, by a host channel adapter (HCA), a packet stream comprising a plurality of packets;

comparing a packet header of each of the plurality of packets to a trigger condition to determine whether the trigger condition has been met;

after the trigger condition has been met, comparing each packet header of the plurality of packets to one or more trace filters stored in the HCA to identify matching packets;

duplicating one or more portions of the matching packets and storing the duplicated one or more portions of the matching packets in a trace buffer, wherein the trace buffer is located in the HCA and is dynamically repurposed from a payload random access memory (RAM) to the trace buffer when a corresponding port of the HCA for transmitting or receiving the packet stream is set to trace mode; and

stopping a trace of the one or more portions of the packets and copying the one or more portions of packets from the trace buffer to host memory.

2. The method of claim 1, further comprising:

setting the corresponding port to trace mode when the corresponding port is idle; and

initiating the trace by writing to a designated control and status register (CSR).

3. The method of claim 1, further comprising:

collecting prestore data in the trace buffer when a prestore percentage is written to a CSR,

wherein a first portion of the trace buffer comprises the prestore data and a second portion of the trace buffer comprises poststore data comprising the one or more portions of packets stored after the trigger condition is met.

4. The method of claim 3, further comprising:

updating a state of the trace buffer from idle to waiting when the prestore data is being collected in the trace buffer;

updating the state of the trace buffer from waiting to sampling when poststore data is being collected in the trace buffer; and

updating the state of the trace buffer from sampling to done when the trace is stopped.

5. The method of claim 1, wherein the one or more trace filters comprises at least a store qualifier.

6. The method of claim 1, wherein the trigger condition and a number of portions of packets to store in the trace buffer after the trigger condition is met are stored in CSRs in the HCA.

7. The method of claim 1, further comprising:

analyzing the one or more portions of packets in the host memory to debug errors observed in network activity.

8. The method of claim 1, further comprising:

receiving a request to start a trace from an embedded processor system (EPS) operatively connected to the HCA.

9. The method of claim 8, wherein the EPS comprises access to the trace buffer and is configured to:

read the one or more portions of packets out from the trace buffer, wherein the EPS is notified by the HCA via an interrupt when the trace buffer is full or when the trace is stopped;

copy the one or more portions of packets to shared memory; and

copy the one or more portions of packets from shared memory to the host memory.

10. The method of claim 1, wherein the one or more portions of packets are continuously read out by the HCA and written to host memory in real-time during the trace.

11. A host channel adapter (HCA), comprising:

at least two ports for transmitting and receiving a packet stream, wherein each of the at least two ports operate in one selected from a group consisting of normal mode and trace mode;

a trace buffer configured to store trace data corresponding to each of the at least two ports of the HCA, wherein the trace buffer is dynamically repurposed from a payload random access memory (RAM) to the trace buffer when the corresponding port of the HCA is set to trace mode;

a packet receive module (PRM) comprising a set of control and status registers (CSRs) for storing trace parameters, and at least one multiplexer for duplicating the packet stream from one of the at least two ports and storing the duplicated packet stream to a corresponding trace buffer when the one of the at least two ports is in trace mode; and

a direct memory access (DMA) engine configured to continuously read trace data out of the trace buffer and write the trace data to host memory,

wherein the HCA is configured to;

receive the packet stream comprising a plurality of packets;

compare a packet header of each of the plurality of packets to a trigger condition stored in one CSR of the set of CSRs to determine whether the trigger condition has been met;

after the trigger condition has been met, compare each packet header of the plurality of packets to one or more trace filters stored in the set of CSRs to identify matching packets;

duplicate and store, via the multiplexer, one or more portions of the matching packets to the corresponding trace buffer; and

stop a trace of the one or more portions of the packets, wherein after the trace is stopped, the one or more portions of packets in host memory are analyzed for debugging errors.

12. The HCA of claim 11, wherein the HCA is further configured to:

collect prestore data in the trace buffer when a prestore percentage is written to one CSR of the set of CSRs,

wherein a first portion of the trace buffer corresponds to prestore data and a second portion of the trace buffer corresponds to poststore data comprising the one or more portions of packets stored after the trigger condition is met.

13. The HCA of claim 12, wherein the trace is stopped automatically when a maximum address of the trace buffer is reached.

14. The HCA of claim 12, wherein the DMA engine is configured to read out data from the trace buffer in real-time.

15. A system, comprising:

a host comprising memory;

a host channel adapter (HCA) operatively connected to the host and comprising;

at least two ports for transmitting and receiving a packet stream, wherein each of the at least two ports operate in one selected from a group consisting of normal mode and trace mode;

a trace buffer configured to store trace data corresponding to each of the at least two ports of the HCA, wherein the trace buffer is dynamically repurposed from a payload random access memory (RAM) to the trace buffer when the corresponding port of the HCA is set to trace mode;

a set of control and status registers (CSRs) for storing trace parameters; and

at least one multiplexer for duplicating the packet stream from one of the at least two ports and storing the duplicated packet stream to a corresponding trace buffer when the one of the at least two ports is in trace mode; and

an embedded processing system (EPS) configured to;

request a trace to be initiated on the HCA; and

read trace data out of the trace buffer and write the trace data to host memory,

wherein the HCA is configured to;

receive the packet stream comprising a plurality of packets;

compare a packet header of each of the plurality of packets to a trigger condition stored in one CSR of the set of CSRs to determine whether the trigger condition has been met;

after the trigger condition has been met, compare each packet header of the plurality of packets to one or more trace filters stored in the set of CSRs to identify matching packets;

duplicating and storing, via the multiplexer, one or more portions of the matching packets to the corresponding trace buffer; and

stop a trace of the one or more portions of the packets.

16. The system of claim 15, wherein the HCA is further configured to:

stop the trace automatically when a maximum address of the trace buffer is reached.

17. The system of claim 16, wherein the HCA is further configured to:

notify the EPS when the maximum address of the trace buffer is reached by an interrupt.

18. The system of claim 15, wherein the host is configured to:

display and analyze the one or more portions of the matching packets to debug errors observed in network activity.

19. The system of claim 18, wherein the network is the Infiniband®

network and the HCA is an Infiniband®

HCA.