Visualizing data transfers in distributed file system

• US 9,159,149 B2
• Filed: 03/14/2013
• Issued: 10/13/2015
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brodsky, Stephen A., De Pauw, Wim, Yang, Eric C.
• Primary Examiner(s): DANG, THANH HA T
• Application Number:
  US13/804,192
• Publication Number:
  US 20140280399A1
• Time in Patent Office: 943 Days
• Field of Search: 707/827
• US Class Current: 1/1
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/3323 using document space presentation or visualization, e.g. category, hierarchy or range presentation and selection
• CPC Class Codes: G06F 16/9027 Trees
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs

Abstract

Data transfers within and between nodes in a distributed computing environment are visualized. In one aspect, the nodes are represented as geometrical shapes. The first part of the geometrical shape represents logic of the corresponding node. The second part of the geometrical shape represents storage of the corresponding node. A line connecting the first part and the second part represents data transfer between logic of a node and storage of a node. Both inter-node and intra-node transfers may be shown.

Claims

1. A method of visualizing data transfers in a distributed computing environment, comprising:

receiving information associated with the data transfers comprising at least information associated with a plurality of nodes in the distributed computing environment; and

visualizing, by a processor, the plurality of nodes in the distributed computing environment, a node in the plurality of nodes represented as a geometrical shape, a first part of the geometrical shape representing logic of the node and a second part of the geometrical shape representing storage of the node, wherein a line connecting the first part and the second part represents data transfer between the first part and the second part, and wherein a spline connecting two of the geometrical shapes each of which represents a different node in the plurality of nodes, represents inter-node data transfer between logic and storage of said two different nodes.

2. The method of claim 1, wherein the plurality of nodes is organized and visualized as a matrix wherein each row of the matrix contains nodes in a rack in the distributed computing environment.

3. The method of claim 2, wherein data transfer between two nodes of different racks are visualized as a spline between said two nodes of different racks drawn outside the matrix.

4. The method of claim 1, wherein the plurality of nodes is organized and visualized as a circular layout wherein data transfer within a node of the plurality of nodes is presented as a radial line between the first part and the second part of the node.

5. The method of claim 4, wherein data transfer between two nodes of the plurality of nodes in a same rack in the distributed computing environment is presented as a spline between two geometrical shapes representing the two nodes and shown along the circular layout.

6. The method of claim 4, wherein data transfer between two nodes of the plurality of nodes in different racks in the distributed computing environment is presented as a spline between two geometrical shapes representing the two nodes and shown inside the circular layout.

7. The method of claim 1, wherein the geometrical shape is a rectangle, the first part and the second part are smaller rectangles shown inside the geometrical shape.

8. The method of claim 1, wherein a thickness of the line represents an amount of the data transfer.

9. The method of claim 1, wherein a size of the first part or the second part represents an amount of data transferred or to be transferred.

10. The method of claim 1, wherein the information comprises at least a type of data transfer, wherein different types of data transfers are shown in different colors in a single visualization.

11. The method of claim 1, wherein the data transfers comprise Hadoop Distributed File System (HDFS) writes, HDFS reads, or HDFS shuffles, or combinations thereof.

12. The method of claim 1, wherein the data transfers represent financial flows within one or more departments of an organization, between said departments, or between a plurality of organizations, or combinations thereof.

13. The method of claim 1, further comprising:

allowing interaction with the visualized data transfers among the plurality of nodes.

14. The method of claim 13, wherein the interaction comprises filtering the data transfers for the visualizing according to a criterion.

15. The method of claim 13, wherein the interaction comprises displaying statistics associated with the data transfers.

16. The method of claim 13, wherein the interaction comprises pattern extraction.