VISUALIZING DATA TRANSFERS IN DISTRIBUTED FILE SYSTEM

• US 20140267293A1
• Filed: 08/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Brodsky, Stephen A., De Pauw, Wim, Yang, Eric C.
• Granted Patent: US 9,142,047 B2
• Time in Patent Office: Days
• US Class Current: 345/440
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/3323 using document space presentation or visualization, e.g. category, hierarchy or range presentation and selection
• CPC Class Codes: G06F 16/9027 Trees
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs

Abstract

Data transfers within and between nodes in a distributed computing environment are visualized. In one aspect, the nodes are represented as geometrical shapes. The first part of the geometrical shape represents logic of the corresponding node. The second part of the geometrical shape represents storage of the corresponding node. A line connecting the first part and the second part represents data transfer between logic of a node and storage of a node. Both inter-node and intra-node transfers may be shown.

Claims

1. A system for visualizing data transfers in a distributed computing environment, comprising:

a processor;

a visualization module operable to receive information associated with the data transfers comprising at least information associated with a plurality of nodes in the distributed computing environment, the visualization module further operable to visualize the plurality of nodes in the distributed computing environment, a node in the plurality of nodes represented as a geometrical shape, a first part of the geometrical shape representing logic of the node and a second part of the geometrical shape representing storage of the node, wherein a line connecting the first part and the second part represents data transfer between the first part and the second part, and wherein a spline connecting two of the geometrical shape each of which represents a different node in the plurality of nodes, represents inter-node data transfer between logic and storage of said two different nodes; and

a graphical user interface module operable to present the visualized data transfers among the plurality of nodes, and further allow interaction with the visualized data transfers among the plurality of nodes.

2. The system of claim 1, wherein the plurality of nodes is organized and visualized as a matrix wherein each row of the matrix contains nodes in a rack in the distributed computing environment, and wherein data transfer between two nodes of different racks are visualized as a spline between said two nodes of different racks drawn outside the matrix.

3. The system of claim 1, wherein the plurality of nodes is organized and visualized as a circular layout wherein data transfer within a node of the plurality of nodes is presented as a radial line between the first part and the second part of the node, wherein data transfer between two nodes of the plurality of nodes in a same rack in the distributed computing environment is presented as a spline between two geometrical shapes representing the two nodes and shown along the circular layout, and wherein data transfer between two nodes of the plurality of nodes in different racks in the distributed computing environment is presented as a spline between two geometrical shapes representing the two nodes and shown inside the circular layout.

4. The system of claim 1, wherein the geometrical shape is a rectangle, the first part and the second part are smaller rectangles shown inside the geometrical shape, and wherein a thickness of the line and a thickness of the spline represent amounts of the data transfer.

5. The system of claim 1, wherein a size of the first part or the second part represents an amount of data transferred or to be transferred.

6. The system of claim 1, wherein the information comprises at least a type of data transfer, wherein different types of data transfers are shown in different colors in a single visualization.

7. The system of claim 1, wherein the data transfers comprise HDFS writes, HDFS reads, or HDFS shuffles, or combinations thereof.

8. A computer readable storage medium storing a program of instructions executable by a machine to perform a method of visualizing data transfers in a distributed computing environment, comprising:

receiving information associated with the data transfers comprising at least information associated with a plurality of nodes in the distributed computing environment; and

visualizing, by a processor, the plurality of nodes in the distributed computing environment, a node in the plurality of nodes represented as a geometrical shape, a first part of the geometrical shape representing logic of the node and a second part of the geometrical shape representing storage of the node, wherein a line connecting the first part and the second part represents data transfer between the first part and the second part, and wherein a spline connecting two of the geometrical shape each of which represents a different node in the plurality of nodes, represents inter-node data transfer between logic and storage of said two different nodes.

9. The computer readable storage medium of claim 8, wherein the data transfers comprise HDFS writes, HDFS reads, or HDFS shuffles, or combinations thereof.