Version 10, last updated by pbaclace at Jun 29 03:14 2011 UTC
Phatvis (PHAse Time VISualizer) is based on the Java nlv from the NetLogger project at dsd.lbl.gov which has been used to tune parallel super computer batch jobs since 1999.
Phatvis was demo-ed for Hadoop Map-Reduce as a performance tuning tool at the ScaleCamp2009 on June 9, 2009. See screenshot1.png and screenshot2.png.
The code here is based on a 2004 snapshot of the Java nlv source from the NetLogger project. Although a binary of the Java nlv was released at some point, the source was never previously released since the interactive nlv part of the project was dropped due to lack of resources. The NetLogger project switched to static diagrams generated with the R stats software. The R diagrams offer more sophistication, and better presentation quality, but the interactive visualization is much better suited to quick exploration and performance tuning. Moreover, using a Java application is easier/faster than installing, setting up, and learning R.
The visualizer is driven by a powerful declarative configuration file that selectively maps log info to chart glyphs, phase ordering, and annotations. The config is more like an SQL select statement than a settings file.
An XML markup config syntax was adopted by NetLogger processing tools (python scripts), but the code base here reads an older, simple syntax style that is nice to hand-edit with a text editor. This format is translated to the xml format internally, so there is a possibility that both config forms could be supported at some point.
The particular kind of diagram has been called a "lifeline" diagram at the NetLogger project, but that does not fully capture how lines can converge or interconnect and is based on what Edward Tufte would call a type of "space-time" diagram. The horizontal axis is absolute time and the vertical axis is computational phase. The semi-descriptive Phatvis was conjured up to reflect Phase-Time Visualization as a memorable and planet wide unique name to distinguish it from the old nlv variants which are not necessarily campatible, and are not being maintained.
CURRENT STATUS: Initial commit done, has samples to run, project Page created, datapoint annotations generalized, rule based dependency generation added..
NEXT: Clean up scripts for scraping Hadoop 0.20.2
Meta-Plans: Be compatible with NetLogger logging methods (which have microsecond resolution in Java), inter-operate with provide working examples, and migrate away from use of a snapshot of ThunderGraph which has an LGPL license that is incompatible with the Apache 2.0 license.
LICENSE: BSD-style (California Regents...) and LGPL.
More Info on NetLogger: