Over decades of continued investment, developer tooling has come a long way for popular programming languages. With computation graphs on the rise, we need specialized tools to achieve the same levels of efficiency we enjoy with IDEs, debuggers and visual editors when writing traditional procedural code. Here are a few of my must use tools when building models.
Netron is a neural network visualization tool. It can open a number of common computational graph model formats including CoreML, TensorFlow, PyTorch, CNTK and more. It allows you to explore the structure and nodes within a graph, along with metadata for nodes (sizes etc.). It also includes documentation for the computational nodes, describing how they transform incoming data. It’s kind of like the equivalent of GNU binutils.
Lobe is still in early beta, and I’ve traditionally used my own hand-spun platform to perform many of the same tasks, although with much less breadth of features. I consider it the equivalent of an IDE, letting developers quickly build, train and debug models within a single workflow with visualization tools to assist with the large volumes of data involved.
Tensorboard is specific to TensorFlow, and achieves many of the same functions lobe does – think of it as a GDB equivalent. You can introspect accuracy and progress of training cycles, visualize model weights and parameters, and visualize and debug the loaded computation graph.