Visualizing Individual Neuron Activations in Deep Neural Networks

Each of the pictures above corresponds to the activations of single neurons chosen from within the hidden layers of a deep neural network trained to generate and image of the Mona Lisa (a task which I've previously described here and here.)

The global inputs to the neural network consist of 2-dimensional tuples of (x, y) coordinates, and this arrangement allows the behavior of single neurons to be visualized completely. Each pixel location corresponds to a different (x, y) input, and the pixel intensity at that location represents the scalar activation of the neuron when the neural network as a whole is exposed to that particular coordinate-pair.

In order to generate each of the images, which when full size are 429-by-300 pixels, the neural network must be run 128,700 times.

Here is the end result: