Memory-based black box optimization

Experimenting with a memory-based approach to black-box optimization to help avoid being trapped in local optima.

The image on the left is a two-dimensional slice out of a (much higher dimensional) fitness landscape from a recurrent neural network. The optimal point is just above the lower left hand corner, and it is discovered after exactly 808 samples. The sampling process is shown on the right.

The size of the entire image being searched is 500x500 (250,000) pixels.