Sunday, March 06, 2011
Non-Markovian Double Pole Balancing with Multiple Carts
Above is an animation depicting the results of having evolved a single recurrent neural network (with 10 hidden neurons) to act as a balancing controller for two carts simultaneously.
The RNN has no access to velocity information, thus making the task non-Markovian, and hence, significantly more difficult.
Cart-pole balancing is a fairly standard benchmark problem in the Reinforcement Learning literature, although to the best of my knowledge this is the first example of controlling multiple carts at the same time.