This post is the first one in the series of posts where I will describe the creation of a neural network, modelling the behaviour of a small organism. Why model such a system, what functions it will perform and what the results I will get will be described in the next posts.
The only thing I want to describe in this post is the solution which helps accelerate the training of neural networks.
I will not describe the basic algorithm and work of neural networks of such type in general here. You can find the information about it in the article CS231n – Neural networks and in the example of Andrej Karpathy in Deep Reinforcement Learning: Pong from Pixels.
In order to train the neural network, one needs to collect a large pool of data. But if you work with systems that are training in real time, it is impossible to collect the data beforehand (e.g. Markov decision process). Every action of the neural network influences its environment and changes it. It makes it very difficult to generate a necessary amount of data. In this case I decided to use parallel training by dividing the network.
I divided the system into two components: a Trainer and a Player.
A Trainer is a class incapsulating the functions of backward training and actual weights. A Trainer decides how many Players can play simultaneously.
A Player is a class that plays a game by getting a snapshot of actual weights from the Trainer.
After finishing the game a Player returns a set of hidden layer states and input data to the Trainer. As all Players play in the same homogenous environment we can assume that their games are of equal value.
There exists some discrepancy related to the lag of players from actual weights but it is negligible compared to the increase in speed of training.
Below you can see the diagrams of the weights distribution before and after the training and the diagram of the speed of training of such model.
The training takes about an hour on an iMac late 2015 using only CPU and Accelerate Framework.
Author: Volodymyr Pavliukevych
Senior Software Engineer, Data Scientist.