Neural gas

Not to be confused with Nerve gas.

Neural gas is an artificial neural network, inspired by the self-organizing map and introduced in 1991 by Thomas Martinetz and Klaus Schulten. The neural gas is a simple algorithm for finding optimal data representations based on feature vectors. The algorithm was coined "neural gas" because of the dynamics of the feature vectors during the adaptation process, which distribute themselves like a gas within the data space. It is applied where data compression or vector quantization is an issue, for example speech recognition, image processing or pattern recognition. As a robustly converging alternative to the k-means clustering it is also used for cluster analysis.

Algorithm

Given a probability distribution P(x) of data vectors x and a finite number of feature vectors w_i, i=1,...,N.

With each time step t a data vector randomly chosen from P is presented. Subsequently, the distance order of the feature vectors to the given data vector x is determined. i₀ denotes the index of the closest feature vector, i₁ the index of the second closest feature vector etc. and i_N-1 the index of the feature vector most distant to x. Then each feature vector (k=0,...,N-1) is adapted according to

$w_{i_k}^{t+1} = w_{i_k}^{t} + \varepsilon\cdot e^{-k/\lambda}\cdot (x-w_{i_k}^{t})$

with ε as the adaptation step size and λ as the so-called neighborhood range. ε and λ are reduced with increasing t. After sufficiently many adaptation steps the feature vectors cover the data space with minimum representation error.

The adaptation step of the neural gas can be interpreted as gradient descent on a cost function. By adapting not only the closest feature vector but all of them with a step size decreasing with increasing distance order, compared to (online) k-means clustering a much more robust convergence of the algorithm can be achieved. The neural gas model does not delete a node and also does not create new nodes.

Further reading

• T. Martinetz, S. Berkovich, and K. Schulten. "Neural-gas" Network for Vector Quantization and its Application to Time-Series Prediction. IEEE-Transactions on Neural Networks, 4(4):558-569, 1993.
• T. Martinetz and K. Schulten. Topology representing networks. Neural Networks, 7(3):507-522, 1994.

External links

• DemoGNG Java applet which demonstrates neural gas, growing neural gas, self-organizing maps and other methods related to competitive learning.
• Java Competitive Learning Applications Unsupervised Neural Networks (including Self-organizing map) in Java with source codes.
• Neural gas algorithm

• http://wwwold.ini.rub.de/VDM/research/gsn/JavaPaper/img187.gif