On-Line Learning Processes
On-Line Learning Processes
Introduction
Neural networks, like humans, learn from examples. In supervised learning, training examples consist of input-output pairs, e.g., inputs are the visual images of the target and of the robot arm, and outputs are the required controls in the form of pressure values for the robot actuators. A well-known example of unsupervised learning is the self-organization of feature maps as discussed elsewhere in this report. Practical applications of both supervised and unsupervised learning are based on sequential presentation of training patterns. A learning step takes place at each presentation of a single training example (so-called "on-line learning") or on account of a whole set of training patterns. From a biological point of view, learning from sets of training patterns is implausible. Therefore, we investigate the usefulness and advantages of on-line learning. This research is conducted by Tom Heskes, a post-doctoral researcher who joined our group in August 1993, coming from the University of Nijmegen.
Description
Learning after each presentation leads to some kind of randomness in the learning process. We showed that this stochasticity can help to prevent the network from getting stuck in suboptimal configurations, so-called "local minima" of the energy function that the learning rule tries to minimize. An example of such an energy function is the average squared distance between the robot's end effector and the target. A well-known technique to find the best possible configuration, the global minimum of the energy function, is simulated annealing. The randomness resulting from on-line learning has a similar effect, but in general, will lead to even better performance.Through learning, neural networks build an internal representation of their environment. This representation is coded in the network's weights or synapses. To speed up learning, one often adds a so-called momentum term to the learning rule. With such a momentum term, the weight changes do not only depend on the current training example, but also on previous weight changes. Incorporation of the momentum term accelerates learning in regions where the energy function is relatively flat. We proved that, whereas the momentum term does help in speeding up learning from sets of examples, it has no effect whatsoever on learning from a sequence of single examples.
Supervised learning can be viewed as a process where a "teacher" (generating the desired network outputs given a particular input) tells the "student" (the neural network) what to do. Previous work on learning processes in neural networks has focussed on perfect conditions, that is, completely reliable "teachers". However, in practical applications, the assumption that teachers are completely reliable no longer holds. Noise in the learning process may be due to inaccuracy of the input data, output data, or to the noisy processing of the network weights. Studying a simple learning problem, we showed that different types of noise lead to different effects, e.g., output noise tends to be more destructive than input or processing noise. Furthermore, we suggested with an algorithm to improve the learning performance in these "noisy environments". The algorithm is "self-tuning", i.e., it uses information in the learning characteristics to optimize its performance. This makes the algorithm fairly insensitive to the type and magnitude of the noise. Further studies should aim at application of the algorithm to real-world problems.
