Least Mean Squares Iterative
The Least Mean Squares or LMS rule is a classic form of supervised learning, which means that the user must supply desired output values for each of a list of input values.
The lms iterative trainer can be configured in the trainer dialog. It is primarily used by the least mean square network and echo state network, though it can also be used by scripts and in other ways. The following parameter can be modified.
Learning Rate: A standard learning rate. This determines how quickly synapses change.
Basic operation
The LMS rule works as follows. The change in a weight is equal to the product of a learning rate ε, the pre-synaptic source activation, and the difference between the post-synaptic activation aj and a desired activation tj. The error is the difference between the desired and actual activation of the target neuron.
Repeated application of this rule minimizes reduces mean squared error on a set of training data.
This rule is also known as the "Widrow-Hoff" rule, and the "Delta Rule." Networks that use these rules are sometimes called "adalines" or "madalines" (for the multilayer case, which these networks do not currently implement). They are descendants of an early form of network studied by Rosenblatt called a "perceptron."