Hebbian

This type of synapse implements Hebbian learning, where the change of the synapse is proportional to the product of the activation of the source neuron a_s and target neuron a_t:

The strength of this synapse is clipped so as to remain between the lower and upper bounds specified for this synapse.

In effect, neurons that "fire together, wire together."

Note that in most cases synapses of this type will rapidly achieve their maximum or minimum value. To prevent this a very small learning rate can be used, or after sufficient training the weights can be clamped. Other learning rules and networks attempt to overcome this limitation of Hebbian learning.

Learning rate

This value changes the rate of the change of the synapse, denoted by epsilon in the equation above.