Archive: the longer working version of this page, kept for reference. It includes material cut from the current page: the two-input panel, why committing works, strategy statistics and hard mode.
A competitive network sorts its inputs into groups by letting output units compete for them. This page starts with a two-input version you can see completely, then plays the game from the Simbrain simulation: give each of five patterns its own output unit.
Here is a competitive network small enough to see completely: two input nodes, five patterns (P1 to P5) and five output units (O1 to O5), as in the Simbrain game. Each output unit’s two weights make a red arrow from the origin, and each pattern is a blue arrow. Press a pattern to present it. The unit with the largest weighted input wins, and only the winner learns: it turns toward that input.
Under Sum to 1, every unit sits on the straight segment. Under Length 1, every unit sits on the quarter circle. Dashed lines join each pattern to its winner, and faint red lines trace where each unit has been. You can also click a pattern on the plot.
Weighted input to each unit from each pattern. Outlined cells are the winners. The last column gives each unit’s weight length.
Under Sum to 1, O1 wins P1 to P3 and O5 wins P4 and P5. O2, O3 and O4 win nothing, even though each points almost straight at a pattern. Training can’t fix this: with two inputs, only the unit furthest toward each end of the segment can win, so at most two units ever have patterns. Switch to Length 1 and each pattern goes to the unit pointing closest to it, which here gives every pattern its own unit.
The reason: a unit’s weighted input is (weight length) × (input length) × cos(angle between them). For a given input, its own length is the same for every unit, so the winner is the unit with the largest weight length × cos(angle). If every unit has length 1, only the angle matters. The unit pointing closest wins, which is also the nearest point, and that is the textbook picture. Simbrain, following Rumelhart and Zipser, scales each unit’s weights to sum to 1 instead. That puts them on the straight segment, where lopsided units are longer (up to 1.0) than balanced ones (0.71). With two inputs this is enough for the most lopsided unit on each side to win everything on that side.
Rumelhart and Zipser said so themselves. Their sphere picture “is only approximate. We have used the constraint that Σw = 1 rather than the constraint that Σw² = 1. This latter constraint would ensure that all vectors are in fact the same length” (Feature discovery by competitive learning, 1985, footnote 3). With more inputs the effect is much milder. In the seven-input game below, with fresh random weights, the winner is not the nearest unit for about one pattern in five, and once units are trained the two rules mostly agree.
The Simbrain simulation uses the same rule with seven input nodes and five output units. Each pattern turns on three of the seven input nodes, and any two patterns share exactly one active node.
Each output unit has seven fan-in weights, so its weight vector is a point in the same 7-dimensional space as the inputs. The weights start random and are scaled to sum to 1. When a pattern is presented, the unit with the largest weighted input wins. Only the winner learns: its weights move a fraction of the way (the learning rate, 0.1 here) toward the input, scaled to sum to 1 as well.
Blue points are the five patterns, red points are the five units’ weight vectors, and dashed lines join each pattern to the unit that currently wins it. Clicking a pattern trains on it, like selecting a pattern in Simbrain and pressing Train. Try taking turns first, then try something else.
Outlined cells are the winner for each pattern. The right column lists the patterns each unit wins now, which is what the labels in Simbrain track.
An untrained unit has its weight spread over all seven inputs, so it gives every pattern about 0.43 (three of seven equal shares). That dashed line is the bar any trained unit has to stay under to let a pattern go.
A unit trained all the way onto P1 responds 1.0 to P1 and 0.33 to every other pattern, because each shares one input with P1. That is under the line, so the other patterns go to other units.
A unit that wins both P1 and P2 and gets trained on them in turn settles halfway between them. Halfway gives each of the two patterns about 0.67, well over the line, so the unit keeps both. Training P1 many times in a row pulls it off the halfway point, and its response to P2 sinks until some other unit takes P2.
How many repeats count as many depends on the learning rate. Each step closes a tenth of the remaining distance at 0.1, so ten to twenty steps on one pattern usually does it.
Measured with Simbrain’s own competitive network code: 2,000 random starts per row, five units, learning rate 0.1 unless noted, then each pattern tested without learning. The bars show how many patterns ended up with their own unit.
Switch the patterns above to Sliding window. Now each pattern is three neighboring inputs (123, 234, …, 567), so neighbors share two of their three inputs. A unit trained onto P3 gives P2 and P4 about 0.67, which is over the line, so it holds on to its neighbors no matter how you train. Committing no longer helps, and the result is mostly decided by the random starting weights. The Competitive grid network simulation in Simbrain is a hard mode as well: its pictures overlap heavily, so committing usually gets four units but almost never five.
The general rule: a trained unit gives another pattern (shared inputs ÷ active inputs). An untrained one gives about (active inputs ÷ total inputs). A strategy can separate two patterns only when the first number is smaller.
Try it in Simbrain under Simulations › Competitive › Competitive network (simple).