The Hidden Pitfall of Neural Network Weight Averaging
Why averaging neural network weights can break model performance due to permutation symmetry
1The Problem with Naive Averaging
When averaging weights from two neural networks trained with different random seeds, the results often perform worse than either original model. This occurs because neural networks have permutation symmetry - the same function can be represented with neurons in different orders. Averaging weights without aligning these permutations is like averaging spreadsheet columns that represent different metrics.
2Understanding Permutation Symmetry
Permutation symmetry means a neural network's loss landscape contains multiple equivalent solutions that are identical except for neuron ordering. For a layer with m neurons, there are m! equivalent configurations. A 512-neuron layer has about 10^1166 equivalent solutions. This symmetry creates barriers between solutions that make direct weight averaging problematic.
3Visualizing the Loss Landscape
A simple two-neuron ReLU network demonstrates the symmetry clearly. The loss landscape shows two global minima (e.g., at (2,6) and (6,2)) that are mirror images. The midpoint between them has 6.4× higher loss, creating a barrier. Gradient descent converges to either minimum randomly, with roughly 50% probability for each configuration.
4The Adaptive Basis Trade-off
Neural networks differ from classical models by learning both basis functions and coefficients simultaneously. While this adaptivity is powerful, it comes at the cost of convexity. Fixed basis functions (like in polynomial regression) yield convex problems with single solutions, but learned bases introduce permutation symmetry and multiple equivalent solutions.
5Real-world Impact on Model Merging
This symmetry explains why techniques like model soups or federated averaging sometimes fail. Without proper alignment, averaging weights from different training runs combines unrelated parameters. The paper shows a case where naive averaging produced a model 6.5× worse, while permutation-aligned averaging maintained full performance.
6Current Research Directions
Recent work by Entezari et al. (2021) suggests most SGD solutions may belong to the same basin after accounting for permutations. Ainsworth et al. (2022) developed algorithms to find aligning permutations, supporting the single-basin hypothesis. These advances are crucial for practical LLM engineering in 2026.
7Mathematical Foundations
The core issue stems from the neural network loss function being symmetric under neuron permutations. For two neurons w₁ and w₂, swapping them leaves the function unchanged. This symmetry creates multiple global minima in parameter space, despite corresponding to the same function in output space.
8Experimental Evidence
In experiments with a two-neuron network, gradient descent converged to (2,6) 47% of the time and (6,2) 53% of the time from random initializations. The choice depended entirely on the random seed. This demonstrates how symmetry leads to solution multiplicity in even simple networks.
9Implications for Ensemble Methods
Traditional ensemble methods that average model outputs remain valid, but weight averaging requires caution. The paper shows output averaging always works, while weight averaging only succeeds with proper permutation alignment. This distinction is crucial for practitioners combining multiple trained models.
10Historical Context
The permutation symmetry problem has been recognized since early neural network research, but gained importance with modern large models. As neural networks grew deeper and wider, the number of equivalent permutations exploded, making weight averaging without alignment increasingly unreliable.
11Practical Solutions
Researchers have developed several approaches to handle permutation symmetry, including weight matching algorithms and activation-based alignment methods. These techniques attempt to find the permutation that makes two models' weights most similar before averaging, significantly improving merged model performance.
12Future of Model Merging
Understanding permutation symmetry is becoming essential as model merging grows in importance for federated learning, continual learning, and collaborative AI development. Future research may yield more efficient alignment methods or architectures that inherently reduce symmetry-related issues in weight space.
Original article: Towards Data Science — The Hidden Pitfall of Neural Network Weight Averaging