# Vanishing gradient problem

> The tendency for the gradients of early hidden layers of some DNNs to become surprisingly flat (low). Increasingly lower gradients result in increasingly smaller changes to the weights on neurons in a DNN, leading to little or no learning. Models suffering from the vanishing gradient problem become difficult or impossible to train. LSTM cells address this issue.[^1]

The tendency for the [gradients](https://wiki.g15e.com/pages/Gradient%20(machine%20learning.txt)) of early [hidden layers](https://wiki.g15e.com/pages/Hidden%20layer.txt) of some [DNNs](https://wiki.g15e.com/pages/Deep%20neural%20network.txt) to become surprisingly flat (low). Increasingly lower gradients result in increasingly smaller changes to the [weights](https://wiki.g15e.com/pages/Weight%20(machine%20learning.txt)) on [neurons](https://wiki.g15e.com/pages/Artificial%20neuron.txt) in a DNN, leading to little or no learning. [Models](https://wiki.g15e.com/pages/Model%20(machine%20learning.txt)) suffering from the vanishing gradient problem become difficult or impossible to [train](https://wiki.g15e.com/pages/Training%20(machine%20learning.txt)). [LSTM](https://wiki.g15e.com/pages/Long%20short-term%20memory.txt) cells address this issue.[^1]

## See also

- [Exploding gradient problem](https://wiki.g15e.com/pages/Exploding%20gradient%20problem.txt)

## Footnotes

[^1]: https://developers.google.com/machine-learning/glossary#vanishing-gradient-problem