AdaGrad

A sophisticated gradient descent algorithm that rescales the gradients of each parameter, effectively giving each parameter an independent learning rate. For a full explanation, see this AdaGrad paper.¹

Footnotes

1. developers.google.com/machine-learning/glossary#adagrad ↩