Optimizer (machine learning)

A specific implementation of the gradient descent algorithm. Popular optimizers include:¹

AdaGrad, which stands for ADAptive GRADient descent.
Adam, which stands for ADAptive with Momentum.

Footnotes

developers.google.com/machine-learning/glossary#optimizer

Nested Learning: The Illusion of Deep Learning Architectures “In addition to its neuroscientifically plausible and mathematically white-box nature, we advocate for its impor…”
ML crash course - Linear regression “Define and describe how gradient descent finds the optimal model parameters.”
Clipping (machine learning) 주제어가 겹침
역전파 “The algorithm that implements gradient descent in neural networks.”
Machine Learning Systems “Machine Learning Systems provides a systematic framework for understanding and engineering machine learning (ML…”
Mean squared error 이 글을 인용: “Mean Squared Error is a popular training optimizer, particularly for linear regression.”
Introduction to Machine Learning “Machine learning crash course: If you're ready for an in-depth and hands-on approach to learning more about ML.”
Learning rate “A floating-point number that tells the gradient descent algorithm how strongly to adjust weights and biases on …”
Convex optimization “The process of using mathematical techniques such as gradient descent to find the minimum of a convex function.”
배치 크기 “Mini-batch stochastic gradient descent, in which the batch size is usually between 10 and 1000.”
Vanishing gradient problem “Models suffering from the vanishing gradient problem become difficult or impossible to train.”
Convex function 관련 절: See also
Gradient (machine learning) “In machine learning, the gradient is the vector of partial derivatives of the model function.”
Epoch (machine learning) 주제어가 겹침
Exploding gradient problem “The tendency for gradient in DNNs (especially RNNs) to become surprisingly steep (high).”
Gradient boosting “Gradient boosting is a powerful ensemble machine learning technique used for both regression and classification…”
Iteration (machine learning) 주제어가 겹침
Downsampling (machine learning) 주제어가 겹침
Generalization (machine learning) 주제어가 겹침
기계학습 “2024-08-24 - What's Really Going On in Machine Learning?”