> For the complete documentation index, see [llms.txt](https://luweikxy.gitbook.io/machine-learning-notes/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://luweikxy.gitbook.io/machine-learning-notes/gradient-descent-algorithm.md).

# 梯度下降算法

## 梯度下降算法

* [返回顶层目录](https://luweikxy.gitbook.io/machine-learning-notes/pages/-LpO5vE88qYwjk5WM_Qf#目录)
* [各类梯度下降算法的演化](https://luweikxy.gitbook.io/machine-learning-notes/pages/-Lug-BTI1fM-jbJ2y9mU#各类梯度下降算法的演化)
* [随机梯度下降SGD](broken://pages/-LurJKgHz3aekzbgvSiM)
* [动量法Momentum](/machine-learning-notes/gradient-descent-algorithm/momentum.md)
* [牛顿动量Nesterov](/machine-learning-notes/gradient-descent-algorithm/nesterov.md)
* [AdaGrad](/machine-learning-notes/gradient-descent-algorithm/adagrad.md)
* [RMSprop](/machine-learning-notes/gradient-descent-algorithm/rmsprop.md)
* [Adadelta](/machine-learning-notes/gradient-descent-algorithm/adadelta.md)
* [Adam](/machine-learning-notes/gradient-descent-algorithm/adam.md)
* [Nadam](/machine-learning-notes/gradient-descent-algorithm/nadam.md)
* [AMSGrad](/machine-learning-notes/gradient-descent-algorithm/amsgrad.md)
* [AdasMax](/machine-learning-notes/gradient-descent-algorithm/adamax.md)

![optimization-on-loss-surface-contours](/files/-LurJLn_2Peov9ITKLo3)

## 各类梯度下降算法的演化

![revolution-of-gradient-descent](/files/-Lug-Cj8eWQsYrsBcgKw)

## 参考资料

* [Deep Learning 之 最优化方法](https://blog.csdn.net/BVL10101111/article/details/72614711)
* [从 SGD 到 Adam —— 深度学习优化算法概览(一)](https://zhuanlan.zhihu.com/p/32626442)
* [深度学习最全优化方法总结比较（SGD，Adagrad，Adadelta，Adam，Adamax，Nadam）](https://zhuanlan.zhihu.com/p/22252270)
* [10个梯度下降优化算法+备忘单](https://ai.yanxishe.com/page/TextTranslation/1603?from=singlemessage)

  [深度学习——优化器算法Optimizer详解（BGD、SGD、MBGD、Momentum、NAG、Adagrad、Adadelta、RMSprop、Adam）](https://www.cnblogs.com/guoyaohua/p/8542554.html)