FirstOrderOptimizationMethods

Library containing implementation of First-Order Optimization methods.

Algorithms Implementations

• Adam Optimizer
• Conjugate Gradient
• Brent's Line Search

References

• J. Ba, Adam: A method for stochastic optimization DP Kingma, arXiv preprint arXiv:1412.6980, 2014 - arxiv.org
• Y. Kobayashi, H. Iiduka, Conjugate-gradient-based Adam for stochastic optimization and its application to deep learning, arXiv preprint arXiv:2003.00231, 2020 - arxiv.org
• R. P. Brent, An algorithm with guaranteed convergence for finding a zero of a function