No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models (ICLR 2022)
-
Updated
Feb 9, 2022 - Python
#
adaptive-learning-rate
Here are 7 public repositories matching this topic...
AdaTask: A Task-Aware Adaptive Learning Rate Approach to Multi-Task Learning. AAAI, 2023.
-
Updated
Sep 29, 2023 - Python
A Julia package for adaptive proximal gradient and primal-dual algorithms
machine-learning optimization convex-optimization primal-dual-algorithms proximal-gradient-method adaptive-learning-rate linesearch-free-methods
-
Updated
Jan 18, 2024 - Julia
A Julia package for adaptive proximal gradient for convex bilevel optimization
machine-learning bilevel-optimization optimization-algorithms convex-optimization adaptive-learning-rate adaptive-proximal-algorithms
-
Updated
Jun 15, 2023 - Julia
Deep Filtering with adaptive learning rates
-
Updated
Jun 21, 2022 - Jupyter Notebook
Implementation of a two-layer perceptron (from scratch) with four back-propagation methods in Python
python stochastic-gradient-descent back-propagation mlp-regressor levenberg-marquardt-algorithm adaptive-learning-rate emotional-learning
-
Updated
Dec 13, 2021 - Python
Implementation of the WAME (Weight-wise Adaptive learning rates with Moving average Estimator) optimization algorithm for TensorFlow version 2.0 or higher.
-
Updated
Aug 16, 2021 - Python
Improve this page
Add a description, image, and links to the adaptive-learning-rate topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the adaptive-learning-rate topic, visit your repo's landing page and select "manage topics."