DeepLearning.AI

Gradient Descent

8 Posts

Gradient Descent: It’s All Downhill - Gradient Descent for Machine Learning
Gradient Descent

Gradient Descent: It’s All Downhill - Gradient Descent for Machine Learning

Imagine hiking in the mountains past dusk and finding that you can’t see much beyond your feet. And your phone’s battery died so you can’t use a GPS app to find your way home. You might find the quickest path down via gradient descent.

May 25, 20222 min read
Gradient Descent: It’s All Downhill - Gradient Descent for Machine Learning
Gradient Descent

Gradient Descent: It’s All Downhill - Gradient Descent for Machine Learning

Imagine hiking in the mountains past dusk and finding that you can’t see much beyond your feet. And your phone’s battery died so you can’t use a GPS app to find your way home. You might find the quickest path down via gradient descent.

May 25, 20222 min read
Striding Toward the Minimum: A faster way to optimize the loss function for deep learning.
Gradient Descent

Striding Toward the Minimum: A faster way to optimize the loss function for deep learning.

When you’re training a deep learning model, it can take days for an optimization algorithm to minimize the loss function. A new approach could save time.

January 13, 20212 min read
Striding Toward the Minimum: A faster way to optimize the loss function for deep learning.
Gradient Descent

Striding Toward the Minimum: A faster way to optimize the loss function for deep learning.

When you’re training a deep learning model, it can take days for an optimization algorithm to minimize the loss function. A new approach could save time.

January 13, 20212 min read
Optimizer Shootout: An evaluation of 14 deep learning optimizers
Gradient Descent

Optimizer Shootout: An evaluation of 14 deep learning optimizers

Everyone has a favorite optimization method, but it’s not always clear which one works best in a given situation. New research aims to establish a set of benchmarks. Researchers evaluated 14 popular optimizers using the Deep Optimization Benchmark Suite some of them introduced last year.

September 9, 20202 min read
Optimizer Shootout: An evaluation of 14 deep learning optimizers
Gradient Descent

Optimizer Shootout: An evaluation of 14 deep learning optimizers

Everyone has a favorite optimization method, but it’s not always clear which one works best in a given situation. New research aims to establish a set of benchmarks. Researchers evaluated 14 popular optimizers using the Deep Optimization Benchmark Suite some of them introduced last year.

September 9, 20202 min read
When Optimization is Suboptimal: How gradient descent can sometimes lead to model bias
Gradient Descent

When Optimization is Suboptimal: How gradient descent can sometimes lead to model bias

Bias arises in machine learning when we fit an overly simple function to a more complex problem. A theoretical study shows that gradient descent itself may introduce such bias and render algorithms unable to fit data properly.

August 12, 20202 min read
When Optimization is Suboptimal: How gradient descent can sometimes lead to model bias
Gradient Descent

When Optimization is Suboptimal: How gradient descent can sometimes lead to model bias

Bias arises in machine learning when we fit an overly simple function to a more complex problem. A theoretical study shows that gradient descent itself may introduce such bias and render algorithms unable to fit data properly.

August 12, 20202 min read

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox

Choose Your Plan

Planning for more users?
Learn More