Optimizer

It determines how the model's weights are updated to minimize the loss function. It drives the learning process, bridging the gap between the loss function and the model's predictive ability.

What does the Optimizer do?

1. Minimizes the loss function
2. Updates weights via Gradient Descent
3. Navigates the Loss Landscape

How it works

1. Compute Gradients
2. Apply Update Rule
3. Iterate

Optimizer's impact on Prediction Ability

1. Learning Quality
2. Generalization
3. Speed vs Accuracy Trade-Off