early stopping

The validation set is the one you tune and select on; the test set is the one you only look at once. Confusing them, or tuning so hard you overfit the validation set, is how a model that aced offline metrics dies in production.

Gradient descent is walking downhill on the loss surface, one step proportional to the slope. Knowing when to stop is the part that separates people who have trained models from people who have read about them: it is the validation curve, not the training curve, that tells you.

Grid search is the answer that signals you have never paid for compute. The efficient answer is random or Bayesian over a small set of parameters that actually move the metric, with Hyperband killing bad runs early, on a validation set you never let leak.