Core
Convex vs Non-Convex Optimization
A convex loss has one global minimum, so gradient descent from anywhere reaches the best solution: this is why logistic regression, linear regression, and linear SVMs are reliable to train. Deep nets are non-convex, with many local minima and far more saddle points, yet SGD still finds good solutions. Understanding why changes how you set initialization, learning rate, and restarts.
a free account unlocks the core curriculum tier · no card
Related concepts
Practise this in questions
Machine Learning & Data ScienceWalk me through SGD vs mini-batch vs Adam, learning-rate schedules, and what vanishing or exploding gradients look like in practice. →medium · PremiumML Infrastructure & GPUsExplain memory coalescing and shared-memory bank conflicts. How would you fix a kernel that has both? →medium · FreeMachine Learning & Data ScienceExplain gradient descent to me like I'm a smart engineer who's never done ML. What can go wrong? →easy · FreeLLM & GenAI FundamentalsYour prompt plus retrieved documents exceed the context window. What are your options and tradeoffs? →medium · Free with sign-inLLM & GenAI FundamentalsA customer says your LLM app is too slow. Give me five levers to reduce latency, and their tradeoffs. →medium · Free with sign-inSystem Design & Production EngineeringDecompose: design a system to route a fleet of delivery trucks. →hard · Free with sign-in