← 🗄️ Data & SQL Engineering
Advanced
Spark Internals and Performance Tuning
Spark is fast until a shuffle, and almost every Spark performance problem is a shuffle problem wearing a disguise. Knowing which transformations force data across the network, and how to spot skew in the stage view, is the difference between a job that finishes in four minutes and the same job running for an hour.
Unlock the full curriculum — ₹2,000 / $25every concept + every answer · 6 months · no auto-renew
RELATED CONCEPTS
LESSONS THAT TEACH THIS
PRACTICE THIS IN REAL QUESTIONS
ML Infrastructure & GPUsWhat is occupancy, and how do you balance it against register and shared-memory usage when choosing block size?→ML Infrastructure & GPUsHow do you determine whether a kernel is memory-bound or compute-bound?→System Design & Production EngineeringA customer's Next.js site has a 3-second TTFB. Diagnose it and walk me through fixes, and how you'd prove each one worked.→SQL & Data EngineeringA customer's query on a 2-billion-row table takes 40 minutes. Walk me through your diagnosis.→SQL & Data EngineeringExplain Snowflake micro-partitions and clustering, and when a bigger warehouse will NOT make a query faster.→MLOps & ML EngineeringWhat is training-serving skew, and how do you keep online and offline features consistent?→