18Your Spark join is skewed, 199 tasks finish in seconds, one runs for an hour. Fix it.▼hardDatabricksPalantirMicrosoft2 replies○ sign inThe Databricks signature deep-dive. Everyone says 'salting', almost nobody can write the salt correctly or explain why AQE might make hand-salting unnecessary. The full playbook, in the order a senior engineer actually tries it.Open full answer →
41Join two petabyte fact tables where both are skewed on the same key. Neither fits in memory. Make it finish.▼hardDatabricksPalantirSnowflake1 replies◆ premiumBasic salting assumes one side is a small dimension you can explode. When both sides are huge and skewed, exploding either one blows up the cluster. The staff move is to split hot keys out and handle them on a different plan entirely.Open full answer →