A real infra-loop question that candidates fumble by guessing. The correct map, TP on NVLink, PP and DP across the scale-out fabric, falls out of three traffic profiles, and the reasoning is the answer.
more free answers with an account · no card
← ML Infrastructure & GPUs / 12
hardxAIOpenAINVIDIA
Pipeline parallelism, does it run over the scale-out network or stay inside the node? Map TP, PP, and DP onto the fabric.
A real infra-loop question that candidates fumble by guessing. The correct map, TP on NVLink, PP and DP across the scale-out fabric, falls out of three traffic profiles, and the reasoning is the answer.
Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.
UP NEXT ON YOUR JOURNEY
Next in this trackWhat does NCCL actually do, and why can GPU utilization read 100% while the job is communication-bound?Next in this trackInfiniBand vs RoCE for a GPU training cluster, how do you choose?Next in this trackHow do you handle expert load balancing in an MoE model? What happens if 90% of tokens route to 10% of experts?
DISCUSSION · 0
No comments yet — be the first to share your approach.