FSDP2 TP CP FP8 pretraining with DCP resume and MFU tracking
-
Updated
Nov 15, 2026 - Python
FSDP2 TP CP FP8 pretraining with DCP resume and MFU tracking
Compare ZeRO-3-style sharding and FSDP2 with per-rank memory and step-time timelines
FSDP2 tensor/context-parallel FP8 pretraining with DCP resume and packed data
To associate your repository with the fsdp2 topic, visit your repo's landing page and select "manage topics."