DeepSpeed
DeepSpeed' is an open-source optimization library for the distributed training and inference of deep learning models using PyTorch. == Library == The library is designed to reduce computing power and memory use and to train large distributed models with better parallelism on existing computer hardware.