319KiB, 2560x1441, 1400x788_deepspeed_update_figure_nologo_Still-1-scaled.jpg
https://www.microsoft.com/en-us/research/blog/zero-infinity-and-deepspeed-unlocking-unprecedented-model-scale-for-deep-learning-training/
>Microsoft's ZeRO-Infinity can now run a model with over a trillion parameters on a single NVIDIA DGX-2 node and over 30 trillion parameters on 32 nodes (512 GPUs). With a hundred DGX-2 nodes in a cluster, Microsoft projects ZeRO-Infinity can train models with over a hundred trillion parameters
AGI INCOMING STAND ASIDE NIGGERS
>Microsoft's ZeRO-Infinity can now run a model with over a trillion parameters on a single NVIDIA DGX-2 node and over 30 trillion parameters on 32 nodes (512 GPUs). With a hundred DGX-2 nodes in a cluster, Microsoft projects ZeRO-Infinity can train models with over a hundred trillion parameters
AGI INCOMING STAND ASIDE NIGGERS
