Parallel Training
=================

.. 整体说一下并行配置使用方式，接下来再分模块详细说明

Tensor Parallel
-----------------


Pipeline Parallel
-----------------


Sequence Parallel
-----------------


Data Parallel
-----------------


ZeRO1.5
-----------------