InternLM/doc/code-docs/source/parallel.rst

23 lines
288 B
ReStructuredText

Parallel Training
=================
.. 整体说一下并行配置使用方式,接下来再分模块详细说明
Tensor Parallel
-----------------
Pipeline Parallel
-----------------
Sequence Parallel
-----------------
Data Parallel
-----------------
ZeRO1.5
-----------------