ColossalAI/colossalai
HELSON 2d1a7dfe5f
[zero] add strict ddp mode (#2508)
* [zero] add strict ddp mode

* [polish] add comments for strict ddp mode

* [zero] fix test error
2023-01-20 14:04:38 +08:00
..
_C [setup] support pre-build and jit-build of cuda kernels (#2374) 2023-01-06 20:50:26 +08:00
amp [setup] support pre-build and jit-build of cuda kernels (#2374) 2023-01-06 20:50:26 +08:00
auto_parallel [autoparallel] support origin activation ckpt on autoprallel system (#2468) 2023-01-16 16:25:13 +08:00
autochunk [autochunk] support parsing blocks (#2506) 2023-01-20 11:18:17 +08:00
builder
cli [cli] fixed hostname mismatch error (#2465) 2023-01-12 14:52:09 +08:00
communication [NFC] polish communication/p2p_v2.py code style (#2303) 2023-01-04 15:09:57 +08:00
context Revert "Update parallel_context.py (#2408)" 2023-01-19 12:27:48 +08:00
device [autoparallel] integrate device mesh initialization into autoparallelize (#2393) 2023-01-11 14:03:49 +08:00
engine
fx [utils] lazy init. (#2148) 2023-01-20 10:49:00 +08:00
gemini [zero] add warning for ignored parameters (#2446) 2023-01-11 15:30:09 +08:00
kernel [example] integrate seq-parallel tutorial with CI (#2463) 2023-01-13 14:40:05 +08:00
logging [logger] hotfix, missing _FORMAT (#2231) 2022-12-29 22:59:39 +08:00
nn [zero] add strict ddp mode (#2508) 2023-01-20 14:04:38 +08:00
pipeline polish pp middleware (#2476) 2023-01-13 16:56:01 +08:00
registry
tensor [hotfix] fix implement error in diffusers 2023-01-07 07:56:39 +08:00
testing [amp] add gradient clipping for unit tests (#2283) 2023-01-04 11:59:56 +08:00
trainer [polish] remove useless file _mem_tracer_hook.py (#1963) 2022-11-16 15:55:10 +08:00
utils [utils] lazy init. (#2148) 2023-01-20 10:49:00 +08:00
zero [zero] add unit testings for hybrid parallelism (#2486) 2023-01-18 10:36:10 +08:00
__init__.py [setup] supported conda-installed torch (#2048) 2022-11-30 16:45:15 +08:00
constants.py updated tp layers 2022-11-02 12:19:38 +08:00
core.py
global_variables.py updated tp layers 2022-11-02 12:19:38 +08:00
initialize.py Fix False warning in initialize.py (#2456) 2023-01-12 13:49:01 +08:00