github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/colossalai
History
oahzxl 05671fcb42
[autochunk] support multi outputs chunk search (#2538) 
Support multi outputs chunk search. Previously we only support single output chunk search. It is more flexible and improve performance by a large margin. For transformer, we reduce memory by 40% than previous search strategy.

1. rewrite search strategy to support multi outputs chunk search
2. fix many, many bugs
3. update tests
 		2023-02-01 13:18:51 +08:00
..
_C [setup] support pre-build and jit-build of cuda kernels (#2374) 2023-01-06 20:50:26 +08:00
amp [zero] fix gradient clipping in hybrid parallelism (#2521) 2023-01-29 15:09:57 +08:00
auto_parallel [autoparallel] accelerate gpt2 training (#2495) 2023-01-29 11:13:15 +08:00
autochunk [autochunk] support multi outputs chunk search (#2538) 2023-02-01 13:18:51 +08:00
builder [NFC] polish colossalai/builder/__init__.py code style (#1560) 2022-09-08 22:11:04 +08:00
cli [cli] fixed hostname mismatch error (#2465) 2023-01-12 14:52:09 +08:00
communication [NFC] polish communication/p2p_v2.py code style (#2303) 2023-01-04 15:09:57 +08:00
context Revert "Update parallel_context.py (#2408)" 2023-01-19 12:27:48 +08:00
device [autoparallel] accelerate gpt2 training (#2495) 2023-01-29 11:13:15 +08:00
engine [engin/schedule] use p2p_v2 to recontruct pipeline_schedule (#1408) 2022-08-12 11:33:26 +08:00
fx [hotfix] meta tensor default device. (#2510) 2023-01-29 16:28:10 +08:00
gemini [gemini] update ddp strict mode (#2518) 2023-01-28 14:35:25 +08:00
kernel [example] integrate seq-parallel tutorial with CI (#2463) 2023-01-13 14:40:05 +08:00
logging [logger] hotfix, missing _FORMAT (#2231) 2022-12-29 22:59:39 +08:00
nn [hotfix] fix lightning error (#2529) 2023-01-31 10:40:39 +08:00
pipeline polish pp middleware (#2476) 2023-01-13 16:56:01 +08:00
registry Remove duplication registry (#1078) 2022-06-08 07:47:24 +08:00
tensor [autoparallel] accelerate gpt2 training (#2495) 2023-01-29 11:13:15 +08:00
testing [amp] add gradient clipping for unit tests (#2283) 2023-01-04 11:59:56 +08:00
trainer [polish] remove useless file _mem_tracer_hook.py (#1963) 2022-11-16 15:55:10 +08:00
utils [utils] lazy init. (#2148) 2023-01-20 10:49:00 +08:00
zero [zero] add zero wrappers (#2523) 2023-01-29 17:52:58 +08:00
__init__.py [setup] supported conda-installed torch (#2048) 2022-11-30 16:45:15 +08:00
constants.py updated tp layers 2022-11-02 12:19:38 +08:00
core.py [Tensor] distributed view supports inter-process hybrid parallel (#1169) 2022-06-27 09:45:26 +08:00
global_variables.py updated tp layers 2022-11-02 12:19:38 +08:00
initialize.py Fix False warning in initialize.py (#2456) 2023-01-12 13:49:01 +08:00