github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,019 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

1019 Commits (f1e1836218a4ecc6e8acf85c7031492cc4e6b590)

Author SHA1 Message Date
ver217 c5cd3b0f35
[zero] zero optim copy chunk rather than copy tensor (#1070) 2022-06-07 10:30:46 +08:00
Ziyue Jiang 4fc748f69b
[Tensor] fix optimizer for CPU parallel (#1069) 2022-06-06 17:36:11 +08:00
Jiarui Fang 49832b2344
[refactory] add nn.parallel module (#1068) 2022-06-06 15:34:41 +08:00
Ziyue Jiang 6754f1b77f
fix module utils bug (#1066) 2022-06-06 12:11:48 +08:00
Jiarui Fang a00644079e
reorgnize colotensor directory (#1062) 
* reorgnize colotensor directory

* polish code
 2022-06-03 18:04:22 +08:00
Frank Lee 3d10be33bd
[cudnn] set False to cudnn benchmark by default (#1063) 2022-06-03 17:58:06 +08:00
Ziyue Jiang df9dcbbff6
[Tensor] add hybrid device demo and fix bugs (#1059) 2022-06-03 12:09:49 +08:00
YuliangLiu0306 b167258b6a
[pipeline]refactor ppschedule to support tensor list (#1050) 
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4.

* refactor ppschedule to support tensor list

* polish
 2022-06-02 13:48:59 +08:00
ver217 e3fde4ee6b
fix import error in sharded model v2 (#1053) 2022-06-02 13:48:22 +08:00
ver217 e1922ea4f6
[zero] add chunk size search for chunk manager (#1052) 2022-06-02 13:20:20 +08:00
アマデウス 2c42b230f3
updated collective ops api (#1054) 2022-06-02 12:52:27 +08:00
ver217 51b9a49655
[zero] add zero optimizer for ColoTensor (#1046) 
* add zero optimizer

* torch ok

* unit test ok

* polish code

* fix bugs

* polish unit test

* polish zero optim

* polish colo ddp v2

* refactor folder structure

* add comment

* polish unit test

* polish zero optim

* polish unit test
 2022-06-02 12:13:15 +08:00
github-actions[bot] e32470b6de
Automated submodule synchronization (#1049) 
Co-authored-by: github-actions <github-actions@github.com>
 2022-06-01 11:04:32 +08:00
Frank Lee 0ccb8c6141
[release] update version.txt (#1048) 2022-05-31 22:14:14 +08:00
binmakeswell 626dd187e4
add inference submodule (#1047) 2022-05-31 19:57:39 +08:00
ver217 7faef93326
fix dist spec mgr (#1045) 2022-05-31 12:14:39 +08:00
ver217 9492a561c3
[tensor] ColoTensor supports ZeRo (#1015) 
* impl chunk manager

* impl param op hook

* add reduce_chunk

* add zero hook v2

* add zero dp

* fix TensorInfo

* impl load balancing when using zero without chunk

* fix zero hook

* polish chunk

* fix bugs

* ddp ok

* zero ok

* polish code

* fix bugs about load balancing

* polish code

* polish code

* add ene-to-end test

* polish code

* polish code

* polish code

* fix typo

* add test_chunk

* fix bugs

* fix bugs

* polish code
 2022-05-31 12:00:12 +08:00
Frank Lee cfa6c1b46b
[ci] fixed nightly build workflow (#1040) 2022-05-31 10:43:18 +08:00
YuliangLiu0306 9feff0f760
[titans]remove model zoo (#1042) 
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4.

* rm model zoo
 2022-05-31 10:40:47 +08:00
binmakeswell 0dac86866b
[NFC] add inference (#1044) 2022-05-30 23:06:49 +08:00
Ziyue Jiang 7c530b9de2
[Tensor] add Parameter inheritance for ColoParameter (#1041) 
* add Parameter inheritance for ColoParameter

* remove tricks

* remove tricks

* polish

* polish
 2022-05-30 17:23:44 +08:00
github-actions[bot] 4d8a574cd3
Automated submodule synchronization (#1034) 
Co-authored-by: github-actions <github-actions@github.com>
 2022-05-27 17:12:48 +08:00
ver217 7cfd6c827e
[zero] add load_state_dict for sharded model (#894) 
* add load_state_dict for sharded model

* fix bug

* fix bug

* fix ckpt dtype and device

* support load state dict in zero init ctx

* fix bugs
 2022-05-27 10:25:08 +08:00
Ziyue Jiang 6c5996a56e
[Tensor] add module check and bert test (#1031) 
* add Embedding

* Add bert test

* polish

* add check module test

* polish

* polish

* polish

* polish
 2022-05-26 18:15:42 +08:00
YuliangLiu0306 7106bd671d
[p2p]add object list send/recv (#1024) 
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4.

* [p2p]add object list send recv

* refactor for code reusability

* polish
 2022-05-26 14:28:46 +08:00
Frank Lee e4685832f8
[engine] fixed bug in gradient accumulation dataloader to keep the last step (#1030) 2022-05-26 14:28:23 +08:00
Ziyue Jiang 32291dd73f
[Tensor] add module handler for linear (#1021) 
* add module spec for linear

* polish

* polish

* polish
 2022-05-26 11:50:44 +08:00
Frank Lee ee50497db2
[ci] fixed nightly build workflow (#1029) 2022-05-26 11:42:50 +08:00
Ryan Russell 9b0c037027
fix typo in constants (#1027) 2022-05-26 08:45:08 +08:00
ver217 007ca0df92
fix colo init context (#1026) 2022-05-25 20:41:58 +08:00
Frank Lee 58a7dd2ede
[ci] fixed nightly build workflow (#1022) 
* [ci] fixed nightly build workflow

* [ci] fixed nightly build workflow

* [ci] fixed nightly build workflow
 2022-05-24 22:38:56 +08:00
Frank Lee 1a76c88aba
[ci] added nightly build (#1018) (#1019) 2022-05-24 17:56:01 +08:00
Frank Lee 8d06186ff9
[doc] update docker instruction (#1020) 2022-05-24 17:51:50 +08:00
Frank Lee e17a43184b
[ci] update the docker image name (#1017) 2022-05-24 16:53:39 +08:00
YuliangLiu0306 d182b0bd47
[hotfix] fix some bugs caused by size mismatch. (#1011) 
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4.

* [hotfix]fix some bugs caused by size mismatch.

* add warning logs

* polish
 2022-05-23 14:02:28 +08:00
binmakeswell 9833d814d5 [NFC] fix paper link 2022-05-21 18:34:36 +08:00
ver217 cefc29ff06
[tensor] impl ColoDDP for ColoTensor (#1009) 
* impl ColoDDP for ColoTensor

* polish code
 2022-05-21 13:52:04 +08:00
zhengzangw ae7c338105 [NFC] polish colossalai/kernel/cuda_native/csrc/colossal_C_frontend.cpp code style 2022-05-20 23:57:38 +08:00
ver217 a3b66f6def
[tensor] refactor parallel action (#1007) 
* refactor parallel action

* polish unit tests
 2022-05-20 20:19:58 +08:00
github-actions[bot] 9e3d602dba
Automated submodule synchronization (#1003) 
Co-authored-by: github-actions <github-actions@github.com>
 2022-05-20 17:08:44 +08:00
ver217 8e3d0ad8f1
[unit test] refactor test tensor (#1005) 
* polish test_gpt

* update op unit tests

* update test model
 2022-05-19 18:57:56 +08:00
ver217 ad536e308e
[tensor] refactor colo-tensor (#992) 
* refactor colo-tensor and update linear op

* polish code

* polish code

* update ops and unit tests

* update unit tests

* polish code

* rename dist_spec module

* polish code

* polish code

* remove unneeded import

* fix pipelinable
 2022-05-19 12:44:59 +08:00
Frank Lee 1467d83edf
[cli] remove unused imports (#1001) 2022-05-18 23:27:18 +08:00
Frank Lee 533d0c46d8
[kernel] fixed the include bug in dropout kernel (#999) 2022-05-18 21:43:18 +08:00
binmakeswell c27ea0d980
fix download link (#998) 2022-05-18 18:05:18 +08:00
Jiarui Fang 802ac297cc
[Tensor] remove useless import in tensor dir (#997) 2022-05-18 14:54:51 +08:00
Ziheng Qin 571f12eff3 [NFC] polish colossalai/nn/layer/utils/common.py code style (#983) 2022-05-17 10:25:06 +08:00
puck_WCR bda70b4b66 [NFC] polish colossalai/kernel/cuda_native/layer_norm.py code style (#980) 2022-05-17 10:25:06 +08:00
Kai Wang (Victor Kai) c50c08dcbb [NFC] polish colossalai/kernel/cuda_native/csrc/kernels/dropout_kernels.cu code style (#979) 2022-05-17 10:25:06 +08:00
binmakeswell f28c021376 [NFC] polish colossalai/kernel/cuda_native/csrc/multi_tensor_sgd_kernel.cu code style (#978) 2022-05-17 10:25:06 +08:00