github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
579 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

579 Commits (7edb38193a0fab158186bb16bbb27ef0e0a36a03)

Author SHA1 Message Date
github-actions[bot] 7edb38193a
Automated submodule synchronization (#932) 
Co-authored-by: github-actions <github-actions@github.com>
 2022-05-13 10:22:51 +08:00
Ziyue Jiang d73c2b1d79
[Tensor] fix init context (#931) 
* change torch.Parameter to ColoParameter

* fix post assignment for init context

* polish

* polish
 2022-05-11 15:48:12 +08:00
Ziyue Jiang dfc88b85ea
[Tensor] simplify named param (#928) 
* simplify ColoModulize

* simplify ColoModulize

* polish

* polish
 2022-05-11 10:54:19 +08:00
YuliangLiu0306 32a45cd7ef
[pipelinable]use pipelinable to support GPT model. (#903) 
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4.

* [pipelinable]use pipelinable to support GPT model.

* fix a bug caused by ShardedModel

* polish

* fix front func list
 2022-05-11 09:23:58 +08:00
github-actions[bot] b61d64685f
Automated submodule synchronization (#929) 
Co-authored-by: github-actions <github-actions@github.com>
 2022-05-11 09:13:06 +08:00
ver217 4ca732349e
[tensor] colo tensor overrides mul (#927) 
* colo tensor overrides mul

* polish code
 2022-05-10 16:04:08 +08:00
ver217 45b9124df4
[tensor] hijack addmm for colo tensor (#923) 
* hijack addmm for colo tensor

* fix bugs

* polish unit test

* polish comments
 2022-05-09 18:55:49 +08:00
Jiarui Fang 534afb018a
test pretrain loading on multi-process (#922) 2022-05-09 17:07:35 +08:00
Ziyue Jiang c195d2814c
[Tensor] add from_pretrained support and bert pretrained test (#921) 
* add from_pretrained support and test

* polish

* polish

* polish

* polish
 2022-05-09 16:11:47 +08:00
ver217 1d625fcd36
[setup] support more cuda architectures (#920) 
* support more cuda archs

* polish code
 2022-05-09 10:56:45 +08:00
ver217 5d8f1262fb
update cuda ext cc flags (#919) 2022-05-07 18:01:04 +08:00
Jiarui Fang 845856ea29
[Graph] building computing graph with ColoTensor, Linear only (#917) 2022-05-07 17:10:37 +08:00
Ziyue Jiang 75d221918a
[Tensor] add 1d vocab loss (#918) 
* add 1d vocab loss

* polish
 2022-05-07 15:49:14 +08:00
Ziyue Jiang dfaff4e243
[Tensor] fix test_model (#916) 
* polish test_model

* polish
 2022-05-06 18:06:22 +08:00
Jiarui Fang ed6426c300
[Tensor] polish model test (#915) 2022-05-06 17:07:56 +08:00
Ziyue Jiang 0fab86b12a
[Tensor] add a basic bert. (#911) 
* add base bert test

* Add bert test

* polish

* remove test_bert

* polish
 2022-05-06 15:03:43 +08:00
Jiarui Fang ab95ec9aea
[Tensor] init ColoParameter (#914) 2022-05-06 12:57:14 +08:00
Ziyue Jiang 193d629311
update pytest.mark.parametrize in tensor tests (#913) 2022-05-06 11:16:40 +08:00
github-actions[bot] 1cf7fb3cd9
Automated submodule synchronization (#912) 
Co-authored-by: github-actions <github-actions@github.com>
 2022-05-06 10:10:56 +08:00
Frank Lee f0f35216f1
[ci] added wheel build scripts (#910) 
* [ci] added wheel build scripts

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* polish code and workflow

* [ci] polish wheel build scripts
 2022-05-05 16:06:39 +08:00
ver217 150b1a7453
update local version format (#909) 2022-05-05 14:59:12 +08:00
github-actions[bot] 3b1f5f07ce
Automated submodule synchronization (#907) 
Co-authored-by: github-actions <github-actions@github.com>
 2022-05-03 13:14:48 +08:00
Ziyue Jiang f593a5637e
[Tensor] add embedding tp1d row (#904) 2022-04-29 14:10:05 +08:00
ver217 16122d5fac
update release bdist CI (#902) 2022-04-28 17:52:57 +08:00
Ziyue Jiang 2c0d19d755
[Tensor] add ColoTensor TP1Dcol Embedding (#899) 2022-04-28 17:45:06 +08:00
ver217 e46e423c00
add CI for releasing bdist wheel (#901) 2022-04-28 17:40:53 +08:00
Jiarui Fang e1108caf7d
change version to 0.1.4 (#900) 2022-04-28 15:51:25 +08:00
Jiarui Fang d16671da75
[Tensor] initialize the ColoOptimizer (#898) 
* [Tensor] activation is an attr of ColoTensor

* [Tensor] add optimizer

* only detach parameters in context

* polish code
 2022-04-28 15:23:40 +08:00
Jiarui Fang 676f191532
[Tensor] activation is an attr of ColoTensor (#897) 2022-04-28 14:43:22 +08:00
Jiarui Fang e76f76c08b
[Tensor] test parameters() as member function (#896) 2022-04-28 10:57:14 +08:00
Ziyue Jiang cb182da7c5
[tensor] refine linear and add gather for laynorm (#893) 
* refine linear and add function to ColoTensor

* add gather for layernorm

* polish

* polish
 2022-04-28 10:55:40 +08:00
Jiarui Fang 26c49639d8
[Tensor] overriding paramters() for Module using ColoTensor (#889) 2022-04-27 15:28:59 +08:00
ver217 daf59ff72e
[setup] add local version label (#890) 2022-04-27 15:26:12 +08:00
Ziyue Jiang 1d0aba4153
[tensor] add ColoTensor 1Dcol (#888) 2022-04-27 14:13:55 +08:00
Jiarui Fang a0e5971692
[Tensor] test model check results for a simple net (#887) 2022-04-27 12:00:18 +08:00
Jiarui Fang 72cdc06875
[Tensor] make ColoTensor more robust for getattr (#886) 
* [Tensor] make ColoTensor more robust for getattr

* polish

* polish
 2022-04-27 10:57:49 +08:00
Ziyue Jiang 9bc5a77c31
[tensor] wrap function in the torch_tensor to ColoTensor (#881) 2022-04-26 20:13:56 +08:00
ver217 4df6471f5d
fix import error (#880) 2022-04-26 19:28:40 +08:00
Jiarui Fang 7f76517a85
[Tensor] make a simple net works with 1D row TP (#879) 2022-04-26 18:11:47 +08:00
ver217 c4d903e64a
[gemini] accelerate adjust_layout() (#878) 
* add lru cache

* polish code

* update unit test

* fix sharded optim
 2022-04-26 18:08:31 +08:00
Jiarui Fang 909211453b
[Tensor] Add some attributes to ColoTensor (#877) 
* [Tensor] add some function to ColoTensor

* torch.allclose

* rm torch.add
 2022-04-26 15:10:47 +08:00
HELSON 425b4a96b8
[gemini] polish stateful_tensor_mgr (#876) 2022-04-26 15:05:03 +08:00
Jiarui Fang e43f83aa5c
[Tensor] get named parameters for model using ColoTensors (#874) 2022-04-26 14:08:01 +08:00
LuGY 2883040286
[example] change qkv processing (#870) 2022-04-26 13:33:27 +08:00
Jiarui Fang 96211c2cc8
[tensor] customized op returns ColoTensor (#875) 
* [tensor] customized op returns ColoTensor

* polish

* polish code
 2022-04-26 13:23:59 +08:00
Ziyue Jiang 26d4ab8b03
[Tensor] Add function to spec and update linear 1Drow and unit tests (#869) 2022-04-26 10:15:26 +08:00
Frank Lee 11f54c7b6b
[doc] improved docstring and assertion messages for the engine module (#871) 2022-04-26 10:00:18 +08:00
Frank Lee 1c34382678
[doc] improved assertion messages in trainer (#873) 2022-04-26 10:00:12 +08:00
Frank Lee 7a64fae33a
[doc] improved error messages in initialize (#872) 2022-04-26 10:00:03 +08:00
Jiarui Fang 1190b2c4a4
[tensor] add cross_entrophy_loss (#868) 2022-04-25 16:01:52 +08:00