github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
826 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

42 Commits (c95e18cdb9ad8788cf5145593a4d9d6d2e647082)

Author SHA1 Message Date
Jiarui Fang 79fe7b027a
[hotfix] test model unittest hotfix (#1281) 2022-07-12 23:45:29 +08:00
HELSON abba4d84e1
[hotfix] fix bert model test in unitests (#1272) 2022-07-12 23:26:45 +08:00
Jiarui Fang 9bcd2fd4af
[tensor] a shorter shard and replicate spec (#1245) 2022-07-11 15:51:48 +08:00
Jiarui Fang 3b500984b1
[tensor] fix some unittests (#1234) 2022-07-08 14:18:30 +08:00
Jiarui Fang 0e199d71e8
[hotfix] fx get comm size bugs (#1233) 
* init a checkpoint dir

* [checkpoint]support resume for cosinewarmuplr

* [checkpoint]add unit test

* fix some bugs but still not OK

* fix bugs

* make it faster

* [checkpoint]support generalized scheduler

* polish

* [tensor] torch function return colotensor

* polish

* fix bugs

* remove debug info

* polish

* polish

* [tensor] test_model pass unittests

* polish

* [hotfix] fx get comm size bug

Co-authored-by: ZhaoYi1222 <zhaoyi9499@gmail.com>
 2022-07-08 10:54:41 +08:00
Jiarui Fang a98319f023
[tensor] torch function return colotensor (#1229) 2022-07-07 18:09:18 +08:00
Jiarui Fang ae7d3f4927
[refactor] move process group from _DistSpec to ColoTensor. (#1203) 2022-07-06 16:15:16 +08:00
Jiarui Fang 060b917daf
[refactor] remove gpc dependency in colotensor's _ops (#1189) 2022-07-04 18:54:37 +08:00
Jiarui Fang c463f8adf9
[tensor] remove gpc in tensor tests (#1186) 2022-06-29 14:08:40 +08:00
Jiarui Fang 7487215b95
[ColoTensor] add independent process group (#1179) 2022-06-29 10:03:09 +08:00
Jiarui Fang 4b9bba8116
[ColoTensor] rename APIs and add output_replicate to ComputeSpec (#1168) 2022-06-24 13:08:54 +08:00
Jiarui Fang f4ef224358
[Tensor] remove ParallelAction, use ComputeSpec instread (#1166) 2022-06-23 17:34:59 +08:00
Jiarui Fang 177c374401
remove gather out in parallel action (#1163) 2022-06-23 16:35:05 +08:00
Ziyue Jiang b3a03e4bfd
[Tensor] fix equal assert (#1091) 
* fix equal assert

* polish
 2022-06-09 17:36:15 +08:00
Frank Lee 65ee6dcc20
[test] ignore 8 gpu test (#1080) 
* [test] ignore 8 gpu test

* polish code

* polish workflow

* polish workflow
 2022-06-08 23:14:18 +08:00
Jiarui Fang 49832b2344
[refactory] add nn.parallel module (#1068) 2022-06-06 15:34:41 +08:00
Jiarui Fang a00644079e
reorgnize colotensor directory (#1062) 
* reorgnize colotensor directory

* polish code
 2022-06-03 18:04:22 +08:00
Ziyue Jiang 7c530b9de2
[Tensor] add Parameter inheritance for ColoParameter (#1041) 
* add Parameter inheritance for ColoParameter

* remove tricks

* remove tricks

* polish

* polish
 2022-05-30 17:23:44 +08:00
ver217 a3b66f6def
[tensor] refactor parallel action (#1007) 
* refactor parallel action

* polish unit tests
 2022-05-20 20:19:58 +08:00
ver217 8e3d0ad8f1
[unit test] refactor test tensor (#1005) 
* polish test_gpt

* update op unit tests

* update test model
 2022-05-19 18:57:56 +08:00
ver217 ad536e308e
[tensor] refactor colo-tensor (#992) 
* refactor colo-tensor and update linear op

* polish code

* polish code

* update ops and unit tests

* update unit tests

* polish code

* rename dist_spec module

* polish code

* polish code

* remove unneeded import

* fix pipelinable
 2022-05-19 12:44:59 +08:00
ver217 c2fdc6a011
[tensor] derive compute pattern from dist spec (#971) 
* derive compute pattern from dist spec

* polish code
 2022-05-16 14:58:08 +08:00
Ziyue Jiang 797a9dc5a9
add DistSpec for loss and test_model (#947) 2022-05-13 20:29:50 +08:00
ver217 67c33f57eb
[tensor] design DistSpec and DistSpecManager for ColoTensor (#934) 
* add dist spec

* update linear op

* polish code

* polish code

* update embedding op

* polish unit tests

* polish unit tests

* polish comments

* polish code

* add test_dist_spec_mgr

* polish code

* refactor folder structure

* polish unit tests

* add get_process_group() for TensorSpec

* polish code
 2022-05-13 15:13:52 +08:00
Ziyue Jiang 830d3bca26
[Tensor] add optimizer to bert test (#933) 
* add optimizer to bert test

* polish
 2022-05-13 11:37:23 +08:00
Ziyue Jiang d73c2b1d79
[Tensor] fix init context (#931) 
* change torch.Parameter to ColoParameter

* fix post assignment for init context

* polish

* polish
 2022-05-11 15:48:12 +08:00
Ziyue Jiang dfc88b85ea
[Tensor] simplify named param (#928) 
* simplify ColoModulize

* simplify ColoModulize

* polish

* polish
 2022-05-11 10:54:19 +08:00
Jiarui Fang 534afb018a
test pretrain loading on multi-process (#922) 2022-05-09 17:07:35 +08:00
Ziyue Jiang c195d2814c
[Tensor] add from_pretrained support and bert pretrained test (#921) 
* add from_pretrained support and test

* polish

* polish

* polish

* polish
 2022-05-09 16:11:47 +08:00
Jiarui Fang 845856ea29
[Graph] building computing graph with ColoTensor, Linear only (#917) 2022-05-07 17:10:37 +08:00
Ziyue Jiang 75d221918a
[Tensor] add 1d vocab loss (#918) 
* add 1d vocab loss

* polish
 2022-05-07 15:49:14 +08:00
Ziyue Jiang dfaff4e243
[Tensor] fix test_model (#916) 
* polish test_model

* polish
 2022-05-06 18:06:22 +08:00
Jiarui Fang ed6426c300
[Tensor] polish model test (#915) 2022-05-06 17:07:56 +08:00
Ziyue Jiang 0fab86b12a
[Tensor] add a basic bert. (#911) 
* add base bert test

* Add bert test

* polish

* remove test_bert

* polish
 2022-05-06 15:03:43 +08:00
Jiarui Fang ab95ec9aea
[Tensor] init ColoParameter (#914) 2022-05-06 12:57:14 +08:00
Ziyue Jiang f593a5637e
[Tensor] add embedding tp1d row (#904) 2022-04-29 14:10:05 +08:00
Ziyue Jiang 2c0d19d755
[Tensor] add ColoTensor TP1Dcol Embedding (#899) 2022-04-28 17:45:06 +08:00
Jiarui Fang d16671da75
[Tensor] initialize the ColoOptimizer (#898) 
* [Tensor] activation is an attr of ColoTensor

* [Tensor] add optimizer

* only detach parameters in context

* polish code
 2022-04-28 15:23:40 +08:00
Jiarui Fang e76f76c08b
[Tensor] test parameters() as member function (#896) 2022-04-28 10:57:14 +08:00
Ziyue Jiang cb182da7c5
[tensor] refine linear and add gather for laynorm (#893) 
* refine linear and add function to ColoTensor

* add gather for layernorm

* polish

* polish
 2022-04-28 10:55:40 +08:00
Jiarui Fang 26c49639d8
[Tensor] overriding paramters() for Module using ColoTensor (#889) 2022-04-27 15:28:59 +08:00
Jiarui Fang a0e5971692
[Tensor] test model check results for a simple net (#887) 2022-04-27 12:00:18 +08:00