Jiarui Fang
|
49832b2344
|
[refactory] add nn.parallel module (#1068)
|
2022-06-06 15:34:41 +08:00 |
Jiarui Fang
|
a00644079e
|
reorgnize colotensor directory (#1062)
* reorgnize colotensor directory
* polish code
|
2022-06-03 18:04:22 +08:00 |
Ziyue Jiang
|
7c530b9de2
|
[Tensor] add Parameter inheritance for ColoParameter (#1041)
* add Parameter inheritance for ColoParameter
* remove tricks
* remove tricks
* polish
* polish
|
2022-05-30 17:23:44 +08:00 |
ver217
|
a3b66f6def
|
[tensor] refactor parallel action (#1007)
* refactor parallel action
* polish unit tests
|
2022-05-20 20:19:58 +08:00 |
ver217
|
8e3d0ad8f1
|
[unit test] refactor test tensor (#1005)
* polish test_gpt
* update op unit tests
* update test model
|
2022-05-19 18:57:56 +08:00 |
ver217
|
ad536e308e
|
[tensor] refactor colo-tensor (#992)
* refactor colo-tensor and update linear op
* polish code
* polish code
* update ops and unit tests
* update unit tests
* polish code
* rename dist_spec module
* polish code
* polish code
* remove unneeded import
* fix pipelinable
|
2022-05-19 12:44:59 +08:00 |
ver217
|
c2fdc6a011
|
[tensor] derive compute pattern from dist spec (#971)
* derive compute pattern from dist spec
* polish code
|
2022-05-16 14:58:08 +08:00 |
Ziyue Jiang
|
797a9dc5a9
|
add DistSpec for loss and test_model (#947)
|
2022-05-13 20:29:50 +08:00 |
ver217
|
67c33f57eb
|
[tensor] design DistSpec and DistSpecManager for ColoTensor (#934)
* add dist spec
* update linear op
* polish code
* polish code
* update embedding op
* polish unit tests
* polish unit tests
* polish comments
* polish code
* add test_dist_spec_mgr
* polish code
* refactor folder structure
* polish unit tests
* add get_process_group() for TensorSpec
* polish code
|
2022-05-13 15:13:52 +08:00 |
Ziyue Jiang
|
830d3bca26
|
[Tensor] add optimizer to bert test (#933)
* add optimizer to bert test
* polish
|
2022-05-13 11:37:23 +08:00 |
Ziyue Jiang
|
d73c2b1d79
|
[Tensor] fix init context (#931)
* change torch.Parameter to ColoParameter
* fix post assignment for init context
* polish
* polish
|
2022-05-11 15:48:12 +08:00 |
Ziyue Jiang
|
dfc88b85ea
|
[Tensor] simplify named param (#928)
* simplify ColoModulize
* simplify ColoModulize
* polish
* polish
|
2022-05-11 10:54:19 +08:00 |
Jiarui Fang
|
534afb018a
|
test pretrain loading on multi-process (#922)
|
2022-05-09 17:07:35 +08:00 |
Ziyue Jiang
|
c195d2814c
|
[Tensor] add from_pretrained support and bert pretrained test (#921)
* add from_pretrained support and test
* polish
* polish
* polish
* polish
|
2022-05-09 16:11:47 +08:00 |
Jiarui Fang
|
845856ea29
|
[Graph] building computing graph with ColoTensor, Linear only (#917)
|
2022-05-07 17:10:37 +08:00 |
Ziyue Jiang
|
75d221918a
|
[Tensor] add 1d vocab loss (#918)
* add 1d vocab loss
* polish
|
2022-05-07 15:49:14 +08:00 |
Ziyue Jiang
|
dfaff4e243
|
[Tensor] fix test_model (#916)
* polish test_model
* polish
|
2022-05-06 18:06:22 +08:00 |
Jiarui Fang
|
ed6426c300
|
[Tensor] polish model test (#915)
|
2022-05-06 17:07:56 +08:00 |
Ziyue Jiang
|
0fab86b12a
|
[Tensor] add a basic bert. (#911)
* add base bert test
* Add bert test
* polish
* remove test_bert
* polish
|
2022-05-06 15:03:43 +08:00 |
Jiarui Fang
|
ab95ec9aea
|
[Tensor] init ColoParameter (#914)
|
2022-05-06 12:57:14 +08:00 |
Ziyue Jiang
|
f593a5637e
|
[Tensor] add embedding tp1d row (#904)
|
2022-04-29 14:10:05 +08:00 |
Ziyue Jiang
|
2c0d19d755
|
[Tensor] add ColoTensor TP1Dcol Embedding (#899)
|
2022-04-28 17:45:06 +08:00 |
Jiarui Fang
|
d16671da75
|
[Tensor] initialize the ColoOptimizer (#898)
* [Tensor] activation is an attr of ColoTensor
* [Tensor] add optimizer
* only detach parameters in context
* polish code
|
2022-04-28 15:23:40 +08:00 |
Jiarui Fang
|
e76f76c08b
|
[Tensor] test parameters() as member function (#896)
|
2022-04-28 10:57:14 +08:00 |
Ziyue Jiang
|
cb182da7c5
|
[tensor] refine linear and add gather for laynorm (#893)
* refine linear and add function to ColoTensor
* add gather for layernorm
* polish
* polish
|
2022-04-28 10:55:40 +08:00 |
Jiarui Fang
|
26c49639d8
|
[Tensor] overriding paramters() for Module using ColoTensor (#889)
|
2022-04-27 15:28:59 +08:00 |
Jiarui Fang
|
a0e5971692
|
[Tensor] test model check results for a simple net (#887)
|
2022-04-27 12:00:18 +08:00 |