Jiarui Fang
|
9bcd2fd4af
|
[tensor] a shorter shard and replicate spec (#1245)
|
2022-07-11 15:51:48 +08:00 |
|
Jiarui Fang
|
ae7d3f4927
|
[refactor] move process group from _DistSpec to ColoTensor. (#1203)
|
2022-07-06 16:15:16 +08:00 |
|
Jiarui Fang
|
c463f8adf9
|
[tensor] remove gpc in tensor tests (#1186)
|
2022-06-29 14:08:40 +08:00 |
|
Jiarui Fang
|
372f791444
|
[refactor] move chunk and chunkmgr to directory gemini (#1182)
|
2022-06-29 13:31:02 +08:00 |
|
Jiarui Fang
|
7487215b95
|
[ColoTensor] add independent process group (#1179)
|
2022-06-29 10:03:09 +08:00 |
|
Jiarui Fang
|
f4ef224358
|
[Tensor] remove ParallelAction, use ComputeSpec instread (#1166)
|
2022-06-23 17:34:59 +08:00 |
|
ver217
|
895c1c5ee7
|
[tensor] refactor param op hook (#1097)
* refactor param op hook
* add docstr
* fix bug
|
2022-06-13 16:11:53 +08:00 |
|
Jiarui Fang
|
a00644079e
|
reorgnize colotensor directory (#1062)
* reorgnize colotensor directory
* polish code
|
2022-06-03 18:04:22 +08:00 |
|
ver217
|
9492a561c3
|
[tensor] ColoTensor supports ZeRo (#1015)
* impl chunk manager
* impl param op hook
* add reduce_chunk
* add zero hook v2
* add zero dp
* fix TensorInfo
* impl load balancing when using zero without chunk
* fix zero hook
* polish chunk
* fix bugs
* ddp ok
* zero ok
* polish code
* fix bugs about load balancing
* polish code
* polish code
* add ene-to-end test
* polish code
* polish code
* polish code
* fix typo
* add test_chunk
* fix bugs
* fix bugs
* polish code
|
2022-05-31 12:00:12 +08:00 |
|
Ziyue Jiang
|
6c5996a56e
|
[Tensor] add module check and bert test (#1031)
* add Embedding
* Add bert test
* polish
* add check module test
* polish
* polish
* polish
* polish
|
2022-05-26 18:15:42 +08:00 |
|
Ziyue Jiang
|
32291dd73f
|
[Tensor] add module handler for linear (#1021)
* add module spec for linear
* polish
* polish
* polish
|
2022-05-26 11:50:44 +08:00 |
|
ver217
|
ad536e308e
|
[tensor] refactor colo-tensor (#992)
* refactor colo-tensor and update linear op
* polish code
* polish code
* update ops and unit tests
* update unit tests
* polish code
* rename dist_spec module
* polish code
* polish code
* remove unneeded import
* fix pipelinable
|
2022-05-19 12:44:59 +08:00 |
|
ver217
|
67c33f57eb
|
[tensor] design DistSpec and DistSpecManager for ColoTensor (#934)
* add dist spec
* update linear op
* polish code
* polish code
* update embedding op
* polish unit tests
* polish unit tests
* polish comments
* polish code
* add test_dist_spec_mgr
* polish code
* refactor folder structure
* polish unit tests
* add get_process_group() for TensorSpec
* polish code
|
2022-05-13 15:13:52 +08:00 |
|
Jiarui Fang
|
ab95ec9aea
|
[Tensor] init ColoParameter (#914)
|
2022-05-06 12:57:14 +08:00 |
|
Jiarui Fang
|
d16671da75
|
[Tensor] initialize the ColoOptimizer (#898)
* [Tensor] activation is an attr of ColoTensor
* [Tensor] add optimizer
* only detach parameters in context
* polish code
|
2022-04-28 15:23:40 +08:00 |
|
Ziyue Jiang
|
cb182da7c5
|
[tensor] refine linear and add gather for laynorm (#893)
* refine linear and add function to ColoTensor
* add gather for layernorm
* polish
* polish
|
2022-04-28 10:55:40 +08:00 |
|
Jiarui Fang
|
e43f83aa5c
|
[Tensor] get named parameters for model using ColoTensors (#874)
|
2022-04-26 14:08:01 +08:00 |
|
Ziyue Jiang
|
26d4ab8b03
|
[Tensor] Add function to spec and update linear 1Drow and unit tests (#869)
|
2022-04-26 10:15:26 +08:00 |
|
Jiarui Fang
|
0ce8924ceb
|
[tensor] reorganize files (#820)
|
2022-04-21 14:15:48 +08:00 |