Jiarui Fang
|
7675792100
|
[builder] raise Error when CUDA_HOME is not set (#2213)
|
2022-12-28 16:07:08 +08:00 |
|
Jiarui Fang
|
d5e3e3ec01
|
[example] update gpt example for larger model scale (#2211)
|
2022-12-28 13:54:08 +08:00 |
|
Boyuan Yao
|
24246f7aa5
|
[autoparallel] Attach input, buffer and output tensor to MetaInfo class (#2162)
* [fx] metainfo class for auto parallel
* [fx] add unit test for linear metainfo
* [fx] fix bwd param for linear
* [fx] modify unit test
* [fx] modify unit test
* [fx] modify import
* [fx] modify import
* [fx] modify import
* [fx] move meta profiler to auto parallel
* [fx] add conv metainfo class
* [fx] restore profiler
* [fx] restore meta profiler
* [autoparallel] modify unit test
* [fx] modify unit test
* [autoparallel] add batchnorm metainfo class
* [autoparallel] fix batchnorm unit test function declaration
* [fx] restore profiler
* [fx] add relu metainfo class
* [fx] restore profiler
* [autoparallel] modify metainfo input
* [autoparallel] add pooling metainfo
* [autoparallel] add F.linear metainfo generator
* [autoparallel] add binary elementwise metainfo
* [fx] recover profiler
* [autoparallel] fix forward memory calculation
* [autoparallel] modify constants.py
* [autoparallel] remove redundant print
* [autoparallel] add F.conv metainfo
* [autoparallel] linear fix
* [autoparallel] memory estimation for communication actions
* [autoparallel] fix docstring
* [autoparallel] fix variables name
* [autoparallel] attach tensor to metainfo class
* [autoparallel] fix dangerous try except
* [autoparallel] attach memory cost to shape consistency node
* [autoparallel] attach shape consistency node's metainfo to the node
* [autoparallel] remove todo in shape consistency memory estimation
* [autoparallel] fix the annotation
|
2022-12-28 13:37:40 +08:00 |
|
Boyuan Yao
|
d0bc5a1b34
|
[autoparallel] new metainfoprop based on metainfo class (#2179)
* [autoparallel] new metainfoprop to combine SPMD solver and checkpoint solver
* [autoparallel] new metainfoprop to combine SPMD solver and checkpoint solver
* [autoparallel] modify placeholder handler
* [autoparallel] modify metainfoprop
* [autoparallel] fix function typo
* [autoparallel] fix placeholder handler
|
2022-12-28 13:35:08 +08:00 |
|
YuliangLiu0306
|
78509124d3
|
[autoparallel] update getitem handler (#2207)
|
2022-12-27 19:58:32 +08:00 |
|
Jiarui Fang
|
1cb532ffec
|
[builder] multihead attn runtime building (#2203)
* [hotfix] correcnt cpu_optim runtime compilation
* [builder] multihead attn
* fix bug
* fix a bug
|
2022-12-27 16:06:09 +08:00 |
|
Tongping Liu
|
8e22c38b89
|
[hotfix] Fixing the bug related to ipv6 support
Co-authored-by: ByteDance <tongping.liu@bytedance.com>
|
2022-12-27 12:42:46 +08:00 |
|
YuliangLiu0306
|
4851f2d607
|
[autoparallel] update_getattr_handler (#2193)
|
2022-12-26 21:57:39 +08:00 |
|
Jiarui Fang
|
5682e6d346
|
[hotfix] correcnt cpu_optim runtime compilation (#2197)
|
2022-12-26 16:45:14 +08:00 |
|
HELSON
|
2458659919
|
[zero] fix error for BEiT models (#2169)
* [zero] fix error for BEiT models
* [ColoParameter] add unpack operation for tuple arguments
* fix bugs
* fix chunkv2 unit testing
* add assertion for gradient state
|
2022-12-26 15:03:54 +08:00 |
|
Jiarui Fang
|
355ffb386e
|
[builder] unified cpu_optim fused_optim inferface (#2190)
|
2022-12-23 20:57:41 +08:00 |
|
Jiarui Fang
|
9587b080ba
|
[builder] use runtime builder for fused_optim (#2189)
|
2022-12-23 17:07:03 +08:00 |
|
Jiarui Fang
|
bc0e271e71
|
[buider] use builder() for cpu adam and fused optim in setup.py (#2187)
|
2022-12-23 16:05:13 +08:00 |
|
Jiarui Fang
|
d42afd30f8
|
[builder] runtime adam and fused_optim builder (#2184)
|
2022-12-23 14:14:21 +08:00 |
|
YuliangLiu0306
|
550f8f8905
|
[autoparallel] integrate_gpt_related_tests (#2134)
* [autoparallel] integrate_gpt_related_tests
* polish code
* polish code
* add GPT2Model into runtime test
|
2022-12-23 12:36:59 +08:00 |
|
Ziyue Jiang
|
59e343328d
|
[Pipeline Middleware ] Fix deadlock when num_microbatch=num_stage (#2156)
* add splitter
* polish code
* remove comment
* fix async nan by moving to cpu first
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
|
2022-12-23 11:38:43 +08:00 |
|
Tongping Liu
|
ab54fed292
|
[hotfix] add kwargs for colo_addmm (#2171)
|
2022-12-22 13:25:30 +08:00 |
|
アマデウス
|
622f863291
|
[hotfix] Jit type hint #2161 (#2164)
|
2022-12-22 10:17:03 +08:00 |
|
Zihao
|
12e7bcd720
|
register meta func for rnn (#2159)
|
2022-12-21 23:06:18 +08:00 |
|
Boyuan Yao
|
cfe2a9bd90
|
[autoparallel] memory estimation for shape consistency (#2144)
* [fx] metainfo class for auto parallel
* [fx] add unit test for linear metainfo
* [fx] fix bwd param for linear
* [fx] modify unit test
* [fx] modify unit test
* [fx] modify import
* [fx] modify import
* [fx] modify import
* [fx] move meta profiler to auto parallel
* [fx] add conv metainfo class
* [fx] restore profiler
* [fx] restore meta profiler
* [autoparallel] modify unit test
* [fx] modify unit test
* [autoparallel] add batchnorm metainfo class
* [autoparallel] fix batchnorm unit test function declaration
* [fx] restore profiler
* [fx] add relu metainfo class
* [fx] restore profiler
* [autoparallel] modify metainfo input
* [autoparallel] add pooling metainfo
* [autoparallel] add F.linear metainfo generator
* [autoparallel] add binary elementwise metainfo
* [fx] recover profiler
* [autoparallel] fix forward memory calculation
* [autoparallel] modify constants.py
* [autoparallel] remove redundant print
* [autoparallel] add F.conv metainfo
* [autoparallel] linear fix
* [autoparallel] memory estimation for communication actions
* [autoparallel] fix docstring
* [autoparallel] fix variables name
|
2022-12-21 10:39:37 +08:00 |
|
Jiarui Fang
|
b87496a66b
|
[hotfix] fix auto policy of test_sharded_optim_v2 (#2157)
|
2022-12-20 23:03:18 +08:00 |
|
YuliangLiu0306
|
16335cb537
|
[hotfix] fix aten default bug (#2158)
|
2022-12-20 22:40:46 +08:00 |
|
HELSON
|
a7d95b7024
|
[example] add zero1, zero2 example in GPT examples (#2146)
* [example] add zero1 and zero2 for GPT
* update readme in gpt example
* polish code
* change init value
* update readme
|
2022-12-20 14:30:27 +08:00 |
|
YuliangLiu0306
|
1cce6e36ca
|
[autoparallel] use metainfo in handler (#2149)
|
2022-12-20 10:31:22 +08:00 |
|
Jiarui Fang
|
2827f41898
|
[Gemini] GeminiDPP convert to PyTorch Module. (#2151)
|
2022-12-20 10:19:36 +08:00 |
|
Jiarui Fang
|
bdef9dfdbe
|
[NFC] remove useless graph node code (#2150)
|
2022-12-20 00:33:58 +08:00 |
|
BlueRum
|
b3f73ce1c8
|
[Gemini] Update coloinit_ctx to support meta_tensor (#2147)
|
2022-12-19 22:37:07 +08:00 |
|
Zihao
|
a128eec9d5
|
register aten._convolution.default (#2137)
|
2022-12-18 19:27:01 +08:00 |
|
Jiarui Fang
|
ee287620f0
|
[Gemini] revert ZeROInitCtx related tracer (#2138)
|
2022-12-16 12:37:06 +08:00 |
|
アマデウス
|
077a66dd81
|
updated attention kernel (#2133)
|
2022-12-16 10:54:03 +08:00 |
|
YuliangLiu0306
|
a3c6924deb
|
[autoparallel] process size nodes in runtime pass (#2130)
* [autoparallel] process size nodes in runtime pass
* polish code
|
2022-12-14 16:10:50 +08:00 |
|
YuliangLiu0306
|
536560ccc0
|
[autoparallel] implement softmax handler (#2132)
|
2022-12-14 16:09:53 +08:00 |
|
Jiarui Fang
|
c89c66a858
|
[Gemini] update API of the chunkmemstatscollector. (#2129)
|
2022-12-14 00:47:06 +08:00 |
|
Jiarui Fang
|
2938edf446
|
[Gemini] update the non model data record method in runtime memory tracer (#2128)
|
2022-12-13 17:11:31 +08:00 |
|
Jiarui Fang
|
8fac837679
|
[Gemini] update non model data calculation method (#2126)
|
2022-12-13 15:44:07 +08:00 |
|
Jiarui Fang
|
5efda69735
|
[Gemini] hotfix the unittest bugs (#2125)
|
2022-12-13 14:14:55 +08:00 |
|
Jiarui Fang
|
05bb28aacf
|
[Gemini] mapping of preop timestep and param (#2124)
|
2022-12-13 12:50:24 +08:00 |
|
YuliangLiu0306
|
cd0af9f7f6
|
[autoparallel] gpt2lp runtimee test (#2113)
|
2022-12-12 18:06:40 +08:00 |
|
Jiarui Fang
|
9214d1fe28
|
[Gemini] chunk init using runtime visited param order (#2115)
|
2022-12-12 18:06:16 +08:00 |
|
HELSON
|
e7d3afc9cc
|
[optimizer] add div_scale for optimizers (#2117)
* [optimizer] add div_scale for optimizers
* [zero] use div_scale in zero optimizer
* fix testing error
|
2022-12-12 17:58:57 +08:00 |
|
Jiarui Fang
|
e5aa8333e4
|
[NFC] update chunk manager API (#2119)
|
2022-12-12 16:57:22 +08:00 |
|
Jiarui Fang
|
e99edfcb51
|
[NFC] polish comments for Chunk class (#2116)
|
2022-12-12 15:39:31 +08:00 |
|
Ziyue Jiang
|
09d69e1c25
|
[PP Middleware] Add bwd and step for PP middleware (#2111)
* add bwd and step for PP middleware
* pre-commit
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
|
2022-12-12 12:40:03 +08:00 |
|
Jiarui Fang
|
8afc001f4f
|
[Gemini] chunk init use OrderedParamGenerator (#2110)
|
2022-12-11 21:41:13 +08:00 |
|
HELSON
|
63fbba3c19
|
[zero] add L2 gradient clipping for ZeRO (#2112)
* [zero] add L2 gradient clipping
* [testing] add MlpModel
* [zero] add unit test for grad clipping
* fix atol
|
2022-12-09 18:09:17 +08:00 |
|
Jiarui Fang
|
70a8556946
|
[gemini] get the param visited order during runtime (#2108)
|
2022-12-09 16:13:03 +08:00 |
|
Jiarui Fang
|
61f31c3cf0
|
[Gemini] NFC, polish search_chunk_configuration (#2107)
|
2022-12-09 15:00:39 +08:00 |
|
Jiarui Fang
|
8e14344ec9
|
[hotfix] fix a type in ColoInitContext (#2106)
|
2022-12-09 11:44:39 +08:00 |
|
Jiarui Fang
|
05545bfee9
|
[ColoTensor] throw error when ColoInitContext meets meta parameter. (#2105)
|
2022-12-09 11:39:46 +08:00 |
|
YuliangLiu0306
|
d87baa85d9
|
[autoparallel] support linear function bias addition (#2104)
|
2022-12-09 10:31:36 +08:00 |