30dd13c450
[autochunk] support complete benchmark ( #3121 )
...
* refact memory code
* dont log free var memory
* add memory align
* update chunk target
* update setting for new memory
* finish test
* update tracer
* update typo
* update test
* add unet test
* add bench
* update bench
* update bench
* init
* support vit
* move to cpu
* add cpu benchmark
2023-03-13 17:42:37 +08:00
68577fbc43
[chatgpt]Fix examples ( #3116 )
...
* fix train_dummy
* fix train-prompts
2023-03-13 11:12:22 +08:00
0672b5afac
[chatgpt] fix lora support for gpt ( #3113 )
...
* fix gpt-actor
* fix gpt-critic
* fix opt-critic
2023-03-13 10:37:41 +08:00
0aa92c0409
Automated submodule synchronization ( #3105 )
...
Co-authored-by: github-actions <github-actions@github.com>
2023-03-13 08:58:06 +08:00
453f7ae5a0
prevent op_builder being installed in site-pkgs ( #3104 )
2023-03-13 08:50:31 +08:00
191daf7411
[chatgpt] type miss of kwargs ( #3107 )
2023-03-13 00:00:02 +08:00
145ccfd7d1
[doc] add Intel cooperation for biomedicine ( #3108 )
...
* [doc] add Intel cooperation for biomedicine
2023-03-11 15:21:45 +08:00
c9dd036592
[chatgpt] fix lora save bug ( #3099 )
...
* fix colo-stratergy
* polish
* fix lora
* fix ddp
* polish
* polish
2023-03-10 17:58:10 +08:00
018936a3f3
[tutorial] update notes for TransformerEngine ( #3098 )
2023-03-10 16:30:52 +08:00
65a4dbda6c
[NVIDIA] Add FP8 example using TE ( #3080 )
...
Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com>
2023-03-10 16:24:08 +08:00
26db1cb57b
[release] v0.2.7 ( #3094 )
2023-03-10 14:54:37 +08:00
02ae80bf9c
[chatgpt]add flag of action mask in critic( #3086 )
2023-03-10 14:40:14 +08:00
95a36eae63
[kernel] added kernel loader to softmax autograd function ( #3093 )
...
* [kernel] added kernel loader to softmax autograd function
* [release] v0.2.6
2023-03-10 14:27:09 +08:00
fff98f06ed
[analyzer] a minimal implementation of static graph analyzer ( #2852 )
...
* [hotfix] meta tensor default device.
* [siu] add experimental submodules to main branch.
* [siu]
* [siu]
* [analyzer] init.
* [analyzer] readme.
* [analyzer] readme.
* [analyzer] readme.
* [analyzer] readme.
* [test] add test.
* Update symbolic_trace.py
* mark skip tests.
* try except.
* try except.
* try except.
* s
* init
* init
* fix
* skip
* skip
---------
Co-authored-by: Daniel Shao <superdainiu@MININT-PVARVID.fareast.corp.microsoft.com>
Co-authored-by: Daniel Shao <superdainiu@Daniels-Mac.local>
2023-03-10 13:21:05 +08:00
5d5f475d75
[diffusers] fix ci and docker ( #3085 )
2023-03-10 10:35:15 +08:00
3213347b49
[doc] fixed typos in docs/README.md ( #3082 )
2023-03-10 10:32:14 +08:00
10c61de2f7
[autochunk] support vit ( #3084 )
...
support vit for autochunk
* support some new ops for vit
* fix some bugs
* add test for vit
2023-03-10 10:23:26 +08:00
e58a3c804c
Fix the version of lightning and colossalai in Stable Diffusion environment requirement ( #3073 )
...
1. Modify the README of stable diffusion
2. Fix the version of pytorch lightning&lightning and colossalai version to enable codes running successfully.
2023-03-10 09:55:58 +08:00
8e4e8601b7
[DTensor] implement layout converter ( #3055 )
...
* [DTensor] refactor LayoutConverter for DTensor
* polish code
* polish docstring
2023-03-10 09:53:52 +08:00
89aa7926ac
[release] v0.2.6 ( #3057 )
2023-03-10 09:47:20 +08:00
416a50dbd7
[doc] moved doc test command to bottom ( #3075 )
2023-03-09 18:10:45 +08:00
91ccf97514
[workflow] fixed doc build trigger condition ( #3072 )
2023-03-09 17:31:41 +08:00
f19b49e164
[booster] init module structure and definition ( #3056 )
2023-03-09 11:27:46 +08:00
faa8526b85
Automated submodule synchronization ( #3062 )
...
Co-authored-by: github-actions <github-actions@github.com>
2023-03-09 11:22:56 +08:00
360674283d
[example] fix redundant note ( #3065 )
2023-03-09 10:59:28 +08:00
af3888481d
[example] fixed opt model downloading from huggingface
2023-03-09 10:47:41 +08:00
2ca9728cbb
[autochunk] refactor chunk memory estimation ( #2762 )
...
* refact memory code
* dont log free var memory
* add memory align
* update chunk target
* update setting for new memory
* finish test
* update tracer
* update typo
* update test
2023-03-08 16:22:30 +08:00
b51bfec357
[chatgpt] change critic input as state ( #3042 )
...
* fix Critic
* fix Critic
* fix Critic
* fix neglect of attention mask
* fix neglect of attention mask
* fix neglect of attention mask
* add return
---------
Co-authored-by: yangwenjun <yangwenjun@soyoung.com>
Co-authored-by: yangwjd <yangwjd@chanjet.com>
2023-03-08 15:18:02 +08:00
2ef855c798
support shardinit option to avoid OPT OOM initializing problem ( #3037 )
...
Co-authored-by: poe <poe@nemoramo>
2023-03-08 13:45:15 +08:00
29386a54e6
[DTensor] refactor CommSpec ( #3034 )
2023-03-08 10:45:31 +08:00
ea0b52c12e
[doc] specified operating system requirement ( #3019 )
...
* [doc] specified operating system requirement
* polish code
2023-03-07 18:04:10 +08:00
378d827c6b
[doc] update nvme offload doc ( #3014 )
...
* [doc] update nvme offload doc
* [doc] add doc testing cmd and requirements
* [doc] add api reference
* [doc] add dependencies
2023-03-07 17:49:01 +08:00
c21b11edce
change nn to models ( #3032 )
2023-03-07 16:34:22 +08:00
4269196c79
[hotfix] skip auto checkpointing tests ( #3029 )
...
* [hotfix] skip auto checkpointing tests
* fix test name issue
2023-03-07 15:50:00 +08:00
8fedc8766a
[workflow] supported conda package installation in doc test ( #3028 )
...
* [workflow] supported conda package installation in doc test
* polish code
* polish code
* polish code
* polish code
* polish code
* polish code
2023-03-07 14:21:26 +08:00
2cd6ba3098
[workflow] fixed the post-commit failure when no formatting needed ( #3020 )
...
* [workflow] fixed the post-commit failure when no formatting needed
* polish code
* polish code
* polish code
2023-03-07 13:35:45 +08:00
2e427ddf42
[revert] recover "[refactor] restructure configuration files ( #2977 )" ( #3022 )
...
This reverts commit 35c8f4ce47
2023-03-07 13:31:23 +08:00
e86d9bb2e1
[format] applied code formatting on changed files in pull request 3025 ( #3026 )
...
Co-authored-by: github-actions <github-actions@github.com>
2023-03-07 12:55:17 +08:00
cd2b0eaa8d
[DTensor] refactor sharding spec ( #2987 )
...
* [autoparallel] refactor sharding spec
* rename function name
2023-03-07 11:08:11 +08:00
400f63012e
[pipeline] Add Simplified Alpa DP Partition ( #2507 )
...
* add alpa dp split
* add alpa dp split
* use fwd+bwd instead of fwd only
---------
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
2023-03-07 10:34:31 +08:00
b42d3d28ed
[fx] remove depreciated algorithms. ( #2312 ) ( #2313 )
2023-03-07 10:30:35 +08:00
55dcd3051a
[chatgpt] fix readme ( #3025 )
2023-03-07 10:21:25 +08:00
287d60499e
[chatgpt] Add saving ckpt callback for PPO ( #2880 )
...
* add checkpoint callback for chatgpt
* add save ckpt callbacks for ppo
---------
Co-authored-by: Fazzie-Maqianli <55798671+Fazziekey@users.noreply.github.com>
2023-03-07 10:13:25 +08:00
e588703454
[chatgpt]fix inference model load ( #2988 )
...
* fix lora bug
* polish
* fix lora gemini
* fix inference laod model bug
2023-03-07 09:17:52 +08:00
82503a96f2
[format] applied code formatting on changed files in pull request 2997 ( #3008 )
...
Co-authored-by: github-actions <github-actions@github.com>
2023-03-06 10:42:22 +08:00
52a5078988
[doc] add ISC tutorial ( #2997 )
...
* [doc] add ISC tutorial
* [doc] add ISC tutorial
* [doc] add ISC tutorial
* [doc] add ISC tutorial
2023-03-06 10:36:38 +08:00
35c8f4ce47
[refactor] restructure configuration files ( #2977 )
...
* gh: move CONTRIBUTING to .github
* chore: move isort config to pyproject
* chore: move pytest config to pyproject
* chore: move yapf config to pyproject
* chore: move clang-format config to pre-commit
2023-03-05 20:29:34 +08:00
823f3b9cf4
[doc] add deepspeed citation and copyright ( #2996 )
...
* [doc] add deepspeed citation and copyright
* [doc] add deepspeed citation and copyright
* [doc] add deepspeed citation and copyright
2023-03-04 20:08:11 +08:00
e0a1c1321c
[doc] added reference to related works ( #2994 )
...
* [doc] added reference to related works
* polish code
2023-03-04 17:32:22 +08:00
19fa0e57f6
Remove extraneous comma ( #2993 )
...
Prevents `TypeError: category must be a Warning subclass, not 'str'`.
2023-03-04 14:44:06 +08:00
Xuanlei Zhao