github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,616 Commits
32 Branches
46 Tags
228 MiB
Commit Graph

1616 Commits (48d33b1b1753f19361e7e54a68a7ac5999dc02e4)

Author SHA1 Message Date
oahzxl 25952b67d7
[feat] add flash attention (#1762) 2022-10-26 16:15:52 +08:00
Super Daniel 0584654c79
[fx] refactor memory utils and extend shard utils. (#1754) 
* [fx] change memory.py to memory_utils.py.

* [fx] add shard utils.

* [fx] fix import.

* [fx] check code style.

* [fx] add comment.

* [autoparallel] first move.

* [fx] add time computations.
 2022-10-26 14:24:41 +08:00
Ziyue Jiang 63f250bbd4
fix file name (#1759) 
Co-authored-by: Ziyue Jiang <ziyue.jiang@gmail.com>
 2022-10-25 16:48:48 +08:00
YuliangLiu0306 314d8c497f
[autoparallel] refactor the runtime apply pass and add docstring to passes (#1757) 
* [autoparallel] refactor the runtime apply pass and add doc string to passes

* fix unit test

* polish
 2022-10-25 14:32:22 +08:00
Frank Lee f9a613d660
[autoparallel] added binary elementwise node handler (#1758) 
* [autoparallel] added binary elementwise node handler

* polish code
 2022-10-25 14:32:01 +08:00
YuliangLiu0306 d2fc067231
[autoparallel] fix param hook issue in transform pass (#1755) 2022-10-24 13:13:38 +08:00
Frank Lee 262652c8bc
[autoparallel] added addbmm handler (#1751) 2022-10-21 18:55:48 +08:00
YuliangLiu0306 980ed21723
[autoparallel] shard param and buffer as expected (#1753) 
* [autoparallel] shard param and buffer as expected

* fix unit test issue
 2022-10-21 15:45:13 +08:00
YuliangLiu0306 cdb7d5e7d2
[hotfix] autoparallel unit test (#1752) 2022-10-20 19:51:38 +08:00
YuliangLiu0306 a4ce180e85
[autoparallel] add sequential order to communication actions (#1735) 2022-10-20 18:48:18 +08:00
Super Daniel b893342f95
[fx] test tracer on diffuser modules. (#1750) 
* [fx] test tracer on diffuser modules.

* [fx] shorter seq_len.

* Update requirements-test.txt
 2022-10-20 18:25:05 +08:00
Frank Lee b80b6eaa88
[autoparallel] recovered skipped test cases (#1748) 2022-10-20 16:37:33 +08:00
Frank Lee 474111ecb5
[autoparallel] fixed wrong sharding strategy in conv handler (#1747) 
* [autoparallel] fixed wrong sharding strategy in conv handler

* polish code
 2022-10-20 16:12:39 +08:00
Frank Lee 8b8937d901
[autoparallel] fixed wrong generated strategy for dot op (#1746) 
* [autoparallel] fixed wrong generated strategy for dot op

* polish code
 2022-10-20 15:18:16 +08:00
Frank Lee 993b8875b6
[autoparallel] handled illegal sharding strategy in shape consistency (#1744) 
* [autoparallel] handled illegal sharding strategy in shape consistency

* polish code
 2022-10-20 12:06:25 +08:00
Frank Lee 88a79814fb
[autoparallel] handled illegal strategy in node handler (#1743) 
* [autoparallel] handled illegal strategy in node handler

* polish code
 2022-10-19 17:08:52 +08:00
Super Daniel 30874f1692
[fx/profiler] debug the fx.profiler / add an example test script for fx.profiler (#1730) 
* [fx/profiler] add test.

* [fx] fix file names.

* [fx] add docstring and comment.

* [fx] polish profiler.py.

* [fx] fix import errors.

* [fx] fix profiler.

* [fx] fix names.
 2022-10-19 14:24:51 +08:00
Frank Lee eee84908d4
[autoparallel] handled illegal sharding strategy (#1728) 
* [autoparallel] handled illegal sharding strategy

* polish code
 2022-10-19 12:53:06 +08:00
Ziheng Qin cbe9a4cb45 [NFC] polish tests/test_layers/test_3d/test_3d.py code style (#1740) 2022-10-19 12:20:51 +08:00
lucasliunju 912eb58ea0 [NFC] polish tests/test_layers/test_3d/checks_3d/common.py code style (#1733) 2022-10-19 12:20:51 +08:00
Sze-qq 23703c9dd6 [NFC] polish colossalai/nn/metric/_utils.py code style (#1727) 2022-10-19 12:20:51 +08:00
Xue Fuzhao 754aa7c81f [NFC] polish tests/test_layers/test_3d/checks_3d/check_layer_3d.py code style (#1731) 2022-10-19 12:20:51 +08:00
xyupeng ff373a11eb [NFC] polish tests/test_layers/test_sequence/checks_seq/check_layer_seq.py code style (#1723) 2022-10-19 12:20:51 +08:00
Ofey Chan 7e62af28a0 [NFC] polish accuracy_2d.py code style (#1719) 2022-10-19 12:20:51 +08:00
Arsmart1 8860d37846 [NFC] polish .github/workflows/scripts/build_colossalai_wheel.py code style (#1721) 2022-10-19 12:20:51 +08:00
LuGY 730f88f8e1 [NFC] polish _checkpoint_hook.py code style (#1722) 2022-10-19 12:20:51 +08:00
Kai Wang (Victor Kai) b38efe4e8a [NFC] polish test_2p5d/checks_2p5d/check_operation_2p5d.py code style (#1718) 2022-10-19 12:20:51 +08:00
CsRic ea961d8fd1 [NFC] polish colossalai/zero/sharded_param/__init__.py code style (#1717) 
Co-authored-by: ric <mkkt_bkkt@mail.ustc.edu.cn>
 2022-10-19 12:20:51 +08:00
yuxuan-lou 2b49ca80a3 [NFC] polish colossalai/nn/lr_scheduler/linear.py code style (#1716) 2022-10-19 12:20:51 +08:00
binmakeswell f6389d0813 [NFC] polish tests/test_layers/test_2d/checks_2d/check_operation_2d.py code style (#1715) 2022-10-19 12:20:51 +08:00
shenggan e1d780030d [NFC] polish colossalai/nn/metric/accuracy_2p5d.py code style (#1714) 2022-10-19 12:20:51 +08:00
Frank Lee 9d0560af9c
[workflow] handled the git directory ownership error (#1741) 2022-10-19 11:59:11 +08:00
YuliangLiu0306 d373e67b99
[hotfix] resharding cost issue (#1742) 2022-10-19 11:33:43 +08:00
Jiarui Fang 24e84eba60
upgrade version to 0.1.11rc1 (#1739) 2022-10-19 11:26:00 +08:00
Frank Lee d2e0e39c9d
[release] update to v0.1.11 (#1736) 2022-10-19 00:28:00 +08:00
binmakeswell 0d87c4e20d
[doc] update recommendation system catalogue (#1732) 2022-10-19 00:25:56 +08:00
HELSON f69f9bf223
[zero] add chunk init function for users (#1729) 
* add chunk manager init function

* fix unit tests

* add comment

* add flush=True
 2022-10-18 16:31:22 +08:00
HELSON 2e1dbfb463
[pre-commit] update pre-commit (#1726) 
* update pre-commit

* update pre-commit
 2022-10-18 14:35:37 +08:00
Jiarui Fang c626b23960
[doc] update recommedation system urls (#1725) 2022-10-18 13:38:41 +08:00
YuliangLiu0306 51b89d2202
[autoparallel] runtime_backward_apply (#1720) 2022-10-18 10:44:58 +08:00
Super Daniel 393f594051
[fx/meta/rpc] move _meta_registration.py to fx folder / register fx functions with compatibility checks / remove color debug (#1710) 
* [fx] move meta registration

* [fx] fix tests.

* [fx] fix test.

* [fx] fix.

* [meta] refactor meta registration.py.

* [fx] add compatibility descriptions.

* [fx] polish import.

* [fx] add a decorator.

* [fx] fix tests.

* [fx] remove print.

* [fx] edit raise error.

* [fx] edit raise error.

* [fx] add type hint.

* [fx] fix import in experimental.

* [rpc] remove color debug.

* [meta] fix naming.
 2022-10-18 10:44:23 +08:00
Frank Lee e8d8eda5e7
[autoparallel] moved tests to test_tensor_shard (#1713) 2022-10-17 13:54:20 +08:00
YuliangLiu0306 845ff4a47a
[autoparallel] resnet block runtime apply (#1709) 
* [autoparallel] resnet block runtime apply

* seperate buffer and parameter in MemoryCost

* polish code

* add comments and todos

* fix test issue
 2022-10-17 13:37:38 +08:00
Jiarui Fang b0a23dc4fc
[embeddings] add doc in readme (#1711) 2022-10-16 21:57:50 +08:00
Frank Lee 22a115406b
[autoparallel] fixed broken node handler tests (#1708) 2022-10-14 18:25:59 +08:00
HELSON 1468e4bcfc
[zero] add constant placement policy (#1705) 
* fixes memory leak when paramter is in fp16 in ZeroDDP init.
* bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release.
* adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters.
 2022-10-14 17:53:16 +08:00
binmakeswell 5f41463a76
add optimizer README for tutorials (#1707) 2022-10-14 09:10:18 +00:00
Frank Lee 6c331a5a09
[autoparallel] refactored the autoparallel module for organization (#1706) 
* [autoparallel] refactored the autoparallel module for organization

* polish code
 2022-10-14 13:27:00 +08:00
Frank Lee 91cd34e6e0
[unittest] added doc for the pytest wrapper (#1704) 2022-10-14 10:56:17 +08:00
YuliangLiu0306 451cd72dea
[autoparallel] adapt runtime passes (#1703) 
* [autoparallel] adapt runtime passes v2

* polish code
 2022-10-14 10:14:07 +08:00