7e62af28a0
[NFC] polish accuracy_2d.py code style ( #1719 )
2022-10-19 12:20:51 +08:00
8860d37846
[NFC] polish .github/workflows/scripts/build_colossalai_wheel.py code style ( #1721 )
2022-10-19 12:20:51 +08:00
730f88f8e1
[NFC] polish _checkpoint_hook.py code style ( #1722 )
2022-10-19 12:20:51 +08:00
b38efe4e8a
[NFC] polish test_2p5d/checks_2p5d/check_operation_2p5d.py code style ( #1718 )
2022-10-19 12:20:51 +08:00
ea961d8fd1
[NFC] polish colossalai/zero/sharded_param/__init__.py code style ( #1717 )
...
Co-authored-by: ric <mkkt_bkkt@mail.ustc.edu.cn>
2022-10-19 12:20:51 +08:00
2b49ca80a3
[NFC] polish colossalai/nn/lr_scheduler/linear.py code style ( #1716 )
2022-10-19 12:20:51 +08:00
f6389d0813
[NFC] polish tests/test_layers/test_2d/checks_2d/check_operation_2d.py code style ( #1715 )
2022-10-19 12:20:51 +08:00
e1d780030d
[NFC] polish colossalai/nn/metric/accuracy_2p5d.py code style ( #1714 )
2022-10-19 12:20:51 +08:00
9d0560af9c
[workflow] handled the git directory ownership error ( #1741 )
2022-10-19 11:59:11 +08:00
d373e67b99
[hotfix] resharding cost issue ( #1742 )
2022-10-19 11:33:43 +08:00
24e84eba60
upgrade version to 0.1.11rc1 ( #1739 )
2022-10-19 11:26:00 +08:00
d2e0e39c9d
[release] update to v0.1.11 ( #1736 )
2022-10-19 00:28:00 +08:00
0d87c4e20d
[doc] update recommendation system catalogue ( #1732 )
2022-10-19 00:25:56 +08:00
f69f9bf223
[zero] add chunk init function for users ( #1729 )
...
* add chunk manager init function
* fix unit tests
* add comment
* add flush=True
2022-10-18 16:31:22 +08:00
2e1dbfb463
[pre-commit] update pre-commit ( #1726 )
...
* update pre-commit
* update pre-commit
2022-10-18 14:35:37 +08:00
c626b23960
[doc] update recommedation system urls ( #1725 )
2022-10-18 13:38:41 +08:00
51b89d2202
[autoparallel] runtime_backward_apply ( #1720 )
2022-10-18 10:44:58 +08:00
393f594051
[fx/meta/rpc] move _meta_registration.py to fx folder / register fx functions with compatibility checks / remove color debug ( #1710 )
...
* [fx] move meta registration
* [fx] fix tests.
* [fx] fix test.
* [fx] fix.
* [meta] refactor meta registration.py.
* [fx] add compatibility descriptions.
* [fx] polish import.
* [fx] add a decorator.
* [fx] fix tests.
* [fx] remove print.
* [fx] edit raise error.
* [fx] edit raise error.
* [fx] add type hint.
* [fx] fix import in experimental.
* [rpc] remove color debug.
* [meta] fix naming.
2022-10-18 10:44:23 +08:00
e8d8eda5e7
[autoparallel] moved tests to test_tensor_shard ( #1713 )
2022-10-17 13:54:20 +08:00
845ff4a47a
[autoparallel] resnet block runtime apply ( #1709 )
...
* [autoparallel] resnet block runtime apply
* seperate buffer and parameter in MemoryCost
* polish code
* add comments and todos
* fix test issue
2022-10-17 13:37:38 +08:00
b0a23dc4fc
[embeddings] add doc in readme ( #1711 )
2022-10-16 21:57:50 +08:00
22a115406b
[autoparallel] fixed broken node handler tests ( #1708 )
2022-10-14 18:25:59 +08:00
1468e4bcfc
[zero] add constant placement policy ( #1705 )
...
* fixes memory leak when paramter is in fp16 in ZeroDDP init.
* bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release.
* adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters.
2022-10-14 17:53:16 +08:00
5f41463a76
add optimizer README for tutorials ( #1707 )
2022-10-14 09:10:18 +00:00
6c331a5a09
[autoparallel] refactored the autoparallel module for organization ( #1706 )
...
* [autoparallel] refactored the autoparallel module for organization
* polish code
2022-10-14 13:27:00 +08:00
91cd34e6e0
[unittest] added doc for the pytest wrapper ( #1704 )
2022-10-14 10:56:17 +08:00
451cd72dea
[autoparallel] adapt runtime passes ( #1703 )
...
* [autoparallel] adapt runtime passes v2
* polish code
2022-10-14 10:14:07 +08:00
21962e1593
[embedding] rename FreqAwareEmbedding -> CachedEmbedding ( #1699 )
2022-10-13 22:22:27 +08:00
0e52f3d3d5
[unittest] supported condititonal testing based on env var ( #1701 )
...
polish code
2022-10-13 19:38:45 +08:00
8283e95db3
[autoparallel] collated all deprecated files ( #1700 )
...
* [autoparallel] collated all deprecated files
* polish code
2022-10-13 18:24:11 +08:00
e2355d01b9
[autoparallel] init new folder structure ( #1696 )
2022-10-13 14:18:55 +08:00
81f7530ee7
[autoparallel] adapt solver and CostGraph with new handler ( #1695 )
...
* [autoparallel] adapt solver and CostGraph with new handler
* fix test issue
2022-10-13 14:04:15 +08:00
42b882ef06
[autoparallel] add output handler and placeholder handler ( #1694 )
...
* [autoparallel] add output handler and placeholder handler
* Delete test_solver_with_resnet.py
* fix test bugs
2022-10-13 13:42:36 +08:00
56088e6d98
[autoparallel] add pooling handler ( #1690 )
...
* [autoparallel] add pooling handler
* polish code
2022-10-13 13:42:13 +08:00
319d654f79
[autoparallel] where_handler_v2 ( #1688 )
...
* where generator
* [autoparallel] where_handler_v2
2022-10-13 11:02:22 +08:00
31d2f03d27
[autoparallel] fix C version rotor inconsistency ( #1691 )
2022-10-12 15:21:58 +08:00
363fc2861a
[embeddings] more detailed timer ( #1692 )
2022-10-12 12:01:21 +08:00
4973157ad7
[autoparallel] added sharding spec conversion for linear handler ( #1687 )
2022-10-12 11:16:18 +08:00
af718e83f2
[autoparallel] add reshape handler v2 and fix some previous bug ( #1683 )
2022-10-11 18:12:59 +08:00
6878e42248
[hotfix] solver bug caused by dict type comm cost ( #1686 )
2022-10-11 17:57:03 +08:00
3dd6994427
[fx/profiler] assigned UUID to each unrecorded tensor/ improved performance on GPT-2 ( #1679 )
...
* [fx/profiler] modify data_ptr into uuid for all tensors.
* [fx] modify uuid.
* [fx/profiler] tune performance on GPT-2.
* [fx] updates.
* [fx] debug.
* [fx] debug.
* [fx] cuda.
2022-10-11 11:03:35 +08:00
0df5034a36
[pipeline/fix-bug] num_microbatches support any integrate | stable chimera | launch tool for rpc pp framework ( #1684 )
...
* [pipeline/tuning] improve dispatch performance both time and space cost
* [pipeline/converge] add interface for testing convergence
* [NFC] polish colossalai/utils/multi_tensor_apply/multi_tensor_apply.py code style
* Update PipelineBase.py
* [pipeline/chimera] reconstruct PipelineBase and Worker to support more feasible custom schedule | finish Chimera
* [pipeline/chimera] test chimera | fix bug of initializing
* [pipeline/pytree] add pytree to process args and kwargs | provide to process args and kwargs after forward
* [pipeline/fix-bug] num_microbatches support any integrate | stable chimera | launch tool for rpc pp framework
2022-10-10 16:01:02 +08:00
e5ab6be72e
[hotfix[ fix colotensor.type() raise NotImplementedError ( #1682 )
2022-10-10 10:13:31 +08:00
3b2a59b0ba
[pipeline/rank_recorder] fix bug when process data before backward | add a tool for multiple ranks debug ( #1681 )
...
* [pipeline/tuning] improve dispatch performance both time and space cost
* [pipeline/converge] add interface for testing convergence
* [NFC] polish colossalai/utils/multi_tensor_apply/multi_tensor_apply.py code style
* Update PipelineBase.py
* [pipeline/chimera] reconstruct PipelineBase and Worker to support more feasible custom schedule | finish Chimera
* [pipeline/chimera] test chimera | fix bug of initializing
* [pipeline/pytree] add pytree to process args and kwargs | provide to process args and kwargs after forward
2022-10-09 17:32:57 +08:00
517b63939a
[autoparallel] add unary element wise handler v2 ( #1674 )
2022-10-09 17:30:42 +08:00
f6c6a932b8
[autoparallel] add following node generator ( #1673 )
...
* [autoparallel] add following node generator
* polish code
* polish code
* update name of arguments
2022-10-09 14:49:18 +08:00
52fda88796
[autoparallel] add layer norm handler v2 ( #1671 )
...
* [autoparallel] add layer norm handler v2
* polish code
* polish code
2022-10-09 14:23:22 +08:00
87c5ad352a
update version to 0.1.10 ( #1676 )
2022-10-09 10:43:29 +08:00
b28991dd0a
[feature] A new ZeRO implementation ( #1644 )
2022-10-09 09:18:51 +08:00
b1be5b88bd
[autoparallel] fix insecure subprocess ( #1680 )
...
* [autoparallel] fix insecure subprocess
* [fx] fix insecure subprocess
2022-10-06 15:07:03 +08:00
Ofey Chan