ColossalAI

Commit Graph

Author	SHA1	Message	Date
Hongxin Liu	079bf3cb26	[misc] update pre-commit and run all files (#4752 ) * [misc] update pre-commit * [misc] run pre-commit * [misc] remove useless configuration files * [misc] ignore cuda for clang-format	2023-09-19 14:20:26 +08:00
digger-yu	b9a8dff7e5	[doc] Fix typo under colossalai and doc(#3618 ) * Fixed several spelling errors under colossalai * Fix the spelling error in colossalai and docs directory * Cautious Changed the spelling error under the example folder * Update runtime_preparation_pass.py revert autograft to autograd * Update search_chunk.py utile to until * Update check_installation.py change misteach to mismatch in line 91 * Update 1D_tensor_parallel.md revert to perceptron * Update 2D_tensor_parallel.md revert to perceptron in line 73 * Update 2p5D_tensor_parallel.md revert to perceptron in line 71 * Update 3D_tensor_parallel.md revert to perceptron in line 80 * Update README.md revert to resnet in line 42 * Update reorder_graph.py revert to indice in line 7 * Update p2p.py revert to megatron in line 94 * Update initialize.py revert to torchrun in line 198 * Update routers.py change to detailed in line 63 * Update routers.py change to detailed in line 146 * Update README.md revert random number in line 402	2023-04-26 11:38:43 +08:00
Xuanlei Zhao	2ca9728cbb	[autochunk] refactor chunk memory estimation (#2762 ) * refact memory code * dont log free var memory * add memory align * update chunk target * update setting for new memory * finish test * update tracer * update typo * update test	2023-03-08 16:22:30 +08:00
oahzxl	05671fcb42	[autochunk] support multi outputs chunk search (#2538 ) Support multi outputs chunk search. Previously we only support single output chunk search. It is more flexible and improve performance by a large margin. For transformer, we reduce memory by 40% than previous search strategy. 1. rewrite search strategy to support multi outputs chunk search 2. fix many, many bugs 3. update tests	2023-02-01 13:18:51 +08:00
oahzxl	c04f183237	[autochunk] support parsing blocks (#2506 )	2023-01-20 11:18:17 +08:00
oahzxl	72341e65f4	[auto-chunk] support extramsa (#3 ) (#2504 )	2023-01-20 10:13:03 +08:00
oahzxl	ecccc91f21	[autochunk] support autochunk on evoformer (#2497 )	2023-01-19 11:41:00 +08:00
oahzxl	7ab2db206f	adapt new fx	2023-01-10 11:56:00 +08:00
oahzxl	7d4abaa525	add doc	2023-01-10 09:59:47 +08:00
oahzxl	4d223e18a2	fix typo	2023-01-09 13:46:17 +08:00

10 Commits (af952673f758c71126b27de8b32bdf5df8f74b69)