1173 Commits (b0a23dc4fc8a0651268e98568a95ab87452ce754)

 

All Branches   

Search

Author	SHA1	Message	Date
Jiarui Fang	b0a23dc4fc	[embeddings] add doc in readme (#1711)	2 years ago
Frank Lee	22a115406b	[autoparallel] fixed broken node handler tests (#1708)	2 years ago
HELSON	1468e4bcfc	[zero] add constant placement policy (#1705) ... * fixes memory leak when paramter is in fp16 in ZeroDDP init. * bans chunk releasement in CUDA. Only when a chunk is about to offload, it is allowed to release. * adds a constant placement policy. With it, users can allocate a reserved caching memory space for parameters.	2 years ago
binmakeswell	5f41463a76	add optimizer README for tutorials (#1707)	2 years ago
Frank Lee	6c331a5a09	[autoparallel] refactored the autoparallel module for organization (#1706) ... * [autoparallel] refactored the autoparallel module for organization * polish code	2 years ago
Frank Lee	91cd34e6e0	[unittest] added doc for the pytest wrapper (#1704)	2 years ago
YuliangLiu0306	451cd72dea	[autoparallel] adapt runtime passes (#1703) ... * [autoparallel] adapt runtime passes v2 * polish code	2 years ago
Jiarui Fang	21962e1593	[embedding] rename FreqAwareEmbedding -> CachedEmbedding (#1699)	2 years ago
Frank Lee	0e52f3d3d5	[unittest] supported condititonal testing based on env var (#1701) ... polish code	2 years ago
Frank Lee	8283e95db3	[autoparallel] collated all deprecated files (#1700) ... * [autoparallel] collated all deprecated files * polish code	2 years ago
Frank Lee	e2355d01b9	[autoparallel] init new folder structure (#1696)	2 years ago
YuliangLiu0306	81f7530ee7	[autoparallel] adapt solver and CostGraph with new handler (#1695) ... * [autoparallel] adapt solver and CostGraph with new handler * fix test issue	2 years ago
YuliangLiu0306	42b882ef06	[autoparallel] add output handler and placeholder handler (#1694) ... * [autoparallel] add output handler and placeholder handler * Delete test_solver_with_resnet.py * fix test bugs	2 years ago
YuliangLiu0306	56088e6d98	[autoparallel] add pooling handler (#1690) ... * [autoparallel] add pooling handler * polish code	2 years ago
YuliangLiu0306	319d654f79	[autoparallel] where_handler_v2 (#1688) ... * where generator * [autoparallel] where_handler_v2	2 years ago
Boyuan Yao	31d2f03d27	[autoparallel] fix C version rotor inconsistency (#1691)	2 years ago
Jiarui Fang	363fc2861a	[embeddings] more detailed timer (#1692)	2 years ago
Frank Lee	4973157ad7	[autoparallel] added sharding spec conversion for linear handler (#1687)	2 years ago
YuliangLiu0306	af718e83f2	[autoparallel] add reshape handler v2 and fix some previous bug (#1683)	2 years ago
YuliangLiu0306	6878e42248	[hotfix] solver bug caused by dict type comm cost (#1686)	2 years ago
Super Daniel	3dd6994427	[fx/profiler] assigned UUID to each unrecorded tensor/ improved performance on GPT-2 (#1679) ... * [fx/profiler] modify data_ptr into uuid for all tensors. * [fx] modify uuid. * [fx/profiler] tune performance on GPT-2. * [fx] updates. * [fx] debug. * [fx] debug. * [fx] cuda.	2 years ago
Kirigaya Kazuto	0df5034a36	[pipeline/fix-bug] num_microbatches support any integrate | stable chimera | launch tool for rpc pp framework (#1684) ... * [pipeline/tuning] improve dispatch performance both time and space cost * [pipeline/converge] add interface for testing convergence * [NFC] polish colossalai/utils/multi_tensor_apply/multi_tensor_apply.py code style * Update PipelineBase.py * [pipeline/chimera] reconstruct PipelineBase and Worker to support more feasible custom schedule | finish Chimera * [pipeline/chimera] test chimera | fix bug of initializing * [pipeline/pytree] add pytree to process args and kwargs | provide to process args and kwargs after forward * [pipeline/fix-bug] num_microbatches support any integrate | stable chimera | launch tool for rpc pp framework	2 years ago
jim	e5ab6be72e	[hotfix[ fix colotensor.type() raise NotImplementedError (#1682)	2 years ago
Kirigaya Kazuto	3b2a59b0ba	[pipeline/rank_recorder] fix bug when process data before backward | add a tool for multiple ranks debug (#1681) ... * [pipeline/tuning] improve dispatch performance both time and space cost * [pipeline/converge] add interface for testing convergence * [NFC] polish colossalai/utils/multi_tensor_apply/multi_tensor_apply.py code style * Update PipelineBase.py * [pipeline/chimera] reconstruct PipelineBase and Worker to support more feasible custom schedule | finish Chimera * [pipeline/chimera] test chimera | fix bug of initializing * [pipeline/pytree] add pytree to process args and kwargs | provide to process args and kwargs after forward	2 years ago
YuliangLiu0306	517b63939a	[autoparallel] add unary element wise handler v2 (#1674)	2 years ago
YuliangLiu0306	f6c6a932b8	[autoparallel] add following node generator (#1673) ... * [autoparallel] add following node generator * polish code * polish code * update name of arguments	2 years ago
YuliangLiu0306	52fda88796	[autoparallel] add layer norm handler v2 (#1671) ... * [autoparallel] add layer norm handler v2 * polish code * polish code	2 years ago
Fazzie-Maqianli	87c5ad352a	update version to 0.1.10 (#1676)	2 years ago
HELSON	b28991dd0a	[feature] A new ZeRO implementation (#1644)	2 years ago
Boyuan Yao	b1be5b88bd	[autoparallel] fix insecure subprocess (#1680) ... * [autoparallel] fix insecure subprocess * [fx] fix insecure subprocess	2 years ago
Boyuan Yao	d8420f81a4	[hotfix] fix wrong type name in profiler (#1678)	2 years ago
Boyuan Yao	132b4306b7	[fx] Add concrete info prop (#1677) ... * [fx] concreteinfoprop * [fx] add concreteinfoprop * [fx] modify docstring of ConcreteInfoProp * [fx] fix device error * [fx] modify parameter calculation * [fx] modify parameters calculation	2 years ago
Boyuan Yao	1df98d5b66	[autoparallel] add rotor C version (#1658) ... * [autoparallel] add rotor c version * [fx] remove metainfoprop in rotor solver * [autoparallel] modify C code format * [autoparallel] remove build.py * [autoparallel] fix C extension build * [autoparallel] add C solver consistency test * [autoparallel] remove some unused imports * [autoparallel] refactor rotor solver code * [autoparallel] replace print with colossalai logger * [autoparallel] ranks fixed	2 years ago
YuliangLiu0306	11ec070e53	[hotfix]unit test (#1670)	2 years ago
Frank Lee	a60024e77a	[autoparallel] added utils for broadcast operation (#1665) ... * [autoparallel] added utils for broadcast operation * polish code	2 years ago
YuliangLiu0306	3f068d1409	[autoparallel] update CommSpec (#1667)	2 years ago
Frank Lee	247a9dbca9	[autoparallel] added bias comm spec to matmul strategy (#1664)	2 years ago
YuliangLiu0306	746f8f979d	[autoparallel] add batch norm handler v2 (#1666)	2 years ago
Kirigaya Kazuto	9708638ded	[pipeline/pytree] add pytree to process args and kwargs | provide `data_process_func` to process args and kwargs after forward (#1642) ... * [pipeline/tuning] improve dispatch performance both time and space cost * [pipeline/converge] add interface for testing convergence * [NFC] polish colossalai/utils/multi_tensor_apply/multi_tensor_apply.py code style * Update PipelineBase.py * [pipeline/chimera] reconstruct PipelineBase and Worker to support more feasible custom schedule | finish Chimera * [pipeline/chimera] test chimera | fix bug of initializing * [pipeline/pytree] add pytree to process args and kwargs | provide to process args and kwargs after forward	2 years ago
YuliangLiu0306	c27e701cb2	[autoparallel] remove no strategy nodes (#1652) ... * [autoparallel] remove no strategy nodes * fix none object iteration issue	2 years ago
Frank Lee	50f16a2850	[autoparallel] added compute resharding costs for node handler (#1662)	2 years ago
Frank Lee	9ec401a722	[autoparallel] added new strategy constructor template (#1661) ... * [autoparallel] added new strategy constructor template * polish code	2 years ago
Frank Lee	3a4d6f63a8	[autoparallel] added node handler for bmm (#1655)	2 years ago
YuliangLiu0306	095854477f	[autoparallel] add conv handler v2 (#1663)	2 years ago
YuliangLiu0306	1e7816a460	[autoparallel] adapt solver with gpt (#1653)	2 years ago
Jiarui Fang	c638bec028	[embedding] polish async copy (#1657)	2 years ago
Jiarui Fang	988570e4a6	[embedding] add more detail profiling (#1656)	2 years ago
Jiarui Fang	e1f97fd2b8	[embedding] print profiling results (#1654)	2 years ago
Frank Lee	30e50c8b4a	[autoparallel] implemented all matmul strategy generator (#1650)	2 years ago
YuliangLiu0306	03978aad45	[autoparallel] change the following nodes strategies generation logic (#1636) ... * [autoparallel] change the following nodes strategies generation logic * fix unit test	2 years ago