ColossalAI

Commit Graph

Author	SHA1	Message	Date
Kirigaya Kazuto	6159d45417	[pipeline/tuning] improve dispatch performance both time and space cost (#1544 )	2 years ago
Kirigaya Kazuto	f1e1836218	[pipeline/pipleline_process_group] finish PipelineProcessGroup to manage local abd global rank in TP,DP and PP (#1508 ) * support p2p communication with any type of object \| pass test * reconstruct pipeline schedule with p2p_v2.py(support communication with List[Any]) \| pass test * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * [pipeline/rpc] implement a demo for PP with cuda rpc framework * [pipeline/rpc] support interleaving \| fix checkpoint bug \| change logic when dispatch data in work_list to ensure steady 1F1B * [pipeline/rpc] implement distributed optimizer \| test with assert_close * [pipeline/rpc] implement distributed optimizer \| test with assert_close * [pipeline/rpc] update outstanding mechanism \| optimize dispatching strategy * [pipeline/rpc] update outstanding mechanism \| optimize dispatching strategy * [pipeline/rpc] update outstanding mechanism \| optimize dispatching strategy * [pipeline/pipleline_process_group] finish PipelineProcessGroup to manage local abd global rank in TP,DP and PP * [pipeline/pipleline_process_group] remove comment * [pipeline/pipleline_process_group] remove comment * [pipeline/pipleline_process_group] skip process group test * [pipeline/pipleline_process_group] remove test named function	2 years ago
Kirigaya Kazuto	5a6fd71f90	[pipeline/rpc] update outstanding mechanism \| optimize dispatching strategy (#1497 ) * support p2p communication with any type of object \| pass test * reconstruct pipeline schedule with p2p_v2.py(support communication with List[Any]) \| pass test * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * [pipeline/rpc] implement a demo for PP with cuda rpc framework * [pipeline/rpc] support interleaving \| fix checkpoint bug \| change logic when dispatch data in work_list to ensure steady 1F1B * [pipeline/rpc] implement distributed optimizer \| test with assert_close * [pipeline/rpc] implement distributed optimizer \| test with assert_close * [pipeline/rpc] update outstanding mechanism \| optimize dispatching strategy * [pipeline/rpc] update outstanding mechanism \| optimize dispatching strategy * [pipeline/rpc] update outstanding mechanism \| optimize dispatching strategy	2 years ago
Kirigaya Kazuto	9145aef2b4	[pipeline/rpc] implement distributed optimizer \| test with assert_close (#1486 ) * support p2p communication with any type of object \| pass test * reconstruct pipeline schedule with p2p_v2.py(support communication with List[Any]) \| pass test * [engin/schedule] use p2p_v2 to recontruct pipeline_schedule * [pipeline/rpc] implement a demo for PP with cuda rpc framework * [pipeline/rpc] support interleaving \| fix checkpoint bug \| change logic when dispatch data in work_list to ensure steady 1F1B * [pipeline/rpc] implement distributed optimizer \| test with assert_close * [pipeline/rpc] implement distributed optimizer \| test with assert_close	2 years ago

4 Commits (c7d4932956175bb42461d63b00e31ecca113a4d7)