github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/examples/language
History
Hongxin Liu c068ef0fa0
[zero] support all-gather overlap (#5898) 
* [zero] support all-gather overlap

* [zero] add overlap all-gather flag

* [misc] fix typo

* [zero] update api
 		2024-07-11 18:59:59 +08:00
..
bert [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
commons [example] make gpt example directory more clear (#2353) 2023-01-06 11:11:26 +08:00
gpt [chore] refactor profiler utils 2024-05-28 12:41:42 +00:00
grok-1 [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
llama [Hoxfix] Fix CUDA_DEVICE_MAX_CONNECTIONS for comm overlap 2024-07-05 20:02:36 +08:00
openmoe [MoE/ZeRO] Moe refactor with zero refactor (#5821) 2024-06-28 14:00:08 +08:00
opt [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
palm [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
__init__.py [example]add gpt2 benchmark example script. (#5295) 2024-03-04 16:18:13 +08:00
data_utils.py [devops] remove post commit ci (#5566) 2024-04-08 15:09:40 +08:00
model_utils.py [example]add gpt2 benchmark example script. (#5295) 2024-03-04 16:18:13 +08:00
performance_evaluator.py [zero] support all-gather overlap (#5898) 2024-07-11 18:59:59 +08:00