github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/colossalai/nn/layer
History
HELSON e6d50ec107
[zero] adapt zero for unsharded parameters (#561) 
* support existing sharded and unsharded parameters in zero

* add unitest for moe-zero model init

* polish moe gradient handler
 		2022-03-31 18:34:11 +08:00
..
colossalai_layer [TP] Add gather_out arg to Linear (#541) 2022-03-30 09:35:46 +08:00
moe [zero] adapt zero for unsharded parameters (#561) 2022-03-31 18:34:11 +08:00
parallel_1d update code format 2022-03-31 17:15:08 +08:00
parallel_2d Refactored docstring to google style 2022-03-29 17:17:47 +08:00
parallel_2p5d Refactored docstring to google style 2022-03-29 17:17:47 +08:00
parallel_3d html refactor (#555) 2022-03-31 11:36:56 +08:00
parallel_sequence Refactored docstring to google style 2022-03-29 17:17:47 +08:00
utils Refactored docstring to google style 2022-03-29 17:17:47 +08:00
vanilla Refactored docstring to google style 2022-03-29 17:17:47 +08:00
wrapper Refactored docstring to google style 2022-03-29 17:17:47 +08:00
__init__.py [MOE] changed parallelmode to dist process group (#460) 2022-03-19 13:46:29 +08:00
base_layer.py Migrated project 2021-10-28 18:21:23 +02:00