ColossalAI/colossalai/nn/_ops
Jiarui Fang 1aad903c15
[tensor] redistribute among different process groups (#1247)
* make it faster

* [tensor] rename convert_to_dist -> redistribute

* [tensor] ShardSpec and ReplicaSpec

* [tensor] redistribute among diff pgs

* polish code
2022-07-12 10:24:05 +08:00
..
__init__.py [tensor] add embedding bag op (#1156) 2022-06-22 15:54:03 +08:00
_utils.py [refactor] move process group from _DistSpec to ColoTensor. (#1203) 2022-07-06 16:15:16 +08:00
addmm.py [tensor] redistribute among different process groups (#1247) 2022-07-12 10:24:05 +08:00
element_wise.py [tensor] add zero_like colo op, important for Optimizer (#1236) 2022-07-08 14:55:27 +08:00
embedding.py [tensor] redistribute among different process groups (#1247) 2022-07-12 10:24:05 +08:00
embedding_bag.py [tensor] a shorter shard and replicate spec (#1245) 2022-07-11 15:51:48 +08:00
layernorm.py [tensor] a shorter shard and replicate spec (#1245) 2022-07-11 15:51:48 +08:00
linear.py [tensor] redistribute among different process groups (#1247) 2022-07-12 10:24:05 +08:00
loss.py [tensor] fix a assertion in colo_tensor cross_entropy (#1232) 2022-07-08 11:18:00 +08:00