167 Commits (152162a80e95c6e30d335f6af805f1107bc6e35e)

Author SHA1 Message Date
Jiarui Fang 1190b2c4a4
[tensor] add cross_entrophy_loss (#868) 3 years ago
Jiarui Fang d01d3b8cb0
colo init context add device attr. (#866) 3 years ago
Jiarui Fang 126ba573a8
[Tensor] add layer norm Op (#852) 3 years ago
Ziyue Jiang bcc8655021
[Tensor ] Add 1Drow weight reshard by spec (#854) 3 years ago
Jiarui Fang 62f059251b
[Tensor] init a tp network training unittest (#849) 3 years ago
Ziyue Jiang 2a0a427e04
[tensor]add assert for colo_tensor 1Drow (#846) 3 years ago
Ziyue Jiang 05023ecfee
[Tensor] TP Linear 1D row (#843) 3 years ago
Jiarui Fang ea0a2ed25f
[hotfix] the bug of numel() in ColoTensor (#845) 3 years ago
Jiarui Fang 8789850eea
Init Conext supports lazy allocate model memory (#842) 3 years ago
Jiarui Fang cb5a4778e1
Revert "[WIP] Applying ColoTensor on TP-1D-row Linear. (#831)" (#835) 3 years ago
Jiarui Fang ac88de6dfc
[WIP] Applying ColoTensor on TP-1D-row Linear. (#831) 3 years ago
Jiarui Fang 294a6060d0
[tensor] ZeRO use ColoTensor as the base class. (#828) 3 years ago
Ziyue Jiang 8e6fdb4f29
[tensor]fix test_linear (#826) 3 years ago
Ziyue Jiang 1a9e2c2dff
[tensor] fix kwargs in colo_tensor torch_funtion (#825) 3 years ago
Jiarui Fang 2ecc3d7a55
[tensor] lazy init (#823) 3 years ago
Jiarui Fang 660d2d1f1b
[Tensor] apply ColoTensor on Torch functions (#821) 3 years ago
Jiarui Fang 0ce8924ceb
[tensor] reorganize files (#820) 3 years ago