2653 Commits (726541afe2cde5c6f547968a3d232bbb8b3f5f14)

 

All Branches   

Search

Author	SHA1	Message	Date
CZYCW	dee1c96344	[NFC] policy applications/Chat/examples/ray/mmmt_prompt.py code style (#4250)	1 year ago
Junming Wu	77c469e1ba	[NFC] polish applications/Chat/coati/models/base/actor.py code style (#4248)	1 year ago
Camille Zhong	915ed8bed1	[NFC] polish applications/Chat/inference/requirements.txt code style (#4265)	1 year ago
Michelle	86cf6aed5b	Fix/format (#4261) ... * revise shardformer readme (#4246) * [example] add llama pretraining (#4257) * [NFC] polish colossalai/communication/p2p.py code style --------- Co-authored-by: Jianghai <72591262+CjhHa1@users.noreply.github.com> Co-authored-by: binmakeswell <binmakeswell@gmail.com> Co-authored-by: Qianran Ma <qianranm@luchentech.com>	1 year ago
Jianghai	b366f1d99f	[NFC] Fix format for mixed precision (#4253) ... * [NFC] polish colossalai/booster/mixed_precision/mixed_precision_base.py code style	1 year ago
Hongxin Liu	02192a632e	[ci] support testmon core pkg change detection (#4305)	1 year ago
Baizhou Zhang	c6f6005990	[checkpointio] Sharded Optimizer Checkpoint for Gemini Plugin (#4302) ... * sharded optimizer checkpoint for gemini plugin * modify test to reduce testing time * update doc * fix bug when keep_gatherd is true under GeminiPlugin	1 year ago
Hongxin Liu	fc5cef2c79	[lazy] support init on cuda (#4269) ... * [lazy] support init on cuda * [test] update lazy init test * [test] fix transformer version	1 year ago
Cuiqing Li	4b977541a8	[Kernels] added triton-implemented of self attention for colossal-ai (#4241) ... * added softmax kernel * added qkv_kernel * added ops * adding tests * upload tets * fix tests * debugging * debugging tests * debugging * added * fixed errors * added softmax kernel * clean codes * added tests * update tests * update tests * added attention * add * fixed pytest checking * add cuda check * fix cuda version * fix typo	1 year ago
binmakeswell	7ff11b5537	[example] add llama pretraining (#4257)	1 year ago
Jianghai	9a4842c571	revise shardformer readme (#4246)	1 year ago
github-actions[bot]	4e9b09c222	Automated submodule synchronization (#4217) ... Co-authored-by: github-actions <github-actions@github.com>	1 year ago
Frank Lee	c1cf752021	[docker] fixed ninja build command (#4203) ... * [docker] fixed ninja build command * polish code	1 year ago
Baizhou Zhang	58913441a1	Next commit [checkpointio] Unsharded Optimizer Checkpoint for Gemini Plugin (#4141) ... * [checkpointio] unsharded optimizer checkpoint for Gemini plugin * [checkpointio] unsharded optimizer checkpoint for Gemini using all_gather	1 year ago
Frank Lee	fee32a3b78	[docker] added ssh and rdma support for docker (#4192)	1 year ago
Frank Lee	190a6ea9c2	[dtensor] fixed readme file name and removed deprecated file (#4162)	1 year ago
Frank Lee	cc3cbe9f6f	[workflow] show test duration (#4159)	1 year ago
Hongxin Liu	1908caad38	[cli] hotfix launch command for multi-nodes (#4165)	1 year ago
digger yu	2ac24040eb	fix some typo colossalai/shardformer (#4160)	1 year ago
github-actions[bot]	c77b3b19be	[format] applied code formatting on changed files in pull request 4152 (#4157) ... Co-authored-by: github-actions <github-actions@github.com>	1 year ago
Frank Lee	f447ca1811	[chat] removed cache file (#4155)	1 year ago
Frank Lee	89f45eda5a	[shardformer] added development protocol for standardization (#4149)	1 year ago
Frank Lee	1fb0d95df0	[shardformer] made tensor parallelism configurable (#4144) ... * [shardformer] made tensor parallelism configurable * polish code	1 year ago
Frank Lee	74257cb446	[shardformer] refactored some doc and api (#4137) ... * [shardformer] refactored some doc and api * polish code	1 year ago
jiangmingyan	7f9b30335b	[shardformer] write an shardformer example with bert finetuning (#4126) ... * [shardformer] add benchmark of shardformer * [shardformer] add benchmark of shardformer	1 year ago
Frank Lee	ae035d305d	[shardformer] added embedding gradient check (#4124)	1 year ago
Frank Lee	44a190e6ac	[shardformer] import huggingface implicitly (#4101)	1 year ago
Frank Lee	6a88bae4ec	[shardformer] integrate with data parallelism (#4103)	1 year ago
Frank Lee	f3b6aaa6b7	[shardformer] supported fused normalization (#4112)	1 year ago
Frank Lee	b1c2901530	[shardformer] supported bloom model (#4098)	1 year ago
Kun Lin	8af29ee47a	[shardformer] support vision transformer (#4096) ... * first v of vit shardformer * keep vit * update * vit shard add vitattention vitlayer * update num head shard para * finish test for vit * add new_model_class & postprocess * add vit readme * delete old files & fix the conflict * fix sth	1 year ago
jiangmingyan	ac80937138	[shardformer] shardformer support opt models (#4091) ... * [shardformer] shardformer support opt models * [shardformer] shardformer support opt models, fix * [shardformer] shardformer support opt models, fix * [shardformer] shardformer support opt models, fix	1 year ago
Frank Lee	d33a44e8c3	[shardformer] refactored layernorm (#4086)	1 year ago
Frank Lee	c4b1b65931	[test] fixed tests failed due to dtensor change (#4082) ... * [test] fixed tests failed due to dtensor change * polish code	1 year ago
FoolPlayer	92f6791095	[shardformer] Add layernorm (#4072) ... * add layernorm to bert * add layernorm test * add layernorm test with load state dict * add use_mixedfusedLN in shard config * refactor policy to support fused_layernorm	1 year ago
Frank Lee	70c58cfd4f	[shardformer] supported fused qkv checkpoint (#4073)	1 year ago
FoolPlayer	0803a61412	[shardformer] add linearconv1d test (#4067) ... * add linearconv1d test * add linearconv1d test	1 year ago
Frank Lee	8eb09a4c69	[shardformer] support module saving and loading (#4062) ... * [shardformer] support module saving and loading * polish code	1 year ago
FoolPlayer	7740c55c55	support kit use for bert/gpt test (#4055) ... * support kit use for bert test * support kit test for gpt2	1 year ago
Frank Lee	f22ddacef0	[shardformer] refactored the shardformer layer structure (#4053)	1 year ago
Frank Lee	58df720570	[shardformer] adapted T5 and LLaMa test to use kit (#4049) ... * [shardformer] adapted T5 and LLaMa test to use kit * polish code	1 year ago
FoolPlayer	4021b9a8a2	[shardformer] add gpt2 test and layer class refactor (#4041) ... * add gpt2 test and layer class refactor * add dropout in gpt2 policy	1 year ago
Frank Lee	d857f3dbba	[shardformer] supported T5 and its variants (#4045)	1 year ago
Frank Lee	c1d5453e9f	[shardformer] adapted llama to the new API (#4036)	1 year ago
FoolPlayer	74d176c8d8	[shardformer] fix bert and gpt downstream with new api (#4024) ... * fix bert downstream with new api * remove comment line	1 year ago
Frank Lee	e253a07007	[shardformer] updated doc (#4016)	1 year ago
FoolPlayer	df018fc305	support bert with new api	1 year ago
FoolPlayer	507c0ad368	add vocabembedding layer	1 year ago
Frank Lee	45d9384346	[shardformer] removed inplace tensor sharding (#4018)	1 year ago
Frank Lee	3893fa1a8d	[shardformer] refactored embedding and dropout to parallel module (#4013) ... * [shardformer] refactored embedding and dropout to parallel module * polish code	1 year ago