github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
3,689 Commits
33 Branches
48 Tags
232 MiB
Commit Graph

45 Commits (fafe049b83bad3a6aa6e3a31c68b38ac63167b53)

Author SHA1 Message Date
duanjunwen fafe049b83 [fix] fix handle name; rm useless comments; 2024-10-29 03:24:15 +00:00
duanjunwen d0ec221b38 [fix\ fix fail case test_shard_llama 2024-10-25 02:28:55 +00:00
duanjunwen 2eca112c90 [feat] support meta cache, meta_grad_send, meta_tensor_send; fix runtime too long in Recv Bwd; benchmark for llama + Hybrid(tp+pp); 2024-10-24 07:30:19 +00:00
duanjunwen 9912cc8c07 [fix] fix bwd b; now bwd w only for Layer replaced by Linear1D_Col/Row; other layer perform a fully bwd; 2024-10-15 06:26:01 +00:00
duanjunwen cfade4c36d [feat] Linear1D_COL/ROW support zbv WeightGradStore; 2024-10-14 07:02:43 +00:00
duanjunwen 0ca16d5cbe [fix] fix llama, mixtral benchmark zbv loss none bug; update mixtral & llama policy and modeling; 2024-10-11 07:32:43 +00:00
duanjunwen 1342a983b1 [fix] rm print & comments; 2024-09-26 11:05:27 +00:00
duanjunwen 64ceea746f [fix] remove chunk 0 stage 0 bwd b; u don't have to cal micrbatch's dx; 2024-09-26 10:50:44 +00:00
duanjunwen a92e16719b [fix] fix zerobubble; support shardformer model type; 2024-09-26 06:11:56 +00:00
duanjunwen 83163fa70c [fix] fix traverse; traverse dict --> traverse tensor List; 2024-09-25 06:38:11 +00:00
duanjunwen fc8b016887 [fix] fix stage_indices; 2024-09-25 06:15:45 +00:00
duanjunwen 7e6f793c51 [fix] fix detach_output_obj clone; 2024-09-24 08:08:32 +00:00
duanjunwen 6c1e1550ae [fix] fix dumb clone; 2024-09-23 06:43:49 +00:00
duanjunwen c114d1429a [fix] fix detach clone release order; 2024-09-23 04:00:24 +00:00
duanjunwen da3220f48c [fix] fix pipeline util func deallocate --> release_tensor_data; fix bwd_b loss bwd branch; 2024-09-20 09:48:35 +00:00
duanjunwen 1739df423c [fix] fix fwd branch, fwd pass both micro_batch & internal_inputs' 2024-09-20 07:34:43 +00:00
duanjunwen b6616f544e [fix] rm comments; 2024-09-20 07:29:41 +00:00
duanjunwen c6d6ee39bd [fix] use tree_flatten replace dict traverse; 2024-09-20 07:18:49 +00:00
duanjunwen 26783776f1 [fix] fix input_tensors buffer append input_obj(dict) --> Tuple (microbatch, input_obj) , and all bwd b related cal logic; 2024-09-20 06:41:19 +00:00
duanjunwen a115106f8d [fix] fix bwd w input; 2024-09-19 08:10:05 +00:00
duanjunwen 349272c71f [fix] updatw bwd b&w input; dict --> list[torch.Tensor] 2024-09-19 07:47:01 +00:00
duanjunwen 6ee9584b9a [fix] fix require_grad & deallocate call; 2024-09-19 05:53:03 +00:00
duanjunwen 3dbad102cf [fix] fix zerobubble pp for shardformer type input; 2024-09-18 07:14:34 +00:00
duanjunwen ce58d8e8bf [fix] add output_obj_grad assert None at bwd b step; replace input_obj.require_grad_ with treemap; 2024-09-09 08:19:58 +00:00
duanjunwen 7568b34626 [fix] fix redundant detach & clone; add buffer assertation in the end; 2024-09-09 08:04:28 +00:00
duanjunwen e6e1a97a6d [fix] fix requir grad position and detach position and input&output local buffer append position; 2024-09-04 03:31:08 +00:00
duanjunwen 20503cdfdf [fix] rm requir_grad for output; 2024-09-03 09:24:40 +00:00
duanjunwen b4103f125c [fix] fix detach output & release output; 2024-09-03 09:09:41 +00:00
duanjunwen 4c1f81c683 [fix] fix bwd step if condition; remove useless comments and format info; 2024-09-03 08:56:08 +00:00
duanjunwen ab643c9af7 [fix] rm output.data after send fwd; 2024-09-03 14:12:17 +08:00
duanjunwen 591a13bf7e [fix] fix optim bwd; 2024-09-02 11:19:42 +00:00
duanjunwen 6d18d38d5c [feat] update test; rm comments; 2024-09-02 09:50:47 +00:00
duanjunwen a7b767b071 [fix] fix communication_map; 2024-08-30 05:56:02 +00:00
duanjunwen 8eb6eac225 [fix] fix optim bwd; add license for v_schedule; remove redundant attributes; fix schedule loop "while"--> "for"; add communication dict; 2024-08-30 05:42:43 +00:00
duanjunwen 6af81d8c0d [feat] add fwd_bwd_step, run_fwd_only; 2024-08-30 02:47:52 +00:00
duanjunwen 48ba22dbfd [feat] fix optimizer bwd b & w; support return accum loss & output 2024-08-29 08:54:45 +00:00
duanjunwen 4c4b01b859 [feat] add optim backward_b_by_grad 2024-08-29 03:16:59 +00:00
duanjunwen 8b37323f16 [feat] add run_fwd_bwd_with_microbatch (replace input) & test; add p&p.grad assert close test & all pass; 2024-08-27 09:31:38 +00:00
duanjunwen 9e0bd1af00 [fix] fix ci test; add pytest; 2024-08-27 08:00:23 +00:00
duanjunwen 283c9ff5d2 [fix] rm useless assign and comments; 2024-08-27 07:31:58 +00:00
duanjunwen 1b4bb2beeb [feat] add comments for ZBV func; 2024-08-27 07:11:50 +00:00
duanjunwen 5e09c8b4e1 [feat] split communication and calculation; fix pop empty send_bwd_buffer error; 2024-08-27 06:29:13 +00:00
duanjunwen 1d75045c37 [feat] add test run_fwd_bwd automatic scheduling; 2024-08-26 11:21:56 +00:00
duanjunwen c18ef060cf [feat] add dw test; 2024-08-23 06:04:12 +00:00
duanjunwen ee9baedadf [feat] add zerobubble pp (just a frame now); add POC test for dx_dw; add test for zerobubble; 2024-08-22 10:25:34 +00:00