github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
ColossalAI/docs/source/zh-Hans/features
History
duanjunwen a9bedc7a43
[Sharderformer] Support zbv in Sharderformer Policy (#6150) 
* [feat] Sharderformer support zbv

* [feat] support chatglm2, command, deepseek for zbv

* [feat] support zbv in shardformer policy:
falcon,gptj,mistral,opt,qwen2,t5, vit, whisper

* [feat] support GPT2FusedLinearConv1D

* [feat] support GPT2FusedLinear (without tp)

* [fix] debug FusedConvLinear

* [shardfromer] support gpt2 policy for zbv, support GPT2FusedLinearConv
Col and Row.

* [Shardformer] support FusedLinear1D base for zbv

* [shardformer] support zbv in FusedLinear1D base, Col, Row

* [shardformer] support zbv in blip2 and sam policy

* [shardformer] fix bug incorrect number of gradients; add fusedLinear
base testcase;

* [fix] fix incorrect number of gradients ;

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [Shardformer] add en doc for zbv;

* [fix] fix typo in Model compatibility table

* [fix] fix API Reference typo

* [Shardformer] add zh-Han doc for zbv

* [fix] fix Linear name; update en & zh doc

* [fix] fix shardformer doc import err

* [fix] fix shardconfig import in doc

* [fix] fix shardformer doc

* [fix] fix shardconfig doc

* [fix] fix config

* [fix] remove shardconfig

* [fix] fix doc

* [feat] add zbv doc string

* [fix] rm doc

* [fix] fix doc

* [fix] empty zbv doc

* [fix] ifx torch version

* [fix] fix torch version

* [fix] fix torch versions

* [fix] fix torch versions

* [fix] fix pyramid versions

* [fix] fix pyramid, zope version

* [fix] try fix workflow

* [fix] try import ShardConfig in yml

* [fix] fix workflow

* [fix] fix workflow

* [fix] fix workflow

* [fix] fix workflow

* [fix] fix ci

* [fix] fix zbv doc

* [fix] fix param for qkv linear, gpt2fused linear; fix requirments;

* [fix] fix policy use fused_linear

* [fix] fix weight grad none, err caused by  weight ptr change

* [fix] fix comm in WeightGradStore

* [fix] fix WeightGradStore pop param

* [fix] remove useless param in doc; fix gpt2 qkv test;

* [shardformer] simplify execute_w_pass_grad_accum;

* [fix] rm useless comments

* [shardformer] simplify execute_w_pass_grad_accum & execute_w_pass

* [shardformer] Run meaningful doc test

* [shadformer] fix doc test cmd;

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 		2025-01-02 10:22:26 +08:00
..
1D_tensor_parallel.md [doc] Fix typo (#5361) 2024-02-19 16:53:28 +08:00
2D_tensor_parallel.md [doc] clean up outdated docs (#4765) 2023-09-21 11:36:20 +08:00
2p5D_tensor_parallel.md [doc] clean up outdated docs (#4765) 2023-09-21 11:36:20 +08:00
3D_tensor_parallel.md [doc] clean up outdated docs (#4765) 2023-09-21 11:36:20 +08:00
cluster_utils.md [doc] add booster docstring and fix autodoc (#3789) 2023-05-22 10:56:47 +08:00
distributed_optimizers.md [doc] Update llama + sp compatibility; fix dist optim table 2024-07-01 17:07:22 +08:00
gradient_accumulation_with_booster.md [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
gradient_clipping_with_booster.md [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
lazy_init.md [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
mixed_precision_training_with_booster.md [misc] fit torch api upgradation and remove legecy import (#6093) 2024-10-18 16:48:52 +08:00
nvme_offload.md [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
pipeline_parallel.md [hotfix] set return_outputs=False in examples and polish code (#5404) 2024-03-25 12:31:09 +08:00
sequence_parallelism.md [doc] update sp doc (#6055) 2024-09-11 17:25:14 +08:00
shardformer.md [doc] Update llama + sp compatibility; fix dist optim table 2024-07-01 17:07:22 +08:00
zero_with_chunk.md [misc] refactor launch API and tensor constructor (#5666) 2024-04-29 10:40:11 +08:00
zerobubble_pipeline_parallelism.md [Sharderformer] Support zbv in Sharderformer Policy (#6150) 2025-01-02 10:22:26 +08:00