Commit Graph

6 Commits (307c4741d1de14f3f7e8fbbebbf5ab70e2075dd7)

Author SHA1 Message Date
ytxiong 307c4741d1
fix(initialize/launch.py): set default value for use_flash_attn (#158)
* add default for use_flash_attn

* fix lint
2023-08-01 16:03:06 +08:00
ytxiong 5ee651c2f1
feat(*): support not-flash-attn for pp and no-pp (#145)
* support not flash attention for no-pp

* support pipeline

* modify the config

* refactor the code

* refactor the code

* remove some unnecessary code
2023-07-28 16:13:04 +08:00
huangting4201 762ab297ee
feat(core/scheduler): support pipeline parallel (#98)
* feat(utils/writer.py): support tensorboard writer

* feat(utils/writer.py): add class comment

* feat(core): support pipeline parallel

* fix(core): fix demo running error

* feat(solver/optimizer): add pp zero optimizer

* fix(solver/optimizer): fix word spelling error

* feat(core/scheduler): add new dir scheduler in core/

* fix(core): fix ci lint error

* feat(solver/optimizer): merge pp and nopp optimizer

* doc(usage.md): update usage doc

* feat(core/scheduler): support post func

* feat(core/scheduler): add dtype para in pp sche and update func get_tensor_shape

* feat(core/scheduler): add _load_micro_batch in base scheduler

* feat(core/scheduler): support optimizer overlap communication in pp scheduler

* feat(core/scheduler): delete data process func code

* feat(core/trainer): schedule pre processing for all schedule

---------

Co-authored-by: 黄婷 <huangting3@CN0014010744M.local>
Co-authored-by: huangting.p <huangting@sensetime.com>
2023-07-24 20:52:09 +08:00
huangting4201 0d3d27cdf4
feat(utils/writer.py): support tensorboard writer (#63)
* feat(utils/writer.py): support tensorboard writer

* feat(utils/writer.py): add class comment

---------

Co-authored-by: 黄婷 <huangting3@CN0014010744M.local>
2023-07-21 15:53:24 +08:00
Sun Peng 912fc8f8aa
doc: update the training examples (#27)
* doc: update the training examples

* update README

* change all "++++" log

* Update pylint

* solve lint err
2023-07-07 15:54:09 +08:00
Sun Peng fa7337b37b initial commit 2023-07-06 12:55:23 +08:00