github
/
InternLM
mirror of https://github.com/InternLM/InternLM
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
299 Commits
4 Branches
8 Tags
10 MiB
Commit Graph

299 Commits (7e505f3c595dc961b88ed507207dbbf2ffb0f1eb)

Author SHA1 Message Date
Wenwen Qu 7e505f3c59 refactor code for sync_model_param() 2023-09-27 18:06:42 +08:00
Wenwen Qu 00478761f7 update 7B-sft.py 2023-09-27 17:57:30 +08:00
Wenwen Qu 9a1bd616d0 Merge branch 'feature_add_moe_data' of https://github.com/blankde/InternLM into feature_add_moe_data 2023-09-27 17:55:05 +08:00
Wenwen Qu e2b7a7fa89 set default expert parallel size 2023-09-27 17:51:58 +08:00
Wenwen Qu f5caa1c048 Merge branch 'feature_add_moe' into feature_add_moe_data 2023-09-27 15:01:35 +08:00
Wenwen Qu f96764868d change condition for compatibility 2023-09-27 12:36:03 +08:00
Wenwen Qu 630c713ef5 Merge branch 'feature_add_moe' into feature_add_moe_data 2023-09-27 11:30:15 +08:00
Wenwen Qu 591b4edb1d update moe config file 2023-09-27 11:28:36 +08:00
Wenwen Qu 2a70262ceb fix merge bugs 2023-09-27 11:17:03 +08:00
Wenwen Qu 8a63cb51ef modify moe config file 2023-09-26 21:01:06 +08:00
Wenwen Qu 54af6ba297 merge develop into feature_add_moe 2023-09-26 21:00:22 +08:00
Wenwen Qu 655e9dae40
Feat(norm)/support fused precision (#319) 
* add fused precision support for norm

* refactor code

* refactor code

* change the granularity of hook

* fix bugs if self.model is ModuleList

* add dtype condition for post hook

* refactor code for split group

* refactor code for pre/post hook

* refactor code for split group

* remove fp32 hook for norm

* unit tests for fused precision

* add doc for fused precision

* add doc for En. version

* reformat docs

* Update mixed_precision.rst

* Update mixed_precision.po

* update mixed_precision.po
 2023-09-26 20:39:55 +08:00
YWMditto 96b20cd43f
doc(usage): add dynamic ntk into doc (#367) 
* add long text generation in doc/usage.md

* add long text generation in doc/usage.md

* add long text generation in doc/usage.md

---------

Co-authored-by: YWMditto <862779238@qq.com>
 2023-09-26 16:58:46 +08:00
Wenwen Qu 55ae973cf0 update moe config file 2023-09-26 12:17:39 +08:00
Wenwen Qu 75774e0b5e Merge branch 'feature_add_moe' of https://github.com/blankde/InternLM into feature_add_moe 2023-09-26 11:52:52 +08:00
Wenwen Qu 3c8fee01b2 add compatible code for old version 2023-09-26 11:51:34 +08:00
jiaxingli c1e30cff2c
feat(numa): bind numa if possible (#320) 
* feat:add numa

* feat:add bind numa

* feat:add bind numa

* feat:add bind numa

* feat: bind numa

* feat: bind numa

* feat: add numa

* feat:add numa

* feat:add numa

* try_bind_numa should not raise exception

---------

Co-authored-by: 877825076@qq.com <877825076@qq.com>
 2023-09-25 19:34:52 +08:00
jiaopenglong 9284303a6d
doc(monitor): add light monitoring doc (#352) 
* add light monitoring doc

* update light monitoring doc

* update light monitoring doc

* update light monitoring doc

* update light monitoring doc continue

* update light monitoring doc continue

* update monitor config doc

* update monitor config doc continue

* update monitor config doc continue
 2023-09-25 19:28:09 +08:00
jiaopenglong 847cc819dd
fix(monitor): add volc and aliyun jobid (#338) 
* add volc and aliyun jobid

* rm workspaceid
 2023-09-25 17:58:32 +08:00
jiaopenglong 064965527b
fix(config): monitor config key error when args_check is False (#362) 
* add monitor switch

* add switch to light monitor

* fix alert_address is empty

* fix light monitor heartbeat

* init light_monitor on rank_log only

* add comments to the monitoring config

* optimize config

* fix monitor config key error when args_check is False
 2023-09-25 17:30:36 +08:00
zhanglei 1986116527 add moe module to `__init__.py` 2023-09-25 15:36:35 +08:00
Guoteng 26a7397752
fix(storage): fix try_get_storage_backend (#359) 
* fix(storage): fix try_get_storage_backend

* fix typo and print infos only in log rank

* fix typo and print infos only in log rank

---------

Co-authored-by: gaoyang07 <Gary1546308416AL@gmail.com>
 2023-09-25 15:16:25 +08:00
huangting4201 a86c4bbbfd Merge branch 'main' into develop 2023-09-22 19:24:03 +08:00
Guoteng d1e52f0c03
feat(doc/code-docs): add checkpoint save/load usage doc (#311) 
* feat(doc): add checkpoint doc

* fix checkpoint doc

* fix comment

* fix(doc/code-docs): remove fuzzy

* fix(doc/code-docs): fix some errors

* fix(doc/code-docs): minor fix

---------

Co-authored-by: li126com <li126com2@126.com>
Co-authored-by: huangting4201 <1538303371@qq.com>
 2023-09-22 18:45:33 +08:00
Wenwen Qu 85f4d4af58 fix bugs in save/load moe checkpoint 2023-09-22 17:54:33 +08:00
Wenwen Qu f6cadcafa2 fix bugs with split group 2023-09-22 16:05:21 +08:00
Wenwen Qu 1fdc7107b4 Merge branch 'feature_add_moe' of https://github.com/blankde/InternLM into feature_add_moe 2023-09-22 15:57:36 +08:00
zhanglei fd932cfc09 refactor 2023-09-22 15:52:37 +08:00
zhanglei ccdaf8ec45 fix the moe_loss for ci and val 2023-09-22 15:45:36 +08:00
Wenwen Qu a5e80bcc56 refactor code for split param group 2023-09-22 15:24:28 +08:00
zhanglei 3df0a51555 fix logger for moe 2023-09-22 14:56:43 +08:00
zhanglei 36d1bd2f41 Merge branch 'develop' of github.com:InternLM/InternLM into feature_add_moe 2023-09-22 14:36:58 +08:00
Ryan (张磊) aa7645a831
Merge pull request #4 from blankde/feature_add_moe_refactor_zl 
refactor code
 2023-09-22 14:22:45 +08:00
Wenwen Qu 9e6e7986b6 refactor code for log 2023-09-22 14:14:58 +08:00
huangting4201 1ed36754df
feat(.github/workflows): update ci e2e tests and add ci unit tests (#324) 
* feat(.github/workflows/e2e_test.yaml): update e2e yaml

* feat(.github/workflows/e2e_test.yaml): update e2e yaml

* test e2e

* test e2e

* test e2e

* test e2e

* test e2e

* fix(ci): test ci

* fix(ci): test ci

* fix(ci): test ci

* fix(ci): test ci

* fix(ci): test ci

* fix(ci): add weekly tests

---------

Co-authored-by: huangting4201 <huangting3@sensetime.com>
 2023-09-22 14:07:14 +08:00
jiaxingli f5337f6e02
Feat(PythonGC): Do garbage collection manually (#326) 
* feat:add gc control

* feat:add gc control

* feat:add gc control

* feat:add gc

* re-lint
 2023-09-22 13:52:25 +08:00
Wenwen Qu 3607548265 refactor code for split group 2023-09-22 13:00:26 +08:00
zhanglei 548d1bd7af refactor code 2023-09-22 12:30:02 +08:00
zhanglei 80972ff314 refactor code 2023-09-22 11:47:05 +08:00
Wenwen Qu 4ab2ef93af
Update 7B_sft.py 2023-09-21 16:41:28 +08:00
Wenwen Qu 1b25bc0325 fix merge bugs 2023-09-21 16:18:26 +08:00
Qu Wenwen b596255356 merge feature_add_moe into feature_add_moe_data 2023-09-21 15:47:31 +08:00
Qu Wenwen 17bc5f562b refactor code 2023-09-21 15:00:28 +08:00
huangting4201 3b0eff0c8a
fix(model/embedding.py): ci lint check error (#345) 
* fix(ci): fix ci lint error

* fix(ci): fix ci lint error
 2023-09-21 14:46:22 +08:00
Qu Wenwen 9665321745 refactor code 2023-09-21 11:51:17 +08:00
YWMditto 8464425a7b
feat(mdoel): add DynamicNTKScalingRotaryEmbedding (#339) 
* add dynamic ntk rope

* update dynamic ntk rope

* fix lint check

* fix lint check

* add more desc

---------

Co-authored-by: YWMditto <862779238@qq.com>
 2023-09-20 23:31:47 +08:00
Shuo Zhang e611817442
fix(doc): add 20b releasing info to readme (#330) 
* fix(eval): StreamingDataset does not have an __len__ method.

* doc(readme): update readme

* update readme

* update readme

* update readme

* update readme

* update readme

* update readme
 2023-09-20 16:46:45 +08:00
Shuo Zhang 5e5d160685
fix(readme): fix readme about 20B releasing (#329) 
* fix(eval): StreamingDataset does not have an __len__ method.

* doc(readme): update readme

* update readme

* update readme

* update readme

* update readme

* update readme
 2023-09-20 16:26:43 +08:00
Shuo Zhang 2a09ebd5c1
doc(readme): update readme, add 20B releasing info (#328) 
* fix(eval): StreamingDataset does not have an __len__ method.

* doc(readme): update readme

* update readme
 2023-09-20 16:04:43 +08:00
huangting4201 67eda4cbe1 fix(.github/workflows/e2e_test.yaml): update ci runner name 2023-09-19 18:13:20 +08:00