Commit Graph

  • 71f705d9bb refactor zero FrankLeeeee 2022-03-07 07:40:06 +0000
  • 9fccdb4986 refactored testing components FrankLeeeee 2022-03-04 05:13:51 +0000
  • b8e7675a22 polish code jiaruifang 2022-03-07 15:27:49 +0800
  • d63d20165d Added profiler communication operations 1SAA 2022-03-04 10:17:45 +0800
  • cc1d2fca9d add more flags for zero init context fix bug of repeated converting param to ShardedParamV2 jiaruifang 2022-03-07 14:55:09 +0800
  • 8e534b462d Added profiler communication operations 1SAA 2022-03-04 10:17:45 +0800
  • 3b63733665 add zero init context jiaruifang 2022-03-07 14:22:00 +0800
  • c7a7174bda WIP use sharded param v2 jiaruifang 2022-03-07 11:53:21 +0800
  • 94b8c3f384 Merge branch 'jiaruifang/zero_hook' into jiaruifang/use_sharded_param_v2 jiaruifang 2022-03-07 10:42:31 +0800
  • b38ed3934a add badge and contributor list binmakeswell 2022-03-04 18:04:51 +0800
  • d57d020bff add badge and contributor list binmakeswell 2022-03-04 18:04:51 +0800
  • 36374e90f8 fix sharded optim with offload and add unit test ver217 2022-03-04 19:07:40 +0800
  • cae686ffa8 add sharded optim v3 ver217 2022-03-04 17:21:31 +0800
  • a3e153a4d1 add zero hook. jiaruifang 2022-03-04 16:15:27 +0800
  • b73a048ad8
    [zero] cpu adam kernel (#288) LuGY 2022-03-04 16:05:15 +0800
  • 3ce353c31a add more shard strategy FrankLeeeee 2022-03-04 07:54:19 +0000
  • 29521cba0a
    [zero] yet an improved sharded param (#311) Jiarui Fang 2022-03-04 15:49:23 +0800
  • da3dfeeb0c
    Merge branch 'develop' into jiaruifang/sharded_param_v2 Jiarui Fang 2022-03-04 15:43:59 +0800
  • 2f6295bf78
    [zero] polish shard strategy (#310) Jiarui Fang 2022-03-04 15:35:07 +0800
  • b95f9b4670 polish code ver217 2022-03-04 13:44:38 +0800
  • 2aa440358d fix sharded param hook and unit test ver217 2022-03-04 13:40:48 +0800
  • 8c2327b93c impl shard optim v2 and add unit test ver217 2022-03-04 11:49:02 +0800
  • b21eb07aa0 removed useless codes lclgy 2022-03-04 15:23:39 +0800
  • aa144c9dca deleted some useless codes lclgy 2022-03-04 15:15:49 +0800
  • b697af88e2 sharded param v2. Use sharded tensor. jiaruifang 2022-03-04 15:11:48 +0800
  • d53118253b polish code jiaruifang 2022-03-04 14:46:28 +0800
  • f1601784b0 modified the method off cpu adam unittest lclgy 2022-03-04 14:43:44 +0800
  • 8c8dce46cf Merge branch 'develop' of https://github.com/hpcaitech/ColossalAI into jiaruifang/shard_strategy jiaruifang 2022-03-04 14:41:34 +0800
  • 1a06ce688e delete useless parameters, removed resnet lclgy 2022-03-04 14:13:48 +0800
  • f47c6ac255 updated the license lclgy 2022-03-04 13:46:22 +0800
  • 58134e1dd1 polish code ver217 2022-03-04 13:44:38 +0800
  • 37f5e474df fix sharded param hook and unit test ver217 2022-03-04 13:40:48 +0800
  • 78b933f265 Merge branch 'develop' of github.com:Gy-Lu/ColossalAI into develop lclgy 2022-03-04 13:16:41 +0800
  • afac304d61
    Merge branch 'hpcaitech:develop' into develop LuGY 2022-03-04 13:16:22 +0800
  • 536a4ecda8 finished the cpu adam lclgy 2022-03-04 12:31:33 +0800
  • 88496b5b31
    [zero] a shard strategy in granularity of tensor (#307) Jiarui Fang 2022-03-04 11:59:35 +0800
  • b73e198ebf
    Merge branch 'hpcaitech:develop' into develop LuGY 2022-03-04 11:56:37 +0800
  • b880b87a39 impl shard optim v2 and add unit test ver217 2022-03-04 11:49:02 +0800
  • 4ba5f9e0a2
    Merge branch 'develop' into jiaruifang/shard_strategy Jiarui Fang 2022-03-04 11:13:30 +0800
  • 3ad6d07c12 move shard and gather logic to shard strategy from shard tensor. jiaruifang 2022-03-04 11:09:51 +0800
  • 0db71cd20a Merge branch 'develop' of https://github.com/hpcaitech/ColossalAI into jiaruifang/shard_strategy jiaruifang 2022-03-04 10:50:25 +0800
  • 408cba655b
    [zero] sharded tensor (#305) Jiarui Fang 2022-03-04 10:46:13 +0800
  • 7016db6d11 add shard stratgy jiaruifang 2022-03-04 10:22:20 +0800
  • ce5d94a604
    [profiler] primary memory tracer Jie Zhu 2022-03-04 09:35:23 +0800
  • 7fa2bcb8ff polish code jiaruifang 2022-03-03 18:15:03 +0800
  • 14b67a3d37 [zero] add shareded tensor class jiaruifang 2022-03-03 18:10:49 +0800
  • fac5d05a8d update unit testing CI rules FrankLeeeee 2022-03-03 07:42:46 +0000
  • 0cd67a8dc0 added compatibility CI and options for release ci FrankLeeeee 2022-02-28 08:40:06 +0000
  • 725d81ad21 added pypi publication CI and remove formatting CI FrankLeeeee 2022-02-28 07:17:37 +0000
  • 34619700e4
    fix bug: add conditional initialization to `MemTracerOpHook` Jie Zhu 2022-03-03 17:36:37 +0800
  • 5cc84d94dc rename shared adam to sharded optim v2 ver217 2022-03-03 15:55:27 +0800
  • df34bd0c7f fix master params dtype ver217 2022-03-03 15:50:30 +0800
  • 6c290dbb08 add fp32 master params in sharded adam ver217 2022-03-03 15:42:53 +0800
  • 6185b9772d add sharded adam ver217 2022-03-03 15:06:18 +0800
  • 8892b727f7 add set_payload method for ShardedParam jiaruifang 2022-03-03 16:14:44 +0800
  • 747bcae851 Merge branch 'develop' of https://github.com/hpcaitech/ColossalAI into develop jiaruifang 2022-03-03 16:14:00 +0800
  • 6b87dd74dc update unit testing CI rules FrankLeeeee 2022-03-03 07:42:46 +0000
  • d2b63754eb
    bug-fix: alternate `add_global_rank()` to `get_global_rank() Jie Zhu 2022-03-03 15:55:42 +0800
  • 8f4c3c2816 rename shared adam to sharded optim v2 ver217 2022-03-03 15:55:27 +0800
  • 06a545fe64 fix master params dtype ver217 2022-03-03 15:50:30 +0800
  • 79603718a2 add fp32 master params in sharded adam ver217 2022-03-03 15:42:53 +0800
  • 1902dfac4c add sharded adam ver217 2022-03-03 15:06:18 +0800
  • 09d6cacfea
    mem_tracer: multi-node support Jie Zhu 2022-03-03 15:48:24 +0800
  • ff83f6f474 added compatibility CI and options for release ci FrankLeeeee 2022-02-28 08:40:06 +0000
  • ba724c7618 added pypi publication CI and remove formatting CI FrankLeeeee 2022-02-28 07:17:37 +0000
  • de11a91007
    polish license (#300) Jiarui Fang 2022-03-03 14:11:45 +0800
  • 3c0ebd2a70 Merge branch 'develop' of https://github.com/hpcaitech/ColossalAI into develop jiaruifang 2022-03-03 14:07:27 +0800
  • 6c78946fdd
    Polish sharded parameter (#297) Jiarui Fang 2022-03-03 12:42:57 +0800
  • b931e4972a
    modify unit test Jie Zhu 2022-03-03 11:04:45 +0800
  • a380328117
    implement primary memory hook Jie Zhu 2022-03-02 21:24:59 +0800
  • 0123b8ad42
    modify `MemTraceOpHook` to make it work Jie Zhu 2022-03-02 16:39:39 +0800
  • f520f689c8
    add type hint to func `get_cuda_memory_used` Jie Zhu 2022-03-02 16:09:03 +0800
  • cf34135b9e
    add func `current_time` and `lap` to `Timer` Jie Zhu 2022-03-02 15:58:33 +0800
  • 946d41b3c7
    [zero] add sharded grad and refactor grad hooks for ShardedModel (#287) ver217 2022-03-02 18:28:29 +0800
  • ebe5380805
    fixed typo in ShardParam (#294) Frank Lee 2022-03-02 17:26:23 +0800
  • 9577231e32
    added unit test for sharded optimizer (#293) Frank Lee 2022-03-02 17:15:54 +0800
  • cbf8e2915a
    added buffer sync to naive amp model wrapper (#291) Frank Lee 2022-03-02 16:47:17 +0800
  • b895de6800 Merge branch 'develop' of https://github.com/hpcaitech/ColossalAI into jiaruifang/polish_sharded_param jiaruifang 2022-03-03 11:50:49 +0800
  • 4945f50f06
    modify unit test Jie Zhu 2022-03-03 11:04:45 +0800
  • 43dcb6b121 add more unittests to shareded param jiaruifang 2022-03-03 10:49:19 +0800
  • 4ab08286be add more unitest for shard param jiaruifang 2022-03-03 10:05:24 +0800
  • 01eb0f5074
    implement primary memory hook Jie Zhu 2022-03-02 21:24:59 +0800
  • 9cf9a35045 init shard param from shape tuple jiaruifang 2022-03-02 18:50:58 +0800
  • 9b07ac81d4
    [zero] add sharded grad and refactor grad hooks for ShardedModel (#287) ver217 2022-03-02 18:28:29 +0800
  • b207f79d35 Merge remote-tracking branch 'origin/develop' into feature/zero3 ver217 2022-03-02 18:09:13 +0800
  • 6bf64ad824 polish comments ver217 2022-03-02 18:01:00 +0800
  • d6ee011861 polish comments ver217 2022-03-02 18:00:16 +0800
  • c8c812b973 simplify sharded grad and fix unit test ver217 2022-03-02 17:59:13 +0800
  • 4fbb8db586
    fixed typo in ShardParam (#294) Frank Lee 2022-03-02 17:26:23 +0800
  • a463980aab
    added unit test for sharded optimizer (#293) Frank Lee 2022-03-02 17:15:54 +0800
  • b16515f58c fixed typo in ShardParam FrankLeeeee 2022-03-02 09:15:46 +0000
  • 193af3a8b7
    added buffer sync to naive amp model wrapper (#291) Frank Lee 2022-03-02 16:47:17 +0800
  • 1ea660df80 refactor for elegance FrankLeeeee 2022-03-02 08:46:38 +0000
  • ede2a9b538
    modify `MemTraceOpHook` to make it work Jie Zhu 2022-03-02 16:39:39 +0800
  • 59583a7118
    add type hint to func `get_cuda_memory_used` Jie Zhu 2022-03-02 16:09:03 +0800
  • af18c97cb0
    add func `current_time` and `lap` to `Timer` Jie Zhu 2022-03-02 15:58:33 +0800
  • f3c14e8314 polish comments ver217 2022-03-02 15:33:05 +0800
  • cf021e34d6 added unit test for sharded optimizer FrankLeeeee 2022-03-02 07:23:21 +0000
  • 004d8d5719 use base param hook mgr to manage grad hooks ver217 2022-03-02 14:51:26 +0800
  • 9f83fb22c4 Merge remote-tracking branch 'origin/develop' into feature/zero3 ver217 2022-03-02 14:40:41 +0800