ColossalAI

Commit Graph

Author	SHA1	Message	Date
Jiarui Fang	4d9332b4c5	[refactor] moving memtracer to gemini (#801 )	2022-04-19 10:13:08 +08:00
yuxuan-lou	cfb41297ff	'fix/format' (#573 )	2022-04-06 11:40:59 +08:00
Liang Bowen	ec5086c49c	Refactored docstring to google style	2022-03-29 17:17:47 +08:00
Jie Zhu	73d36618a6	[profiler] add MemProfiler (#356 ) * add memory trainer hook * fix bug * add memory trainer hook * fix import bug * fix import bug * add trainer hook * fix #370 git log bug * modify `to_tensorboard` function to support better output * remove useless output * change the name of `MemProfiler` * complete memory profiler * replace error with warning * finish trainer hook * modify interface of MemProfiler * modify `__init__.py` in profiler * remove unnecessary pass statement * add usage to doc string * add usage to trainer hook * new location to store temp data file	2022-03-29 12:48:34 +08:00
yuxuan-lou	3b88eb2259	Flake8 code restyle	2022-03-11 15:50:28 +08:00
Jiarui Fang	10e2826426	move async memory to an individual directory (#345 )	2022-03-11 15:50:28 +08:00
Jie Zhu	3213554cc2	[profiler] add adaptive sampling to memory profiler (#330 ) * fix merge conflict modify unit test remove unnessesary log info reformat file * remove unused module * remove unnecessary sync function * change doc string style from Google to Sphinx	2022-03-11 15:50:28 +08:00
Jiarui Fang	11bddb6e55	[zero] update zero context init with the updated test utils (#327 )	2022-03-11 15:50:28 +08:00
Jie Zhu	d344689274	[profiler] primary memory tracer	2022-03-11 15:50:28 +08:00
Jiarui Fang	5a560a060a	Feature/zero (#279 ) * add zero1 (#209) * add zero1 * add test zero1 * update zero stage 1 develop (#212) * Implement naive zero3 (#240) * naive zero3 works well * add zero3 param manager * add TODOs in comments * add gather full param ctx * fix sub module streams * add offload * fix bugs of hook and add unit tests * fix bugs of hook and add unit tests (#252) * add gather full param ctx * fix sub module streams * add offload * fix bugs of hook and add unit tests * polish code and add state dict hook * fix bug * update unit test * refactor reconstructed zero code * clip_grad support zero3 and add unit test * add unit test for Zero3ParameterManager * [WIP] initialize the shard param class * [WIP] Yet another sharded model implementation (#274) * [WIP] initialize the shard param class * [WIP] Yes another implementation of shardModel. Using a better hook method. * torch.concat -> torch.cat * fix test_zero_level_1.py::test_zero_level_1 unitest * remove deepspeed implementation and refactor for the reconstructed zero module * polish zero dp unittests Co-authored-by: ver217 <lhx0217@gmail.com> Co-authored-by: Frank Lee <somerlee.9@gmail.com>	2022-03-11 15:50:28 +08:00
Jiarui Fang	569357fea0	add pytorch hooks (#179 ) * add pytorch hooks fix #175 * remove licenses in src code * add gpu memory tracer * replacing print with logger in ophooks.	2022-01-25 22:20:54 +08:00

11 Commits (dfaff4e24345396f006488a9a9891f3a8b68756d)