ytxiong c219065348 feat(*): support sequence_parallel (#180) ... * support sequence_parallel for no pipeline * sequence_parallel does not support no-flash-attn * support sequence parallel for pipeline * add memory profiler * Update 13B.py * add memory profiler * fix evaluation bug * remove some unnecessary code * remove some unnecessary code * Update parallel_context.py * modify the config * remove memory profiler * modify the config * support selective dropout		2023-08-07 16:42:52 +08:00
..
__init__.py	initial commit	2023-07-06 12:55:23 +08:00
checkpoint.py	initial commit	2023-07-06 12:55:23 +08:00
common.py	feat(core/scheduler): support pipeline parallel (#98)	2023-07-24 20:52:09 +08:00
evaluation.py	feat(*): support sequence_parallel (#180)	2023-08-07 16:42:52 +08:00
logger.py	feat(utils/evaluation.py): support evaluate (#154)	2023-08-02 19:03:59 +08:00
megatron_timers.py	initial commit	2023-07-06 12:55:23 +08:00
model_checkpoint.py	feat(core/scheduler): support pipeline parallel (#98)	2023-07-24 20:52:09 +08:00
parallel.py	feat(utils/logger.py): support uniscale logger (#152)	2023-08-01 17:37:32 +08:00
registry.py	feat(core/scheduler): support pipeline parallel (#98)	2023-07-24 20:52:09 +08:00
storage_manager.py	initial commit	2023-07-06 12:55:23 +08:00
writer.py	feat(utils/logger.py): support uniscale logger (#152)	2023-08-01 17:37:32 +08:00