4 Commits (b10e5132fe2bf54883b250c91c806f2a76a32588)

Author	SHA1	Message	Date
Wenwen Qu	cd6b28b073	use dummy mode to generate random numbers in model construction	2023-09-08 17:56:42 +08:00
Wenwen Qu	c357288a8b	feat(XXX): add moe	2023-08-07 20:17:49 +08:00
ytxiong	c219065348	feat(*): support sequence_parallel (#180) ... * support sequence_parallel for no pipeline * sequence_parallel does not support no-flash-attn * support sequence parallel for pipeline * add memory profiler * Update 13B.py * add memory profiler * fix evaluation bug * remove some unnecessary code * remove some unnecessary code * Update parallel_context.py * modify the config * remove memory profiler * modify the config * support selective dropout	2023-08-07 16:42:52 +08:00
Sun Peng	fa7337b37b	initial commit	2023-07-06 12:55:23 +08:00