ColossalAI

Commit Graph

Author	SHA1	Message	Date
YuliangLiu0306	c7ac0f4ab2	[autoparallel] add elementwise handler (#1622 ) * [autoparallel] add elementwise handler * polish code * polish code * reduce skipped strategies range * polish code	2 years ago
YuliangLiu0306	3a46215135	[autoparallel] add embedding handler (#1620 )	2 years ago
YuliangLiu0306	69448f64c4	[autoparallel] protect bcast handler from invalid strategies (#1631 )	2 years ago
YuliangLiu0306	0c703189b9	[autoparallel] add layernorm handler (#1629 )	2 years ago
YuliangLiu0306	bf77d3ab65	[autoparallel] recover the merged node strategy index (#1613 )	2 years ago
Boyuan Yao	d6b01feb66	[fx] Modify offload codegen (#1618 ) * [fx] modify offload codegen * [fx] remove repeated hook definitions * [fx] modify offload test	2 years ago
YuliangLiu0306	9eae855408	[hotfix] add recompile after graph manipulatation (#1621 )	2 years ago
Super Daniel	d967779a32	[fx/profiler] tuned the calculation of memory estimation (#1619 ) * [fx] tuned the meta info and rotor solver. * [fx] remove import. * [fx] remove import. * [fx] remove import. * [fx] tune the meta calculations. * [fx] polish comments. * [fx] remove assertions. * [fx] modify test cases. * [fx] modify test cases. * [fx] optimize import. * [fx	2 years ago
HELSON	f7f2248771	[moe] fix MoE bugs (#1628 ) * remove forced FP32 modules * correct no_shard-contexts' positions	2 years ago
Jiarui Fang	38c68b5b9a	[embedding] rollback for better FAW performance (#1625 )	2 years ago
Frank Lee	d925122020	[autoparallel] added new linear module handler (#1616 )	2 years ago
Kirigaya Kazuto	170fa81095	[pipeline/chimera] test chimera \| fix bug of initializing (#1615 ) * [pipeline/tuning] improve dispatch performance both time and space cost * [pipeline/converge] add interface for testing convergence * [NFC] polish colossalai/utils/multi_tensor_apply/multi_tensor_apply.py code style * Update PipelineBase.py * [pipeline/chimera] reconstruct PipelineBase and Worker to support more feasible custom schedule \| finish Chimera * [pipeline/chimera] test chimera \| fix bug of initializing	2 years ago
Jiarui Fang	504ff1d101	[embeddings] use cache_ratio instead of cuda_row_num (#1611 )	2 years ago
YuliangLiu0306	6a8f8cc05e	[hotfix] got sliced types (#1614 )	2 years ago
Frank Lee	d397842fa8	[autoparallel] added new node handler (#1612 )	2 years ago
YuliangLiu0306	7d1bb71d5d	[fx] PoC of runtime shape consistency application (#1607 ) * [fx] PoC of runtime shape consistency application * polish code	2 years ago
YuliangLiu0306	47b11c432c	[autoparallel]add bcast matmul strategies (#1605 )	2 years ago
Frank Lee	edb67cb378	[autoparallel] refactored the data structure for sharding strategy (#1610 )	2 years ago
Boyuan Yao	933b6c6367	[fx] Add pofo solver (#1608 ) * [fx] add pofo algorithm * [fx] Add pofo solver * [fx] code refactor * [fx] fix test_linearize import	2 years ago
github-actions[bot]	d32cf84c46	Automated submodule synchronization (#1609 ) Co-authored-by: github-actions <github-actions@github.com>	2 years ago
Frank Lee	725666d6a9	[workflow] deactivate conda environment before removing (#1606 )	2 years ago
Kirigaya Kazuto	edc9e419ad	[pipeline/chimera] reconstruct PipelineBase and Worker to support more feasible custom schedule \| finish Chimera (#1595 ) * [pipeline/tuning] improve dispatch performance both time and space cost * [pipeline/converge] add interface for testing convergence * [NFC] polish colossalai/utils/multi_tensor_apply/multi_tensor_apply.py code style * Update PipelineBase.py * [pipeline/chimera] reconstruct PipelineBase and Worker to support more feasible custom schedule \| finish Chimera	2 years ago
ver217	c9e8ce67b8	fix move fp32 shards (#1604 )	2 years ago
YuliangLiu0306	eac1b79371	[autoparallel] add bcast op handler (#1600 ) * [autoparallel] add bcast op handler * polish code * add more BCAST FUNC OP * polish code * add exception handler * polish	2 years ago
Frank Lee	3abf98a633	[autoparallel] added all non-bcast matmul strategies (#1603 )	2 years ago
Frank Lee	db98b695b2	[autoparallel] added strategy generator and bmm strategies (#1602 )	2 years ago
Jiarui Fang	a19eb80998	[embedding] updates some default parameters	2 years ago
Super Daniel	cd5cf2bcc9	[fx/tuning] tune performance on rotor with meta info. (#1599 )	2 years ago
Boyuan Yao	a7cda6f57d	[fx] Add offload codegen (#1598 ) * [fx] add input activation offload to codegen * [fx] modify unit test * [fx] remove two skips in torch11 * [fx] use all_input_nodes instead of _input_nodes	2 years ago
Super Daniel	c8e9b2ad78	[hotfix/rotor] fix variable names (#1597 ) * [fx] add some comment and docstrings. * [fx] add dataflow analysis for an autograd graph. * add intepretation for graph analysis. * [fx] before doing save_tensor_hooks. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] a very accurate version on GPT-2. * [fx] refactor code. * [fx] remove redundant inplace=True. * [fx] refactor code. * [fx] refactor code. * [fx] refactor code. * [fx] dive into backward memory. * [fx] fix variable names in ckpt_solvers and unskip tests. * [fx] commit my changes. * [fx] restore skips. * [fx] restore skips. * [fx] chaange stage into phase. * [fx] chaange stage into phase. * [fx] chaange stage into phase.	2 years ago
YuliangLiu0306	faa23b9d9a	[autoparallel] add reshape handler (#1594 ) * [autoparallel] add reshape handler * polish code	2 years ago
github-actions[bot]	c938dda028	Automated submodule synchronization (#1596 ) Co-authored-by: github-actions <github-actions@github.com>	2 years ago
Super Daniel	5c494d4540	[fx] provide an accurate estimation of memory. (#1587 ) * [fx] add some comment and docstrings. * [fx] add dataflow analysis for an autograd graph. * add intepretation for graph analysis. * [fx] before doing save_tensor_hooks. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] provide an accurate estimation of memory except for GPT-2. * [fx] a very accurate version on GPT-2. * [fx] refactor code. * [fx] remove redundant inplace=True. * [fx] refactor code. * [fx] refactor code. * [fx] refactor code. * [fx] dive into backward memory.	2 years ago
Frank Lee	27fe8af60c	[autoparallel] refactored shape consistency to remove redundancy (#1591 ) * [autoparallel] refactored shape consistency to remove redundancy * polish code * polish code * polish code	2 years ago
YuliangLiu0306	d164449d00	[autoparallel] add resnet autoparallel unit test and add backward weight communication cost (#1589 )	2 years ago
Frank Lee	7c18a588c8	[autoparallel] added generate_sharding_spec to utils (#1590 )	2 years ago
Boyuan Yao	49ccf8b5f8	[fx] Improve linearize and rotor solver (#1586 ) * [fx] add nested activation_checkpoint codegen * undo algorithms commits * solver * undo some commits * [fx] torch11 add nested activation checkpoint codegen * remove some imports * [fx] add some comments in activation codegen * [fx] codegen instance error fix * [fx] imporve linearize and rotor solver * [fx] some comments and format modification	2 years ago
Frank Lee	219f66c571	[autoparallel] added solver option dataclass (#1588 )	2 years ago
YuliangLiu0306	82d4376c23	[autoparallel] adapt solver with resnet (#1583 ) * [autoparallel]adapt solver with resnet * polish code * polish code	2 years ago
CsRic	f3403ff98e	[embeddings] add already_split_along_rank flag for tablewise mode (#1584 )	2 years ago
github-actions[bot]	77399dc91b	Automated submodule synchronization (#1550 ) Co-authored-by: github-actions <github-actions@github.com>	2 years ago
Boyuan Yao	f3687e4ee2	[fx] Add nested checkpoint in activation checkpoint codegen (#1585 ) * [fx] add nested activation_checkpoint codegen * undo algorithms commits * solver * undo some commits * [fx] torch11 add nested activation checkpoint codegen * remove some imports * [fx] add some comments in activation codegen * [fx] codegen instance error fix	2 years ago
binmakeswell	1c9ec32734	[NFC] add OPT serving (#1581 )	2 years ago
Boyuan Yao	20e466527b	[NFC] polish ./colossalai/trainer/hooks/_lr_scheduler_hook.py code style (#1576 )	2 years ago
Fazzie-Maqianli	06dccdde44	[NFC] polish colossalai/zero/sharded_model/reduce_scatter.py code style (#1554 )	2 years ago
CsRic	2ac46f7be4	[NFC] polish utils/tensor_detector/__init__.py code style (#1573 ) Co-authored-by: ric <mkkt_bkkt@mail.ustc.edu.cn>	2 years ago
Sze-qq	2144cbae8c	[NFC] polish colossalai/nn/lr_scheduler/multistep.py code style (#1572 )	2 years ago
superhao1995	e4bf7ae667	[NFC] polish colossalai/nn/lr_scheduler/torch.py code style (#1571 ) Co-authored-by: Research <research@soccf-snr3-017.comp.nus.edu.sg>	2 years ago
Jiatong Han	3263cdf57f	[NFC] polish colossalai/nn/parallel/data_parallel.py code style (#1570 ) Co-authored-by: JThh <jiatong.han@u.nus.edu>	2 years ago
Zirui Zhu	f566c9b98d	[NFC] polish colossalai/pipeline/utils.py code style (#1562 )	2 years ago

1 2 3 4 5 ...

1107 Commits (c7ac0f4ab2f07f40487dc852e5ce19202a96a64a) All Branches Search

1107 Commits (c7ac0f4ab2f07f40487dc852e5ce19202a96a64a)

All Branches