ColossalAI

Commit Graph

Author	SHA1	Message	Date
Jiarui Fang	21962e1593	[embedding] rename FreqAwareEmbedding -> CachedEmbedding (#1699 )	2022-10-13 22:22:27 +08:00
Jiarui Fang	363fc2861a	[embeddings] more detailed timer (#1692 )	2022-10-12 12:01:21 +08:00
Jiarui Fang	c638bec028	[embedding] polish async copy (#1657 )	2022-09-27 14:37:03 +08:00
Jiarui Fang	988570e4a6	[embedding] add more detail profiling (#1656 )	2022-09-27 13:43:59 +08:00
Jiarui Fang	e1f97fd2b8	[embedding] print profiling results (#1654 )	2022-09-27 12:50:33 +08:00
Jiarui Fang	04443605a5	[embedding] non-blocking cpu-gpu copy (#1647 )	2022-09-26 14:57:57 +08:00
CsRic	0767f67a0f	[embedding] isolate cache_op from forward (#1645 ) Co-authored-by: ric <mkkt_bkkt@mail.ustc.edu.cn>	2022-09-26 11:18:59 +08:00
Jiarui Fang	e57df80325	[embeddings] cache option (#1635 )	2022-09-23 16:40:18 +08:00
Jiarui Fang	38c68b5b9a	[embedding] rollback for better FAW performance (#1625 )	2022-09-22 11:16:25 +08:00
Jiarui Fang	504ff1d101	[embeddings] use cache_ratio instead of cuda_row_num (#1611 )	2022-09-20 14:33:04 +08:00
Jiarui Fang	a19eb80998	[embedding] updates some default parameters	2022-09-15 15:45:17 +08:00
CsRic	f3403ff98e	[embeddings] add already_split_along_rank flag for tablewise mode (#1584 )	2022-09-13 10:50:34 +08:00
CsRic	a389ac4ec9	[embedding] cache_embedding small improvement (#1564 )	2022-09-08 16:41:19 +08:00
Jiarui Fang	64169f3e8f	[embedding] polish parallel embedding tablewise (#1545 )	2022-09-06 10:41:20 +08:00
CsRic	964123ae0f	[embedding] freq_aware_embedding: add small functions for caller application (#1537 )	2022-09-05 15:12:53 +08:00
Jiarui Fang	521078ffc9	[embedding] fix a bug in table wise sharding (#1538 )	2022-09-02 15:48:35 +08:00
Jiarui Fang	87134524fd	[embedding] tablewise sharding polish (#1535 )	2022-09-02 11:09:37 +08:00
CsRic	5156d5b4f8	[embedding] add tablewise sharding for FAW (#1526 )	2022-09-01 17:55:41 +08:00
Jiarui Fang	4537d39df9	[doc] docstring for FreqAwareEmbeddingBag (#1525 )	2022-08-31 13:52:30 +08:00
Jiarui Fang	9a9ef65313	[FAW] cpu caching operations (#1520 )	2022-08-30 14:50:02 +08:00
Jiarui Fang	af5438caa2	[FAW] refactor reorder() for CachedParamMgr (#1514 )	2022-08-29 14:22:07 +08:00
Jiarui Fang	9feee6d06b	[FAW] LFU initialize with dataset freq (#1513 )	2022-08-29 12:52:53 +08:00
CsRic	1b8fee8e9c	[FAW] shrink freq_cnter size (#1509 )	2022-08-29 11:44:55 +08:00
Jiarui Fang	ba61109b6c	[FAW] remove code related to chunk (#1501 )	2022-08-26 14:23:30 +08:00
Jiarui Fang	d5085bb317	[FAW] add more docs and fix a warning (#1500 )	2022-08-26 14:10:21 +08:00
CsRic	0ed2f46131	[FAW] FAW embedding use LRU as eviction strategy intialized with dataset stats (#1494 )	2022-08-26 11:24:12 +08:00
CsRic	b8d0e39eaf	[FAW] LFU cache for the FAW	2022-08-25 13:08:46 +08:00
Jiarui Fang	cde7b8a5b8	[FAW] init an LFU implementation for FAW (#1488 )	2022-08-24 17:37:22 +08:00
Geng Zhang	0aad53c62b	[FCE] update interface for frequency statistics in FreqCacheEmbedding (#1462 )	2022-08-23 17:38:24 +08:00
Jiarui Fang	a1476ea882	[NFC] polish doc style for ColoTensor (#1457 )	2022-08-16 09:21:05 +08:00
Geng Zhang	9f3eed66eb	[FAW] reorganize the inheritance struct of FreqCacheEmbedding (#1448 )	2022-08-12 15:55:46 +08:00
Jiarui Fang	30b4dd17c0	[FAW] export FAW in _ops (#1438 )	2022-08-11 13:43:24 +08:00
HELSON	1b41686461	[hotfix] fix unit test test_module_spec (#1321 )	2022-07-15 14:02:32 +08:00
Jiarui Fang	9bcd2fd4af	[tensor] a shorter shard and replicate spec (#1245 )	2022-07-11 15:51:48 +08:00
Jiarui Fang	ae7d3f4927	[refactor] move process group from _DistSpec to ColoTensor. (#1203 )	2022-07-06 16:15:16 +08:00
Jiarui Fang	060b917daf	[refactor] remove gpc dependency in colotensor's _ops (#1189 )	2022-07-04 18:54:37 +08:00
Ziyue Jiang	dd0420909f	[Tensor] rename parallel_action (#1174 ) * rename parallel_action * polish	2022-06-27 10:04:45 +08:00
Jiarui Fang	4b9bba8116	[ColoTensor] rename APIs and add output_replicate to ComputeSpec (#1168 )	2022-06-24 13:08:54 +08:00
Jiarui Fang	f4ef224358	[Tensor] remove ParallelAction, use ComputeSpec instread (#1166 )	2022-06-23 17:34:59 +08:00
Jiarui Fang	49832b2344	[refactory] add nn.parallel module (#1068 )	2022-06-06 15:34:41 +08:00

40 Commits (4ee311c0262dfbca9b5da7e18f04dd8f1f23fe4c)