3 Commits (fda99947aded9d4b47e20b8ad7e5e8aa22fc28f4)

Author	SHA1	Message	Date
Yang Gao	555ed60a2c	fix(tokenizer): refactor tokenizer and update usage in readme (#51) ... * update tokenizer example	2023-07-13 17:16:27 +08:00
yhcc	745d2b911a	Fix readme about conversion to transformers (#25) ... * add links for 8k * fix acknowledgement * modified readme for convert_hf	2023-07-07 13:38:06 +08:00
Sun Peng	fa7337b37b	initial commit	2023-07-06 12:55:23 +08:00