History

Zian(Andy) Zheng 7768afbad0 Update flash_attention_patch.py To be compatible with the new change in the Transformers library, where a new argument 'padding_mask' was added to forward function of attention layer. https://github.com/huggingface/transformers/pull/25598		2023-10-16 14:00:45 +08:00
..
Chat	[doc] update slack link (#4823 )	2023-09-27 17:37:39 +08:00
Colossal-LLaMA-2	Update flash_attention_patch.py	2023-10-16 14:00:45 +08:00
ColossalEval	Update Qwen-7B results (#4821 )	2023-09-27 17:33:54 +08:00
README.md	update Colossal (#4832 )	2023-09-28 16:05:05 +08:00

Applications

This directory contains the applications that are powered by Colossal-AI.

The list of applications include:

Colossal-LLaMA-2: Continual Pre-training of LLaMA-2.
ColossalEval: Evaluation Pipeline for LLMs.
ColossalChat: Replication of ChatGPT with RLHF.
FastFold: Optimizing AlphaFold (Biomedicine) Training and Inference on GPU Clusters.

Please note that the Chatbot application is migrated from the original ChatGPT folder.

You can find more example code for base models and functions in the Examples directory.