From a88ed0f83a1d98fe6f7b9064551201773ac9c99c Mon Sep 17 00:00:00 2001
From: Fazzie-Maqianli <55798671+Fazziekey@users.noreply.github.com>
Date: Wed, 29 Mar 2023 00:53:23 +0800
Subject: [PATCH] add limit (#3293)

---
 applications/Chat/README.md | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/applications/Chat/README.md b/applications/Chat/README.md
index cf54e7e7f..80e1f3657 100644
--- a/applications/Chat/README.md
+++ b/applications/Chat/README.md
@@ -20,6 +20,8 @@
 - [Coati7B examples](#coati7b-examples)
   - [Generation](#generation)
   - [Open QA](#open-qa)
+  - [Limitation for LLaMA-finetuned models](#limitation-for-llama-finetuned-models)
+  - [Limitation of dataset](#limitation-of-dataset)
 - [FAQ](#faq)
   - [How to save/load checkpoint](#how-to-saveload-checkpoint)
 - [The Plan](#the-plan)
@@ -214,6 +216,19 @@ We also support training reward model with true-world data. See `examples/train_
 
 </details>
 
+### Limitation for LLaMA-finetuned models
+- Both Alpaca and ColossalChat are based on LLaMA. It is hard to compensate for the missing knowledge in the pre-training stage.
+- Lack of counting ability: Cannot count the number of items in a list.
+- Lack of Logics (reasoning and calculation)
+- Tend to repeat the last sentence (fail to produce the end token).
+- Poor multilingual results: LLaMA is mainly trained on English datasets (Generation performs better than QA).
+### Limitation of dataset
+- Lack of summarization ability: No such instructions in finetune datasets.
+- Lack of multi-turn chat: No such instructions in finetune datasets
+- Lack of self-recognition: No such instructions in finetune datasets
+- Lack of Safety:
+  - When the input contains fake facts, the model makes up false facts and explanations.
+  - Cannot abide by OpenAI's policy: When generating prompts from OpenAI API, it always abides by its policy. So no violation case is in the datasets.
 ## FAQ
 
 ### How to save/load checkpoint