ColossalAI/applications/ColossalQA/data/tests/test.md

README Format File for Testing

Table of Contents

• Table of Contents
• Install
• How to Use
• Examples
  • Local Chinese Retrieval QA + Chat
  • Local English Retrieval QA + Chat
  • Local Bi-lingual Retrieval QA + Chat
  • Experimental AI Agent Based on Chatgpt + Chat

As Colossal-AI is undergoing some major updates, this project will be actively maintained to stay in line with the Colossal-AI project.

Install

Install colossalqa

# python==3.8.17
cd ColossalAI/applications/ColossalQA
pip install -e .

To use the vllm server, please refer to the official guide here for installation instruction. Simply run the following command from another terminal.

cd ./vllm/entrypoints
python api_server.py --host localhost --port $PORT_NUMBER --model $PATH_TO_MODEL --swap-space $SWAP_SPACE_IN_GB

How to use

Collect your data

For ChatGPT based Agent we support document retrieval and simple sql search. If you want to run the demo locally, we provided document retrieval based conversation system built upon langchain. It accept a wide range of documents.

Read comments under ./colossalqa/data_loader for more detail

Serving

Currently use vllm will replace with colossal inference when ready. Please refer class VllmLLM.

Run the script

We provided scripts for Chinese document retrieval based conversation system, English document retrieval based conversation system, Bi-lingual document retrieval based conversation system and an experimental AI agent with document retrieval and SQL query functionality.

To run the bi-lingual scripts, set the following environmental variables before running the script.

export ZH_MODEL_PATH=XXX
export ZH_MODEL_NAME: chatglm2
export EN_MODEL_PATH: XXX
export EN_MODEL_NAME: llama
python retrieval_conversation_universal.py

To run retrieval_conversation_en.py. set the following environmental variables.

export EN_MODEL_PATH=XXX
export EN_MODEL_NAME: llama
python retrieval_conversation_en.py

To run retrieval_conversation_zh.py. set the following environmental variables.

export ZH_MODEL_PATH=XXX
export ZH_MODEL_NAME: chatglm2
python retrieval_conversation_en.py

It will ask you to provide the path to your data during the execution of the script. You can also pass a glob path to load multiple files at once. If csv files are provided, please use ',' as delimiter and '"' as quotation mark. There are no other formatting constraints for loading documents type files. For loading table type files, we use pandas, please refer to Pandas-Input/Output for file format details.

The Plan

• build document retrieval QA tool
• Add long + short term memory
• Add demo for AI agent with SQL query
• Add customer retriever for fast construction and retrieving (with incremental mode)