|
|
|
@ -225,6 +225,28 @@ curl -X POST "http://127.0.0.1:8000" \
|
|
|
|
|
"time":"2023-03-23 21:38:40"
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
感谢 [@hiyouga]() 实现了 OpenAI 格式的流式 API 部署,可以作为任意基于 ChatGPT的应用的后端,比如 [ChatGPT-Next-Web](https://github.com/Yidadaa/ChatGPT-Next-Web)。可以通过运行仓库中的[openai_api.py](openai_api.py) 进行部署:
|
|
|
|
|
```shell
|
|
|
|
|
python openai_api.py
|
|
|
|
|
```
|
|
|
|
|
进行 API 调用的示例代码为
|
|
|
|
|
```python
|
|
|
|
|
import openai
|
|
|
|
|
if __name__ == "__main__":
|
|
|
|
|
openai.api_base = "http://localhost:8000/v1"
|
|
|
|
|
openai.api_key = "none"
|
|
|
|
|
for chunk in openai.ChatCompletion.create(
|
|
|
|
|
model="chatglm2-6b",
|
|
|
|
|
messages=[
|
|
|
|
|
{"role": "user", "content": "你好"}
|
|
|
|
|
],
|
|
|
|
|
stream=True
|
|
|
|
|
):
|
|
|
|
|
if hasattr(chunk.choices[0].delta, "content"):
|
|
|
|
|
print(chunk.choices[0].delta.content, end="", flush=True)
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## 低成本部署
|
|
|
|
|
|
|
|
|
|
### 模型量化
|
|
|
|
|