github
/
ColossalAI
mirror of https://github.com/hpcaitech/ColossalAI
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

[doc] fix typo in opt inference tutorial (#2849)

pull/2860/head
Zheng Zeng 2023-02-21 17:16:13 +08:00 committed by GitHub
parent 935346430f
commit 597914317b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
examples/tutorial/opt/inference/README.md
View File

@ -50,7 +50,7 @@ python opt_fastapi.py <model> --queue_size <QueueSize>
```
The `<QueueSize>` can be an integer in `[0, MAXINT]`. If it's `0`, the request queue size is infinite. If it's a positive integer, when the request queue is full, incoming requests will be dropped (the HTTP status code of response will be 406).
### Configure bathcing
### Configure batching
```shell
python opt_fastapi.py <model> --max_batch_size <MaxBatchSize>
```
@ -85,4 +85,4 @@ Then open the web interface link which is on your console.
See [script/processing_ckpt_66b.py](./script/processing_ckpt_66b.py).
## OPT-175B
See [script/process-opt-175b](./script/process-opt-175b/).
See [script/process-opt-175b](./script/process-opt-175b/).