mirror of https://github.com/hpcaitech/ColossalAI
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1.1 KiB
1.1 KiB
File Structure
|- sd3_generation.py: an example of how to use Colossalai Inference Engine to generate result by loading Diffusion Model.
|- compute_metric.py: compare the quality of images w/o some acceleration method like Distrifusion
|- benchmark_sd3.py: benchmark the performance of our InferenceEngine
|- run_benchmark.sh: run benchmark command
Note: compute_metric.py need some dependencies which need pip install -r requirements.txt, requirements.txt is in examples/inference/stable_diffusion/
Run Inference
The provided example sd3_generation.py is an example to configure, initialize the engine, and run inference on provided model. We've added DiffusionPipeline as model class, and the script is good to run inference with StableDiffusion 3.
For a basic setting, you could run the example by:
colossalai run --nproc_per_node 1 sd3_generation.py -m PATH_MODEL -p "hello world"
Run multi-GPU inference (Patched Parallelism), as in the following example using 2 GPUs:
colossalai run --nproc_per_node 2 sd3_generation.py -m PATH_MODEL