[misc] refactor launch API and tensor constructor (#5666)
* [misc] remove config arg from initialize
* [misc] remove old tensor contrusctor
* [plugin] add npu support for ddp
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [devops] fix doc test ci
* [test] fix test launch
* [doc] update launch doc
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@ -106,20 +105,11 @@ First, we need to set the launch method in our code. As this is a wrapper of the
use `colossalai.launch_from_torch`. The arguments required for distributed environment such as rank, world size, host and port are all set by the PyTorch
use `colossalai.launch_from_torch`. The arguments required for distributed environment such as rank, world size, host and port are all set by the PyTorch
launcher and can be read from the environment variable directly.
launcher and can be read from the environment variable directly.
config.py
```python
BATCH_SIZE = 512
LEARNING_RATE = 3e-3
WEIGHT_DECAY = 0.3
NUM_EPOCHS = 2
```
train.py
train.py
```python
```python
import colossalai
import colossalai
colossalai.launch_from_torch(
colossalai.launch_from_torch()
config="./config.py",
)
...
...
```
```
@ -203,7 +193,6 @@ Do this in your training script:
import colossalai
import colossalai
colossalai.launch_from_slurm(
colossalai.launch_from_slurm(
config=<CONFIG>,
host=args.host,
host=args.host,
port=args.port
port=args.port
)
)
@ -224,7 +213,6 @@ use them to start the distributed backend.
Hongxin Liu
7 months ago
committed by