ColossalAI/examples/simclr_cifar10_data_parallel/config.py

from colossalai.amp import AMP_TYPE


LOG_NAME = 'cifar-simclr' 

BATCH_SIZE = 512
NUM_EPOCHS = 801
LEARNING_RATE = 0.03*BATCH_SIZE/256
WEIGHT_DECAY = 0.0005
MOMENTUM = 0.9


fp16 = dict(
    mode=AMP_TYPE.TORCH,
)

dataset = dict(
    root='./dataset',
)

gradient_accumulation=2
clip_grad_norm=1.0
add example of self-supervised SimCLR training - V2 (#50) * add example of self-supervised SimCLR training * simclr v2, replace nvidia dali dataloader * updated * sync to latest code writing style * sync to latest code writing style and modify README * detail README & standardize dataset path 2021-12-21 00:07:18 +00:00			`from colossalai.amp import AMP_TYPE`


			`LOG_NAME = 'cifar-simclr'`

			`BATCH_SIZE = 512`
			`NUM_EPOCHS = 801`
			`LEARNING_RATE = 0.03*BATCH_SIZE/256`
			`WEIGHT_DECAY = 0.0005`
			`MOMENTUM = 0.9`


			`fp16 = dict(`
			`mode=AMP_TYPE.TORCH,`
			`)`

			`dataset = dict(`
			`root='./dataset',`
			`)`

			`gradient_accumulation=2`
Layer integration (#83) * integrated parallel layers for ease of building models * integrated 2.5d layers * cleaned codes and unit tests * added log metric by step hook; updated imagenet benchmark; fixed some bugs * reworked initialization; cleaned codes Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com> 2021-12-27 07:04:32 +00:00			`clip_grad_norm=1.0`
add example of self-supervised SimCLR training - V2 (#50) * add example of self-supervised SimCLR training * simclr v2, replace nvidia dali dataloader * updated * sync to latest code writing style * sync to latest code writing style and modify README * detail README & standardize dataset path 2021-12-21 00:07:18 +00:00