ColossalAI/examples/simclr_cifar10_data_parallel/le_config.py

from colossalai.amp import AMP_TYPE


LOG_NAME = 'cifar-simclr'
EPOCH = 800

BATCH_SIZE = 512
NUM_EPOCHS = 51
LEARNING_RATE = 0.03*BATCH_SIZE/256
WEIGHT_DECAY = 0.0005
MOMENTUM = 0.9


fp16 = dict(
    mode=AMP_TYPE.TORCH,
)

dataset = dict(
    root='./dataset',
)

gradient_accumulation=1
clip_grad_norm=1.0
add example of self-supervised SimCLR training - V2 (#50) * add example of self-supervised SimCLR training * simclr v2, replace nvidia dali dataloader * updated * sync to latest code writing style * sync to latest code writing style and modify README * detail README & standardize dataset path 2021-12-21 00:07:18 +00:00			`from colossalai.amp import AMP_TYPE`


			`LOG_NAME = 'cifar-simclr'`
			`EPOCH = 800`

			`BATCH_SIZE = 512`
			`NUM_EPOCHS = 51`
			`LEARNING_RATE = 0.03*BATCH_SIZE/256`
			`WEIGHT_DECAY = 0.0005`
			`MOMENTUM = 0.9`


			`fp16 = dict(`
			`mode=AMP_TYPE.TORCH,`
			`)`

			`dataset = dict(`
			`root='./dataset',`
			`)`

			`gradient_accumulation=1`
Layer integration (#83) * integrated parallel layers for ease of building models * integrated 2.5d layers * cleaned codes and unit tests * added log metric by step hook; updated imagenet benchmark; fixed some bugs * reworked initialization; cleaned codes Co-authored-by: BoxiangW <45734921+BoxiangW@users.noreply.github.com> 2021-12-27 07:04:32 +00:00			`clip_grad_norm=1.0`