ColossalAI/examples/language/llama/scripts/benchmark_70B/gemini.sh

#!/bin/bash

################
#Load your environments and modules here
################

HOSTFILE=$(realpath hosts.txt)

cd ../..

export OMP_NUM_THREADS=8

colossalai run --nproc_per_node 8 --hostfile $HOSTFILE benchmark.py -c 70b -g -x -b 2
[example] add llama2 example (#4527) * [example] transfer llama-1 example * [example] fit llama-2 * [example] refactor scripts folder * [example] fit new gemini plugin * [cli] fix multinode runner * [example] fit gemini optim checkpoint * [example] refactor scripts * [example] update requirements * [example] update requirements * [example] rename llama to llama2 * [example] update readme and pretrain script * [example] refactor scripts 2023-08-28 09:59:11 +00:00			`#!/bin/bash`

			`################`
			`#Load your environments and modules here`
			`################`

			`HOSTFILE=$(realpath hosts.txt)`

			`cd ../..`

			`export OMP_NUM_THREADS=8`

			`colossalai run --nproc_per_node 8 --hostfile $HOSTFILE benchmark.py -c 70b -g -x -b 2`