ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Add llama 3.1 in igpu perf (#12194)

Browse source
This commit is contained in:
Jin, Qiao 2024-10-14 15:14:34 +08:00 committed by GitHub
parent a768d71581
commit 8e35800abe
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 7 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
python/llm/test/benchmark/igpu-perf/1024-128_443.yaml
View file

@ -1,6 +1,7 @@
repo_id: repo_id:
- 'google/gemma-2-2b-it' - 'google/gemma-2-2b-it'
- 'google/gemma-2-9b-it' - 'google/gemma-2-9b-it'
- 'meta-llama/Llama-3.1-8B-Instruct'
local_model_hub: 'path to your local model hub' local_model_hub: 'path to your local model hub'
warm_up: 1 warm_up: 1
num_trials: 3 num_trials: 3

1
python/llm/test/benchmark/igpu-perf/1024-128_int4_fp16_443.yaml
View file

@ -1,6 +1,7 @@
repo_id: repo_id:
- 'google/gemma-2-2b-it' - 'google/gemma-2-2b-it'
- 'google/gemma-2-9b-it' - 'google/gemma-2-9b-it'
- 'meta-llama/Llama-3.1-8B-Instruct'
local_model_hub: 'path to your local model hub' local_model_hub: 'path to your local model hub'
warm_up: 1 warm_up: 1
num_trials: 3 num_trials: 3

1
python/llm/test/benchmark/igpu-perf/1024-128_int4_fp16_loadlowbit_443.yaml
View file

@ -1,6 +1,7 @@
repo_id: repo_id:
- 'google/gemma-2-2b-it' - 'google/gemma-2-2b-it'
- 'google/gemma-2-9b-it' - 'google/gemma-2-9b-it'
- 'meta-llama/Llama-3.1-8B-Instruct'
local_model_hub: 'path to your local model hub' local_model_hub: 'path to your local model hub'
warm_up: 1 warm_up: 1
num_trials: 3 num_trials: 3

1
python/llm/test/benchmark/igpu-perf/2048-256_int4_fp16_443.yaml
View file

@ -1,6 +1,7 @@
repo_id: repo_id:
- 'google/gemma-2-2b-it' - 'google/gemma-2-2b-it'
- 'google/gemma-2-9b-it' - 'google/gemma-2-9b-it'
- 'meta-llama/Llama-3.1-8B-Instruct'
local_model_hub: 'path to your local model hub' local_model_hub: 'path to your local model hub'
warm_up: 1 warm_up: 1
num_trials: 3 num_trials: 3

1
python/llm/test/benchmark/igpu-perf/3072-384_int4_fp16_443.yaml
View file

@ -1,6 +1,7 @@
repo_id: repo_id:
- 'google/gemma-2-2b-it' - 'google/gemma-2-2b-it'
# - 'google/gemma-2-9b-it' # - 'google/gemma-2-9b-it'
- 'meta-llama/Llama-3.1-8B-Instruct'
local_model_hub: 'path to your local model hub' local_model_hub: 'path to your local model hub'
warm_up: 1 warm_up: 1
num_trials: 3 num_trials: 3

1
python/llm/test/benchmark/igpu-perf/32-32_int4_fp16_443.yaml
View file

@ -1,6 +1,7 @@
repo_id: repo_id:
- 'google/gemma-2-2b-it' - 'google/gemma-2-2b-it'
- 'google/gemma-2-9b-it' - 'google/gemma-2-9b-it'
- 'meta-llama/Llama-3.1-8B-Instruct'
local_model_hub: 'path to your local model hub' local_model_hub: 'path to your local model hub'
warm_up: 3 warm_up: 3
num_trials: 5 num_trials: 5

1
python/llm/test/benchmark/igpu-perf/4096-512_int4_fp16_443.yaml
View file

@ -1,6 +1,7 @@
repo_id: repo_id:
- 'google/gemma-2-2b-it' - 'google/gemma-2-2b-it'
# - 'google/gemma-2-9b-it' # - 'google/gemma-2-9b-it'
- 'meta-llama/Llama-3.1-8B-Instruct'
local_model_hub: 'path to your local model hub' local_model_hub: 'path to your local model hub'
warm_up: 1 warm_up: 1
num_trials: 3 num_trials: 3