ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
ipex-llm/python/llm/dev/benchmark/all-in-one
History
Xin Qiu 5d9942a3ca transformer int4 and native int4's benchmark script for 32 256 1k 2k input (#8871) 
* transformer

* move

* update

* add header

* update all-in-one

* clean up
2023-09-07 09:49:55 +08:00
..
prompt transformer int4 and native int4's benchmark script for 32 256 1k 2k input (#8871) 2023-09-07 09:49:55 +08:00
config.yaml transformer int4 and native int4's benchmark script for 32 256 1k 2k input (#8871) 2023-09-07 09:49:55 +08:00
README.md [LLM] auto perf test to output to csv (#8846) 2023-09-01 10:48:00 +08:00
run-spr.sh [LLM] auto perf test to output to csv (#8846) 2023-09-01 10:48:00 +08:00
run.py transformer int4 and native int4's benchmark script for 32 256 1k 2k input (#8871) 2023-09-07 09:49:55 +08:00

README.md

All in One Benchmark Test

All in one benchmark test allows users to test all the benchmarks and record them in a result CSV. Users can provide models and related information in config.yaml.

Before running, make sure to have bigdl-llm installed.

Config

Config YAML file has following format

model_name: model_path
# following is an example, with model name llama2
llama2: /path/to/llama2

Run

run python run.py, this will output results to results.csv.

For SPR performance, run bash run-spr.sh.