ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
ipex-llm/python/llm/dev/benchmark/all-in-one
History
Xin Qiu e9de9d9950 benchmark for native int4 (#8918) 
* native4

* update

* update

* update
2023-09-07 15:56:15 +08:00
..
prompt transformer int4 and native int4's benchmark script for 32 256 1k 2k input (#8871) 2023-09-07 09:49:55 +08:00
config.yaml benchmark for native int4 (#8918) 2023-09-07 15:56:15 +08:00
README.md [LLM] auto perf test to output to csv (#8846) 2023-09-01 10:48:00 +08:00
run-spr.sh [LLM] auto perf test to output to csv (#8846) 2023-09-01 10:48:00 +08:00
run.py benchmark for native int4 (#8918) 2023-09-07 15:56:15 +08:00

README.md

All in One Benchmark Test

All in one benchmark test allows users to test all the benchmarks and record them in a result CSV. Users can provide models and related information in config.yaml.

Before running, make sure to have bigdl-llm installed.

Config

Config YAML file has following format

model_name: model_path
# following is an example, with model name llama2
llama2: /path/to/llama2

Run

run python run.py, this will output results to results.csv.

For SPR performance, run bash run-spr.sh.