ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

LLM: Enable jemalloc in benchmark scripts. (#9058)

Browse source 
* enable jemalloc.

* fix readme.
This commit is contained in:
Cengguang Zhang 2023-09-26 15:37:49 +08:00 committed by GitHub
parent ecee02b34d
commit ad62c58b33
2 changed files with 1 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
python/llm/dev/benchmark/all-in-one/README.md
View file

@ -37,8 +37,6 @@ run `python run.py`, this will output results to `results.csv`.
For SPR performance, run `bash run-spr.sh`.
> **Note**
>
> In `run-spr.sh`, we set optimal environment varaible by `source bigdl-nano-init -c`, `-c` stands for disabling jemalloc. Enabling jemalloc may lead to latency increasement after multiple trials.
>
> The value of `OMP_NUM_THREADS` should be the same as the cpu cores specified by `numactl -C`.
> **Note**

3
python/llm/dev/benchmark/all-in-one/run-spr.sh
View file

@ -1,7 +1,6 @@
#!/bin/bash
source bigdl-nano-init -c
source bigdl-nano-init
export OMP_NUM_THREADS=48
export TRANSFORMERS_OFFLINE=1
# set following parameters according to the actual specs of the test machine
numactl -C 0-47 -m 0 python $(dirname "$0")/run.py