ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Openvino all in one benchmark small fix (#12269)

Browse source 
* Small update for all-in-one benchmark readme to support OpenVINO tests

* Small fix
This commit is contained in:
Yuwen Hu 2024-10-25 14:13:52 +08:00 committed by GitHub
parent f7f62a3fef
commit 93895b2ac2
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 4 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
python/llm/dev/benchmark/all-in-one/README.md
View file

@ -2,7 +2,9 @@
All in one benchmark test allows users to test all the benchmarks and record them in a result CSV. Users can provide models and related information in `config.yaml`.
Before running, make sure to have [ipex-llm](../../../../../README.md) installed.
Before running, make sure you have [ipex-llm](../../../../../README.md) installed.
If you would like to use all-in-one benchmark for testing OpenVINO, please directly refer to [this section](#optional-save-model-for-openvino) for environment setup.
> The prompts for benchmarking are from datasets [abisee/cnn_dailymail](https://huggingface.co/datasets/abisee/cnn_dailymail), [Open-Orca/OpenOrca](https://huggingface.co/datasets/Open-Orca/OpenOrca), [THUDM/LongBench](https://huggingface.co/datasets/THUDM/LongBench), etc.
@ -82,7 +84,7 @@ Follow commands below to set up the environment for testing OpenVINO on Intel GP
conda create -n test-ov python=3.11
pip install -r requirements.txt
pip install --pre --upgrade ipex-llm # only for IPEX-LLM BenchmarkWrapper
pip install accelerate
pip install accelerate omegaconf pandas
```
Then, running `python save_openvino.py` will save all models declared in `repo_id` list into OpenVINO models with `low_bit` precision under `local_model_hub` folder.