ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update README.md (#11824)

Browse source
This commit is contained in:
Jason Dai 2024-08-16 10:22:02 +08:00 committed by GitHub
parent 9e9086cc2a
commit 6a8d07ddb4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 9 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
README.md
View file

@ -158,9 +158,15 @@ See the demo of running [*Text-Generation-WebUI*](https://ipex-llm.readthedocs.i
Please see the **Perplexity** result below (tested on Wikitext dataset using the script [here](https://github.com/intel-analytics/ipex-llm/tree/main/python/llm/dev/benchmark/perplexity)).
|Perplexity |sym_int4 |q4_k |fp6 |fp8_e5m2 |fp8_e4m3 |fp16 |
|---------------------------|---------|-------|-------|---------|---------|-------|
|Llama-2-7B-chat-hf |6.3638 |6.2179 |6.0924 |6.1796 |6.0980 |6.0963 |
|Mistral-7B-Instruct-v0.1 |6.0025 |5.9581 |5.8930 |5.8884 |5.8820 |5.8734 |
|Qwen1.5-7B-chat |8.8652 |8.8163 |8.5573 |8.8463 |8.5304 |8.6065 |
|Llama-2-7B-chat-hf |6.364 |6.218 |6.092 |6.180 |6.098 |6.096 |
|Mistral-7B-Instruct-v0.2 |5.365 |5.320 |5.270 |5.273 |5.246 |5.244 |
|Baichuan2-7B-chat |6.734 |6.727 |6.527 |6.539 |6.488 |6.508 |
|Qwen1.5-7B-chat |8.865 |8.816 |8.557 |8.846 |8.530 |8.607 |
|Llama-3.1-8B-Instruct |6.705 |6.566 |6.338 |6.383 |6.325 |6.267 |
|gemma-2-9b-it |7.541 |7.412 |7.269 |7.380 |7.268 |7.270 |
|Baichuan2-13B-Chat |6.313 |6.160 |6.070 |6.145 |6.086 |6.031 |
|Llama-2-13b-chat-hf |5.449 |5.422 |5.341 |5.384 |5.332 |5.329 |
|Qwen1.5-14B-Chat |7.529 |7.520 |7.367 |7.504 |7.297 |7.334 |
[^1]: Performance varies by use, configuration and other factors. `ipex-llm` may not optimize to the same degree for non-Intel products. Learn more at www.Intel.com/PerformanceIndex.