ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update README.md (#10700)

Browse source 
Edit "summarize the results"
This commit is contained in:
Wenjing Margaret Mao 2024-04-09 16:01:12 +08:00 committed by GitHub
parent 3e4fbee87c
commit 289cc99cd6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
python/llm/dev/benchmark/harness/README.md
View file

@ -30,6 +30,6 @@ Taking example above, the script will fork 3 processes, each for one xpu, to exe
## Results ## Results
We follow [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) to record our metrics, `acc_norm` for `hellaswag` and `arc_challenge`, `mc2` for `truthful_qa` and `acc` for `mmlu`. For `mmlu`, there are 57 subtasks which means users may need to average them manually to get final result. We follow [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) to record our metrics, `acc_norm` for `hellaswag` and `arc_challenge`, `mc2` for `truthful_qa` and `acc` for `mmlu`. For `mmlu`, there are 57 subtasks which means users may need to average them manually to get final result.
## Summarize the results ## Summarize the results
"""python ```python
python make_table.py <input_dir> python make_table.py <input_dir>
""" ```