ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

LLM: fix fp16 mem record in benchmark (#10461)

Browse source 
* LLM: fix fp16 mem record in benchmark

* change style
This commit is contained in:
Jin Qiao 2024-03-19 16:17:23 +08:00 committed by GitHub
parent 0e388f4b91
commit e9055c32f9
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
python/llm/dev/benchmark/all-in-one/run.py
View file

@ -115,7 +115,7 @@ def run_model(repo_id, test_api, in_out_pairs, local_model_hub=None, warm_up=1,
low_bit,
cpu_embedding if 'win' in test_api else 'N/A',
round(result[in_out_pair][-1][5], 2),
result[in_out_pair][-1][6] if 'int4_gpu' in test_api or 'int4_loadlowbit_gpu' in test_api else 'N/A',
result[in_out_pair][-1][6] if any(keyword in test_api for keyword in ['int4_gpu', 'int4_fp16_gpu_win', 'int4_loadlowbit_gpu' ]) else 'N/A',
streaming if 'win' in test_api else 'N/A'],
)