ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
ipex-llm/python/llm/test/benchmark/igpu-perf
History
Yuwen Hu 0d58c2fdf9
Update performance test regarding updated default transformers==4.37.0 (#11869) 
* Update igpu performance from transformers 4.36.2 to 4.37.0 (#11841)

* upgrade arc perf test to transformers 4.37 (#11842)

* fix load low bit com dtype (#11832)

* feat: add mixed_precision argument on ppl longbench evaluation

* fix: delete extra code

* feat: upgrade arc perf test to transformers 4.37

* fix: add missing codes

* fix: keep perf test for qwen-vl-chat in transformers 4.36

* fix: remove extra space

* fix: resolve pr comment

* fix: add empty line

* fix: add pip install for spr and core test

* fix: delete extra comments

* fix: remove python -m for pip

* Revert "fix load low bit com dtype (#11832)"

This reverts commit 6841a9ac8f.

---------

Co-authored-by: Zhao Changmin <changmin.zhao@intel.com>
Co-authored-by: Jinhe Tang <jin.tang1337@gmail.com>

* add transformers==4.36 for qwen vl in igpu-perf (#11846)

* add transformers==4.36.2 for qwen-vl

* Small update

---------

Co-authored-by: Yuwen Hu <yuwen.hu@intel.com>

* fix: remove qwen-7b on core test (#11851)

* fix: remove qwen-7b on core test

* fix: change delete to comment

---------

Co-authored-by: Jinhe Tang <jin.tang1337@gmail.com>

* replce filename (#11854)

* fix: remove qwen-7b on core test

* fix: change delete to comment

* fix: replace filename

---------

Co-authored-by: Jinhe Tang <jin.tang1337@gmail.com>

* fix: delete extra comments (#11863)

* Remove transformers installation for temp test purposes

* Small fix

* Small update

---------

Co-authored-by: Chu,Youcheng <70999398+cranechu0131@users.noreply.github.com>
Co-authored-by: Zhao Changmin <changmin.zhao@intel.com>
Co-authored-by: Jinhe Tang <jin.tang1337@gmail.com>
Co-authored-by: Zijie Li <michael20001122@gmail.com>
Co-authored-by: Chu,Youcheng <1340390339@qq.com>
2024-08-20 17:59:28 +08:00
..
32-32_int4_fp16.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
32-32_int4_fp16_436.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
32-32_int4_fp16_438.yaml Add Yi-6B and StableLM to iGPU perf test (#11546) 2024-07-09 18:51:23 +08:00
32-32_int4_fp16_443.yaml Add gemma-2-2b-it and gemma-2-9b-it to igpu nightly performance test (#11778) 2024-08-13 15:39:56 +08:00
1024-128.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
1024-128_436.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
1024-128_438.yaml Add Yi-6B and StableLM to iGPU perf test (#11546) 2024-07-09 18:51:23 +08:00
1024-128_443.yaml Add gemma-2-2b-it and gemma-2-9b-it to igpu nightly performance test (#11778) 2024-08-13 15:39:56 +08:00
1024-128_int4_fp16.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
1024-128_int4_fp16_436.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
1024-128_int4_fp16_438.yaml Add Yi-6B and StableLM to iGPU perf test (#11546) 2024-07-09 18:51:23 +08:00
1024-128_int4_fp16_443.yaml Add gemma-2-2b-it and gemma-2-9b-it to igpu nightly performance test (#11778) 2024-08-13 15:39:56 +08:00
1024-128_int4_fp16_loadlowbit.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
1024-128_int4_fp16_loadlowbit_436.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
1024-128_int4_fp16_loadlowbit_438.yaml Add Yi-6B and StableLM to iGPU perf test (#11546) 2024-07-09 18:51:23 +08:00
1024-128_int4_fp16_loadlowbit_443.yaml Add gemma-2-2b-it and gemma-2-9b-it to igpu nightly performance test (#11778) 2024-08-13 15:39:56 +08:00
2048-256_int4_fp16.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
2048-256_int4_fp16_436.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
2048-256_int4_fp16_438.yaml Add Yi-6B and StableLM to iGPU perf test (#11546) 2024-07-09 18:51:23 +08:00
2048-256_int4_fp16_443.yaml Add gemma-2-2b-it and gemma-2-9b-it to igpu nightly performance test (#11778) 2024-08-13 15:39:56 +08:00
3072-384_int4_fp16.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
3072-384_int4_fp16_436.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
3072-384_int4_fp16_438.yaml add 3k and 4k input of nightly perf test on iGPU (#11701) 2024-08-01 14:17:46 +08:00
3072-384_int4_fp16_443.yaml Remove gemma-2-9b-it 3k input from igpu-perf (#11834) 2024-08-17 13:10:05 +08:00
4096-512_int4_fp16.yaml Update performance test regarding updated default transformers==4.37.0 (#11869) 2024-08-20 17:59:28 +08:00
4096-512_int4_fp16_438.yaml add 3k and 4k input of nightly perf test on iGPU (#11701) 2024-08-01 14:17:46 +08:00
4096-512_int4_fp16_443.yaml Remove 4k igpu perf on gemma-2-9b-it (#11820) 2024-08-15 18:06:19 +08:00
check_csv_results.py Add check in iGPU perf workflow for results integrity (#11616) 2024-07-18 14:13:16 +08:00