ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
1744 commits 1 branch 0 tags 37 MiB
Commit graph

234 commits

Author SHA1 Message Date
Chen, Zhentao
c8e0c2ed48 Fixed dumped logs in harness (#9549) 
* install transformers==4.34.0

* modify output_path as a directory

* add device and task to output dir parents
 2023-11-30 12:47:56 +08:00
WeiguangHan
5098bc3544 LLM: enable previous models (#9505) 
* enable previous models

* test mistral model

* for test

* run models separately

* test all models

* for test

* revert the llm_performance_test.yaml
 2023-11-28 10:21:07 +08:00
Wang, Jian4
40ec9f7ead Add qlora cpu docker manually build (#9501) 2023-11-21 14:39:16 +08:00
WeiguangHan
c487b53f21 LLM: only run arc perf test nightly (#9448) 
* LLM: only run arc perf test nightly

* deleted unused python scripts

* rebase main
 2023-11-15 19:38:14 +08:00
Chen, Zhentao
d19ca21957 patch bigdl-llm model to harness by binding instead of patch file (#9420) 
* add run_llb.py

* fix args interpret

* modify outputs

* update workflow

* add license

* test mixed 4 bit

* update readme

* use autotokenizer

* add timeout

* refactor workflow file

* fix working directory

* fix env

* throw exception if some jobs failed

* improve terminal outputs

* Disable var which cause the run stuck

* fix unknown precision

* fix key error

* directly output config instead

* rm harness submodule
 2023-11-14 12:51:39 +08:00
Chen, Zhentao
f36d7b2d59 Fix harness stuck (#9435) 
* remove env to avoid being stuck

* use small model for test
 2023-11-13 15:29:53 +08:00
Yuwen Hu
4faf5af8f1 [LLM] Add perf test for core on Windows (#9397) 
* temporary stop other perf test

* Add framework for core performance test with one test model

* Small fix and add platform control

* Comment out lp for now

* Add missing ymal file

* Small fix

* Fix sed contents

* Small fix

* Small path fixes

* Small fix

* Add update to ftp

* Small upload fix

* add chatglm3-6b

* LLM: add model names

* Keep repo id same as ftp and temporary make baichuan2 first priority

* change order

* Remove temp if false and separate pr and nightly results

* Small fix

---------

Co-authored-by: jinbridge <2635480475@qq.com>
 2023-11-13 13:58:40 +08:00
WeiguangHan
2cfef5ef1e LLM: store the nightly test and pr results separately (#9404) 
* LLM: store the csv results separately

* modify the trigger files of LLM Performance Test
 2023-11-11 06:35:27 +08:00
Yuwen Hu
3d107f6d25 [LLM] Separate windows build UT and build runner (#9403) 
* Separate windows build UT and build runner

* Small fix
 2023-11-09 18:47:38 +08:00
WeiguangHan
34449cb4bb LLM: add remaining models to the arc perf test (#9384) 
* add remaining models

* modify the filepath which stores the test result on ftp server

* resolve some comments
 2023-11-09 14:28:42 +08:00
Yuwen Hu
d4b248fcd4 Add windows binary build label AVX_VNNI (#9387) 2023-11-08 18:13:35 +08:00
Chen, Zhentao
298b64217e add auto triggered acc test (#9364) 
* add auto triggered acc test

* use llama 7b instead

* fix env

* debug download

* fix download prefix

* add cut dirs

* fix env of model path

* fix dataset download

* full job

* source xpu env vars

* use matrix to trigger model run

* reset batch=1

* remove redirect

* remove some trigger

* add task matrix

* add precision list

* test llama-7b-chat

* use /mnt/disk1 to store model and datasets

* remove installation test

* correct downloading path

* fix HF vars

* add bigdl-llm env vars

* rename file

* fix hf_home

* fix script path

* rename as harness evalution

* rerun
 2023-11-08 10:22:27 +08:00
WeiguangHan
84ab614aab LLM: add more models and skip runtime error (#9349) 
* add more models and skip runtime error

* upgrade transformers

* temporarily removed Mistral-7B-v0.1

* temporarily disable the upload of arc perf result
 2023-11-08 09:45:53 +08:00
Shaojun Liu
833e4dbc8d fix llm-performance-test-on-arc bug (#9357) 2023-11-06 10:00:25 +08:00
ZehuaCao
ef83c3302e Use to test llm-performance on spr-perf (#9316) 
* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update action.yml

* Create cpu-perf-test.yaml

* Update action.yml

* Update action.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml

* Update llm_performance_tests.yml
 2023-11-03 11:17:16 +08:00
Cheen Hau, 俊豪
8f23fb04dc Add inference test for Whisper model on Arc (#9330) 
* Add inference test for Whisper model

* Remove unnecessary inference time measurement
 2023-11-03 10:15:52 +08:00
Ziteng Zhang
dd3cf2f153 LLM: Add python 3.10 & 3.11 UT 
LLM: Add python 3.10 & 3.11 UT
 2023-11-02 14:09:29 +08:00
Jasonzzt
d1bdc0ef72 spr & arc ut with python 3.9 & 3.10 & 3.11 2023-11-01 22:57:48 +08:00
Jasonzzt
687da21467 test 3.11 2023-11-01 19:14:53 +08:00
WeiguangHan
9722e811be LLM: add more models to the arc perf test (#9297) 
* LLM: add more models to the arc perf test

* remove some old models

* install some dependencies
 2023-11-01 16:56:32 +08:00
Jasonzzt
3c3329010d add conda update -n base conda 2023-11-01 16:36:35 +08:00
Jasonzzt
2fff0e8c21 use runner avx2 with linux 2023-11-01 16:28:29 +08:00
Jasonzzt
964a8e6dc1 update conda 2023-11-01 16:20:19 +08:00
Jasonzzt
cb7ef38e86 rerun 2023-11-01 15:30:34 +08:00
Jasonzzt
8f6e979fad test again 2023-11-01 15:10:11 +08:00
Jasonzzt
b66584f23b test 2023-11-01 14:51:23 +08:00
Jasonzzt
ba148ff3ff test py311 2023-11-01 14:08:49 +08:00
Jasonzzt
6f1cee90a4 test 2023-11-01 13:58:03 +08:00
Jasonzzt
d51821e264 test 2023-11-01 13:49:32 +08:00
Jasonzzt
7c7a7f2ec1 spr & arc ut with python3,9&3.10&3.11 2023-11-01 13:17:13 +08:00
Jasonzzt
4f9fd0dffd arc-ut with 3.10 & 3.11 2023-11-01 10:51:57 +08:00
Cengguang Zhang
d4ab5904ef LLM: Add python 3.10 llm UT (#9302) 
* add py310 test for llm-unit-test.

* add py310 llm-unit-tests

* add llm-cpp-build-py310

* test

* test

* test.

* test

* test

* fix deactivate.

* fix

* fix.

* fix

* test

* test

* test

* add build chatglm for win.

* test.

* fix
 2023-11-01 10:15:32 +08:00
WeiguangHan
03aa368776 LLM: add the comparison between latest arc perf test and last one (#9296) 
* add the comparison between latest test and last one to html

* resolve some comments

* modify some code logics
 2023-11-01 09:53:02 +08:00
Cheen Hau, 俊豪
d638b93dfe Add test script and workflow for qlora fine-tuning (#9295) 
* Add test script and workflow for qlora fine-tuning

* Test fix export model

* Download dataset

* Fix export model issue

* Reduce number of training steps

* Rename script

* Correction
 2023-11-01 09:39:53 +08:00
Yuwen Hu
21631209a9 [LLM] Skip CPU performance test for now (#9291) 
* Skip llm cpu performance test for now

* Add install for wheel package
 2023-10-27 12:55:04 +08:00
Ziteng Zhang
46ab0419b8 Merge pull request #9279 from Jasonzzt/main 
Add bigdl-llm-finetune-cpu to manually_build to upload image on hub
 2023-10-27 09:55:08 +08:00
Yuwen Hu
733df28a2b [LLM] Migrate Arc UT to another runner (#9286) 
* Separate arc llm ut to another runner

* Add dependency for einops
 2023-10-26 19:08:57 +08:00
Ziteng Zhang
916ccc0779 Update manually_build_for_testing.yml 2023-10-26 16:26:14 +08:00
Ziteng Zhang
14a23015f8 Update manually_build.yml 2023-10-26 16:24:03 +08:00
Jasonzzt
37b1708d16 Add bigdl-llm-finetune-cpu to manually_build 2023-10-26 15:53:44 +08:00
Lilac09
4ed7f066d3 add bigdl-llm-finetune-xpu to manually_build (#9278) 2023-10-26 15:30:05 +08:00
Cheen Hau, 俊豪
ab40607b87 Enable unit test workflow on Arc (#9213) 
* Add gpu workflow and a transformers API inference test

* Set device-specific env variables in script instead of workflow

* Fix status message

---------

Co-authored-by: sgwhat <ge.song@intel.com>
 2023-10-25 15:17:18 +08:00
SONG Ge
160a1e5ee7 [WIP] Add UT for Mistral Optimized Model (#9248) 
* add ut for mistral model

* update

* fix model path

* upgrade transformers version for mistral model

* refactor correctness ut for mustral model

* refactor mistral correctness ut

* revert test_optimize_model back

* remove mistral from test_optimize_model

* add to revert transformers version back to 4.31.0
 2023-10-25 15:14:17 +08:00
WeiguangHan
ec9195da42 LLM: using html to visualize the perf result for Arc (#9228) 
* LLM: using html to visualize the perf result for Arc

* deploy the html file

* add python license

* reslove some comments
 2023-10-24 18:05:25 +08:00
Guancheng Fu
f37547249d Refine README/CICD (#9253) 2023-10-24 12:56:03 +08:00
Guancheng Fu
9faa2f1eef Fix bigdl-llm-serving-tdx image (#9251) 2023-10-24 10:49:35 +08:00
Guancheng Fu
6cb884d82d Fix missing manually_build_for_testing entry (#9245) 2023-10-23 16:35:09 +08:00
Guancheng Fu
2ead3f7d54 add manually build (#9244) 2023-10-23 15:53:30 +08:00
WeiguangHan
f87f67ee1c LLM: arc perf test for some popular models (#9188) 2023-10-19 15:56:15 +08:00
ZehuaCao
65dd73b62e Update manually_build.yml (#9138) 
* Update manually_build.yml

fix llm-serving-tdx image build dir

* Update manually_build.yml
 2023-10-11 15:07:09 +08:00