ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
3559 commits 1 branch 0 tags 37 MiB
Commit graph

458 commits

Author SHA1 Message Date
Yuwen Hu
75f836f288
Add extra warmup for THUDM/glm-4-9b-chat in igpu-performance test (#11417) 2024-06-24 18:08:05 +08:00
Shaojun Liu
5e823ef2ce
Fix nightly arc perf (#11404) 
* pip install pytest for arc perf test

* trigger test
 2024-06-24 15:58:41 +08:00
Shaojun Liu
5aa3e427a9
Fix docker images (#11362) 
* Fix docker images

* add-apt-repository requires gnupg, gpg-agent, software-properties-common

* update

* avoid importing ipex again
 2024-06-20 15:44:55 +08:00
Wenjing Margaret Mao
c0e86c523a
Add qwen-moe batch1 to nightly perf (#11369) 
* add moe

* reduce 437 models

* rename

* fix syntax

* add moe check result

* add 430 + 437

* all modes

* 4-37-4 exclud

* revert & comment

---------

Co-authored-by: Yishuo Wang <yishuo.wang@intel.com>
 2024-06-20 14:17:41 +08:00
Wenjing Margaret Mao
b2f62a8561
Add batch 4 perf test (#11355) 
* copy files to this branch

* add tasks

* comment one model

* change the model to test the 4.36

* only test batch-4

* typo

* typo

* typo

* typo

* typo

* typo

* add 4.37-batch4

* change the file name

* revet yaml file

* no print

* add batch4 task

* revert

---------

Co-authored-by: Yishuo Wang <yishuo.wang@intel.com>
 2024-06-20 09:48:52 +08:00
Qiyuan Gong
de4bb97b4f
Remove accelerate 0.23.0 install command in readme and docker (#11333) 
*ipex-llm's accelerate has been upgraded to 0.23.0. Remove accelerate 0.23.0 install command in README and docker。
 2024-06-17 17:52:12 +08:00
Yuwen Hu
a2a5890b48
Make manually-triggered perf test able to choose which test to run (#11324) 2024-06-17 10:23:13 +08:00
Yuwen Hu
1978f63f6b
Fix igpu performance guide regarding html generation (#11328) 2024-06-17 10:21:30 +08:00
Wenjing Margaret Mao
b61f6e3ab1
Add update_parent_folder for nightly_perf_test (#11287) 
* add update_parent_folder and change the workflow file

* add update_parent_folder and change the workflow file

* move to pr mode and comment the test

* use one model per comfig

* revert

---------

Co-authored-by: Yishuo Wang <yishuo.wang@intel.com>
 2024-06-12 17:58:13 +08:00
Wenjing Margaret Mao
70b17c87be
Merge multiple batches (#11264) 
* add merge steps

* move to pr mode

* remove build + add merge.py

* add tohtml and change cp

* change test_batch folder path

* change merge_temp path

* change to html folder

* revert

* change place

* revert 437

* revert space

---------

Co-authored-by: Yishuo Wang <yishuo.wang@intel.com>
 2024-06-07 18:38:45 +08:00
Shaojun Liu
8aabb5bac7
Enable CodeQL Check for CT39 (#11242) 
* Create codeql.yml

* Update codeql.yml

* Update codeql.yml

* Update codeql.yml

* Update codeql.yml
 2024-06-06 17:41:12 +08:00
Wenjing Margaret Mao
c825a7e1e9
change the workflow file to test ftp (#11241) 
* change the workflow to test ftp

* comment some models

* revert file

---------

Co-authored-by: Yishuo Wang <yishuo.wang@intel.com>
 2024-06-06 16:53:19 +08:00
Wenjing Margaret Mao
231b968aba
Modify the check_results.py to support batch 2&4 (#11133) 
* add batch 2&4 and exclude to perf_test

* modify the perf-test&437 yaml

* modify llm_performance_test.yml

* remove batch 4

* modify check_results.py to support batch 2&4

* change the batch_size format

* remove genxir

* add str(batch_size)

* change actual_test_casese in check_results file to support batch_size

* change html highlight

* less models to test html and html_path

* delete the moe model

* split batch html

* split

* use installing from pypi

* use installing from pypi - batch2

* revert cpp

* revert cpp

* merge two jobs into one, test batch_size in one job

* merge two jobs into one, test batch_size in one job

* change file directory in workflow

* try catch deal with odd file without batch_size

* modify pandas version

* change the dir

* organize the code

* organize the code

* remove Qwen-MOE

* modify based on feedback

* modify based on feedback

* modify based on second round of feedback

* modify based on second round of feedback + change run-arc.sh mode

* modify based on second round of feedback + revert config

* modify based on second round of feedback + revert config

* modify based on second round of feedback + remove comments

* modify based on second round of feedback + remove comments

* modify based on second round of feedback + revert arc-perf-test

* modify based on third round of feedback

* change error type

* change error type

* modify check_results.html

* split batch into two folders

* add all models

* move csv_name

* revert pr test

* revert pr test

---------

Co-authored-by: Yishuo Wang <yishuo.wang@intel.com>
 2024-06-05 15:04:55 +08:00
Shaojun Liu
dc4fea7e3f
always cleanup conda env after build (#11211) 2024-06-05 13:46:30 +08:00
Yuwen Hu
9f8074c653
Add extra warmup for chatglm3-6b in igpu-performance test (#11197) 
* Add extra warmup for chatglm3-6b to record more stable performance (int4+fp32)

* Small updates
 2024-06-04 14:06:09 +08:00
Shaojun Liu
401013a630
Remove chatglm_C Module to Eliminate LGPL Dependency (#11178) 
* remove chatglm_C.**.pyd to solve ngsolve weak copyright vunl

* fix style check error

* remove chatglm native int4 from langchain
 2024-05-31 17:03:11 +08:00
Jin Qiao
25b6402315
Add Windows GPU unit test (#11050) 
* Add Windows GPU UT

* temporarily remove ut on arc

* retry

* retry

* retry

* fix

* retry

* retry

* fix

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* fix

* retry

* retry

* retry

* retry

* retry

* retry

* merge into single workflow

* retry inference test

* retry

* retrigger

* try to fix inference test

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* retry

* check lower_bound

* retry

* retry

* try example test

* try fix example test

* retry

* fix

* seperate function into shell script

* remove cygpath

* try remove all cygpath

* retry

* retry

* Revert "try remove all cygpath"

This reverts commit 7ceeff3e48f08429062ecef548c1a3ad3488756f.

* Revert "retry"

This reverts commit 40ea2457843bff6991b8db24316cde5de1d35418.

* Revert "retry"

This reverts commit 817d0db3e5aec3bd449d3deaf4fb01d3ecfdc8a3.

* enable ut

* fix

* retrigger

* retrigger

* update download url

* fix

* fix

* retry

* add comment

* fix
 2024-05-28 13:29:47 +08:00
Yina Chen
b6b70d1ba0
Divide core-xe packages (#11131) 
* temp

* add batch

* fix style

* update package name

* fix style

* add workflow

* use temp version to run uts

* trigger performance test

* trigger win igpu perf

* revert workflow & setup
 2024-05-28 12:00:18 +08:00
Jiao Wang
0a06a6e1d4
Update tests for transformers 4.36 (#10858) 
* update unit test

* update

* update

* update

* update

* update

* fix gpu attention test

* update

* update

* update

* update

* update

* update

* update example test

* replace replit code

* update

* update

* update

* update

* set safe_serialization false

* perf test

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* delete

* update

* update

* update

* update

* update

* update

* revert

* update
 2024-05-24 10:26:38 +08:00
Yuwen Hu
1c5ed9b6cf
Fix arc ut (#11096) 2024-05-22 14:13:13 +08:00
Yuwen Hu
b3027e2d60
Update for cpu install option in performance tests (#11060) 2024-05-17 10:33:43 +08:00
Yuwen Hu
fff067d240
Make install ut for cpu exactly the same as what we want for users (#11051) 2024-05-17 10:11:01 +08:00
Shaojun Liu
c62e828281
Create release-ipex-llm.yaml (#11039) 2024-05-16 11:10:10 +08:00
Qiyuan Gong
4638682140
Fix xpu finetune image path in action (#11037) 
* Fix xpu finetune image path in action
 2024-05-16 10:48:02 +08:00
Xiangyu Tian
612a365479
LLM: Install CPU version torch with extras [all] (#10868) 
Modify setup.py to install CPU version torch with extras [all]
 2024-05-16 10:39:55 +08:00
Wang, Jian4
86cec80b51
LLM: Add llm inference_cpp_xpu_docker (#10933) 
* test_cpp_docker

* update

* update

* update

* update

* add sudo

* update nodejs version

* no need npm

* remove blinker

* new cpp docker

* restore

* add line

* add manually_build

* update and add mtl

* update for workdir llm

* add benchmark part

* update readme

* update 1024-128

* update readme

* update

* fix

* update

* update

* update readme too

* update readme

* no change

* update dir_name

* update readme
 2024-05-15 11:10:22 +08:00
Qiyuan Gong
1e00bd7bbe
Re-org XPU finetune images (#10971) 
* Rename xpu finetune image from `ipex-llm-finetune-qlora-xpu` to `ipex-llm-finetune-xpu`.
* Add axolotl to xpu finetune image.
* Upgrade peft to 0.10.0, transformers to 4.36.0.
* Add accelerate default config to home.
 2024-05-15 09:42:43 +08:00
Yuwen Hu
8010af700f
Update igpu performance test to use pypi installed oneAPI (#11010) 2024-05-14 14:05:33 +08:00
Kai Huang
f8dd2e52ad
Fix Langchain upstream ut (#10985) 
* Fix Langchain upstream ut

* Small fix

* Install bigdl-llm

* Update run-langchain-upstream-tests.sh

* Update run-langchain-upstream-tests.sh

* Update llm_unit_tests.yml

* Update run-langchain-upstream-tests.sh

* Update llm_unit_tests.yml

* Update run-langchain-upstream-tests.sh

* fix git checkout

* fix

---------

Co-authored-by: Zhangky11 <2321096202@qq.com>
Co-authored-by: Keyan (Kyrie) Zhang <79576162+Zhangky11@users.noreply.github.com>
 2024-05-11 14:40:37 +08:00
Yuwen Hu
9f6358e4c2
Deprecate support for pytorch 2.0 on Linux for ipex-llm >= 2.1.0b20240511 (#10986) 
* Remove xpu_2.0 option in setup.py

* Disable xpu_2.0 test in UT and nightly

* Update docs for deprecated pytorch 2.0

* Small doc update
 2024-05-11 12:33:35 +08:00
Wang, Jian4
459b764406
Remove munually_build_for_test push outside (#10968) 2024-05-09 10:40:34 +08:00
Zephyr1101
7e7d969dcb
a experimental for workflow abuse step1 fix a typo (#10965) 
* Update llm_unit_tests.yml

* Update README.md

* Update llm_unit_tests.yml

* Update llm_unit_tests.yml
 2024-05-08 17:12:50 +08:00
Qiyuan Gong
d7ca5d935b
Upgrade Peft version to 0.10.0 for LLM finetune (#10886) 
* Upgrade Peft version to 0.10.0
* Upgrade Peft version in ARC unit test and HF-Peft example.
 2024-05-07 15:09:14 +08:00
Wang, Jian4
33b8f524c2
Add cpp docker manually_test (#10946) 
* add cpp docker

* update
 2024-05-07 11:23:28 +08:00
Yuwen Hu
c936ba3b64
Small fix for supporting workflow dispatch in nightly perf (#10908) 2024-04-29 13:25:14 +08:00
Yuwen Hu
94b4e96fa6
Small updates for workflow-dispatch triggered nightly perf (#10902) 
* Small fix for workflow-dispatch triggerd nightly perf

* Small fix
 2024-04-28 11:27:20 +08:00
Yuwen Hu
7c290d3f92
Add workflow dispatch trigger to nightly perf (#10900) 2024-04-28 09:54:30 +08:00
Yuwen Hu
c7235e34a8
Small update to ut (#10804) 2024-04-19 10:59:00 +08:00
Zhicun
88463cbf47
fix transformer version (#10788) 
* fix transformer version

* uninstall sentence transformer

* uninstall

* uninstall
 2024-04-18 17:37:21 +08:00
Wenjing Margaret Mao
63a9a736be
Merge branch 'intel-analytics:main' into MargarettMao-parent_folder 2024-04-11 07:18:19 +08:00
Wenjing Margaret Mao
50dfcaa8fa
Update llm-ppl-evaluation.yml -- Update llm-ppl-evaluation.yml -- Update HTML file: change from ppl/update_in_parent_folder into harness/update_in_parent_folder 
ppl test and harness test are using the same update_in_parent_folder file. To reduce the repetition, change the ppl update HTML file to the same one under the harness folder and delete the HTML file under the ppl folder.
 2024-04-11 07:15:18 +08:00
Yuwen Hu
97db2492c8
Update setup.py for bigdl-core-xe-esimd-21 on Windows (#10705) 
* Support bigdl-core-xe-esimd-21 for windows in setup.py

* Update setup-llm-env accordingly
 2024-04-09 18:21:21 +08:00
Shaojun Liu
e10040b7f1
upgrade to python 3.11 (#10695) 2024-04-09 17:04:42 +08:00
Chen, Zhentao
d59e0cce5c
Migrate harness to ipexllm (#10703) 
* migrate to ipexlm

* fix workflow

* fix run_multi

* fix precision map

* rename ipexlm to ipexllm

* rename bigdl to ipex  in comments
 2024-04-09 15:48:53 +08:00
Zhicun
f03c029914
pydantic version>=2.0.0 for llamaindex (#10694) 
* pydantic version

* pydantic version

* upgrade version
 2024-04-09 09:48:42 +08:00
Shaojun Liu
db7c5cb78f
update model path for spr perf test (#10687) 
* update model path for spr perf test

* revert
 2024-04-08 10:21:56 +08:00
Shaojun Liu
d18dbfb097
update spr perf test (#10644) 2024-04-03 15:53:55 +08:00
Shaojun Liu
0779ca3db0
Bump ossf/scorecard-action to v2.3.1 (#10639) 
* Bump ossf/scorecard-action to v2.3.1

* revert
 2024-04-03 11:14:18 +08:00
Shaojun Liu
dfcf08c58a
update ossf/scorecard-action to fix TUF invalid key bug (#10635) 2024-04-03 09:55:32 +08:00
Shaojun Liu
a10f5a1b8d
add python style check (#10620) 
* add python style check

* fix style checks

* update runner

* add ipex-llm-finetune-qlora-cpu-k8s to manually_build workflow

* update tag to 2.1.0-SNAPSHOT
 2024-04-02 16:17:56 +08:00
Shaojun Liu
20a5e72da0
refine and verify ipex-llm-serving-xpu docker document (#10615) 
* refine serving on cpu/xpu

* minor fix

* replace localhost with 0.0.0.0 so that service can be accessed through ip address
 2024-04-02 11:45:45 +08:00
Shaojun Liu
c4b533f0e1
nightly build docker images (#10585) 
* nightly build docker images
 2024-03-29 16:12:28 +08:00
Cheen Hau, 俊豪
1c5eb14128
Update pip install to use --extra-index-url for ipex package (#10557) 
* Change to 'pip install .. --extra-index-url' for readthedocs

* Change to 'pip install .. --extra-index-url' for examples

* Change to 'pip install .. --extra-index-url' for remaining files

* Fix URL for ipex

* Add links for ipex US and CN servers

* Update ipex cpu url

* remove readme

* Update for github actions

* Update for dockerfiles
 2024-03-28 09:56:23 +08:00
Shaojun Liu
924e01b842
Create scorecard.yml (#10559) 2024-03-27 16:51:10 +08:00
Shaojun Liu
bb9be70105
replace bigdl-llm with ipex-llm (#10545) 2024-03-26 15:12:38 +08:00
Shaojun Liu
c563b41491
add nightly_build workflow (#10533) 
* add nightly_build workflow

* add create-job-status-badge action

* update

* update

* update

* update setup.py

* release

* revert
 2024-03-26 12:47:38 +08:00
Shaojun Liu
93e6804bfe
update nightly test (#10520) 
* trigger nightly test

* trigger perf test

* update bigdl-llm to ipex-llm

* revert
 2024-03-25 18:22:05 +08:00
Wang, Jian4
a1048ca7f6
Update setup.py and add new actions and add compatible mode (#25) 
* update setup.py

* add new action

* add compatible mode
 2024-03-22 15:44:59 +08:00
Yuwen Hu
1579ee4421 [LLM] Add nightly igpu perf test for INT4+FP16 1024-128 (#10496) 2024-03-21 16:07:06 +08:00
Shaojun Liu
a57fd52a5b pip install notebook (#10444) 2024-03-18 13:56:34 +08:00
Keyan (Kyrie) Zhang
444b11af22 Add LangChain upstream ut test for ipynb (#10387) 
* Add LangChain upstream ut test for ipynb

* Integrate unit test for LangChain upstream ut and ipynb into one file

* Modify file name

* Remove LangChain version update in unit test

* Move Langchain upstream ut job to arc

* Modify path in .yml file

* Modify path in llm_unit_tests.yml

* Avoid create directory repeatedly
 2024-03-15 16:31:01 +08:00
Yuxuan Xia
a90e9b6ec2 Fix C-Eval Workflow (#10359) 
* Fix Baichuan2 prompt format

* Fix ceval workflow errors

* Fix ceval workflow error

* Fix ceval error

* Fix ceval error

* Test ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Add ceval dependency test

* Fix ceval

* Fix ceval

* Test full ceval

* Test full ceval

* Fix ceval

* Fix ceval
 2024-03-13 17:23:17 +08:00
Keyan (Kyrie) Zhang
7cf01e6ec8 Add LangChain upstream ut test (#10349) 
* Add LangChain upstream ut test

* Add LangChain upstream ut test

* Specify version numbers in yml script

* Correct langchain-community version
 2024-03-13 09:52:45 +08:00
binbin Deng
df3bcc0e65 LLM: remove english_quotes dataset (#10370) 2024-03-12 16:57:40 +08:00
WeiguangHan
17bdb1a60b LLM: add whisper models into nightly test (#10193) 
* LLM: add whisper models into nightly test

* small fix

* small fix

* add more whisper models

* test all cases

* test specific cases

* collect the csv

* store the resut

* to html

* small fix

* small test

* test all cases

* modify whisper_csv_to_html
 2024-03-11 20:00:47 +08:00
Chen, Zhentao
a425eaabfc fix from_pretrained when device_map=None (#10361) 
* pr trigger

* fix error when device_map=None

* fix device_map=None
 2024-03-11 16:06:12 +08:00
Keyan (Kyrie) Zhang
f1825d7408 Add RMSNorm unit test (#10190) 2024-03-08 15:51:03 +08:00
Yuxuan Xia
0c8d3c9830 Add C-Eval HTML report (#10294) 
* Add C-Eval HTML report

* Fix C-Eval workflow pr trigger path

* Fix C-Eval workflow typos

* Add permissions to C-Eval workflow

* Fix C-Eval workflow typo

* Add pandas dependency

* Fix C-Eval workflow typo
 2024-03-07 16:44:49 +08:00
hxsz1997
b7db21414e Update llamaindex ut (#10338) 
* add test_llamaindex of gpu

* add llamaindex gpu tests bash

* add llamaindex cpu tests bash

* update name of Run LLM langchain GPU test

* import llama_index in llamaindex gpu ut

* update the dependency of test_llamaindex

* add Run LLM llamaindex GPU test

* modify import dependency of llamaindex cpu test

* add Run LLM llamaindex test

* update llama_model_path

* delete unused model path

* add LLAMA2_7B_ORIGIN_PATH in llamaindex cpu test
 2024-03-07 10:06:16 +08:00
dingbaorong
fc7f10cd12 add langchain gpu example (#10277) 
* first draft

* fix

* add readme for transformer_int4_gpu

* fix doc

* check device_map

* add arc ut test

* fix ut test

* fix langchain ut

* Refine README

* fix gpu mem too high

* fix ut test

---------

Co-authored-by: Ariadne <wyn2000330@126.com>
 2024-03-05 13:33:57 +08:00
Yuwen Hu
5dbbe1a826 [LLM] Support for new arc ut runner (#10311) 
* Support for new arc ut runner

* Comment unnecessary OMP_NUM_THREADS related settings for arc uts
 2024-03-04 18:42:02 +08:00
Yuwen Hu
d45e577d8c [LLM] Test load_low_bit in iGPU perf test on Windows (#10313) 2024-03-04 18:03:57 +08:00
Shaojun Liu
bab2ee5f9e update nightly spr perf test (#10178) 
* update nightly spr perf test

* update

* update runner lable

* update

* update

* update folder

* revert
 2024-03-04 13:46:33 +08:00
Shaojun Liu
57e211dab4 topLevel 'contents' permission set to 'read' (#10295) 2024-03-04 10:33:19 +08:00
hxsz1997
925aff730e Integrate the result of ppl and harness (#10265) 
* modify NIGHTLY_MATRIX_PRECISION

* change ACC_FOLDER of harness

* change ACC_FOLDER of ppl
 2024-02-28 17:53:02 +08:00
Yuwen Hu
d85f7c78df Small fix for better trail (#10256) 2024-02-27 20:00:40 +08:00
hxsz1997
cba61a2909 Add html report of ppl (#10218) 
* remove include and language option, select the corresponding dataset based on the model name in Run

* change the nightly test time

* change the nightly test time of harness and ppl

* save the ppl result to json file

* generate csv file and print table result

* generate html

* modify the way to get parent folder

* update html in parent folder

* add llm-ppl-summary and llm-ppl-summary-html

* modify echo single result

* remove download fp16.csv

* change model name of PR

* move ppl nightly related files to llm/test folder

* reformat

* seperate make_table from make_table_and_csv.py

* separate make_csv from make_table_and_csv.py

* update llm-ppl-html

* remove comment

* add Download fp16.results
 2024-02-27 17:37:08 +08:00
hxsz1997
15ad2fd72e Merge pull request #10226 from zhentaocc/fix_harness 
Fix harness
 2024-02-26 16:49:27 +08:00
Chen, Zhentao
5ad752bae8 Separate llmcpp build of linux and windows (#10136) 
* separate linux window llmcpp build

* harness run on linux only

* fix platform

* skip error

* change to linux only build

* add judgement of platform

* add download args

* remove  ||true
 2024-02-26 15:04:29 +08:00
Chen, Zhentao
62350a36f0 fix if in update html 2024-02-26 13:39:59 +08:00
Yuxuan Xia
0c6aef0f47 Add einops dependency for C-Eval (#10234) 
* Add c-eval workflow and modify running files

* Modify the chatglm evaluator file

* Modify the ceval workflow for triggering test

* Modify the ceval workflow file

* Modify the ceval workflow file

* Modify ceval workflow

* Adjust the ceval dataset download

* Add ceval workflow dependencies

* Modify ceval workflow dataset download

* Add ceval test dependencies

* Add ceval test dependencies

* Correct the result print

* Fix the nightly test trigger time

* Fix ChatGLM loading issue

* Add einops dependency
 2024-02-26 10:13:10 +08:00
Chen, Zhentao
85d13c65de run one job only if triggered by pr 2024-02-24 00:33:33 +08:00
Chen, Zhentao
a55cc91e1f fix make_csv.py 2024-02-23 20:25:46 +08:00
Chen, Zhentao
a204337cad Rename results 2024-02-23 17:12:37 +08:00
Chen, Zhentao
4fdf96dc8b fix ACC_FOLDER 2024-02-23 17:11:03 +08:00
Chen, Zhentao
e838ec9e14 remove dependency 2024-02-23 16:33:40 +08:00
Chen, Zhentao
88f7f56980 rewrite html visualization 2024-02-23 16:33:39 +08:00
Chen, Zhentao
bfa98666a6 fall back to make_table.py 2024-02-23 16:33:38 +08:00
Chen, Zhentao
02cb96e7f6 fix Run Harness job 2024-02-23 16:33:37 +08:00
Chen, Zhentao
e1fcf54a0c reformat 2024-02-23 16:33:36 +08:00
Chen, Zhentao
5399343adc fix harness installation 2024-02-23 16:33:35 +08:00
Chen, Zhentao
9c8e349196 remove harness job output 2024-02-23 16:33:34 +08:00
Chen, Zhentao
8472de90e8 use stable lm to test pr 2024-02-23 16:33:34 +08:00
Chen, Zhentao
f315c7f93a Move harness nightly related files to llm/test folder (#10209) 
* move harness nightly files to test folder

* change workflow file path accordingly

* use arc01 when pr

* fix path

* fix fp16 csv path
 2024-02-23 11:12:36 +08:00
Ovo233
60e11b6739 LLM: Add mlp layer unit tests (#10200) 
* add mlp layer unit tests

* add download baichuan-13b

* exclude llama for now

* install additional packages

* rename bash file

* switch to Baichuan2

* delete attention related code

* fix name errors in yml file
 2024-02-22 13:44:45 +08:00
hxsz1997
5b387bb71a Change the nightly test time of ppl and harness (#10198) 
* remove include and language option, select the corresponding dataset based on the model name in Run

* change the nightly test time

* change the nightly test time of harness and ppl
 2024-02-21 17:39:33 +08:00
yb-peng
b1a97b71a9 Harness eval: Add is_last parameter and fix logical operator in highlight_vals (#10192) 
* Add is_last parameter and fix logical operator in highlight_vals

* Add script to update HTML files in parent folder

* Add running update_html_in_parent_folder.py in summarize step

* Add licence info

* Remove update_html_in_parent_folder.py in Summarize the results for pull request
 2024-02-21 14:45:32 +08:00
Chen, Zhentao
39d37bd042 upgrade harness package version in workflow (#10188) 
* upgrade harness

* update readme
 2024-02-21 11:21:30 +08:00
yb-peng
de3dc609ee Modify harness evaluation workflow (#10174) 
* Modify table head in harness

* Specify the file path of fp16.csv

* change run to run nightly and run pr to debug

* Modify the way to get fp16.csv to downloading from github

* Change the method to calculate diff in html table

* Change the method to calculate diff in html table

* Re-arrange job order

* Re-arrange job order

* Change limit

* Change fp16.csv  path

* Change highlight rules

* Change limit
 2024-02-20 18:55:43 +08:00
hxsz1997
b55fd00fb1 remove include and language option, select the corresponding dataset based on the model name in Run (#10181) 2024-02-20 17:34:52 +08:00