Commit graph

361 commits

Author SHA1 Message Date
Shaojun Liu
0779ca3db0
Bump ossf/scorecard-action to v2.3.1 (#10639)
* Bump ossf/scorecard-action to v2.3.1

* revert
2024-04-03 11:14:18 +08:00
Shaojun Liu
dfcf08c58a
update ossf/scorecard-action to fix TUF invalid key bug (#10635) 2024-04-03 09:55:32 +08:00
Shaojun Liu
a10f5a1b8d
add python style check (#10620)
* add python style check

* fix style checks

* update runner

* add ipex-llm-finetune-qlora-cpu-k8s to manually_build workflow

* update tag to 2.1.0-SNAPSHOT
2024-04-02 16:17:56 +08:00
Shaojun Liu
20a5e72da0
refine and verify ipex-llm-serving-xpu docker document (#10615)
* refine serving on cpu/xpu

* minor fix

* replace localhost with 0.0.0.0 so that service can be accessed through ip address
2024-04-02 11:45:45 +08:00
Shaojun Liu
c4b533f0e1
nightly build docker images (#10585)
* nightly build docker images
2024-03-29 16:12:28 +08:00
Cheen Hau, 俊豪
1c5eb14128
Update pip install to use --extra-index-url for ipex package (#10557)
* Change to 'pip install .. --extra-index-url' for readthedocs

* Change to 'pip install .. --extra-index-url' for examples

* Change to 'pip install .. --extra-index-url' for remaining files

* Fix URL for ipex

* Add links for ipex US and CN servers

* Update ipex cpu url

* remove readme

* Update for github actions

* Update for dockerfiles
2024-03-28 09:56:23 +08:00
Shaojun Liu
924e01b842
Create scorecard.yml (#10559) 2024-03-27 16:51:10 +08:00
Shaojun Liu
bb9be70105
replace bigdl-llm with ipex-llm (#10545) 2024-03-26 15:12:38 +08:00
Shaojun Liu
c563b41491
add nightly_build workflow (#10533)
* add nightly_build workflow

* add create-job-status-badge action

* update

* update

* update

* update setup.py

* release

* revert
2024-03-26 12:47:38 +08:00
Shaojun Liu
93e6804bfe
update nightly test (#10520)
* trigger nightly test

* trigger perf test

* update bigdl-llm to ipex-llm

* revert
2024-03-25 18:22:05 +08:00
Wang, Jian4
a1048ca7f6
Update setup.py and add new actions and add compatible mode (#25)
* update setup.py

* add new action

* add compatible mode
2024-03-22 15:44:59 +08:00
Yuwen Hu
1579ee4421 [LLM] Add nightly igpu perf test for INT4+FP16 1024-128 (#10496) 2024-03-21 16:07:06 +08:00
Shaojun Liu
a57fd52a5b pip install notebook (#10444) 2024-03-18 13:56:34 +08:00
Keyan (Kyrie) Zhang
444b11af22 Add LangChain upstream ut test for ipynb (#10387)
* Add LangChain upstream ut test for ipynb

* Integrate unit test for LangChain upstream ut and ipynb into one file

* Modify file name

* Remove LangChain version update in unit test

* Move Langchain upstream ut job to arc

* Modify path in .yml file

* Modify path in llm_unit_tests.yml

* Avoid create directory repeatedly
2024-03-15 16:31:01 +08:00
Yuxuan Xia
a90e9b6ec2 Fix C-Eval Workflow (#10359)
* Fix Baichuan2 prompt format

* Fix ceval workflow errors

* Fix ceval workflow error

* Fix ceval error

* Fix ceval error

* Test ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Fix ceval

* Add ceval dependency test

* Fix ceval

* Fix ceval

* Test full ceval

* Test full ceval

* Fix ceval

* Fix ceval
2024-03-13 17:23:17 +08:00
Keyan (Kyrie) Zhang
7cf01e6ec8 Add LangChain upstream ut test (#10349)
* Add LangChain upstream ut test

* Add LangChain upstream ut test

* Specify version numbers in yml script

* Correct langchain-community version
2024-03-13 09:52:45 +08:00
binbin Deng
df3bcc0e65 LLM: remove english_quotes dataset (#10370) 2024-03-12 16:57:40 +08:00
WeiguangHan
17bdb1a60b LLM: add whisper models into nightly test (#10193)
* LLM: add whisper models into nightly test

* small fix

* small fix

* add more whisper models

* test all cases

* test specific cases

* collect the csv

* store the resut

* to html

* small fix

* small test

* test all cases

* modify whisper_csv_to_html
2024-03-11 20:00:47 +08:00
Chen, Zhentao
a425eaabfc fix from_pretrained when device_map=None (#10361)
* pr trigger

* fix error when device_map=None

* fix device_map=None
2024-03-11 16:06:12 +08:00
Keyan (Kyrie) Zhang
f1825d7408 Add RMSNorm unit test (#10190) 2024-03-08 15:51:03 +08:00
Yuxuan Xia
0c8d3c9830 Add C-Eval HTML report (#10294)
* Add C-Eval HTML report

* Fix C-Eval workflow pr trigger path

* Fix C-Eval workflow typos

* Add permissions to C-Eval workflow

* Fix C-Eval workflow typo

* Add pandas dependency

* Fix C-Eval workflow typo
2024-03-07 16:44:49 +08:00
hxsz1997
b7db21414e Update llamaindex ut (#10338)
* add test_llamaindex of gpu

* add llamaindex gpu tests bash

* add llamaindex cpu tests bash

* update name of Run LLM langchain GPU test

* import llama_index in llamaindex gpu ut

* update the dependency of test_llamaindex

* add Run LLM llamaindex GPU test

* modify import dependency of llamaindex cpu test

* add Run LLM llamaindex test

* update llama_model_path

* delete unused model path

* add LLAMA2_7B_ORIGIN_PATH in llamaindex cpu test
2024-03-07 10:06:16 +08:00
dingbaorong
fc7f10cd12 add langchain gpu example (#10277)
* first draft

* fix

* add readme for transformer_int4_gpu

* fix doc

* check device_map

* add arc ut test

* fix ut test

* fix langchain ut

* Refine README

* fix gpu mem too high

* fix ut test

---------

Co-authored-by: Ariadne <wyn2000330@126.com>
2024-03-05 13:33:57 +08:00
Yuwen Hu
5dbbe1a826 [LLM] Support for new arc ut runner (#10311)
* Support for new arc ut runner

* Comment unnecessary OMP_NUM_THREADS related settings for arc uts
2024-03-04 18:42:02 +08:00
Yuwen Hu
d45e577d8c [LLM] Test load_low_bit in iGPU perf test on Windows (#10313) 2024-03-04 18:03:57 +08:00
Shaojun Liu
bab2ee5f9e update nightly spr perf test (#10178)
* update nightly spr perf test

* update

* update runner lable

* update

* update

* update folder

* revert
2024-03-04 13:46:33 +08:00
Shaojun Liu
57e211dab4 topLevel 'contents' permission set to 'read' (#10295) 2024-03-04 10:33:19 +08:00
hxsz1997
925aff730e Integrate the result of ppl and harness (#10265)
* modify NIGHTLY_MATRIX_PRECISION

* change ACC_FOLDER of harness

* change ACC_FOLDER of ppl
2024-02-28 17:53:02 +08:00
Yuwen Hu
d85f7c78df Small fix for better trail (#10256) 2024-02-27 20:00:40 +08:00
hxsz1997
cba61a2909 Add html report of ppl (#10218)
* remove include and language option, select the corresponding dataset based on the model name in Run

* change the nightly test time

* change the nightly test time of harness and ppl

* save the ppl result to json file

* generate csv file and print table result

* generate html

* modify the way to get parent folder

* update html in parent folder

* add llm-ppl-summary and llm-ppl-summary-html

* modify echo single result

* remove download fp16.csv

* change model name of PR

* move ppl nightly related files to llm/test folder

* reformat

* seperate make_table from make_table_and_csv.py

* separate make_csv from make_table_and_csv.py

* update llm-ppl-html

* remove comment

* add Download fp16.results
2024-02-27 17:37:08 +08:00
hxsz1997
15ad2fd72e Merge pull request #10226 from zhentaocc/fix_harness
Fix harness
2024-02-26 16:49:27 +08:00
Chen, Zhentao
5ad752bae8 Separate llmcpp build of linux and windows (#10136)
* separate linux window llmcpp build

* harness run on linux only

* fix platform

* skip error

* change to linux only build

* add judgement of platform

* add download args

* remove  ||true
2024-02-26 15:04:29 +08:00
Chen, Zhentao
62350a36f0 fix if in update html 2024-02-26 13:39:59 +08:00
Yuxuan Xia
0c6aef0f47 Add einops dependency for C-Eval (#10234)
* Add c-eval workflow and modify running files

* Modify the chatglm evaluator file

* Modify the ceval workflow for triggering test

* Modify the ceval workflow file

* Modify the ceval workflow file

* Modify ceval workflow

* Adjust the ceval dataset download

* Add ceval workflow dependencies

* Modify ceval workflow dataset download

* Add ceval test dependencies

* Add ceval test dependencies

* Correct the result print

* Fix the nightly test trigger time

* Fix ChatGLM loading issue

* Add einops dependency
2024-02-26 10:13:10 +08:00
Chen, Zhentao
85d13c65de run one job only if triggered by pr 2024-02-24 00:33:33 +08:00
Chen, Zhentao
a55cc91e1f fix make_csv.py 2024-02-23 20:25:46 +08:00
Chen, Zhentao
a204337cad Rename results 2024-02-23 17:12:37 +08:00
Chen, Zhentao
4fdf96dc8b fix ACC_FOLDER 2024-02-23 17:11:03 +08:00
Chen, Zhentao
e838ec9e14 remove dependency 2024-02-23 16:33:40 +08:00
Chen, Zhentao
88f7f56980 rewrite html visualization 2024-02-23 16:33:39 +08:00
Chen, Zhentao
bfa98666a6 fall back to make_table.py 2024-02-23 16:33:38 +08:00
Chen, Zhentao
02cb96e7f6 fix Run Harness job 2024-02-23 16:33:37 +08:00
Chen, Zhentao
e1fcf54a0c reformat 2024-02-23 16:33:36 +08:00
Chen, Zhentao
5399343adc fix harness installation 2024-02-23 16:33:35 +08:00
Chen, Zhentao
9c8e349196 remove harness job output 2024-02-23 16:33:34 +08:00
Chen, Zhentao
8472de90e8 use stable lm to test pr 2024-02-23 16:33:34 +08:00
Chen, Zhentao
f315c7f93a Move harness nightly related files to llm/test folder (#10209)
* move harness nightly files to test folder

* change workflow file path accordingly

* use arc01 when pr

* fix path

* fix fp16 csv path
2024-02-23 11:12:36 +08:00
Ovo233
60e11b6739 LLM: Add mlp layer unit tests (#10200)
* add mlp layer unit tests

* add download baichuan-13b

* exclude llama for now

* install additional packages

* rename bash file

* switch to Baichuan2

* delete attention related code

* fix name errors in yml file
2024-02-22 13:44:45 +08:00
hxsz1997
5b387bb71a Change the nightly test time of ppl and harness (#10198)
* remove include and language option, select the corresponding dataset based on the model name in Run

* change the nightly test time

* change the nightly test time of harness and ppl
2024-02-21 17:39:33 +08:00
yb-peng
b1a97b71a9 Harness eval: Add is_last parameter and fix logical operator in highlight_vals (#10192)
* Add is_last parameter and fix logical operator in highlight_vals

* Add script to update HTML files in parent folder

* Add running update_html_in_parent_folder.py in summarize step

* Add licence info

* Remove update_html_in_parent_folder.py in Summarize the results for pull request
2024-02-21 14:45:32 +08:00