SONG Ge
d2926c7672
[LLM] Unify Langchain Native and Transformers LLM API ( #8752 )
...
* deprecate BigDLNativeTransformers and add specific LMEmbedding method
* deprecate and add LM methods for langchain llms
* add native params to native langchain
* new imple for embedding
* move ut from bigdlnative to casual llm
* rename embeddings api and examples update align with usage updating
* docqa example hot-fix
* add more api docs
* add langchain ut for starcoder
* support model_kwargs for transformer methods when calling causalLM and add ut
* ut fix for transformers embedding
* update for langchain causal supporting transformers
* remove model_family in readme doc
* add model_families params to support more models
* update api docs and remove chatglm embeddings for now
* remove chatglm embeddings in examples
* new refactor for ut to add bloom and transformers llama ut
* disable llama transformers embedding ut
2023-08-25 11:14:21 +08:00
binbin Deng
5582872744
LLM: update chatglm example to be more friendly for beginners ( #8795 )
2023-08-25 10:55:01 +08:00
Yina Chen
7c37424a63
Fix voice assistant example input error on Linux ( #8799 )
...
* fix linux error
* update
* remove alsa log
2023-08-25 10:47:27 +08:00
xingyuan li
9537194b4b
[LLM] Fix llm test workflow repeatedly download model files
2023-08-25 11:20:46 +09:00
Yang Wang
bf3591e2ff
Optimize chatglm2 for bf16 ( #8725 )
...
* make chatglm works with bf16
* fix style
* support chatglm v1
* fix style
* fix style
* add chatglm2 file
2023-08-24 10:04:25 -07:00
Jin Hanyu
a73a3e5ff9
Fix bugs in manually_build_for_testing.yml. ( #8792 )
2023-08-23 15:49:23 +08:00
xingyuan li
c94bdd3791
[LLM] Merge windows & linux nightly test ( #8756 )
...
* fix download statement
* add check before build wheel
* use curl to upload files
* windows unittest won't upload converted model
* split llm-cli test into windows & linux versions
* update tempdir create way
* fix nightly converted model name
* windows llm-cli starcoder test temply disabled
* remove taskset dependency
* rename llm_unit_tests_linux to llm_unit_tests
2023-08-23 12:48:41 +09:00
Jason Dai
dcadd09154
Update llm document ( #8784 )
2023-08-21 22:34:44 +08:00
Yishuo Wang
611c1fb628
[LLM] change default n_threads of native int4 langchain API ( #8779 )
2023-08-21 13:30:12 +08:00
Yishuo Wang
3d1f2b44f8
LLM: change default n_threads of native int4 models ( #8776 )
2023-08-18 15:46:19 +08:00
Yishuo Wang
2ba2133613
fix starcoder chinese output ( #8773 )
2023-08-18 13:37:02 +08:00
binbin Deng
548f7a6cf7
LLM: update convert of llama family to support llama2-70B ( #8747 )
2023-08-18 09:30:35 +08:00
Yina Chen
4afea496ab
support q8_0 ( #8765 )
2023-08-17 15:06:36 +08:00
Shaojun Liu
394304b918
Re organize llm test ( #8766 )
...
* run llm-example-test in llm-nightly-test.yml
* comment out the schedule event
2023-08-17 09:42:25 +08:00
Ruonan Wang
e9aa2bd890
LLM: reduce GPU 1st token latency and update example ( #8763 )
...
* reduce 1st token latency
* update example
* fix
* fix style
* update readme of gpu benchmark
2023-08-16 18:01:23 +08:00
binbin Deng
06609d9260
LLM: add qwen example on arc ( #8757 )
2023-08-16 17:11:08 +08:00
SONG Ge
f4164e4492
[BigDL LLM] Update readme for unifying transformers API ( #8737 )
...
* update readme doc
* fix readthedocs error
* update comment
* update exception error info
* invalidInputError instead
* fix readme typo error and remove import error
* fix more typo
2023-08-16 14:22:32 +08:00
Song Jiaming
c1f9af6d97
[LLM] chatglm example and transformers low-bit examples ( #8751 )
2023-08-16 11:41:44 +08:00
Ruonan Wang
8805186f2f
LLM: add benchmark tool for gpu ( #8760 )
...
* add benchmark tool for gpu
* update
2023-08-16 11:22:10 +08:00
binbin Deng
97283c033c
LLM: add falcon example on arc ( #8742 )
2023-08-15 17:38:38 +08:00
binbin Deng
8c55911308
LLM: add baichuan-13B on arc example ( #8755 )
2023-08-15 15:07:04 +08:00
Shaojie Cui
0a8db3abe0
[PPML]refactor python toolkit ( #8740 )
...
* add dependency and example
* fix stage 3
* downgrade protobuf
* reduce epc memory
* add script
* Readme reduction
* delete unused note
2023-08-15 10:11:53 +08:00
binbin Deng
be2ae6eb7c
LLM: fix langchain native int4 voiceasistant example ( #8750 )
2023-08-14 17:23:33 +08:00
Ruonan Wang
d28ad8f7db
LLM: add whisper example for arc transformer int4 ( #8749 )
...
* add whisper example for arc int4
* fix
2023-08-14 17:05:48 +08:00
Yishuo Wang
77844125f2
[LLM] Support chatglm cache ( #8745 )
2023-08-14 15:10:46 +08:00
Ruonan Wang
faaccb64a2
LLM: add chatglm2 example for Arc ( #8741 )
...
* add chatglm2 example
* update
* fix readme
2023-08-14 10:43:08 +08:00
binbin Deng
b10d7e1adf
LLM: add mpt example on arc ( #8723 )
2023-08-14 09:40:01 +08:00
binbin Deng
e9a1afffc5
LLM: add internlm example on arc ( #8722 )
2023-08-14 09:39:39 +08:00
SONG Ge
aceea4dc29
[LLM] Unify Transformers and Native API ( #8713 )
...
* re-open pr to run on latest runner
* re-add examples and ut
* rename ut and move deprecate to warning instead of raising an error info
* ut fix
2023-08-11 19:45:47 +08:00
xingyuan li
1cb8f5abbd
[LLM] Revert compile OS for llm build workflow ( #8732 )
...
* use almalinux to build
2023-08-11 17:47:45 +09:00
Yishuo Wang
f91035c298
[LLM] fix chatglm native int4 emoji output ( #8739 )
2023-08-11 15:38:41 +08:00
binbin Deng
77efcf7b1d
LLM: fix ChatGLM2 native int4 stream output ( #8733 )
2023-08-11 14:51:50 +08:00
Ruonan Wang
ca3e59a1dc
LLM: support stop for starcoder native int4 stream ( #8734 )
2023-08-11 14:51:30 +08:00
Song Jiaming
e292dfd970
[WIP] LLM transformers api for langchain ( #8642 )
2023-08-11 13:32:35 +08:00
Yishuo Wang
3d5a7484a2
[LLM] fix bloom and starcoder memory release ( #8728 )
2023-08-11 11:18:19 +08:00
xingyuan li
33d9ad234f
[LLM] Linux vnni build with ubuntu 18.04 ( #8710 )
...
* move from almalinux
2023-08-10 19:04:03 +09:00
xingyuan li
02ec01cb48
[LLM] Add bigdl-core-xe dependency when installing bigdl-llm[xpu] ( #8716 )
...
* add bigdl-core-xe dependency
2023-08-10 17:41:42 +09:00
Shengsheng Huang
7c56c39e36
Fix GPU examples READ to use bigdl-core-xe ( #8714 )
...
* Update README.md
* Update README.md
2023-08-10 12:53:49 +08:00
Yina Chen
6d1ca88aac
add voice assistant example ( #8711 )
2023-08-10 12:42:14 +08:00
Kai Huang
4573ff7ffe
Update llm api doc ( #8712 )
...
* change to bigdlllm
* update
2023-08-10 12:17:19 +08:00
Song Jiaming
e717e304a6
LLM first example test and template ( #8658 )
2023-08-10 10:03:11 +08:00
Ruonan Wang
1a7b698a83
[LLM] support ipex arc int4 & add basic llama2 example ( #8700 )
...
* first support of xpu
* make it works on gpu
update setup
update
add GPU llama2 examples
add use_optimize flag to disbale optimize for gpu
fix style
update gpu exmaple readme
fix
* update example, and update env
* fix setup to add cpp files
* replace jit with aot to avoid data leak
* rename to bigdl-core-xe
* update installation in example readme
2023-08-09 22:20:32 +08:00
Jason Dai
d03218674a
Update llm readme ( #8703 )
2023-08-09 14:47:26 +08:00
Kai Huang
1b65288bdb
Add api doc for LLM ( #8605 )
...
* api doc initial
* update desc
2023-08-08 18:17:16 +08:00
binbin Deng
4c44153584
LLM: add Qwen transformers int4 example ( #8699 )
2023-08-08 11:23:09 +08:00
Yishuo Wang
710b9b8982
[LLM] add linux chatglm pybinding binary file ( #8698 )
2023-08-08 11:16:30 +08:00
xingyuan li
4482ccb329
[LLM] Change build system from centos7 to ubuntu18.04 ( #8686 )
...
* centos7 to ubuntu18
* ubuntu git version 2.17 need to update
* use almalinux8 to build avx2 binaries
2023-08-07 19:09:58 +09:00
binbin Deng
ea5d7aff5b
LLM: add chatglm native int4 transformers API ( #8695 )
2023-08-07 17:52:47 +08:00
Yishuo Wang
6da830cf7e
[LLM] add chaglm pybinding binary file in setup.py ( #8692 )
2023-08-07 09:41:03 +08:00
Yuwen Hu
06ed70fa43
Update google site verification for readthedocs ( #8689 )
2023-08-04 16:05:28 +08:00
