binbin Deng
7e96d3e79a
LLM: improve gpu supports key feature doc page ( #9212 )
2023-10-19 18:40:48 +08:00
Shaojun Liu
9dc76f19c0
fix hadolint error ( #9223 )
2023-10-19 16:22:32 +08:00
Chen, Zhentao
5850241423
correct Readme GPU example and API docstring ( #9225 )
...
* update readme to correct GPU usage
* update from_pretrained supported low bit options
* fix stype check
2023-10-19 16:08:47 +08:00
WeiguangHan
f87f67ee1c
LLM: arc perf test for some popular models ( #9188 )
2023-10-19 15:56:15 +08:00
Ziteng Zhang
0d62bd4adb
Added Docker installation guide and modified link in Dockerfile ( #9224 )
...
* changed '/ppml' into '/bigdl' and modified llama-7b
* Added the contents of finetuning in README
* Modified link of qlora_finetuning.py in Dockerfile
2023-10-19 15:28:05 +08:00
Lilac09
160c543a26
README for BigDL-LLM on docker ( #9197 )
...
* add instruction for MacOS/Linux
* modify html label of gif images
* organize structure of README
* change title name
* add inference-xpu, serving-cpu and serving-xpu parts
* revise README
* revise README
* revise README
2023-10-19 13:48:06 +08:00
Yang Wang
b0ddde0410
Fix removing convert dtype bug ( #9216 )
...
* Fix removing convert dtype bug
* fix style
2023-10-18 11:24:22 -07:00
Ruonan Wang
942d6418e7
LLM: fix chatglm kv cache ( #9215 )
2023-10-18 19:09:53 +08:00
SONG Ge
0765f94770
[LLM] Optimize kv_cache for mistral model family ( #9189 )
...
* add kv_cache optimization for mistral model
* kv_cache optimize for mistral
* update stylr
* update
2023-10-18 15:13:37 +08:00
Ruonan Wang
3555ebc148
LLM: fix wrong length in gptj kv_cache optimization ( #9210 )
...
* fix wrong length in gptj kv cache
* update
2023-10-18 14:59:02 +08:00
Shengsheng Huang
6dad8d16df
optimize NormHead for Baichuan2 ( #9205 )
...
* optimize NormHead for Baichuan2
* fix ut and change name
* rename functions
2023-10-18 14:05:07 +08:00
Jin Qiao
a3b664ed03
LLM: add GPU More-Data-Types and Save/Load example ( #9199 )
2023-10-18 13:13:45 +08:00
Ziteng Zhang
2f14f53b1c
changed '/ppml' into '/bigdl' and modified llama-7b ( #9209 )
2023-10-18 10:25:12 +08:00
WeiguangHan
b9194c5786
LLM: skip some model tests using certain api ( #9163 )
...
* LLM: Skip some model tests using certain api
* initialize variable named result
2023-10-18 09:39:27 +08:00
Ruonan Wang
09815f7064
LLM: fix RMSNorm optimization of Baichuan2-13B/Baichuan-13B ( #9204 )
...
* fix rmsnorm of baichuan2-13B
* update baichuan1-13B too
* fix style
2023-10-17 18:40:34 +08:00
binbin Deng
efcda3892f
LLM: add save & load usage in PyTorch API key feature page ( #9183 )
2023-10-17 17:24:51 +08:00
Jin Qiao
d7ce78edf0
LLM: fix portable zip README image link ( #9201 )
...
* LLM: fix portable zip readme img link
* LLM: make README first image center align
2023-10-17 16:38:22 +08:00
binbin Deng
330e67e2c0
LLM: update example doc page ( #9186 )
2023-10-17 16:26:11 +08:00
Cheen Hau, 俊豪
66c2e45634
Add unit tests for optimized model correctness ( #9151 )
...
* Add test to check correctness of optimized model
* Refactor optimized model test
* Use models in llm-unit-test
* Use AutoTokenizer for bloom
* Print out each passed test
* Remove unused tokenizer from import
2023-10-17 14:46:41 +08:00
Jin Qiao
d946bd7c55
LLM: add CPU More-Data-Types and Save-Load examples ( #9179 )
2023-10-17 14:38:52 +08:00
Ruonan Wang
c0497ab41b
LLM: support kv_cache optimization for Qwen-VL-Chat ( #9193 )
...
* dupport qwen_vl_chat
* fix style
2023-10-17 13:33:56 +08:00
binbin Deng
1cd9ab15b8
LLM: fix ChatGLMConfig check ( #9191 )
2023-10-17 11:52:56 +08:00
Yang Wang
7160afd4d1
Support XPU DDP training and autocast for LowBitMatmul ( #9167 )
...
* support autocast in low bit matmul
* Support XPU DDP training
* fix amp
2023-10-16 20:47:19 -07:00
Ruonan Wang
77afb8796b
LLM: fix convert of chatglm ( #9190 )
2023-10-17 10:48:13 +08:00
dingbaorong
af3b575c7e
expose modules_to_not_convert in optimize_model ( #9180 )
...
* expose modules_to_not_convert in optimize_model
* some fixes
2023-10-17 09:50:26 +08:00
Cengguang Zhang
5ca8a851e9
LLM: add fuse optimization for Mistral. ( #9184 )
...
* add fuse optimization for mistral.
* fix.
* fix
* fix style.
* fix.
* fix error.
* fix style.
* fix style.
2023-10-16 16:50:31 +08:00
Jiao Wang
49e1381c7f
update rope ( #9155 )
2023-10-15 21:51:45 -07:00
Jason Dai
b192a8032c
Update llm-readme ( #9176 )
2023-10-16 10:54:52 +08:00
binbin Deng
a164c24746
LLM: add kv_cache optimization for chatglm2-6b-32k ( #9165 )
2023-10-16 10:43:15 +08:00
Lilac09
326ef7f491
add README for llm-inference-cpu ( #9147 )
...
* add README for llm-inference-cpu
* modify README
* add README for llm-inference-cpu on Windows
2023-10-16 10:27:44 +08:00
Yang Wang
7a2de00b48
Fixes for xpu Bf16 training ( #9156 )
...
* Support bf16 training
* Use a stable transformer version
* remove env
* fix style
2023-10-14 21:28:59 -07:00
Cengguang Zhang
51a133de56
LLM: add fuse rope and norm optimization for Baichuan. ( #9166 )
...
* add fuse rope optimization.
* add rms norm optimization.
2023-10-13 17:36:52 +08:00
Jin Qiao
db7f938fdc
LLM: add replit and starcoder to gpu pytorch model example ( #9154 )
2023-10-13 15:44:17 +08:00
Jin Qiao
797b156a0d
LLM: add dolly-v1 and dolly-v2 to gpu pytorch model example ( #9153 )
2023-10-13 15:43:35 +08:00
Yishuo Wang
259cbb4126
[LLM] add initial bigdl-llm-init ( #9150 )
2023-10-13 15:31:45 +08:00
Cengguang Zhang
433f408081
LLM: Add fuse rope and norm optimization for Aquila. ( #9161 )
...
* add fuse norm optimization.
* add fuse rope optimization
2023-10-13 14:18:37 +08:00
SONG Ge
e7aa67e141
[LLM] Add rope optimization for internlm ( #9159 )
...
* add rope and norm optimization for internlm and gptneox
* revert gptneox back and split with pr#9155 #
* add norm_forward
* style fix
* update
* update
2023-10-13 14:18:28 +08:00
Jin Qiao
f754ab3e60
LLM: add baichuan and baichuan2 to gpu pytorch model example ( #9152 )
2023-10-13 13:44:31 +08:00
Ruonan Wang
b8aee7bb1b
LLM: Fix Qwen kv_cache optimization ( #9148 )
...
* first commit
* ut pass
* accelerate rotate half by using common util function
* fix style
2023-10-12 15:49:42 +08:00
binbin Deng
69942d3826
LLM: fix model check before attention optimization ( #9149 )
2023-10-12 15:21:51 +08:00
JIN Qiao
1a1ddc4144
LLM: Add Replit CPU and GPU example ( #9028 )
2023-10-12 13:42:14 +08:00
JIN Qiao
d74834ff4c
LLM: add gpu pytorch-models example llama2 and chatglm2 ( #9142 )
2023-10-12 13:41:48 +08:00
Ruonan Wang
4f34557224
LLM: support num_beams in all-in-one benchmark ( #9141 )
...
* support num_beams
* fix
2023-10-12 13:35:12 +08:00
Ruonan Wang
62ac7ae444
LLM: fix inaccurate input / output tokens of current all-in-one benchmark ( #9137 )
...
* first fix
* fix all apis
* fix
2023-10-11 17:13:34 +08:00
Lilac09
e02fbb40cc
add bigdl-llm-tutorial into llm-inference-cpu image ( #9139 )
...
* add bigdl-llm-tutorial into llm-inference-cpu image
* modify Dockerfile
* modify Dockerfile
2023-10-11 16:41:04 +08:00
ZehuaCao
65dd73b62e
Update manually_build.yml ( #9138 )
...
* Update manually_build.yml
fix llm-serving-tdx image build dir
* Update manually_build.yml
2023-10-11 15:07:09 +08:00
binbin Deng
eb3fb18eb4
LLM: improve PyTorch API doc ( #9128 )
2023-10-11 15:03:39 +08:00
Ziteng Zhang
4a0a3c376a
Add stand-alone mode on cpu for finetuning ( #9127 )
...
* Added steps for finetune on CPU in stand-alone mode
* Add stand-alone mode to bigdl-lora-finetuing-entrypoint.sh
* delete redundant docker commands
* Update README.md
Turn to intelanalytics/bigdl-llm-finetune-cpu:2.4.0-SNAPSHOT and append example outputs to allow users to check the running
* Update bigdl-lora-finetuing-entrypoint.sh
Add some tunable parameters
* Add parameters --cpus and -e WORKER_COUNT_DOCKER
* Modified the cpu number range parameters
* Set -ppn to CCL_WORKER_COUNT
* Add related configuration suggestions in README.md
2023-10-11 15:01:21 +08:00
binbin Deng
995b0f119f
LLM: update some gpu examples ( #9136 )
2023-10-11 14:23:56 +08:00
Ruonan Wang
1c8d5da362
LLM: fix llama tokenizer for all-in-one benchmark ( #9129 )
...
* fix tokenizer for gpu benchmark
* fix ipex fp16
* meet code review
* fix
2023-10-11 13:39:39 +08:00
