ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
2308 commits 1 branch 0 tags 37 MiB
Commit graph

70 commits

Author SHA1 Message Date
Lilac09
a2ed4d714e Fix vllm service error (#10279) 2024-02-29 15:45:04 +08:00
Ziteng Zhang
e08c74f1d1 Fix build error of bigdl-llm-cpu (#10228) 2024-02-23 16:30:21 +08:00
Ziteng Zhang
f7e2591f15 [LLM] change IPEX230 to IPEX220 in dockerfile (#10222) 
* change IPEX230 to IPEX220 in dockerfile
 2024-02-23 15:02:08 +08:00
Shaojun Liu
079f2011ea Update bigdl-llm-finetune-qlora-xpu Docker Image (#10194) 
* Bump oneapi version to 2024.0

* pip install bitsandbytes scipy

* Pin level-zero-gpu version

* Pin accelerate version 0.23.0
 2024-02-21 15:18:27 +08:00
Lilac09
eca69a6022 Fix build error of bigdl-llm-cpu (#10176) 
* fix build error

* fix build error

* fix build error

* fix build error
 2024-02-20 14:50:12 +08:00
Lilac09
f8dcaff7f4 use default python (#10070) 2024-02-05 09:06:59 +08:00
Lilac09
72e67eedbb Add speculative support in docker (#10058) 
* add speculative environment

* add speculative environment

* add speculative environment
 2024-02-01 09:53:53 +08:00
binbin Deng
171fb2d185 LLM: reorganize GPU finetuning examples (#9952) 2024-01-25 19:02:38 +08:00
ZehuaCao
51aa8b62b2 add gradio_web_ui to llm-serving image (#9918) 2024-01-25 11:11:39 +08:00
Lilac09
de27ddd81a Update Dockerfile (#9981) 2024-01-24 11:10:06 +08:00
Lilac09
a2718038f7 Fix qwen model adapter in docker (#9969) 
* fix qwen in docker

* add patch for model_adapter.py in fastchat

* add patch for model_adapter.py in fastchat
 2024-01-24 11:01:29 +08:00
Lilac09
052962dfa5 Using original fastchat and add bigdl worker in docker image (#9967) 
* add vllm worker

* add options in entrypoint
 2024-01-23 14:17:05 +08:00
Shaojun Liu
32c56ffc71 pip install deps (#9916) 2024-01-17 11:03:57 +08:00
ZehuaCao
05ea0ecd70 add pv for llm-serving k8s deployment (#9906) 2024-01-16 11:32:54 +08:00
Guancheng Fu
0396fafed1 Update BigDL-LLM-inference image (#9805) 
* upgrade to oneapi 2024

* Pin level-zero-gpu version

* add flag
 2024-01-03 14:00:09 +08:00
Lilac09
a5c481fedd add chat.py denpendency in Dockerfile (#9699) 2023-12-18 09:00:22 +08:00
Lilac09
3afed99216 fix path issue (#9696) 2023-12-15 11:21:49 +08:00
ZehuaCao
d204125e88 [LLM] Use to build a more slim docker for k8s (#9608) 
* Create Dockerfile.k8s

* Update Dockerfile

More slim standalone image

* Update Dockerfile

* Update Dockerfile.k8s

* Update bigdl-qlora-finetuing-entrypoint.sh

* Update qlora_finetuning_cpu.py

* Update alpaca_qlora_finetuning_cpu.py

Refer to this [pr](https://github.com/intel-analytics/BigDL/pull/9551/files#diff-2025188afa54672d21236e6955c7c7f7686bec9239532e41c7983858cc9aaa89), update the LoraConfig

* update

* update

* update

* update

* update

* update

* update

* update transformer version

* update Dockerfile

* update Docker image name

* fix error
 2023-12-08 10:25:36 +08:00
Heyang Sun
4e70e33934 [LLM] code and document for distributed qlora (#9585) 
* [LLM] code and document for distributed qlora

* doc

* refine for gradient checkpoint

* refine

* Update alpaca_qlora_finetuning_cpu.py

* Update alpaca_qlora_finetuning_cpu.py

* Update alpaca_qlora_finetuning_cpu.py

* add link in doc
 2023-12-06 09:23:17 +08:00
Guancheng Fu
8b00653039 fix doc (#9599) 2023-12-05 13:49:31 +08:00
Heyang Sun
74fd7077a2 [LLM] Multi-process and distributed QLoRA on CPU platform (#9491) 
* [LLM] Multi-process and distributed QLoRA on CPU platform

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* enable llm-init and bind to socket

* refine

* Update Dockerfile

* add all files of qlora cpu example to /bigdl

* fix

* fix k8s

* Update bigdl-qlora-finetuing-entrypoint.sh

* Update bigdl-qlora-finetuing-entrypoint.sh

* Update bigdl-qlora-finetuning-job.yaml

* fix train sync and performance issues

* add node affinity

* disable user to tune cpu per pod

* Update bigdl-qlora-finetuning-job.yaml
 2023-12-01 13:47:19 +08:00
Lilac09
b785376f5c Add vllm-example to docker inference image (#9570) 
* add vllm-serving to cpu image

* add vllm-serving to cpu image

* add vllm-serving
 2023-11-30 17:04:53 +08:00
Lilac09
2554ba0913 Add usage of vllm (#9564) 
* add usage of vllm

* add usage of vllm

* add usage of vllm

* add usage of vllm

* add usage of vllm

* add usage of vllm
 2023-11-30 14:19:23 +08:00
Lilac09
557bb6bbdb add judgement for running serve (#9555) 2023-11-29 16:57:00 +08:00
Guancheng Fu
2b200bf2f2 Add vllm_worker related arguments in docker serving image's entrypoint (#9500) 
* fix entrypoint

* fix missing long mode argument
 2023-11-21 14:41:06 +08:00
Lilac09
566ec85113 add stream interval option to entrypoint (#9498) 2023-11-21 09:47:32 +08:00
Lilac09
13f6eb77b4 Add exec bash to entrypoint.sh to keep container running after being booted. (#9471) 
* add bigdl-llm-init

* boot bash
 2023-11-15 16:09:16 +08:00
Lilac09
24146d108f add bigdl-llm-init (#9468) 2023-11-15 14:55:33 +08:00
Lilac09
b2b085550b Remove bigdl-nano and add ipex into inference-cpu image (#9452) 
* remove bigdl-nano and add ipex into inference-cpu image

* remove bigdl-nano in docker

* remove bigdl-nano in docker
 2023-11-14 10:50:52 +08:00
Wang, Jian4
0f78ebe35e LLM : Add qlora cpu finetune docker image (#9271) 
* init qlora cpu docker image

* update

* remove ipex and update

* update

* update readme

* update example and readme
 2023-11-14 10:36:53 +08:00
Shaojun Liu
0e5ab5ebfc update docker tag to 2.5.0-SNAPSHOT (#9443) 2023-11-13 16:53:40 +08:00
Lilac09
5d4ec44488 Add all-in-one benchmark into inference-cpu docker image (#9433) 
* add all-in-one into inference-cpu image

* manually_build

* revise files
 2023-11-13 13:07:56 +08:00
Lilac09
74a8ad32dc Add entry point to llm-serving-xpu (#9339) 
* add entry point to llm-serving-xpu

* manually build

* manually build

* add entry point to llm-serving-xpu

* manually build

* add entry point to llm-serving-xpu

* add entry point to llm-serving-xpu

* add entry point to llm-serving-xpu
 2023-11-02 16:31:07 +08:00
Ziteng Zhang
4df66f5cbc Update llm-finetune-lora-cpu dockerfile and readme 
* Update README.md

* Update Dockerfile
 2023-11-02 16:26:24 +08:00
Lilac09
2c2bc959ad add tools into previously built images (#9317) 
* modify Dockerfile

* manually build

* modify Dockerfile

* add chat.py into inference-xpu

* add benchmark into inference-cpu

* manually build

* add benchmark into inference-cpu

* add benchmark into inference-cpu

* add benchmark into inference-cpu

* add chat.py into inference-xpu

* add chat.py into inference-xpu

* change ADD to COPY in dockerfile

* fix dependency issue

* temporarily remove run-spr in llm-cpu

* temporarily remove run-spr in llm-cpu
 2023-10-31 16:35:18 +08:00
Lilac09
030edeecac Ubuntu upgrade: fix installation error (#9309) 
* upgrade ubuntu version in llm-inference cpu image

* fix installation issue

* fix installation issue

* fix installation issue
 2023-10-31 09:55:15 +08:00
Lilac09
5842f7530e upgrade ubuntu version in llm-inference cpu image (#9307) 2023-10-30 16:51:38 +08:00
Ziteng Zhang
ca2965fb9f hosted k8s.png on readthedocs (#9258) 2023-10-24 15:07:16 +08:00
Guancheng Fu
7f66bc5c14 Fix bigdl-llm-serving-cpu Dockerfile (#9247) 2023-10-23 16:51:30 +08:00
Shaojun Liu
9dc76f19c0 fix hadolint error (#9223) 2023-10-19 16:22:32 +08:00
Ziteng Zhang
0d62bd4adb Added Docker installation guide and modified link in Dockerfile (#9224) 
* changed '/ppml' into '/bigdl' and modified llama-7b

* Added the contents of finetuning in README

* Modified link of qlora_finetuning.py in Dockerfile
 2023-10-19 15:28:05 +08:00
Lilac09
160c543a26 README for BigDL-LLM on docker (#9197) 
* add instruction for MacOS/Linux

* modify html label of gif images

* organize structure of README

* change title name

* add inference-xpu, serving-cpu and serving-xpu parts

* revise README

* revise README

* revise README
 2023-10-19 13:48:06 +08:00
Ziteng Zhang
2f14f53b1c changed '/ppml' into '/bigdl' and modified llama-7b (#9209) 2023-10-18 10:25:12 +08:00
Lilac09
326ef7f491 add README for llm-inference-cpu (#9147) 
* add README for llm-inference-cpu

* modify README

* add README for llm-inference-cpu on Windows
 2023-10-16 10:27:44 +08:00
Lilac09
e02fbb40cc add bigdl-llm-tutorial into llm-inference-cpu image (#9139) 
* add bigdl-llm-tutorial into llm-inference-cpu image

* modify Dockerfile

* modify Dockerfile
 2023-10-11 16:41:04 +08:00
Ziteng Zhang
4a0a3c376a Add stand-alone mode on cpu for finetuning (#9127) 
* Added steps for finetune on CPU in stand-alone mode

* Add stand-alone mode to bigdl-lora-finetuing-entrypoint.sh

* delete redundant docker commands

* Update README.md

Turn to intelanalytics/bigdl-llm-finetune-cpu:2.4.0-SNAPSHOT and append example outputs to allow users to check the running

* Update bigdl-lora-finetuing-entrypoint.sh

Add some tunable parameters

* Add parameters --cpus and -e WORKER_COUNT_DOCKER

* Modified the cpu number range parameters

* Set -ppn to CCL_WORKER_COUNT

* Add related configuration suggestions in README.md
 2023-10-11 15:01:21 +08:00
Lilac09
30e3c196f3 Merge pull request #9108 from Zhengjin-Wang/main 
Add instruction for chat.py in bigdl-llm-cpu
 2023-10-10 16:40:52 +08:00
Lilac09
1e78b0ac40 Optimize LoRA Docker by Shrinking Image Size (#9110) 
* modify dockerfile

* modify dockerfile
 2023-10-10 15:53:17 +08:00
Heyang Sun
2c0c9fecd0 refine LLM containers (#9109) 2023-10-09 15:45:30 +08:00
Wang
a1aefdb8f4 modify README 2023-10-09 13:36:29 +08:00