ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

update docker tag to 2.5.0-SNAPSHOT (#9443)

Browse source
This commit is contained in:
Shaojun Liu 2023-11-13 16:53:40 +08:00 committed by GitHub
parent 5747e2fe69
commit 0e5ab5ebfc
12 changed files with 41 additions and 41 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

34
docker/llm/README.md
View file

@ -27,11 +27,11 @@ The instructions for installing can be accessed from
To pull image from hub, you can execute command on console: To pull image from hub, you can execute command on console:
```bash ```bash
docker pull intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT docker pull intelanalytics/bigdl-llm-cpu:2.5.0-SNAPSHOT
``` ```
to check if the image is successfully downloaded, you can use: to check if the image is successfully downloaded, you can use:
```powershell ```powershell
docker images | sls intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT docker images | sls intelanalytics/bigdl-llm-cpu:2.5.0-SNAPSHOT
``` ```
@ -42,7 +42,7 @@ To run the image and do inference, you could create and run a bat script on Wind
An example on Windows could be: An example on Windows could be:
```bat ```bat
@echo off @echo off
set DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT set DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.5.0-SNAPSHOT
set CONTAINER_NAME=my_container set CONTAINER_NAME=my_container
set MODEL_PATH=D:/llm/models[change to your model path] set MODEL_PATH=D:/llm/models[change to your model path]
@ -112,7 +112,7 @@ Here is a demostration of how to use tutorial in explorer:
To run container on Linux/MacOS: To run container on Linux/MacOS:
```bash ```bash
#/bin/bash #/bin/bash
export DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT export DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.5.0-SNAPSHOT
export CONTAINER_NAME=my_container export CONTAINER_NAME=my_container
export MODEL_PATH=/llm/models[change to your model path] export MODEL_PATH=/llm/models[change to your model path]
@ -136,13 +136,13 @@ Also, you could use chat.py and bigdl-llm-tutorial for development.
First, pull docker image from docker hub: First, pull docker image from docker hub:
``` ```
docker pull intelanalytics/bigdl-llm-xpu:2.4.0-SNAPSHOT docker pull intelanalytics/bigdl-llm-xpu:2.5.0-SNAPSHOT
``` ```
To map the xpu into the container, you need to specify --device=/dev/dri when booting the container. To map the xpu into the container, you need to specify --device=/dev/dri when booting the container.
An example could be: An example could be:
```bash ```bash
#/bin/bash #/bin/bash
export DOCKER_IMAGE=intelanalytics/bigdl-llm-xpu:2.4.0-SNAPSHOT export DOCKER_IMAGE=intelanalytics/bigdl-llm-xpu:2.5.0-SNAPSHOT
export CONTAINER_NAME=my_container export CONTAINER_NAME=my_container
export MODEL_PATH=/llm/models[change to your model path] export MODEL_PATH=/llm/models[change to your model path]
@ -176,12 +176,12 @@ To run inference using `BigDL-LLM` using xpu, you could refer to this [documenta
Pull image: Pull image:
``` ```
docker pull intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT docker pull intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT
``` ```
You could use the following bash script to start the container. Please be noted that the CPU config is specified for Xeon CPUs, change it accordingly if you are not using a Xeon CPU. You could use the following bash script to start the container. Please be noted that the CPU config is specified for Xeon CPUs, change it accordingly if you are not using a Xeon CPU.
```bash ```bash
export DOCKER_IMAGE=intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT export DOCKER_IMAGE=intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT
export CONTAINER_NAME=my_container export CONTAINER_NAME=my_container
export MODEL_PATH=/llm/models[change to your model path] export MODEL_PATH=/llm/models[change to your model path]
@ -268,7 +268,7 @@ python3 -m fastchat.serve.openai_api_server --host localhost --port 8000
Pull image: Pull image:
``` ```
docker pull intelanalytics/bigdl-llm-serving-xpu:2.4.0-SNAPSHOT docker pull intelanalytics/bigdl-llm-serving-xpu:2.5.0-SNAPSHOT
``` ```
To map the `xpu` into the container, you need to specify `--device=/dev/dri` when booting the container. To map the `xpu` into the container, you need to specify `--device=/dev/dri` when booting the container.
@ -276,7 +276,7 @@ To map the `xpu` into the container, you need to specify `--device=/dev/dri` whe
An example could be: An example could be:
```bash ```bash
#/bin/bash #/bin/bash
export DOCKER_IMAGE=intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT export DOCKER_IMAGE=intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT
export CONTAINER_NAME=my_container export CONTAINER_NAME=my_container
export MODEL_PATH=/llm/models[change to your model path] export MODEL_PATH=/llm/models[change to your model path]
export SERVICE_MODEL_PATH=/llm/models/chatglm2-6b[a specified model path for running service] export SERVICE_MODEL_PATH=/llm/models/chatglm2-6b[a specified model path for running service]
@ -362,7 +362,7 @@ python3 -m fastchat.serve.openai_api_server --host localhost --port 8000
You can download directly from Dockerhub like: You can download directly from Dockerhub like:
```bash ```bash
docker pull intelanalytics/bigdl-llm-finetune-lora-cpu:2.4.0-SNAPSHOT docker pull intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT
``` ```
Or build the image from source: Or build the image from source:
@ -374,7 +374,7 @@ export HTTPS_PROXY=your_https_proxy
docker build \ docker build \
--build-arg http_proxy=${HTTP_PROXY} \ --build-arg http_proxy=${HTTP_PROXY} \
--build-arg https_proxy=${HTTPS_PROXY} \ --build-arg https_proxy=${HTTPS_PROXY} \
-t intelanalytics/bigdl-llm-finetune-lora-cpu:2.4.0-SNAPSHOT \ -t intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT \
-f ./Dockerfile . -f ./Dockerfile .
``` ```
@ -390,7 +390,7 @@ docker run -itd \
-e WORKER_COUNT_DOCKER=your_worker_count \ -e WORKER_COUNT_DOCKER=your_worker_count \
-v your_downloaded_base_model_path:/bigdl/model \ -v your_downloaded_base_model_path:/bigdl/model \
-v your_downloaded_data_path:/bigdl/data/alpaca_data_cleaned_archive.json \ -v your_downloaded_data_path:/bigdl/data/alpaca_data_cleaned_archive.json \
intelanalytics/bigdl-llm-finetune-cpu:2.4.0-SNAPSHOT \ intelanalytics/bigdl-llm-finetune-cpu:2.5.0-SNAPSHOT \
bash bash
``` ```
@ -449,7 +449,7 @@ The following shows how to fine-tune LLM with Quantization (QLoRA built on BigDL
You can download directly from Dockerhub like: You can download directly from Dockerhub like:
```bash ```bash
docker pull intelanalytics/bigdl-llm-finetune-qlora-xpu:2.4.0-SNAPSHOT docker pull intelanalytics/bigdl-llm-finetune-qlora-xpu:2.5.0-SNAPSHOT
``` ```
Or build the image from source: Or build the image from source:
@ -461,7 +461,7 @@ export HTTPS_PROXY=your_https_proxy
docker build \ docker build \
--build-arg http_proxy=${HTTP_PROXY} \ --build-arg http_proxy=${HTTP_PROXY} \
--build-arg https_proxy=${HTTPS_PROXY} \ --build-arg https_proxy=${HTTPS_PROXY} \
-t intelanalytics/bigdl-llm-finetune-qlora-xpu:2.4.0-SNAPSHOT \ -t intelanalytics/bigdl-llm-finetune-qlora-xpu:2.5.0-SNAPSHOT \
-f ./Dockerfile . -f ./Dockerfile .
``` ```
@ -485,7 +485,7 @@ docker run -itd \
-v $BASE_MODE_PATH:/model \ -v $BASE_MODE_PATH:/model \
-v $DATA_PATH:/data/english_quotes \ -v $DATA_PATH:/data/english_quotes \
--shm-size="16g" \ --shm-size="16g" \
intelanalytics/bigdl-llm-fintune-qlora-xpu:2.4.0-SNAPSHOT intelanalytics/bigdl-llm-fintune-qlora-xpu:2.5.0-SNAPSHOT
``` ```
The download and mount of base model and data to a docker container demonstrates a standard fine-tuning process. You can skip this step for a quick start, and in this way, the fine-tuning codes will automatically download the needed files: The download and mount of base model and data to a docker container demonstrates a standard fine-tuning process. You can skip this step for a quick start, and in this way, the fine-tuning codes will automatically download the needed files:
@ -502,7 +502,7 @@ docker run -itd \
-e http_proxy=${HTTP_PROXY} \ -e http_proxy=${HTTP_PROXY} \
-e https_proxy=${HTTPS_PROXY} \ -e https_proxy=${HTTPS_PROXY} \
--shm-size="16g" \ --shm-size="16g" \
intelanalytics/bigdl-llm-fintune-qlora-xpu:2.4.0-SNAPSHOT intelanalytics/bigdl-llm-fintune-qlora-xpu:2.5.0-SNAPSHOT
``` ```
However, we do recommend you to handle them manually, because the automatical download can be blocked by Internet access and Huggingface authentication etc. according to different environment, and the manual method allows you to fine-tune in a custom way (with different base model and dataset). However, we do recommend you to handle them manually, because the automatical download can be blocked by Internet access and Huggingface authentication etc. according to different environment, and the manual method allows you to fine-tune in a custom way (with different base model and dataset).

6
docker/llm/finetune/lora/cpu/docker/README.md
View file

@ -5,7 +5,7 @@
You can download directly from Dockerhub like: You can download directly from Dockerhub like:
```bash ```bash
docker pull intelanalytics/bigdl-llm-finetune-lora-cpu:2.4.0-SNAPSHOT docker pull intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT
``` ```
Or build the image from source: Or build the image from source:
@ -17,7 +17,7 @@ export HTTPS_PROXY=your_https_proxy
docker build \ docker build \
--build-arg http_proxy=${HTTP_PROXY} \ --build-arg http_proxy=${HTTP_PROXY} \
--build-arg https_proxy=${HTTPS_PROXY} \ --build-arg https_proxy=${HTTPS_PROXY} \
-t intelanalytics/bigdl-llm-finetune-lora-cpu:2.4.0-SNAPSHOT \ -t intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT \
-f ./Dockerfile . -f ./Dockerfile .
``` ```
@ -33,7 +33,7 @@ docker run -itd \
-e WORKER_COUNT_DOCKER=your_worker_count \ -e WORKER_COUNT_DOCKER=your_worker_count \
-v your_downloaded_base_model_path:/bigdl/model \ -v your_downloaded_base_model_path:/bigdl/model \
-v your_downloaded_data_path:/bigdl/data/alpaca_data_cleaned_archive.json \ -v your_downloaded_data_path:/bigdl/data/alpaca_data_cleaned_archive.json \
intelanalytics/bigdl-llm-finetune-lora-cpu:2.4.0-SNAPSHOT \ intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT \
bash bash
``` ```

2
docker/llm/finetune/lora/cpu/kubernetes/values.yaml
View file

@ -1,4 +1,4 @@
imageName: intelanalytics/bigdl-llm-finetune-lora-cpu:2.4.0-SNAPSHOT imageName: intelanalytics/bigdl-llm-finetune-lora-cpu:2.5.0-SNAPSHOT
trainerNum: 8 trainerNum: 8
microBatchSize: 8 microBatchSize: 8
nfsServerIp: your_nfs_server_ip nfsServerIp: your_nfs_server_ip

8
docker/llm/finetune/qlora/xpu/docker/README.md
View file

@ -7,7 +7,7 @@ The following shows how to fine-tune LLM with Quantization (QLoRA built on BigDL
You can download directly from Dockerhub like: You can download directly from Dockerhub like:
```bash ```bash
docker pull intelanalytics/bigdl-llm-finetune-qlora-xpu:2.4.0-SNAPSHOT docker pull intelanalytics/bigdl-llm-finetune-qlora-xpu:2.5.0-SNAPSHOT
``` ```
Or build the image from source: Or build the image from source:
@ -19,7 +19,7 @@ export HTTPS_PROXY=your_https_proxy
docker build \ docker build \
--build-arg http_proxy=${HTTP_PROXY} \ --build-arg http_proxy=${HTTP_PROXY} \
--build-arg https_proxy=${HTTPS_PROXY} \ --build-arg https_proxy=${HTTPS_PROXY} \
-t intelanalytics/bigdl-llm-finetune-qlora-xpu:2.4.0-SNAPSHOT \ -t intelanalytics/bigdl-llm-finetune-qlora-xpu:2.5.0-SNAPSHOT \
-f ./Dockerfile . -f ./Dockerfile .
``` ```
@ -43,7 +43,7 @@ docker run -itd \
-v $BASE_MODE_PATH:/model \ -v $BASE_MODE_PATH:/model \
-v $DATA_PATH:/data/english_quotes \ -v $DATA_PATH:/data/english_quotes \
--shm-size="16g" \ --shm-size="16g" \
intelanalytics/bigdl-llm-fintune-qlora-xpu:2.4.0-SNAPSHOT intelanalytics/bigdl-llm-fintune-qlora-xpu:2.5.0-SNAPSHOT
``` ```
The download and mount of base model and data to a docker container demonstrates a standard fine-tuning process. You can skip this step for a quick start, and in this way, the fine-tuning codes will automatically download the needed files: The download and mount of base model and data to a docker container demonstrates a standard fine-tuning process. You can skip this step for a quick start, and in this way, the fine-tuning codes will automatically download the needed files:
@ -60,7 +60,7 @@ docker run -itd \
-e http_proxy=${HTTP_PROXY} \ -e http_proxy=${HTTP_PROXY} \
-e https_proxy=${HTTPS_PROXY} \ -e https_proxy=${HTTPS_PROXY} \
--shm-size="16g" \ --shm-size="16g" \
intelanalytics/bigdl-llm-fintune-qlora-xpu:2.4.0-SNAPSHOT intelanalytics/bigdl-llm-fintune-qlora-xpu:2.5.0-SNAPSHOT
``` ```
However, we do recommend you to handle them manually, because the automatical download can be blocked by Internet access and Huggingface authentication etc. according to different environment, and the manual method allows you to fine-tune in a custom way (with different base model and dataset). However, we do recommend you to handle them manually, because the automatical download can be blocked by Internet access and Huggingface authentication etc. according to different environment, and the manual method allows you to fine-tune in a custom way (with different base model and dataset).

6
docker/llm/inference/cpu/docker/README.md
View file

@ -6,7 +6,7 @@ docker build \
--build-arg http_proxy=.. \ --build-arg http_proxy=.. \
--build-arg https_proxy=.. \ --build-arg https_proxy=.. \
--build-arg no_proxy=.. \ --build-arg no_proxy=.. \
--rm --no-cache -t intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT . --rm --no-cache -t intelanalytics/bigdl-llm-cpu:2.5.0-SNAPSHOT .
``` ```
@ -16,7 +16,7 @@ docker build \
An example could be: An example could be:
```bash ```bash
#/bin/bash #/bin/bash
export DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT export DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.5.0-SNAPSHOT
sudo docker run -itd \ sudo docker run -itd \
--net=host \ --net=host \
@ -41,7 +41,7 @@ You can download models and bind the model directory from host machine to contai
Here is an example: Here is an example:
```bash ```bash
export DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT export DOCKER_IMAGE=intelanalytics/bigdl-llm-cpu:2.5.0-SNAPSHOT
export MODEL_PATH=/home/llm/models export MODEL_PATH=/home/llm/models
sudo docker run -itd \ sudo docker run -itd \

4
docker/llm/inference/xpu/docker/README.md
View file

@ -6,7 +6,7 @@ docker build \
--build-arg http_proxy=.. \ --build-arg http_proxy=.. \
--build-arg https_proxy=.. \ --build-arg https_proxy=.. \
--build-arg no_proxy=.. \ --build-arg no_proxy=.. \
--rm --no-cache -t intelanalytics/bigdl-llm-xpu:2.4.0-SNAPSHOT . --rm --no-cache -t intelanalytics/bigdl-llm-xpu:2.5.0-SNAPSHOT .
``` ```
@ -17,7 +17,7 @@ To map the `xpu` into the container, you need to specify `--device=/dev/dri` whe
An example could be: An example could be:
```bash ```bash
#/bin/bash #/bin/bash
export DOCKER_IMAGE=intelanalytics/bigdl-llm-xpu:2.4.0-SNAPSHOT export DOCKER_IMAGE=intelanalytics/bigdl-llm-xpu:2.5.0-SNAPSHOT
sudo docker run -itd \ sudo docker run -itd \
--net=host \ --net=host \

2
docker/llm/serving/cpu/docker/Dockerfile
View file

@ -1,4 +1,4 @@
FROM intelanalytics/bigdl-llm-cpu:2.4.0-SNAPSHOT FROM intelanalytics/bigdl-llm-cpu:2.5.0-SNAPSHOT
ARG http_proxy ARG http_proxy
ARG https_proxy ARG https_proxy

4
docker/llm/serving/cpu/docker/README.md
View file

@ -6,7 +6,7 @@ docker build \
--build-arg http_proxy=.. \ --build-arg http_proxy=.. \
--build-arg https_proxy=.. \ --build-arg https_proxy=.. \
--build-arg no_proxy=.. \ --build-arg no_proxy=.. \
--rm --no-cache -t intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT . --rm --no-cache -t intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT .
``` ```
@ -17,7 +17,7 @@ You could use the following bash script to start the container. Please be noted
```bash ```bash
#/bin/bash #/bin/bash
export DOCKER_IMAGE=intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT export DOCKER_IMAGE=intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT
sudo docker run -itd \ sudo docker run -itd \
--net=host \ --net=host \

6
docker/llm/serving/cpu/kubernetes/README.md
View file

@ -3,7 +3,7 @@
## Image ## Image
To deploy BigDL-LLM-serving cpu in Kubernetes environment, please use this image: `intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT` To deploy BigDL-LLM-serving cpu in Kubernetes environment, please use this image: `intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT`
## Before deployment ## Before deployment
@ -48,7 +48,7 @@ spec:
dnsPolicy: "ClusterFirst" dnsPolicy: "ClusterFirst"
containers: containers:
- name: fastchat-controller # fixed - name: fastchat-controller # fixed
image: intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT image: intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
env: env:
- name: CONTROLLER_HOST # fixed - name: CONTROLLER_HOST # fixed
@ -119,7 +119,7 @@ spec:
dnsPolicy: "ClusterFirst" dnsPolicy: "ClusterFirst"
containers: containers:
- name: fastchat-worker # fixed - name: fastchat-worker # fixed
image: intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT image: intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
env: env:
- name: CONTROLLER_HOST # fixed - name: CONTROLLER_HOST # fixed

4
docker/llm/serving/cpu/kubernetes/deployment.yaml
View file

@ -9,7 +9,7 @@ spec:
dnsPolicy: "ClusterFirst" dnsPolicy: "ClusterFirst"
containers: containers:
- name: fastchat-controller # fixed - name: fastchat-controller # fixed
image: intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT image: intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
env: env:
- name: CONTROLLER_HOST # fixed - name: CONTROLLER_HOST # fixed
@ -74,7 +74,7 @@ spec:
dnsPolicy: "ClusterFirst" dnsPolicy: "ClusterFirst"
containers: containers:
- name: fastchat-worker # fixed - name: fastchat-worker # fixed
image: intelanalytics/bigdl-llm-serving-cpu:2.4.0-SNAPSHOT image: intelanalytics/bigdl-llm-serving-cpu:2.5.0-SNAPSHOT
imagePullPolicy: IfNotPresent imagePullPolicy: IfNotPresent
env: env:
- name: CONTROLLER_HOST # fixed - name: CONTROLLER_HOST # fixed

2
docker/llm/serving/xpu/docker/Dockerfile
View file

@ -1,4 +1,4 @@
FROM intelanalytics/bigdl-llm-xpu:2.4.0-SNAPSHOT FROM intelanalytics/bigdl-llm-xpu:2.5.0-SNAPSHOT
ARG http_proxy ARG http_proxy
ARG https_proxy ARG https_proxy

4
docker/llm/serving/xpu/docker/README.md
View file

@ -6,7 +6,7 @@ docker build \
--build-arg http_proxy=.. \ --build-arg http_proxy=.. \
--build-arg https_proxy=.. \ --build-arg https_proxy=.. \
--build-arg no_proxy=.. \ --build-arg no_proxy=.. \
--rm --no-cache -t intelanalytics/bigdl-llm-serving-xpu:2.4.0-SNAPSHOT . --rm --no-cache -t intelanalytics/bigdl-llm-serving-xpu:2.5.0-SNAPSHOT .
``` ```
@ -18,7 +18,7 @@ To map the `xpu` into the container, you need to specify `--device=/dev/dri` whe
An example could be: An example could be:
```bash ```bash
#/bin/bash #/bin/bash
export DOCKER_IMAGE=intelanalytics/bigdl-llm-serving-xpu:2.4.0-SNAPSHOT export DOCKER_IMAGE=intelanalytics/bigdl-llm-serving-xpu:2.5.0-SNAPSHOT
sudo docker run -itd \ sudo docker run -itd \
--net=host \ --net=host \