ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

update docker image tag to 2.2.0-SNAPSHOT (#11904)

Browse source
This commit is contained in:
Shaojun Liu 2024-08-23 13:57:41 +08:00 committed by GitHub
parent 650e6e6ce4
commit 4cf640c548
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
15 changed files with 54 additions and 54 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
.github/workflows/manually_build.yml vendored
View file

@ -24,9 +24,9 @@ on:
# - ipex-llm-finetune-qlora-cpu-k8s
# - ipex-llm-finetune-xpu
# tag:
# description: 'docker image tag (e.g. 2.1.0-SNAPSHOT)'
# description: 'docker image tag (e.g. 2.2.0-SNAPSHOT)'
# required: true
# default: '2.1.0-SNAPSHOT'
# default: '2.2.0-SNAPSHOT'
# type: string
workflow_call:
inputs:
@ -40,9 +40,9 @@ on:
default: 'all'
type: string
tag:
description: 'docker image tag (e.g. 2.1.0-SNAPSHOT)'
description: 'docker image tag (e.g. 2.2.0-SNAPSHOT)'
required: true
default: '2.1.0-SNAPSHOT'
default: '2.2.0-SNAPSHOT'
type: string
public:
description: "if the docker image push to public docker hub"

16
docker/llm/README.md
View file

@ -13,20 +13,20 @@ You can run IPEX-LLM containers (via docker or k8s) for inference, serving and f
#### Pull a IPEX-LLM Docker Image
To pull IPEX-LLM Docker images from [Docker Hub](https://hub.docker.com/u/intelanalytics), use the `docker pull` command. For instance, to pull the CPU inference image:
```bash
docker pull intelanalytics/ipex-llm-cpu:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT
```
Available images in hub are:
| Image Name | Description |
| --- | --- |
| intelanalytics/ipex-llm-cpu:2.1.0-SNAPSHOT | CPU Inference |
| intelanalytics/ipex-llm-xpu:2.1.0-SNAPSHOT | GPU Inference |
| intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT | CPU Serving|
| intelanalytics/ipex-llm-serving-xpu:2.1.0-SNAPSHOT | GPU Serving|
| intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.1.0-SNAPSHOT | CPU Finetuning via Docker|
| intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.1.0-SNAPSHOT|CPU Finetuning via Kubernetes|
| intelanalytics/ipex-llm-finetune-qlora-xpu:2.1.0-SNAPSHOT| GPU Finetuning|
| intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT | CPU Inference |
| intelanalytics/ipex-llm-xpu:2.2.0-SNAPSHOT | GPU Inference |
| intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT | CPU Serving|
| intelanalytics/ipex-llm-serving-xpu:2.2.0-SNAPSHOT | GPU Serving|
| intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.2.0-SNAPSHOT | CPU Finetuning via Docker|
| intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.2.0-SNAPSHOT|CPU Finetuning via Kubernetes|
| intelanalytics/ipex-llm-finetune-qlora-xpu:2.2.0-SNAPSHOT| GPU Finetuning|
#### Run a Container
Use `docker run` command to run an IPEX-LLM docker container. For detailed instructions, refer to the [IPEX-LLM Docker Container Guides](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/DockerGuides/index.html).

34
docker/llm/README_backup.md
View file

@ -30,14 +30,14 @@ This guide provides step-by-step instructions for installing and using IPEX-LLM
Run the following command to pull image:
```bash
docker pull intelanalytics/ipex-llm-cpu:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT
```
### 2. Start bigdl-llm-cpu Docker Container
```bash
#/bin/bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-cpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT
export CONTAINER_NAME=my_container
export MODEL_PATH=/llm/models[change to your model path]
@ -156,7 +156,7 @@ Additionally, for examples related to Inference with Speculative Decoding, you c
Run the following command to pull image from dockerhub:
```bash
docker pull intelanalytics/ipex-llm-xpu:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-xpu:2.2.0-SNAPSHOT
```
### 2. Start Chat Inference
@ -167,7 +167,7 @@ To map the xpu into the container, you need to specify --device=/dev/dri when bo
```bash
#/bin/bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-xpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-xpu:2.2.0-SNAPSHOT
export CONTAINER_NAME=my_container
export MODEL_PATH=/llm/models[change to your model path]
@ -189,7 +189,7 @@ Execute a quick performance benchmark by starting the ipex-llm-xpu container, sp
To map the XPU into the container, specify `--device=/dev/dri` when booting the container.
```bash
#/bin/bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-xpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-xpu:2.2.0-SNAPSHOT
export CONTAINER_NAME=my_container
export MODEL_PATH=/llm/models [change to your model path]
@ -226,7 +226,7 @@ IPEX-LLM is integrated into FastChat so that user can use IPEX-LLM as a serving
Run the following command:
```bash
docker pull intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT
```
### 2. Start ipex-llm-serving-cpu Docker Container
@ -234,7 +234,7 @@ docker pull intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT
Please be noted that the CPU config is specified for Xeon CPUs, change it accordingly if you are not using a Xeon CPU.
```bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT
export CONTAINER_NAME=my_container
export MODEL_PATH=/llm/models[change to your model path]
@ -349,7 +349,7 @@ IPEX-LLM is integrated into FastChat so that user can use IPEX-LLM as a serving
Run the following command:
```bash
docker pull intelanalytics/ipex-llm-serving-xpu:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-serving-xpu:2.2.0-SNAPSHOT
```
### 2. Start ipex-llm-serving-xpu Docker Container
@ -357,7 +357,7 @@ docker pull intelanalytics/ipex-llm-serving-xpu:2.1.0-SNAPSHOT
To map the `xpu` into the container, you need to specify `--device=/dev/dri` when booting the container.
```bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:2.2.0-SNAPSHOT
export CONTAINER_NAME=my_container
export MODEL_PATH=/llm/models[change to your model path]
@ -473,10 +473,10 @@ You can download directly from Dockerhub like:
```bash
# For standalone
docker pull intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.2.0-SNAPSHOT
# For k8s
docker pull intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.2.0-SNAPSHOT
```
Or build the image from source:
@ -489,7 +489,7 @@ export HTTPS_PROXY=your_https_proxy
docker build \
--build-arg http_proxy=${HTTP_PROXY} \
--build-arg https_proxy=${HTTPS_PROXY} \
-t intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.1.0-SNAPSHOT \
-t intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.2.0-SNAPSHOT \
-f ./Dockerfile .
# For k8s
@ -499,7 +499,7 @@ export HTTPS_PROXY=your_https_proxy
docker build \
--build-arg http_proxy=${HTTP_PROXY} \
--build-arg https_proxy=${HTTPS_PROXY} \
-t intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.1.0-SNAPSHOT \
-t intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.2.0-SNAPSHOT \
-f ./Dockerfile.k8s .
```
@ -520,7 +520,7 @@ docker run -itd \
-e https_proxy=${HTTPS_PROXY} \
-v $BASE_MODE_PATH:/ipex_llm/model \
-v $DATA_PATH:/ipex_llm/data/alpaca-cleaned \
intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.1.0-SNAPSHOT
intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.2.0-SNAPSHOT
```
The download and mount of base model and data to a docker container demonstrates a standard fine-tuning process. You can skip this step for a quick start, and in this way, the fine-tuning codes will automatically download the needed files:
@ -534,7 +534,7 @@ docker run -itd \
--name=ipex-llm-fintune-qlora-cpu \
-e http_proxy=${HTTP_PROXY} \
-e https_proxy=${HTTPS_PROXY} \
intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.1.0-SNAPSHOT
intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.2.0-SNAPSHOT
```
However, we do recommend you to handle them manually, because the automatical download can be blocked by Internet access and Huggingface authentication etc. according to different environment, and the manual method allows you to fine-tune in a custom way (with different base model and dataset).
@ -593,7 +593,7 @@ The following shows how to fine-tune LLM with Quantization (QLoRA built on IPEX-
Run the following command:
```bash
docker pull intelanalytics/ipex-llm-finetune-xpu:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT
```
### 2. Prepare Base Model, Data and Start Docker Container
@ -606,7 +606,7 @@ export DATA_PATH=your_downloaded_data_path
export HTTP_PROXY=your_http_proxy
export HTTPS_PROXY=your_https_proxy
export CONTAINER_NAME=my_container
export DOCKER_IMAGE=intelanalytics/ipex-llm-finetune-xpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT
docker run -itd \
--net=host \

6
docker/llm/finetune/lora/cpu/docker/README.md
View file

@ -5,7 +5,7 @@
You can download directly from Dockerhub like:
```bash
docker pull intelanalytics/ipex-llm-finetune-lora-cpu:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-finetune-lora-cpu:2.2.0-SNAPSHOT
```
Or build the image from source:
@ -17,7 +17,7 @@ export HTTPS_PROXY=your_https_proxy
docker build \
--build-arg http_proxy=${HTTP_PROXY} \
--build-arg https_proxy=${HTTPS_PROXY} \
-t intelanalytics/ipex-llm-finetune-lora-cpu:2.1.0-SNAPSHOT \
-t intelanalytics/ipex-llm-finetune-lora-cpu:2.2.0-SNAPSHOT \
-f ./Dockerfile .
```
@ -33,7 +33,7 @@ docker run -itd \
-e WORKER_COUNT_DOCKER=your_worker_count \
-v your_downloaded_base_model_path:/ipex_llm/model \
-v your_downloaded_data_path:/ipex_llm/data/alpaca_data_cleaned_archive.json \
intelanalytics/ipex-llm-finetune-lora-cpu:2.1.0-SNAPSHOT \
intelanalytics/ipex-llm-finetune-lora-cpu:2.2.0-SNAPSHOT \
bash
```

2
docker/llm/finetune/lora/cpu/kubernetes/values.yaml
View file

@ -1,4 +1,4 @@
imageName: intelanalytics/ipex-llm-finetune-lora-cpu:2.1.0-SNAPSHOT
imageName: intelanalytics/ipex-llm-finetune-lora-cpu:2.2.0-SNAPSHOT
trainerNum: 8
microBatchSize: 8
nfsServerIp: your_nfs_server_ip

2
docker/llm/finetune/qlora/cpu/kubernetes/values.yaml
View file

@ -1,4 +1,4 @@
imageName: intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.1.0-SNAPSHOT
imageName: intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.2.0-SNAPSHOT
trainerNum: 2
microBatchSize: 8
enableGradientCheckpoint: false # true will save more memory but increase latency

8
docker/llm/finetune/xpu/README.md
View file

@ -19,7 +19,7 @@ With this docker image, we can use all [ipex-llm finetune examples on Intel GPU]
You can download directly from Dockerhub like:
```bash
docker pull intelanalytics/ipex-llm-finetune-xpu:2.1.0-SNAPSHOT
docker pull intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT
```
Or build the image from source:
@ -31,7 +31,7 @@ export HTTPS_PROXY=your_https_proxy
docker build \
--build-arg http_proxy=${HTTP_PROXY} \
--build-arg https_proxy=${HTTPS_PROXY} \
-t intelanalytics/ipex-llm-finetune-xpu:2.1.0-SNAPSHOT \
-t intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT \
-f ./Dockerfile .
```
@ -55,7 +55,7 @@ docker run -itd \
-v $BASE_MODE_PATH:/model \
-v $DATA_PATH:/data/alpaca-cleaned \
--shm-size="16g" \
intelanalytics/ipex-llm-finetune-xpu:2.1.0-SNAPSHOT
intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT
```
The download and mount of base model and data to a docker container demonstrates a standard fine-tuning process. You can skip this step for a quick start, and in this way, the fine-tuning codes will automatically download the needed files:
@ -72,7 +72,7 @@ docker run -itd \
-e http_proxy=${HTTP_PROXY} \
-e https_proxy=${HTTPS_PROXY} \
--shm-size="16g" \
intelanalytics/ipex-llm-finetune-xpu:2.1.0-SNAPSHOT
intelanalytics/ipex-llm-finetune-xpu:2.2.0-SNAPSHOT
```
However, we do recommend you to handle them manually, because the download can be blocked by Internet access and Huggingface authentication etc. according to different environment, and the manual method allows you to fine-tune in a custom way (with different base model and dataset).

6
docker/llm/inference/cpu/docker/README.md
View file

@ -6,7 +6,7 @@ docker build \
--build-arg http_proxy=.. \
--build-arg https_proxy=.. \
--build-arg no_proxy=.. \
--rm --no-cache -t intelanalytics/ipex-llm-cpu:2.1.0-SNAPSHOT .
--rm --no-cache -t intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT .
```
@ -16,7 +16,7 @@ docker build \
An example could be:
```bash
#/bin/bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-cpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT
sudo docker run -itd \
--net=host \
@ -41,7 +41,7 @@ You can download models and bind the model directory from host machine to contai
Here is an example:
```bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-cpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT
export MODEL_PATH=/home/llm/models
sudo docker run -itd \

4
docker/llm/inference/xpu/docker/README.md
View file

@ -6,7 +6,7 @@ docker build \
--build-arg http_proxy=.. \
--build-arg https_proxy=.. \
--build-arg no_proxy=.. \
--rm --no-cache -t intelanalytics/ipex-llm-xpu:2.1.0-SNAPSHOT .
--rm --no-cache -t intelanalytics/ipex-llm-xpu:2.2.0-SNAPSHOT .
```
@ -17,7 +17,7 @@ To map the `xpu` into the container, you need to specify `--device=/dev/dri` whe
An example could be:
```bash
#/bin/bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-xpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-xpu:2.2.0-SNAPSHOT
sudo docker run -itd \
--net=host \

2
docker/llm/serving/cpu/docker/Dockerfile
View file

@ -1,4 +1,4 @@
FROM intelanalytics/ipex-llm-cpu:2.1.0-SNAPSHOT
FROM intelanalytics/ipex-llm-cpu:2.2.0-SNAPSHOT
ARG http_proxy
ARG https_proxy

4
docker/llm/serving/cpu/docker/README.md
View file

@ -6,7 +6,7 @@ docker build \
--build-arg http_proxy=.. \
--build-arg https_proxy=.. \
--build-arg no_proxy=.. \
--rm --no-cache -t intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT .
--rm --no-cache -t intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT .
```
### Use the image for doing cpu serving
@ -16,7 +16,7 @@ You could use the following bash script to start the container. Please be noted
```bash
#/bin/bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT
sudo docker run -itd \
--net=host \

6
docker/llm/serving/cpu/kubernetes/README.md
View file

@ -2,7 +2,7 @@
## Image
To deploy IPEX-LLM-serving cpu in Kubernetes environment, please use this image: `intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT`
To deploy IPEX-LLM-serving cpu in Kubernetes environment, please use this image: `intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT`
## Before deployment
@ -73,7 +73,7 @@ spec:
dnsPolicy: "ClusterFirst"
containers:
- name: fastchat-controller # fixed
image: intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT
image: intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT
imagePullPolicy: IfNotPresent
env:
- name: CONTROLLER_HOST # fixed
@ -146,7 +146,7 @@ spec:
dnsPolicy: "ClusterFirst"
containers:
- name: fastchat-worker # fixed
image: intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT
image: intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT
imagePullPolicy: IfNotPresent
env:
- name: CONTROLLER_HOST # fixed

4
docker/llm/serving/cpu/kubernetes/deployment.yaml
View file

@ -24,7 +24,7 @@ spec:
dnsPolicy: "ClusterFirst"
containers:
- name: fastchat-controller # fixed
image: intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT
image: intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT
imagePullPolicy: IfNotPresent
env:
- name: CONTROLLER_HOST # fixed
@ -91,7 +91,7 @@ spec:
dnsPolicy: "ClusterFirst"
containers:
- name: fastchat-worker # fixed
image: intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT
image: intelanalytics/ipex-llm-serving-cpu:2.2.0-SNAPSHOT
imagePullPolicy: IfNotPresent
env:
- name: CONTROLLER_HOST # fixed

2
docker/llm/serving/xpu/docker/Dockerfile
View file

@ -17,7 +17,7 @@ RUN cd /tmp/ && \
mv /tmp/torch-ccl/dist/oneccl_bind_pt-2.1.100+xpu-cp311-cp311-linux_x86_64.whl /tmp/
FROM intelanalytics/ipex-llm-xpu:2.1.0-SNAPSHOT
FROM intelanalytics/ipex-llm-xpu:2.2.0-SNAPSHOT
ARG http_proxy
ARG https_proxy

4
docker/llm/serving/xpu/docker/README.md
View file

@ -6,7 +6,7 @@ docker build \
--build-arg http_proxy=.. \
--build-arg https_proxy=.. \
--build-arg no_proxy=.. \
--rm --no-cache -t intelanalytics/ipex-llm-serving-xpu:2.1.0-SNAPSHOT .
--rm --no-cache -t intelanalytics/ipex-llm-serving-xpu:2.2.0-SNAPSHOT .
```
@ -18,7 +18,7 @@ To map the `xpu` into the container, you need to specify `--device=/dev/dri` whe
An example could be:
```bash
#/bin/bash
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:2.1.0-SNAPSHOT
export DOCKER_IMAGE=intelanalytics/ipex-llm-serving-xpu:2.2.0-SNAPSHOT
sudo docker run -itd \
--net=host \