ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Enhanced XPU Dockerfiles: Optimized Environment Variables and Documentation (#11506)

Browse source 
* Added SYCL_CACHE_PERSISTENT=1 to xpu Dockerfile

* Update the document to add explanations for environment variables.

* update quickstart
This commit is contained in:
Shaojun Liu 2024-07-04 20:18:38 +08:00 committed by GitHub
parent 60de428b37
commit 72b4efaad4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 36 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docker/llm/README.md
View file

@ -25,7 +25,7 @@ Available images in hub are:
| intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT | CPU Serving|
| intelanalytics/ipex-llm-serving-xpu:2.1.0-SNAPSHOT | GPU Serving|
| intelanalytics/ipex-llm-finetune-qlora-cpu-standalone:2.1.0-SNAPSHOT | CPU Finetuning via Docker|
|intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.1.0-SNAPSHOT|CPU Finetuning via Kubernetes|
| intelanalytics/ipex-llm-finetune-qlora-cpu-k8s:2.1.0-SNAPSHOT|CPU Finetuning via Kubernetes|
| intelanalytics/ipex-llm-finetune-qlora-xpu:2.1.0-SNAPSHOT| GPU Finetuning|
#### Run a Container

3
docker/llm/finetune/xpu/Dockerfile
View file

@ -4,6 +4,9 @@ ARG https_proxy
ENV TZ=Asia/Shanghai
ARG PIP_NO_CACHE_DIR=false
# When cache is enabled SYCL runtime will try to cache and reuse JIT-compiled binaries.
ENV SYCL_CACHE_PERSISTENT=1
# retrive oneapi repo public key
RUN wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | tee /usr/share/keyrings/intel-oneapi-archive-keyring.gpg > /dev/null && \
echo "deb [signed-by=/usr/share/keyrings/intel-oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main " | tee /etc/apt/sources.list.d/oneAPI.list && \

3
docker/llm/inference-cpp/Dockerfile
View file

@ -6,6 +6,9 @@ ARG https_proxy
ENV TZ=Asia/Shanghai
ENV PYTHONUNBUFFERED=1
# When cache is enabled SYCL runtime will try to cache and reuse JIT-compiled binaries.
ENV SYCL_CACHE_PERSISTENT=1
# Disable pip's cache behavior
ARG PIP_NO_CACHE_DIR=false

5
docker/llm/inference/xpu/docker/Dockerfile
View file

@ -5,8 +5,9 @@ ARG https_proxy
ENV TZ=Asia/Shanghai
ENV PYTHONUNBUFFERED=1
ENV USE_XETLA=OFF
ENV SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
# When cache is enabled SYCL runtime will try to cache and reuse JIT-compiled binaries.
ENV SYCL_CACHE_PERSISTENT=1
COPY chat.py /llm/chat.py
COPY benchmark.sh /llm/benchmark.sh

26
docs/mddocs/DockerGuides/docker_pytorch_inference_gpu.md
View file

@ -78,6 +78,32 @@ root@arda-arc12:/# sycl-ls
> bash env-check.sh
> ```
> [!NOTE]
> For optimal performance, it is recommended to set several environment variables according to your hardware environment.
>
> ```bash
> # Disable code related to XETLA; only Intel Data Center GPU Max Series supports XETLA, so non-Max machines should set this to OFF.
> # Recommended for use on Intel Arc™ A-Series Graphics and Intel Data Center GPU Flex Series.
> export USE_XETLA=OFF
>
> # Enable immediate command lists mode for the Level Zero plugin. Improves performance on Intel Arc™ A-Series Graphics and Intel Data Center GPU Max Series; however, it depends on the Linux Kernel, and some Linux kernels may not necessarily provide acceleration.
> # Recommended for use on Intel Arc™ A-Series Graphics and Intel Data Center GPU Max Series, but it depends on the Linux kernel, Non-i915 kernel drivers may cause performance regressions.
> export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
>
> # Controls persistent device compiled code cache. Set to '1' to turn on and '0' to turn off.
> # Recommended for all hardware environments. This environment variable is already set by default in Docker images.
> export SYCL_CACHE_PERSISTENT=1
>
> # Reduce memory accesses by fusing SDP ops.
> # Recommended for use on Intel Data Center GPU Max Series.
> export ENABLE_SDP_FUSION=1
>
> # Disable XMX computation.
> # Recommended for use on integrated GPUs.
> export BIGDL_LLM_XMX_DISABLED=1
> ```
## Run Inference Benchmark
Navigate to benchmark directory, and modify the `config.yaml` under the `all-in-one` folder for benchmark configurations.