diff --git a/docs/mddocs/Overview/install_gpu.md b/docs/mddocs/Overview/install_gpu.md index f6cd5e98..2de066e4 100644 --- a/docs/mddocs/Overview/install_gpu.md +++ b/docs/mddocs/Overview/install_gpu.md @@ -46,7 +46,7 @@ We recommend using [Miniforge](https://conda-forge.org/download/) to create a py The easiest ways to install `ipex-llm` is the following commands. -- For **Intel Core™ Ultra Series 2 with Intel Arc™ Graphics (a.k.a. Lunar Lake)**: +- For **Intel Core™ Ultra Processers (Series 2) with processor number 2xxV (code name Lunar Lake)**: Choose either US or CN website for `extra-index-url`: @@ -94,7 +94,7 @@ The easiest ways to install `ipex-llm` is the following commands. If you encounter network issues when installing IPEX, you can also install IPEX-LLM dependencies for Intel XPU from source archives. First you need to download and install torch/torchvision/ipex from wheels listed below before installing `ipex-llm`. -- For **Intel Core™ Ultra Series 2 with Intel Arc™ Graphics (a.k.a. Lunar Lake)**: +- For **Intel Core™ Ultra Processers (Series 2) with processor number 2xxV (code name Lunar Lake)**: Download the wheels on Windows system: @@ -187,8 +187,12 @@ IPEX-LLM GPU support on Linux has been verified on: * Intel Arc™ A-Series Graphics * Intel Data Center GPU Flex Series * Intel Data Center GPU Max Series +* Intel iGPU > [!IMPORTANT] +> For prerequisite installation on Intel Core™ Ultra Processers (Series 1) with processor number 1xxH/U/HL/UL (code name Meteor Lake), please refer to [this guide](../Quickstart/install_linux_gpu.md#install-prerequisites). + +> [!Note] > IPEX-LLM on Linux supports PyTorch 2.0 and PyTorch 2.1. > > **Warning** @@ -211,29 +215,6 @@ IPEX-LLM GPU support on Linux has been verified on: > > See [release page](https://dgpu-docs.intel.com/releases/index.html) for latest version. - > **Note**: - > - > For Intel Core™ Ultra integrated GPU, please make sure level_zero version >= 1.3.28717. The level_zero version can be checked with ``sycl-ls``, and verison will be tagged be ``[ext_oneapi_level_zero:gpu]``. - > ``` - > [opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device OpenCL 1.2 [2023.16.12.0.12_195853.xmain-hotfix] - > [opencl:cpu:1] Intel(R) OpenCL, Intel(R) Core(TM) Ultra 5 125H OpenCL 3.0 (Build 0) [2023.16.12.0.12_195853.xmain-hotfix] - > [opencl:gpu:2] Intel(R) OpenCL Graphics, Intel(R) Arc(TM) Graphics OpenCL 3.0 NEO [24.09.28717.12] - > [ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) Arc(TM) Graphics 1.3 [1.3.28717] - > ``` - > - > If you have level_zero version < 1.3.28717, you could update as follows: - > - > ```bash - > wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16238.4/intel-igc-core_1.0.16238.4_amd64.deb - > wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16238.4/intel-igc-opencl_1.0.16238.4_amd64.deb - > wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-level-zero-gpu-dbgsym_1.3.28717.12_amd64.ddeb - > wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-level-zero-gpu_1.3.28717.12_amd64.deb - > wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-opencl-icd-dbgsym_24.09.28717.12_amd64.ddeb - > wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-opencl-icd_24.09.28717.12_amd64.deb - > wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/libigdgmm12_22.3.17_amd64.deb - > sudo dpkg -i *.deb - > ``` - - Step 2: Download and install [Intel® oneAPI Base Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html) with version 2024.0. OneDNN, OneMKL and DPC++ compiler are needed, others are optional. Intel® oneAPI Base Toolkit 2024.0 installation methods: diff --git a/docs/mddocs/Quickstart/install_linux_gpu.md b/docs/mddocs/Quickstart/install_linux_gpu.md index 6a5ea4f6..f87b2ff1 100644 --- a/docs/mddocs/Quickstart/install_linux_gpu.md +++ b/docs/mddocs/Quickstart/install_linux_gpu.md @@ -1,12 +1,17 @@ # Install IPEX-LLM on Linux with Intel GPU -This guide demonstrates how to install IPEX-LLM on Linux with Intel GPUs. It applies to Intel Data Center GPU Flex Series and Max Series, as well as Intel Arc Series GPU. +This guide demonstrates how to install IPEX-LLM on Linux with Intel GPUs. It applies to Intel Data Center GPU Flex Series and Max Series, as well as Intel Arc Series GPU and Intel iGPU. IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and supports PyTorch 2.0 and PyTorch 2.1 on Linux. This page demonstrates IPEX-LLM with PyTorch 2.1. Check the [Installation](../Overview/install_gpu.md#linux) page for more details. ## Table of Contents - [Install Prerequisites](./install_linux_gpu.md#install-prerequisites) + - [Install GPU Driver](#install-gpu-driver) + - [For Intel Core™ Ultra Processers (Series 1) with processor number 1xxH/U/HL/UL (code name Meteor Lake)](#for-intel-core-ultra-processers-series-1-with-processor-number-1xxhuhlul-code-name-meteor-lake) + - [For other Intel iGPU and dGPU](#for-other-intel-igpu-and-dgpu) + - [Install oneAPI](#install-oneapi) + - [Setup Python Environment](#setup-python-environment) - [Install ipex-llm](./install_linux_gpu.md#install-ipex-llm) - [Verify Installation](./install_linux_gpu.md#verify-installation) - [Runtime Configurations](./install_linux_gpu.md#runtime-configurations) @@ -17,7 +22,115 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup ### Install GPU Driver -#### For Linux kernel 6.2 +#### For Intel Core™ Ultra Processers (Series 1) with processor number 1xxH/U/HL/UL (code name Meteor Lake) + +> [!NOTE] +> For IPEX-LLM on Linux with Meteor Lake integrated GPU, we have currently verified on Ubuntu 22.04 with kernel `6.5.0-35-generic`. + +##### 1. Check current kernel version + +You could check your current kernel version through: + +```bash +uname -r +``` +If the version displayed is not `6.5.0-35-generic`, downgrade or upgrade the kernel to the recommended version. + +##### 2. (Optional) Downgrade / Upgrade to kernel 6.5.0-35 + +If your current kernel version is not `6.5.0-35-generic`, you could downgrade or upgrade it by: + +```bash +export VERSION="6.5.0-35" +sudo apt-get install -y linux-headers-$VERSION-generic linux-image-$VERSION-generic linux-modules-extra-$VERSION-generic + +sudo sed -i "s/GRUB_DEFAULT=.*/GRUB_DEFAULT=\"1> $(echo $(($(awk -F\' '/menuentry / {print $2}' /boot/grub/grub.cfg \ +| grep -no $VERSION | sed 's/:/\n/g' | head -n 1)-2)))\"/" /etc/default/grub + +sudo update-grub +``` + +And the reboot your machine: + +```bash +sudo reboot +``` + +After rebooting, you can use `uname -r` again to see that your kernel version has been changed to `6.5.0-35-generic`. + +##### 3. Enable GPU driver support through `force_probe` flag + +Next, you need to enable GPU driver support on kernel `6.5.0-35-generic` through `force_probe` parameter： + +```bash +export FORCE_PROBE_VALUE=$(sudo dmesg | grep i915 | grep -o 'i915\.force_probe=[a-zA-Z0-9]\{4\}') + +sed -i "/^GRUB_CMDLINE_LINUX_DEFAULT=/ s/\"\(.*\)\"/\"\1 $FORCE_PROBE_VALUE\"/" /etc/default/grub +``` + +> [!TIP] +> In addition to using the above command, you could also maunally check your `force_probe` flag value through: +> +> ```bash +> sudo dmesg | grep i915 +> ``` +> +> You may get output like `Your graphics device 7d55 is not properly supported by i915 in this kernel version. To force driver probe anyway, use i915.force_probe=7d55`, in which `7d55` is the PCI ID depends on your GPU model. +> +> Then, modify the `/etc/default/grub` file directly. Make sure you add `i915.force_probe=xxxx` to the value of `GRUB_CMDLINE_LINUX_DEFAULT`. For example, before modification, you have`GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"` inside `/etc/default/grub`. You then need to change it to `GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.force_probe=7d55"`. + +Then update grub through: +```bash +sudo update-grub +``` + +Reboot the machine then to make the configuration take effect: + +```bash +sudo reboot +``` + +##### 4. Install compute packages + +Compute packages are also required to be installed for Intel GPU on Ubuntu 22.04 through the following commands: + +```bash +wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | \ + sudo gpg --yes --dearmor --output /usr/share/keyrings/intel-graphics.gpg + +echo "deb [arch=amd64,i386 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy client" | \ + sudo tee /etc/apt/sources.list.d/intel-gpu-jammy.list + +sudo apt update + +apt-get install -y libze1 intel-level-zero-gpu intel-opencl-icd clinfo +``` + + +##### 5. Configure permmision and verify GPU driver setup + +To complete the GPU driver setup, you need to make sure your user is in the render group: + +```bash +sudo gpasswd -a ${USER} render +newgrp render +``` + +You could then verify whether the GPU driver is functioning properly with: + +```bash +clinfo | grep "Device Name" +``` + +whose output should contain `Intel(R) Arc(TM) Graphics`. + +> [!TIP] +> You could refer to the [official driver guide for client GPUS](https://dgpu-docs.intel.com/driver/client/overview.html#installing-client-gpus-on-ubuntu-desktop-22-04-lts) for more information. + + +#### For other Intel iGPU and dGPU + +##### For Linux kernel 6.2 * Choose one option below depending on your CPU type: @@ -80,7 +193,7 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup hwinfo --display ``` -#### For Linux kernel 6.5 +##### For Linux kernel 6.5 * Choose one option below depending on your CPU type: @@ -140,7 +253,7 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup hwinfo --display ``` -#### (Optional) Update Level Zero on Intel Core™ Ultra iGPU + + +### Install oneAPI +IPEX-LLM requires installation of oneAPI 2024.0 for Intel GPU on Linux + +```bash +wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null + +echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list + +sudo apt update + +sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \ + intel-oneapi-common-oneapi-vars=2024.0.0-49406 \ + intel-oneapi-diagnostics-utility=2024.0.0-49093 \ + intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \ + intel-oneapi-dpcpp-ct=2024.0.0-49381 \ + intel-oneapi-mkl=2024.0.0-49656 \ + intel-oneapi-mkl-devel=2024.0.0-49656 \ + intel-oneapi-mpi=2021.11.0-49493 \ + intel-oneapi-mpi-devel=2021.11.0-49493 \ + intel-oneapi-dal=2024.0.1-25 \ + intel-oneapi-dal-devel=2024.0.1-25 \ + intel-oneapi-ippcp=2021.9.1-5 \ + intel-oneapi-ippcp-devel=2021.9.1-5 \ + intel-oneapi-ipp=2021.10.1-13 \ + intel-oneapi-ipp-devel=2021.10.1-13 \ + intel-oneapi-tlt=2024.0.0-352 \ + intel-oneapi-ccl=2021.11.2-5 \ + intel-oneapi-ccl-devel=2021.11.2-5 \ + intel-oneapi-dnnl-devel=2024.0.0-49521 \ + intel-oneapi-dnnl=2024.0.0-49521 \ + intel-oneapi-tcm-1.0=1.0.0-435 ``` +image-20240221102252565 -### Install oneAPI - ```bash - wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null +image-20240221102252565 - echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list - - sudo apt update - - sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \ - intel-oneapi-common-oneapi-vars=2024.0.0-49406 \ - intel-oneapi-diagnostics-utility=2024.0.0-49093 \ - intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \ - intel-oneapi-dpcpp-ct=2024.0.0-49381 \ - intel-oneapi-mkl=2024.0.0-49656 \ - intel-oneapi-mkl-devel=2024.0.0-49656 \ - intel-oneapi-mpi=2021.11.0-49493 \ - intel-oneapi-mpi-devel=2021.11.0-49493 \ - intel-oneapi-dal=2024.0.1-25 \ - intel-oneapi-dal-devel=2024.0.1-25 \ - intel-oneapi-ippcp=2021.9.1-5 \ - intel-oneapi-ippcp-devel=2021.9.1-5 \ - intel-oneapi-ipp=2021.10.1-13 \ - intel-oneapi-ipp-devel=2021.10.1-13 \ - intel-oneapi-tlt=2024.0.0-352 \ - intel-oneapi-ccl=2021.11.2-5 \ - intel-oneapi-ccl-devel=2021.11.2-5 \ - intel-oneapi-dnnl-devel=2024.0.0-49521 \ - intel-oneapi-dnnl=2024.0.0-49521 \ - intel-oneapi-tcm-1.0=1.0.0-435 - ``` - image-20240221102252565 - - image-20240221102252565 +>[!IMPORTANT] +> Please make sure to reboot the machine after the installation of GPU driver and oneAPI is complete: +> +> ```bash +> sudo reboot +> ``` ### Setup Python Environment @@ -258,8 +380,7 @@ To use GPU acceleration on Linux, several environment variables are required or For Intel Arc™ A-Series Graphics and Intel Data Center GPU Flex Series, we recommend: ```bash - # Configure oneAPI environment variables. Required step for APT or offline installed oneAPI. - # Skip this step for PIP-installed oneAPI since the environment has already been configured in LD_LIBRARY_PATH. + # Configure oneAPI environment variables. source /opt/intel/oneapi/setvars.sh # Recommended Environment Variables for optimal performance @@ -274,8 +395,7 @@ To use GPU acceleration on Linux, several environment variables are required or For Intel Data Center GPU Max Series, we recommend: ```bash - # Configure oneAPI environment variables. Required step for APT or offline installed oneAPI. - # Skip this step for PIP-installed oneAPI since the environment has already been configured in LD_LIBRARY_PATH. + # Configure oneAPI environment variables. source /opt/intel/oneapi/setvars.sh # Recommended Environment Variables for optimal performance @@ -288,6 +408,16 @@ To use GPU acceleration on Linux, several environment variables are required or Please note that `libtcmalloc.so` can be installed by `conda install -c conda-forge -y gperftools=2.10` +- For **Intel iGPU**: + + ```bash + # Configure oneAPI environment variables. + source /opt/intel/oneapi/setvars.sh + + export SYCL_CACHE_PERSISTENT=1 + export BIGDL_LLM_XMX_DISABLED=1 + ``` + > [!NOTE] > Please refer to [this guide](../Overview/install_gpu.md#runtime-configuration-1) for more details regarding runtime configuration. diff --git a/docs/mddocs/Quickstart/install_windows_gpu.md b/docs/mddocs/Quickstart/install_windows_gpu.md index c82b8186..4407dffb 100644 --- a/docs/mddocs/Quickstart/install_windows_gpu.md +++ b/docs/mddocs/Quickstart/install_windows_gpu.md @@ -47,7 +47,7 @@ conda activate llm With the `llm` environment active, use `pip` to install `ipex-llm` for GPU: -- For **Intel Core™ Ultra Series 2 with Intel Arc™ Graphics (a.k.a. Lunar Lake)**: +- For **Intel Core™ Ultra Processers (Series 2) with processor number 2xxV (code name Lunar Lake)**: Choose either US or CN website for `extra-index-url`: