Update Linux prerequisites installation guide for MTL iGPU (#12263)

* Update Linux prerequisites installation guide for MTL iGPU

* Further link update

* Small fixes

* Small fix

* Update based on comments

* Small fix

* Make oneAPI installation a shared section for both MTL iGPU and other GPU

* Small fix

* Small fix

* Clarify description
This commit is contained in:
Yuwen Hu 2024-10-28 09:27:14 +08:00 committed by GitHub
parent ec362e6133
commit 16074ae2a4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 177 additions and 66 deletions

View file

@ -46,7 +46,7 @@ We recommend using [Miniforge](https://conda-forge.org/download/) to create a py
The easiest ways to install `ipex-llm` is the following commands. The easiest ways to install `ipex-llm` is the following commands.
- For **Intel Core™ Ultra Series 2 with Intel Arc™ Graphics (a.k.a. Lunar Lake)**: - For **Intel Core™ Ultra Processers (Series 2) with processor number 2xxV (code name Lunar Lake)**:
Choose either US or CN website for `extra-index-url`: Choose either US or CN website for `extra-index-url`:
@ -94,7 +94,7 @@ The easiest ways to install `ipex-llm` is the following commands.
If you encounter network issues when installing IPEX, you can also install IPEX-LLM dependencies for Intel XPU from source archives. First you need to download and install torch/torchvision/ipex from wheels listed below before installing `ipex-llm`. If you encounter network issues when installing IPEX, you can also install IPEX-LLM dependencies for Intel XPU from source archives. First you need to download and install torch/torchvision/ipex from wheels listed below before installing `ipex-llm`.
- For **Intel Core™ Ultra Series 2 with Intel Arc™ Graphics (a.k.a. Lunar Lake)**: - For **Intel Core™ Ultra Processers (Series 2) with processor number 2xxV (code name Lunar Lake)**:
Download the wheels on Windows system: Download the wheels on Windows system:
@ -187,8 +187,12 @@ IPEX-LLM GPU support on Linux has been verified on:
* Intel Arc™ A-Series Graphics * Intel Arc™ A-Series Graphics
* Intel Data Center GPU Flex Series * Intel Data Center GPU Flex Series
* Intel Data Center GPU Max Series * Intel Data Center GPU Max Series
* Intel iGPU
> [!IMPORTANT] > [!IMPORTANT]
> For prerequisite installation on Intel Core™ Ultra Processers (Series 1) with processor number 1xxH/U/HL/UL (code name Meteor Lake), please refer to [this guide](../Quickstart/install_linux_gpu.md#install-prerequisites).
> [!Note]
> IPEX-LLM on Linux supports PyTorch 2.0 and PyTorch 2.1. > IPEX-LLM on Linux supports PyTorch 2.0 and PyTorch 2.1.
> >
> **Warning** > **Warning**
@ -211,29 +215,6 @@ IPEX-LLM GPU support on Linux has been verified on:
> >
> See [release page](https://dgpu-docs.intel.com/releases/index.html) for latest version. > See [release page](https://dgpu-docs.intel.com/releases/index.html) for latest version.
> **Note**:
>
> For Intel Core™ Ultra integrated GPU, please make sure level_zero version >= 1.3.28717. The level_zero version can be checked with ``sycl-ls``, and verison will be tagged be ``[ext_oneapi_level_zero:gpu]``.
> ```
> [opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device OpenCL 1.2 [2023.16.12.0.12_195853.xmain-hotfix]
> [opencl:cpu:1] Intel(R) OpenCL, Intel(R) Core(TM) Ultra 5 125H OpenCL 3.0 (Build 0) [2023.16.12.0.12_195853.xmain-hotfix]
> [opencl:gpu:2] Intel(R) OpenCL Graphics, Intel(R) Arc(TM) Graphics OpenCL 3.0 NEO [24.09.28717.12]
> [ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) Arc(TM) Graphics 1.3 [1.3.28717]
> ```
>
> If you have level_zero version < 1.3.28717, you could update as follows:
>
> ```bash
> wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16238.4/intel-igc-core_1.0.16238.4_amd64.deb
> wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16238.4/intel-igc-opencl_1.0.16238.4_amd64.deb
> wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-level-zero-gpu-dbgsym_1.3.28717.12_amd64.ddeb
> wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-level-zero-gpu_1.3.28717.12_amd64.deb
> wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-opencl-icd-dbgsym_24.09.28717.12_amd64.ddeb
> wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-opencl-icd_24.09.28717.12_amd64.deb
> wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/libigdgmm12_22.3.17_amd64.deb
> sudo dpkg -i *.deb
> ```
- Step 2: Download and install [Intel® oneAPI Base Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html) with version 2024.0. OneDNN, OneMKL and DPC++ compiler are needed, others are optional. - Step 2: Download and install [Intel® oneAPI Base Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html) with version 2024.0. OneDNN, OneMKL and DPC++ compiler are needed, others are optional.
Intel® oneAPI Base Toolkit 2024.0 installation methods: Intel® oneAPI Base Toolkit 2024.0 installation methods:

View file

@ -1,12 +1,17 @@
# Install IPEX-LLM on Linux with Intel GPU # Install IPEX-LLM on Linux with Intel GPU
This guide demonstrates how to install IPEX-LLM on Linux with Intel GPUs. It applies to Intel Data Center GPU Flex Series and Max Series, as well as Intel Arc Series GPU. This guide demonstrates how to install IPEX-LLM on Linux with Intel GPUs. It applies to Intel Data Center GPU Flex Series and Max Series, as well as Intel Arc Series GPU and Intel iGPU.
IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and supports PyTorch 2.0 and PyTorch 2.1 on Linux. This page demonstrates IPEX-LLM with PyTorch 2.1. Check the [Installation](../Overview/install_gpu.md#linux) page for more details. IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and supports PyTorch 2.0 and PyTorch 2.1 on Linux. This page demonstrates IPEX-LLM with PyTorch 2.1. Check the [Installation](../Overview/install_gpu.md#linux) page for more details.
## Table of Contents ## Table of Contents
- [Install Prerequisites](./install_linux_gpu.md#install-prerequisites) - [Install Prerequisites](./install_linux_gpu.md#install-prerequisites)
- [Install GPU Driver](#install-gpu-driver)
- [For Intel Core™ Ultra Processers (Series 1) with processor number 1xxH/U/HL/UL (code name Meteor Lake)](#for-intel-core-ultra-processers-series-1-with-processor-number-1xxhuhlul-code-name-meteor-lake)
- [For other Intel iGPU and dGPU](#for-other-intel-igpu-and-dgpu)
- [Install oneAPI](#install-oneapi)
- [Setup Python Environment](#setup-python-environment)
- [Install ipex-llm](./install_linux_gpu.md#install-ipex-llm) - [Install ipex-llm](./install_linux_gpu.md#install-ipex-llm)
- [Verify Installation](./install_linux_gpu.md#verify-installation) - [Verify Installation](./install_linux_gpu.md#verify-installation)
- [Runtime Configurations](./install_linux_gpu.md#runtime-configurations) - [Runtime Configurations](./install_linux_gpu.md#runtime-configurations)
@ -17,7 +22,115 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup
### Install GPU Driver ### Install GPU Driver
#### For Linux kernel 6.2 #### For Intel Core™ Ultra Processers (Series 1) with processor number 1xxH/U/HL/UL (code name Meteor Lake)
> [!NOTE]
> For IPEX-LLM on Linux with Meteor Lake integrated GPU, we have currently verified on Ubuntu 22.04 with kernel `6.5.0-35-generic`.
##### 1. Check current kernel version
You could check your current kernel version through:
```bash
uname -r
```
If the version displayed is not `6.5.0-35-generic`, downgrade or upgrade the kernel to the recommended version.
##### 2. (Optional) Downgrade / Upgrade to kernel 6.5.0-35
If your current kernel version is not `6.5.0-35-generic`, you could downgrade or upgrade it by:
```bash
export VERSION="6.5.0-35"
sudo apt-get install -y linux-headers-$VERSION-generic linux-image-$VERSION-generic linux-modules-extra-$VERSION-generic
sudo sed -i "s/GRUB_DEFAULT=.*/GRUB_DEFAULT=\"1> $(echo $(($(awk -F\' '/menuentry / {print $2}' /boot/grub/grub.cfg \
| grep -no $VERSION | sed 's/:/\n/g' | head -n 1)-2)))\"/" /etc/default/grub
sudo update-grub
```
And the reboot your machine:
```bash
sudo reboot
```
After rebooting, you can use `uname -r` again to see that your kernel version has been changed to `6.5.0-35-generic`.
##### 3. Enable GPU driver support through `force_probe` flag
Next, you need to enable GPU driver support on kernel `6.5.0-35-generic` through `force_probe` parameter
```bash
export FORCE_PROBE_VALUE=$(sudo dmesg | grep i915 | grep -o 'i915\.force_probe=[a-zA-Z0-9]\{4\}')
sed -i "/^GRUB_CMDLINE_LINUX_DEFAULT=/ s/\"\(.*\)\"/\"\1 $FORCE_PROBE_VALUE\"/" /etc/default/grub
```
> [!TIP]
> In addition to using the above command, you could also maunally check your `force_probe` flag value through:
>
> ```bash
> sudo dmesg | grep i915
> ```
>
> You may get output like `Your graphics device 7d55 is not properly supported by i915 in this kernel version. To force driver probe anyway, use i915.force_probe=7d55`, in which `7d55` is the PCI ID depends on your GPU model.
>
> Then, modify the `/etc/default/grub` file directly. Make sure you add `i915.force_probe=xxxx` to the value of `GRUB_CMDLINE_LINUX_DEFAULT`. For example, before modification, you have`GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"` inside `/etc/default/grub`. You then need to change it to `GRUB_CMDLINE_LINUX_DEFAULT="quiet splash i915.force_probe=7d55"`.
Then update grub through:
```bash
sudo update-grub
```
Reboot the machine then to make the configuration take effect:
```bash
sudo reboot
```
##### 4. Install compute packages
Compute packages are also required to be installed for Intel GPU on Ubuntu 22.04 through the following commands:
```bash
wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | \
sudo gpg --yes --dearmor --output /usr/share/keyrings/intel-graphics.gpg
echo "deb [arch=amd64,i386 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy client" | \
sudo tee /etc/apt/sources.list.d/intel-gpu-jammy.list
sudo apt update
apt-get install -y libze1 intel-level-zero-gpu intel-opencl-icd clinfo
```
##### 5. Configure permmision and verify GPU driver setup
To complete the GPU driver setup, you need to make sure your user is in the render group:
```bash
sudo gpasswd -a ${USER} render
newgrp render
```
You could then verify whether the GPU driver is functioning properly with:
```bash
clinfo | grep "Device Name"
```
whose output should contain `Intel(R) Arc(TM) Graphics`.
> [!TIP]
> You could refer to the [official driver guide for client GPUS](https://dgpu-docs.intel.com/driver/client/overview.html#installing-client-gpus-on-ubuntu-desktop-22-04-lts) for more information.
#### For other Intel iGPU and dGPU
##### For Linux kernel 6.2
* Choose one option below depending on your CPU type: * Choose one option below depending on your CPU type:
@ -80,7 +193,7 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup
hwinfo --display hwinfo --display
``` ```
#### For Linux kernel 6.5 ##### For Linux kernel 6.5
* Choose one option below depending on your CPU type: * Choose one option below depending on your CPU type:
@ -140,7 +253,7 @@ IPEX-LLM currently supports the Ubuntu 20.04 operating system and later, and sup
hwinfo --display hwinfo --display
``` ```
#### (Optional) Update Level Zero on Intel Core™ Ultra iGPU <!-- #### (Optional) Update Level Zero on Intel Core™ Ultra iGPU
For Intel Core™ Ultra integrated GPU, please make sure level_zero version >= 1.3.28717. The level_zero version can be checked with `sycl-ls`, and verison will be tagged behind `[ext_oneapi_level_zero:gpu]`. For Intel Core™ Ultra integrated GPU, please make sure level_zero version >= 1.3.28717. The level_zero version can be checked with `sycl-ls`, and verison will be tagged behind `[ext_oneapi_level_zero:gpu]`.
Here are the sample output of `sycl-ls`: Here are the sample output of `sycl-ls`:
@ -161,41 +274,50 @@ wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/i
wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-opencl-icd_24.09.28717.12_amd64.deb wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/intel-opencl-icd_24.09.28717.12_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/libigdgmm12_22.3.17_amd64.deb wget https://github.com/intel/compute-runtime/releases/download/24.09.28717.12/libigdgmm12_22.3.17_amd64.deb
sudo dpkg -i *.deb sudo dpkg -i *.deb
``` -->
### Install oneAPI
IPEX-LLM requires installation of oneAPI 2024.0 for Intel GPU on Linux
```bash
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list
sudo apt update
sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \
intel-oneapi-common-oneapi-vars=2024.0.0-49406 \
intel-oneapi-diagnostics-utility=2024.0.0-49093 \
intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
intel-oneapi-dpcpp-ct=2024.0.0-49381 \
intel-oneapi-mkl=2024.0.0-49656 \
intel-oneapi-mkl-devel=2024.0.0-49656 \
intel-oneapi-mpi=2021.11.0-49493 \
intel-oneapi-mpi-devel=2021.11.0-49493 \
intel-oneapi-dal=2024.0.1-25 \
intel-oneapi-dal-devel=2024.0.1-25 \
intel-oneapi-ippcp=2021.9.1-5 \
intel-oneapi-ippcp-devel=2021.9.1-5 \
intel-oneapi-ipp=2021.10.1-13 \
intel-oneapi-ipp-devel=2021.10.1-13 \
intel-oneapi-tlt=2024.0.0-352 \
intel-oneapi-ccl=2021.11.2-5 \
intel-oneapi-ccl-devel=2021.11.2-5 \
intel-oneapi-dnnl-devel=2024.0.0-49521 \
intel-oneapi-dnnl=2024.0.0-49521 \
intel-oneapi-tcm-1.0=1.0.0-435
``` ```
<img src="https://llm-assets.readthedocs.io/en/latest/_images/oneapi.png" alt="image-20240221102252565" width=100%; />
### Install oneAPI <img src="https://llm-assets.readthedocs.io/en/latest/_images/basekit.png" alt="image-20240221102252565" width=100%; />
```bash
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list >[!IMPORTANT]
> Please make sure to reboot the machine after the installation of GPU driver and oneAPI is complete:
sudo apt update >
> ```bash
sudo apt install intel-oneapi-common-vars=2024.0.0-49406 \ > sudo reboot
intel-oneapi-common-oneapi-vars=2024.0.0-49406 \ > ```
intel-oneapi-diagnostics-utility=2024.0.0-49093 \
intel-oneapi-compiler-dpcpp-cpp=2024.0.2-49895 \
intel-oneapi-dpcpp-ct=2024.0.0-49381 \
intel-oneapi-mkl=2024.0.0-49656 \
intel-oneapi-mkl-devel=2024.0.0-49656 \
intel-oneapi-mpi=2021.11.0-49493 \
intel-oneapi-mpi-devel=2021.11.0-49493 \
intel-oneapi-dal=2024.0.1-25 \
intel-oneapi-dal-devel=2024.0.1-25 \
intel-oneapi-ippcp=2021.9.1-5 \
intel-oneapi-ippcp-devel=2021.9.1-5 \
intel-oneapi-ipp=2021.10.1-13 \
intel-oneapi-ipp-devel=2021.10.1-13 \
intel-oneapi-tlt=2024.0.0-352 \
intel-oneapi-ccl=2021.11.2-5 \
intel-oneapi-ccl-devel=2021.11.2-5 \
intel-oneapi-dnnl-devel=2024.0.0-49521 \
intel-oneapi-dnnl=2024.0.0-49521 \
intel-oneapi-tcm-1.0=1.0.0-435
```
<img src="https://llm-assets.readthedocs.io/en/latest/_images/oneapi.png" alt="image-20240221102252565" width=100%; />
<img src="https://llm-assets.readthedocs.io/en/latest/_images/basekit.png" alt="image-20240221102252565" width=100%; />
### Setup Python Environment ### Setup Python Environment
@ -258,8 +380,7 @@ To use GPU acceleration on Linux, several environment variables are required or
For Intel Arc™ A-Series Graphics and Intel Data Center GPU Flex Series, we recommend: For Intel Arc™ A-Series Graphics and Intel Data Center GPU Flex Series, we recommend:
```bash ```bash
# Configure oneAPI environment variables. Required step for APT or offline installed oneAPI. # Configure oneAPI environment variables.
# Skip this step for PIP-installed oneAPI since the environment has already been configured in LD_LIBRARY_PATH.
source /opt/intel/oneapi/setvars.sh source /opt/intel/oneapi/setvars.sh
# Recommended Environment Variables for optimal performance # Recommended Environment Variables for optimal performance
@ -274,8 +395,7 @@ To use GPU acceleration on Linux, several environment variables are required or
For Intel Data Center GPU Max Series, we recommend: For Intel Data Center GPU Max Series, we recommend:
```bash ```bash
# Configure oneAPI environment variables. Required step for APT or offline installed oneAPI. # Configure oneAPI environment variables.
# Skip this step for PIP-installed oneAPI since the environment has already been configured in LD_LIBRARY_PATH.
source /opt/intel/oneapi/setvars.sh source /opt/intel/oneapi/setvars.sh
# Recommended Environment Variables for optimal performance # Recommended Environment Variables for optimal performance
@ -288,6 +408,16 @@ To use GPU acceleration on Linux, several environment variables are required or
Please note that `libtcmalloc.so` can be installed by `conda install -c conda-forge -y gperftools=2.10` Please note that `libtcmalloc.so` can be installed by `conda install -c conda-forge -y gperftools=2.10`
- For **Intel iGPU**:
```bash
# Configure oneAPI environment variables.
source /opt/intel/oneapi/setvars.sh
export SYCL_CACHE_PERSISTENT=1
export BIGDL_LLM_XMX_DISABLED=1
```
> [!NOTE] > [!NOTE]
> Please refer to [this guide](../Overview/install_gpu.md#runtime-configuration-1) for more details regarding runtime configuration. > Please refer to [this guide](../Overview/install_gpu.md#runtime-configuration-1) for more details regarding runtime configuration.

View file

@ -47,7 +47,7 @@ conda activate llm
With the `llm` environment active, use `pip` to install `ipex-llm` for GPU: With the `llm` environment active, use `pip` to install `ipex-llm` for GPU:
- For **Intel Core™ Ultra Series 2 with Intel Arc™ Graphics (a.k.a. Lunar Lake)**: - For **Intel Core™ Ultra Processers (Series 2) with processor number 2xxV (code name Lunar Lake)**:
Choose either US or CN website for `extra-index-url`: Choose either US or CN website for `extra-index-url`: