ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

[LLM] Update GPU installation doc (#9693)

Browse source 
* Temp

* Add win 2.1 doc supports

* Reorg layout

* Fix based on comments

* Small fix
This commit is contained in:
Yuwen Hu 2023-12-14 18:32:51 +08:00 committed by GitHub
parent 9a330bfc2b
commit 68d0c255fc
1 changed files with 229 additions and 76 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

305
docs/readthedocs/source/doc/LLM/Overview/install_gpu.md
View file

@ -1,116 +1,269 @@
# BigDL-LLM Installation: GPU # BigDL-LLM Installation: GPU
## Quick Installation
Install BigDL-LLM for GPU supports using pip through: ## PyTorch 2.1
### Prerequisite
```bash
pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-whl-stable-xpu # install bigdl-llm for Intel GPU
```
Please refer to [Environment Setup](#environment-setup) for more information.
```eval_rst ```eval_rst
.. note:: .. tabs::
.. tab:: Linux
The above command will install ``intel_extension_for_pytorch==2.0.110+xpu`` as default. You can install specific ``ipex``/``torch`` version for your need. BigDL-LLM for GPU supports on Linux with PyTorch 2.1 has been verified on:
.. important:: * Intel Arc™ A-Series Graphics
* Intel Data Center GPU Flex Series
* Intel Data Center GPU Max Series
Currently ``https://developer.intel.com/ipex-whl-stable-xpu`` is the only achievable source for ``-f`` option since IPEX 2.0.110+xpu and corresponding torch versions are not released on pypi. .. note::
We currently support the Ubuntu 20.04 operating system or later.
To enable BigDL-LLM for Intel GPUs with PyTorch 2.1, here're several prerequisite steps for tools installation and environment preparation:
* Step 1: Install Intel GPU Driver version >= stable_736_25_20231031.
.. seealso::
Please refer to our `driver installation <https://dgpu-docs.intel.com/driver/installation.html>`_ for general purpose GPU capabilities.
See `release page <https://dgpu-docs.intel.com/releases/index.html>`_ for latest version.
* Step 2: Download and install `Intel® oneAPI Base Toolkit <https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html>`_ with version 2024.0. OneMKL and DPC++ compiler are needed, others are optional.
.. seealso::
We recommend you to use `this offline package <https://registrationcenter-download.intel.com/akdlm/IRC_NAS/20f4e6a1-6b0b-4752-b8c1-e5eacba10e01/l_BaseKit_p_2024.0.0.49564_offline.sh>`_ to install oneapi.
.. tab:: Windows
BigDL-LLM on Windows supports Intel iGPU and dGPU.
To apply Intel GPU acceleration, there're several prerequisite steps for tools installation and environment preparation:
* Step 1: Install `Visual Studio 2022 <https://visualstudio.microsoft.com/downloads/>`_ Community Edition and select "Desktop development with C++" workload
* Step 2: Install or update to latest GPU driver
* Step 3: Install `Intel® oneAPI Base Toolkit <https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html>`_ 2024.0
```
### Install BigDL-LLM
We recommend using [Conda](https://docs.conda.io/en/latest/miniconda.html) to create a python 3.9 enviroment:
```eval_rst
.. important:: .. important::
``bigdl-llm`` is tested with Python 3.9, which is recommended for best practices. ``bigdl-llm`` is tested with Python 3.9, which is recommended for best practices.
``` ```
## Recommended Requirements ```eval_rst
.. tabs::
.. tab:: Linux
BigDL-LLM for GPU supports has been verified on: .. code-block:: bash
conda create -n llm python=3.9
conda activate llm
pip install --pre --upgrade bigdl-llm[xpu_2.1] -f https://developer.intel.com/ipex-whl-stable-xpu
.. tab:: Windows
.. code-block:: cmd
conda create -n llm python=3.9 libuv
conda activate llm
pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-whl-stable-xpu
```
## PyTorch 2.0
```eval_rst
.. note::
BigDL-LLM for GPU with PyTorch 2.0 only supports Ubuntu 20.04 operating system or later.
```
### Prerequisite
BigDL-LLM for GPU supports on Linux with PyTorch 2.0 has been verified on:
* Intel Arc™ A-Series Graphics * Intel Arc™ A-Series Graphics
* Intel Data Center GPU Flex Series * Intel Data Center GPU Flex Series
* Intel Data Center GPU Max Series * Intel Data Center GPU Max Series
To enable BigDL-LLM for Intel GPUs with PyTorch 2.0, here're several prerequisite steps for tools installation and environment preparation:
- Step 1: Install Intel GPU Driver version >= stable_647_21_20230714.
```eval_rst
.. seealso::
Please refer to our `driver installation <https://dgpu-docs.intel.com/driver/installation.html>`_ for general purpose GPU capabilities.
See `release page <https://dgpu-docs.intel.com/releases/index.html>`_ for latest version.
```
- Step 2: Download and install [Intel® oneAPI Base Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html) with version 2023.2.0. OneMKL and DPC++ compiler are needed, others are optional.
```eval_rst
.. seealso::
We recommend you to use `this offline package <https://registrationcenter-download.intel.com/akdlm/IRC_NAS/992857b9-624c-45de-9701-f6445d845359/l_BaseKit_p_2023.2.0.49397_offline.sh>`_ to install oneapi.
```
### Install BigDL-LLM
We recommend using [Conda](https://docs.conda.io/en/latest/miniconda.html) to create a python 3.9 enviroment:
```eval_rst ```eval_rst
.. note:: .. important::
We currently supoort the Ubuntu 20.04 operating system or later. Windows supoort is in progress. ``bigdl-llm`` is tested with Python 3.9, which is recommended for best practices.
``` ```
To apply Intel GPU acceleration, there're several steps for tools installation and environment preparation:
* Step 1, only Linux system is supported now, Ubuntu 22.04 and Linux kernel 5.19.0 is prefered.
```eval_rst
.. note::
Ubuntu 22.04 and Linux kernel 5.19.0-41-generic is mostly used in our test environment. But default linux kernel of ubuntu 22.04.3 is 6.2.0-35-generic, so we recommonded you to downgrade kernel to 5.19.0-41-generic to archive the best performance.
```
* Step 2, please refer to our [driver installation](https://dgpu-docs.intel.com/driver/installation.html) for general purpose GPU capabilities.
```eval_rst
.. note::
IPEX 2.0.110+xpu requires Intel GPU Driver version >= stable_647_21_20230714, see `release page <https://dgpu-docs.intel.com/releases/index.html>`_ for latest version.
```
* Step 3, you also need to download and install [Intel® oneAPI Base Toolkit](https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html). OneMKL and DPC++ compiler are needed, others are optional.
```eval_rst
.. note::
IPEX 2.0.110+xpu requires Intel® oneAPI Base Toolkit's version == 2023.2.0. We recommand you to use `this offline package <https://registrationcenter-download.intel.com/akdlm/IRC_NAS/992857b9-624c-45de-9701-f6445d845359/l_BaseKit_p_2023.2.0.49397_offline.sh>`_ to install oneapi.
```
## Environment Setup
For optimal performance with LLM models using BigDL-LLM optimizations on Intel GPUs, here are some best practices for setting up environment:
First we recommend using [Conda](https://docs.conda.io/en/latest/miniconda.html) to create a python 3.9 enviroment:
```bash ```bash
conda create -n llm python=3.9 conda create -n llm python=3.9
conda activate llm conda activate llm
pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-whl-stable-xpu # install bigdl-llm for GPU pip install --pre --upgrade bigdl-llm[xpu] -f https://developer.intel.com/ipex-whl-stable-xpu
``` ```
If you encounter network issues when installing ipex, you can refer to [Installing bigdl-llm[xpu] dependencies from Wheels](#installing-bigdl-llm[xpu]-dependencies-from-wheels) as an alternative method.
Then for running a LLM model with BigDL-LLM optimizations, several environment variables are recommended:
```bash
# configures OneAPI environment variables
source /opt/intel/oneapi/setvars.sh
export USE_XETLA=OFF
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
```
## Installing bigdl-llm[xpu] dependencies from Wheels
You can also install BigDL-LLM dependencies for Intel XPU from source achieves. First you need to install the target torch/torchvision/ipex versions from downloaded whls [here](http://ec2-52-27-27-201.us-west-2.compute.amazonaws.com/ipex-release.php?device=xpu&repo=us&release=stable) before installing bigdl-llm.
```bash
# get the wheels on Linux system
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/torch-2.0.1a0%2Bcxx11.abi-cp39-cp39-linux_x86_64.whl
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/torchvision-0.15.2a0%2Bcxx11.abi-cp39-cp39-linux_x86_64.whl
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/intel_extension_for_pytorch-2.0.110%2Bxpu-cp39-cp39-linux_x86_64.whl
```
```eval_rst ```eval_rst
.. note:: .. note::
Compatible versions of ``torch`` and ``torchvision`` are prerequisites for installing ``intel_extension_for_pytorch``. BigDL-LLM with PyTorch 2.0 only supports Ubuntu 20.04 operating system or later.
``` ```
Then you may install directly from the wheel archives using following commands: ## Known issues
### 1. For Linux users, Ubuntu 22.04 and Linux kernel 5.19.0 is prefered
```bash Ubuntu 22.04 and Linux kernel 5.19.0-41-generic is mostly used in our test environment. But default linux kernel of ubuntu 22.04.3 is 6.2.0-35-generic, so we recommonded you to downgrade kernel to 5.19.0-41-generic to archive the best performance.
# install the packages from the wheels
python -m pip install torch-2.0.1a0+cxx11.abi-cp39-cp39-linux_x86_64.whl ### 2. Best known configurations
python -m pip install torchvision-0.15.2a0+cxx11.abi-cp39-cp39-linux_x86_64.whl
python -m pip install intel_extension_for_pytorch-2.0.110+xpu-cp39-cp39-linux_x86_64.whl For running a LLM model with BigDL-LLM optimizations, several environment variables are recommended:
```eval_rst
.. tabs::
.. tab:: Linux
For Intel Arc™ A-Series Graphics and Intel Data Center GPU Flex Series, we recommend:
.. code-block:: bash
# configures OneAPI environment variables
source /opt/intel/oneapi/setvars.sh
export USE_XETLA=OFF
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
For Intel Data Center GPU Max Series, we recommend:
.. code-block:: bash
# configures OneAPI environment variables
source /opt/intel/oneapi/setvars.sh
export LD_PRELOAD=${LD_PRELOAD}:${CONDA_PREFIX}/lib/libtcmalloc.so
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=1
export ENABLE_SDP_FUSION=1
Please note that ``libtcmalloc.so`` can installed by ``conda install -c conda-forge -y gperftools=2.10``
.. tab:: Windows
Make sure you are using CMD as PowerShell is not supported:
.. code-block:: cmd
# configures OneAPI environment variables
call "C:\Program Files (x86)\Intel\oneAPI\setvars.bat"
set SYCL_CACHE_PERSISTENT=1
Please also set the following environment variable for iGPU:
.. code-block:: cmd
set BIGDL_LLM_XMX_DISABLED=1
.. note::
For the first time that **each** model runs on **a new machine**, it may take around several minutes to compile.
```
### 3. How to install from wheel
If you encounter network issues when installing IPEX, you can also install BigDL-LLM dependencies for Intel XPU from source achieves. First you need to install the target torch/torchvision/ipex versions from downloaded wheels [here](https://developer.intel.com/ipex-whl-stable-xpu) before installing `bigdl-llm`.
```eval_rst
.. tabs::
.. tab:: PyTorch 2.1 Linux
.. code-block:: bash
# get the wheels on Linux system for IPEX 2.1.10+xpu
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/torch-2.1.0a0%2Bcxx11.abi-cp39-cp39-linux_x86_64.whl
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/torchvision-0.16.0a0%2Bcxx11.abi-cp39-cp39-linux_x86_64.whl
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/intel_extension_for_pytorch-2.1.10%2Bxpu-cp39-cp39-linux_x86_64.whl
Then you may install directly from the wheel archives using following commands:
.. code-block:: bash
# install the packages from the wheels
pip install torch-2.1.0a0+cxx11.abi-cp39-cp39-linux_x86_64.whl
pip install torchvision-0.16.0a0+cxx11.abi-cp39-cp39-linux_x86_64.whl
pip install intel_extension_for_pytorch-2.1.10+xpu-cp39-cp39-linux_x86_64.whl
# install bigdl-llm for Intel GPU
pip install --pre --upgrade bigdl-llm[xpu_2.1]
.. tab:: PyTorch 2.1 Windows
.. code-block:: bash
# get the wheels on Windows system
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/torch-2.1.0a0%2Bcxx11.abi-cp39-cp39-win_amd64.whl
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/torchvision-0.16.0a0%2Bcxx11.abi-cp39-cp39-win_amd64.whl
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/intel_extension_for_pytorch-2.1.10%2Bxpu-cp39-cp39-win_amd64.whl
Then you may install directly from the wheel archives using following commands:
.. code-block:: cmd
# install the packages from the wheels
pip install torch-2.1.0a0+cxx11.abi-cp39-cp39-win_amd64.whl
pip install torchvision-0.16.0a0+cxx11.abi-cp39-cp39-win_amd64.whl
pip install intel_extension_for_pytorch-2.1.10+xpu-cp39-cp39-win_amd64.whl
# install bigdl-llm for Intel GPU
pip install --pre --upgrade bigdl-llm[xpu]
.. tab:: PyTorch 2.0 Linux
.. code-block:: bash
# get the wheels on Linux system for IPEX 2.0.110+xpu
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/torch-2.0.1a0%2Bcxx11.abi-cp39-cp39-linux_x86_64.whl
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/torchvision-0.15.2a0%2Bcxx11.abi-cp39-cp39-linux_x86_64.whl
wget https://intel-extension-for-pytorch.s3.amazonaws.com/ipex_stable/xpu/intel_extension_for_pytorch-2.0.110%2Bxpu-cp39-cp39-linux_x86_64.whl
Then you may install directly from the wheel archives using following commands:
.. code-block:: bash
# install the packages from the wheels
pip install torch-2.0.1a0+cxx11.abi-cp39-cp39-linux_x86_64.whl
pip install torchvision-0.15.2a0+cxx11.abi-cp39-cp39-linux_x86_64.whl
pip install intel_extension_for_pytorch-2.0.110+xpu-cp39-cp39-linux_x86_64.whl
# install bigdl-llm for Intel GPU
pip install --pre --upgrade bigdl-llm[xpu]
# install bigdl-llm for Intel GPU
pip install --pre --upgrade bigdl-llm[xpu]
``` ```