ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

[Doc] Simplify installation on Windows for Intel GPU (#11004)

Browse source 
* Simplify GPU installation guide regarding windows Prerequisites

* Update Windows install quickstart on Intel GPU

* Update for llama.cpp quickstart

* Update regarding minimum driver version

* Small fix

* Update based on comments

* Small fix
This commit is contained in:
Yuwen Hu 2024-05-15 09:55:41 +08:00 committed by GitHub
parent 1e00bd7bbe
commit c34f85e7d0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 36 additions and 55 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

44
docs/readthedocs/source/doc/LLM/Overview/install_gpu.md
View file

@ -12,34 +12,15 @@ IPEX-LLM on Windows supports Intel iGPU and dGPU.
IPEX-LLM on Windows only supports PyTorch 2.1. IPEX-LLM on Windows only supports PyTorch 2.1.
``` ```
To apply Intel GPU acceleration, there're several prerequisite steps for tools installation and environment preparation: To apply Intel GPU acceleration, please first verify your GPU driver version.
* Step 1: Install [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/) Community Edition and select "Desktop development with C++" workload, like [this](https://learn.microsoft.com/en-us/cpp/build/vscpp-step-0-installation?view=msvc-170#step-4---choose-workloads) ```eval_rst
.. note::
* Step 2: Install or update to latest [GPU driver](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html) The GPU driver version of your device can be checked in the "Task Manager" -> GPU 0 (or GPU 1, etc.) -> Driver version.
```
* Step 3 (Recommended): Install [Miniconda](https://docs.anaconda.com/free/miniconda/) for Python environment management. Choose Miniconda installer for Windows. If you have driver version lower than `31.0.101.5122`, it is recommended to [**update your GPU driver to the latest**](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html):
* Step 4: Install Intel® oneAPI Base Toolkit 2024.0:
First, Create a Python 3.11 enviroment and activate it. In Anaconda Prompt:
```cmd
conda create -n llm python=3.11 libuv
conda activate llm
```
```eval_rst
.. important::
``ipex-llm`` is tested with Python 3.9, 3.10 and 3.11. Python 3.11 is recommended for best practices.
```
Then, use `pip` to install the Intel oneAPI Base Toolkit 2024.0:
```cmd
pip install dpcpp-cpp-rt==2024.0.2 mkl-dpcpp==2024.0.0 onednn==2024.0.0
```
<!-- Intel® oneAPI Base Toolkit 2024.0 installation methods: <!-- Intel® oneAPI Base Toolkit 2024.0 installation methods:
@ -64,8 +45,15 @@ To apply Intel GPU acceleration, there're several prerequisite steps for tools i
### Install IPEX-LLM ### Install IPEX-LLM
#### Install IPEX-LLM From PyPI #### Install IPEX-LLM From PyPI
The easiest ways to install `ipex-llm` is the following commands, We recommend using [miniconda](https://docs.conda.io/en/latest/miniconda.html) to create a python 3.11 enviroment.
choosing either US or CN website for `extra-index-url`:
```eval_rst
.. important::
``ipex-llm`` is tested with Python 3.9, 3.10 and 3.11. Python 3.11 is recommended for best practices.
```
The easiest ways to install `ipex-llm` is the following commands, choosing either US or CN website for `extra-index-url`:
```eval_rst ```eval_rst
.. tabs:: .. tabs::
@ -73,6 +61,7 @@ choosing either US or CN website for `extra-index-url`:
.. code-block:: cmd .. code-block:: cmd
conda create -n llm python=3.11 libuv
conda activate llm conda activate llm
pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/ pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
@ -81,6 +70,7 @@ choosing either US or CN website for `extra-index-url`:
.. code-block:: cmd .. code-block:: cmd
conda create -n llm python=3.11 libuv
conda activate llm conda activate llm
pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/cn/ pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/cn/

29
docs/readthedocs/source/doc/LLM/Quickstart/install_windows_gpu.md
View file

@ -2,25 +2,18 @@
This guide demonstrates how to install IPEX-LLM on Windows with Intel GPUs. This guide demonstrates how to install IPEX-LLM on Windows with Intel GPUs.
It applies to Intel Core Ultra and Core 12 - 14 gen integrated GPUs (iGPUs), as well as Intel Arc Series GPU. It applies to Intel Core Ultra and Core 11 - 14 gen integrated GPUs (iGPUs), as well as Intel Arc Series GPU.
## Install Prerequisites ## Install Prerequisites
### Install Visual Studio 2022 ### (Optional) Update GPU Driver
Download and Install Visual Studio 2022 Community Edition from the [official Microsoft Visual Studio website](https://visualstudio.microsoft.com/downloads/). Ensure you select the **Desktop development with C++ workload** during the installation process.
```eval_rst ```eval_rst
.. tip:: .. tip::
The installation could take around 15 minutes, and requires at least 7GB of free disk space. It is recommended to update your GPU driver, if you have driver version lower than ``31.0.101.5122``. Refer to `here <../Overview/install_gpu.html#prerequisites>`_ for more information.
If you accidentally skip adding the **Desktop development with C++ workload** during the initial setup, you can add it afterward by navigating to **Tools > Get Tools and Features...**. Follow the instructions on `this Microsoft guide <https://learn.microsoft.com/en-us/cpp/build/vscpp-step-0-installation?view=msvc-170#step-4---choose-workloads>`_  to update your installation.
``` ```
<img src="https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_1.png" alt="image-20240221102252560" width=100%; />
### Install GPU Driver
Download and install the latest GPU driver from the [official Intel download page](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html). A system reboot is necessary to apply the changes after the installation is complete. Download and install the latest GPU driver from the [official Intel download page](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html). A system reboot is necessary to apply the changes after the installation is complete.
```eval_rst ```eval_rst
@ -44,7 +37,7 @@ Download and install the latest GPU driver from the [official Intel download pag
If the oneAPI installation hangs at the finalization step for more than 10 minutes, the error might be due to a problematic install of Visual Studio. Please reboot your computer and then launch the Visual Studio installer. If you see installation error messages, please repair your Visual Studio installation. After the repair is done, oneAPI installation is completed successfully. If the oneAPI installation hangs at the finalization step for more than 10 minutes, the error might be due to a problematic install of Visual Studio. Please reboot your computer and then launch the Visual Studio installer. If you see installation error messages, please repair your Visual Studio installation. After the repair is done, oneAPI installation is completed successfully.
``` --> ``` -->
### Install Miniconda ### Setup Python Environment
Visit [Miniconda installation page](https://docs.anaconda.com/free/miniconda/), download the **Miniconda installer for Windows**, and follow the instructions to complete the installation. Visit [Miniconda installation page](https://docs.anaconda.com/free/miniconda/), download the **Miniconda installer for Windows**, and follow the instructions to complete the installation.
@ -52,24 +45,18 @@ Visit [Miniconda installation page](https://docs.anaconda.com/free/miniconda/),
<img src="https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_5.png" width=70%/> <img src="https://llm-assets.readthedocs.io/en/latest/_images/quickstart_windows_gpu_5.png" width=70%/>
</div> </div>
After installation, open the **Anaconda Prompt**, create a new python environment `llm`:
### Install oneAPI
Open the **Anaconda Prompt**. Then create a new python environment `llm` and activate it:
```cmd ```cmd
conda create -n llm python=3.11 libuv conda create -n llm python=3.11 libuv
conda activate llm
``` ```
Activate the newly created environment `llm`:
Use `pip` to install the **Intel oneAPI Base Toolkit 2024.0**:
```cmd ```cmd
pip install dpcpp-cpp-rt==2024.0.2 mkl-dpcpp==2024.0.0 onednn==2024.0.0 conda activate llm
``` ```
## Install `ipex-llm` ## Install `ipex-llm`
With the `llm` environment active, use `pip` to install `ipex-llm` for GPU: With the `llm` environment active, use `pip` to install `ipex-llm` for GPU. Choose either US or CN website for `extra-index-url`:
Choose either US or CN website for `extra-index-url`:
```eval_rst ```eval_rst
.. tabs:: .. tabs::

18
docs/readthedocs/source/doc/LLM/Quickstart/llama_cpp_quickstart.md
View file

@ -21,12 +21,13 @@ IPEX-LLM's support for `llama.cpp` now is available for Linux system and Windows
#### Linux #### Linux
For Linux system, we recommend Ubuntu 20.04 or later (Ubuntu 22.04 is preferred). For Linux system, we recommend Ubuntu 20.04 or later (Ubuntu 22.04 is preferred).
Visit the [Install IPEX-LLM on Linux with Intel GPU](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html), follow [Install Intel GPU Driver](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html#install-intel-gpu-driver) and [Install oneAPI](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html#install-oneapi) to install GPU driver and Intel® oneAPI Base Toolkit 2024.0. Visit the [Install IPEX-LLM on Linux with Intel GPU](./install_linux_gpu.html), follow [Install Intel GPU Driver](./install_linux_gpu.html#install-intel-gpu-driver) and [Install oneAPI](./install_linux_gpu.html#install-oneapi) to install GPU driver and Intel® oneAPI Base Toolkit 2024.0.
#### Windows #### Windows (Optional)
Visit the [Install IPEX-LLM on Windows with Intel GPU Guide](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_windows_gpu.html), and follow [Install Prerequisites](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_windows_gpu.html#install-prerequisites) to install [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/) Community Edition and latest [GPU driver](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html).
**Note**: IPEX-LLM backend only supports the more recent GPU drivers. Please make sure your GPU driver version is equal or newer than `31.0.101.5333`, otherwise you might find gibberish output. IPEX-LLM backend for llama.cpp only supports the more recent GPU drivers. Please make sure your GPU driver version is equal or newer than `31.0.101.5333`, otherwise you might find gibberish output.
If you have lower GPU driver version, visit the [Install IPEX-LLM on Windows with Intel GPU Guide](./install_windows_gpu.html), and follow [Update GPU driver](./install_windows_gpu.html#optional-update-gpu-driver).
### 1 Install IPEX-LLM for llama.cpp ### 1 Install IPEX-LLM for llama.cpp
@ -46,13 +47,12 @@ To use `llama.cpp` with IPEX-LLM, first ensure that `ipex-llm[cpp]` is installed
.. note:: .. note::
for Windows, we use pip to install oneAPI. Please run the following command in Anaconda Prompt.
.. code-block:: cmd .. code-block:: cmd
conda create -n llm-cpp python=3.11 conda create -n llm-cpp python=3.11
conda activate llm-cpp conda activate llm-cpp
pip install dpcpp-cpp-rt==2024.0.2 mkl-dpcpp==2024.0.0 onednn==2024.0.0 # install oneapi
pip install --pre --upgrade ipex-llm[cpp] pip install --pre --upgrade ipex-llm[cpp]
``` ```
@ -118,7 +118,9 @@ To use GPU acceleration, several environment variables are required or recommend
export SYCL_CACHE_PERSISTENT=1 export SYCL_CACHE_PERSISTENT=1
.. tab:: Windows .. tab:: Windows
Please run the following command in Anaconda Prompt.
.. code-block:: bash .. code-block:: bash
set SYCL_CACHE_PERSISTENT=1 set SYCL_CACHE_PERSISTENT=1
@ -159,6 +161,8 @@ Before running, you should download or copy community GGUF model to your current
.. tab:: Windows .. tab:: Windows
Please run the following command in Anaconda Prompt.
.. code-block:: bash .. code-block:: bash
main -m mistral-7b-instruct-v0.1.Q4_K_M.gguf -n 32 --prompt "Once upon a time, there existed a little girl who liked to have adventures. She wanted to go to places and meet new people, and have fun" -t 8 -e -ngl 33 --color main -m mistral-7b-instruct-v0.1.Q4_K_M.gguf -n 32 --prompt "Once upon a time, there existed a little girl who liked to have adventures. She wanted to go to places and meet new people, and have fun" -t 8 -e -ngl 33 --color