From c34f85e7d075fb8671857f7fa1268705ede82579 Mon Sep 17 00:00:00 2001 From: Yuwen Hu <54161268+Oscilloscope98@users.noreply.github.com> Date: Wed, 15 May 2024 09:55:41 +0800 Subject: [PATCH] [Doc] Simplify installation on Windows for Intel GPU (#11004) * Simplify GPU installation guide regarding windows Prerequisites * Update Windows install quickstart on Intel GPU * Update for llama.cpp quickstart * Update regarding minimum driver version * Small fix * Update based on comments * Small fix --- .../source/doc/LLM/Overview/install_gpu.md | 44 +++++++------------ .../doc/LLM/Quickstart/install_windows_gpu.md | 29 ++++-------- .../LLM/Quickstart/llama_cpp_quickstart.md | 18 +++++--- 3 files changed, 36 insertions(+), 55 deletions(-) diff --git a/docs/readthedocs/source/doc/LLM/Overview/install_gpu.md b/docs/readthedocs/source/doc/LLM/Overview/install_gpu.md index fd749038..6a60a7d4 100644 --- a/docs/readthedocs/source/doc/LLM/Overview/install_gpu.md +++ b/docs/readthedocs/source/doc/LLM/Overview/install_gpu.md @@ -12,34 +12,15 @@ IPEX-LLM on Windows supports Intel iGPU and dGPU. IPEX-LLM on Windows only supports PyTorch 2.1. ``` -To apply Intel GPU acceleration, there're several prerequisite steps for tools installation and environment preparation: +To apply Intel GPU acceleration, please first verify your GPU driver version. -* Step 1: Install [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/) Community Edition and select "Desktop development with C++" workload, like [this](https://learn.microsoft.com/en-us/cpp/build/vscpp-step-0-installation?view=msvc-170#step-4---choose-workloads) +```eval_rst +.. note:: -* Step 2: Install or update to latest [GPU driver](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html) + The GPU driver version of your device can be checked in the "Task Manager" -> GPU 0 (or GPU 1, etc.) -> Driver version. +``` -* Step 3 (Recommended): Install [Miniconda](https://docs.anaconda.com/free/miniconda/) for Python environment management. Choose Miniconda installer for Windows. - -* Step 4: Install Intel® oneAPI Base Toolkit 2024.0: - - First, Create a Python 3.11 enviroment and activate it. In Anaconda Prompt: - - ```cmd - conda create -n llm python=3.11 libuv - - conda activate llm - ``` - - ```eval_rst - .. important:: - - ``ipex-llm`` is tested with Python 3.9, 3.10 and 3.11. Python 3.11 is recommended for best practices. - ``` - - Then, use `pip` to install the Intel oneAPI Base Toolkit 2024.0: - ```cmd - pip install dpcpp-cpp-rt==2024.0.2 mkl-dpcpp==2024.0.0 onednn==2024.0.0 - ``` +If you have driver version lower than `31.0.101.5122`, it is recommended to [**update your GPU driver to the latest**](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html): -### Install Miniconda +### Setup Python Environment Visit [Miniconda installation page](https://docs.anaconda.com/free/miniconda/), download the **Miniconda installer for Windows**, and follow the instructions to complete the installation. @@ -52,24 +45,18 @@ Visit [Miniconda installation page](https://docs.anaconda.com/free/miniconda/), - -### Install oneAPI - -Open the **Anaconda Prompt**. Then create a new python environment `llm` and activate it: +After installation, open the **Anaconda Prompt**, create a new python environment `llm`: ```cmd conda create -n llm python=3.11 libuv -conda activate llm ``` - -Use `pip` to install the **Intel oneAPI Base Toolkit 2024.0**: +Activate the newly created environment `llm`: ```cmd -pip install dpcpp-cpp-rt==2024.0.2 mkl-dpcpp==2024.0.0 onednn==2024.0.0 +conda activate llm ``` ## Install `ipex-llm` -With the `llm` environment active, use `pip` to install `ipex-llm` for GPU: -Choose either US or CN website for `extra-index-url`: +With the `llm` environment active, use `pip` to install `ipex-llm` for GPU. Choose either US or CN website for `extra-index-url`: ```eval_rst .. tabs:: diff --git a/docs/readthedocs/source/doc/LLM/Quickstart/llama_cpp_quickstart.md b/docs/readthedocs/source/doc/LLM/Quickstart/llama_cpp_quickstart.md index 3110acfb..be86987a 100644 --- a/docs/readthedocs/source/doc/LLM/Quickstart/llama_cpp_quickstart.md +++ b/docs/readthedocs/source/doc/LLM/Quickstart/llama_cpp_quickstart.md @@ -21,12 +21,13 @@ IPEX-LLM's support for `llama.cpp` now is available for Linux system and Windows #### Linux For Linux system, we recommend Ubuntu 20.04 or later (Ubuntu 22.04 is preferred). -Visit the [Install IPEX-LLM on Linux with Intel GPU](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html), follow [Install Intel GPU Driver](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html#install-intel-gpu-driver) and [Install oneAPI](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_linux_gpu.html#install-oneapi) to install GPU driver and Intel® oneAPI Base Toolkit 2024.0. +Visit the [Install IPEX-LLM on Linux with Intel GPU](./install_linux_gpu.html), follow [Install Intel GPU Driver](./install_linux_gpu.html#install-intel-gpu-driver) and [Install oneAPI](./install_linux_gpu.html#install-oneapi) to install GPU driver and Intel® oneAPI Base Toolkit 2024.0. -#### Windows -Visit the [Install IPEX-LLM on Windows with Intel GPU Guide](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_windows_gpu.html), and follow [Install Prerequisites](https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/install_windows_gpu.html#install-prerequisites) to install [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/) Community Edition and latest [GPU driver](https://www.intel.com/content/www/us/en/download/785597/intel-arc-iris-xe-graphics-windows.html). +#### Windows (Optional) -**Note**: IPEX-LLM backend only supports the more recent GPU drivers. Please make sure your GPU driver version is equal or newer than `31.0.101.5333`, otherwise you might find gibberish output. +IPEX-LLM backend for llama.cpp only supports the more recent GPU drivers. Please make sure your GPU driver version is equal or newer than `31.0.101.5333`, otherwise you might find gibberish output. + +If you have lower GPU driver version, visit the [Install IPEX-LLM on Windows with Intel GPU Guide](./install_windows_gpu.html), and follow [Update GPU driver](./install_windows_gpu.html#optional-update-gpu-driver). ### 1 Install IPEX-LLM for llama.cpp @@ -46,13 +47,12 @@ To use `llama.cpp` with IPEX-LLM, first ensure that `ipex-llm[cpp]` is installed .. note:: - for Windows, we use pip to install oneAPI. + Please run the following command in Anaconda Prompt. .. code-block:: cmd conda create -n llm-cpp python=3.11 conda activate llm-cpp - pip install dpcpp-cpp-rt==2024.0.2 mkl-dpcpp==2024.0.0 onednn==2024.0.0 # install oneapi pip install --pre --upgrade ipex-llm[cpp] ``` @@ -118,7 +118,9 @@ To use GPU acceleration, several environment variables are required or recommend export SYCL_CACHE_PERSISTENT=1 .. tab:: Windows - + + Please run the following command in Anaconda Prompt. + .. code-block:: bash set SYCL_CACHE_PERSISTENT=1 @@ -159,6 +161,8 @@ Before running, you should download or copy community GGUF model to your current .. tab:: Windows + Please run the following command in Anaconda Prompt. + .. code-block:: bash main -m mistral-7b-instruct-v0.1.Q4_K_M.gguf -n 32 --prompt "Once upon a time, there existed a little girl who liked to have adventures. She wanted to go to places and meet new people, and have fun" -t 8 -e -ngl 33 --color