ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Update WebUI quickstart (#10305)

Browse source
This commit is contained in:
Jason Dai 2024-03-03 22:18:26 +08:00 committed by GitHub
parent 27d9a14989
commit 4cb4db618d
1 changed files with 4 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
docs/readthedocs/source/doc/LLM/Quickstart/webui_quickstart.md
View file

@ -1,7 +1,7 @@
# Use Text Generation WebUI on Windows with Intel GPU # Use Text Generation WebUI on Windows with Intel GPU
This quickstart guide walks you through setting up and using the **Text Generation WebUI** (a Gradio web UI for Large Language Models) with `bigdl-llm`. This quickstart guide walks you through setting up and using the [Text Generation WebUI](https://github.com/intel-analytics/text-generation-webui) (a Gradio WebUI for running Large Language Models) with `bigdl-llm`.
A preview of the WebUI in action is shown below: A preview of the WebUI in action is shown below:
@ -12,7 +12,7 @@ A preview of the WebUI in action is shown below:
## 1 Install BigDL-LLM ## 1 Install BigDL-LLM
To use the WebUI, first ensure that BigDL-LLM is installed. Follow the instructions on the [BigDL-LLM Installation Quickstart for Windows with Intel GPU](https://github.com/intel-analytics/BigDL/blob/main/docs/readthedocs/source/doc/LLM/Quickstart/install_windows_gpu.md). To use the WebUI, first ensure that BigDL-LLM is installed. Follow the instructions on the [BigDL-LLM Installation Quickstart for Windows with Intel GPU](install_windows_gpu.html).
**After the installation, you should have created a conda environment, named `llm` for instance, for running `bigdl-llm` applications.** **After the installation, you should have created a conda environment, named `llm` for instance, for running `bigdl-llm` applications.**
@ -38,7 +38,7 @@ pip install -r requirements_cpu_only.txt
### Set Environment Variables ### Set Environment Variables
If you're running on iGPUs, set some environment variables by running below commands in **Anaconda Prompt**: If you're running on iGPUs, set some environment variables by running below commands in **Anaconda Prompt**:
> Note: For more details about runtime configurations, refer to [this link](https://bigdl.readthedocs.io/en/latest/doc/LLM/Overview/install_gpu.html#runtime-configuration): > Note: For more details about runtime configurations, refer to [this link](../Overview/install_gpu.html#runtime-configuration):
```bash ```bash
set SYCL_CACHE_PERSISTENT=1 set SYCL_CACHE_PERSISTENT=1
set BIGDL_LLM_XMX_DISABLED=1 set BIGDL_LLM_XMX_DISABLED=1
@ -118,4 +118,4 @@ If there are still errors on missing packages, repeat the installation process f
### Compatiblity issues ### Compatiblity issues
If you encounter **AttributeError** errors like shown below, it may be due to some models being incompatible with the current version of the transformers package because they are outdated. In such instances, using a more recent model is recommended. If you encounter **AttributeError** errors like shown below, it may be due to some models being incompatible with the current version of the transformers package because they are outdated. In such instances, using a more recent model is recommended.
<img src="https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_load_model_error.png" width=80%; /> <img src="https://llm-assets.readthedocs.io/en/latest/_images/webui_quickstart_load_model_error.png" width=80%; />